Architecture

Where this sits. The general system architecture — module layout, layers, tenancy — is Architecture. This doc zooms into the AI submodule; each live agent then has its own detail page under Models (§9). Reading order: general → this → agent.

Scope. This describes the v2 LangGraph.js design, from the multi-agent architecture diagram. The legacy v1 assistant is being retired and is not documented here. Some pieces below are live today; others are the planned expansion — §9 says which.

1. Shape in one line

Supervisor → specialist agents · policy-guarded tools · central checkpoints · a swappable model gateway. One graph per run — tenant-pinned, resumable, metered once.

2. The layered picture

3. Layers, top to bottom

• Channels — REST / WS API, WhatsApp (Redis-backed), Template WS. Every entry point funnels into one graph service.
• AiGraphService — the single front door: enters tenant context, pins the RLS connection, and derives thread_id = tenant:ws:conversation. Chooses how the run is routed (gate / direct / supervisor, below). Shipped as a router that delegates the pin + thread_id + metering to each specialist's facade — see §5.
• Checkpoint store — one central store, composite-keyed by tenant:ws:conversation. Backs both human-in-the-loop pauses and crash-resume.
• Routing — three ways in, cheapest-first: a deterministic gate (is_new_user → Onboarding), Direct mode (frontend pins the agent), and the LLM Supervisor (routes free-form requests to a specialist).
• Model gateway — CortexaChatModel → AiProviderFactory: failover, cache, rate-limit, and usage across OpenAI · Anthropic · Gemini · Kimi, selected by config (AI_DEFAULT_PROVIDER, default OpenAI). Agents never see a raw provider.
• Specialist agents — each owns a narrow domain and 3–8 tools (§9).
• Policy / permission guard — the authorization boundary in front of every tool: risk tier (SAFE / WRITE / DANGER), the caller's A/G/M/D permissions, RLS row-scope, an argument allowlist, and the tenant pin.
• Tool layer — guarded @AiToolDef tools with idempotency keys so a retried write never double-applies.
• Module services → DB — tools call the same records / messaging / workflow / email services the rest of the app uses; the DB is RLS-pinned per tenant / hyper.
• Observability — every hop appends to ai_run_traces (thread · tenant · ws · agent · tool · latency · tokens · cost · provider · approval · error).

4. Design decisions — our architecture & why

Stated as decision → why → trade-off.

Supervisor → specialists (not one mega-agent)

Why: the surface spans many domains (records, messaging, workflow, email, lead-ads, …). One agent holding every tool is unfocused, slower, and more expensive per call. A cheap supervisor routes to a specialist that carries only its own 3–8 tools. Trade-off: an extra LLM hop — paid down by routing the supervisor to a cheap model and by the gate / direct-mode paths that skip it entirely.

Deterministic onboarding gate

Why: a brand-new user should always land in onboarding — there's no reason to pay an LLM to decide that. is_new_user === true routes straight to Onboarding, and Onboarding itself is a deterministic state-machine (the LLM is used only to normalize free-text, ~$0.015/run). Trade-off: a hard-coded route, but it's the cheapest and most reliable path for the most common first action.

Direct mode (frontend-pinned agent)

Why: when the UI already knows the agent (the builder screen, the onboarding screen), it pins it and skips the supervisor. Trade-off: the client holds routing knowledge for those explicit, single-purpose screens — acceptable, and it's what runs today while the supervisor is built.

Central, composite-key checkpoints

Why: human-in-the-loop pauses (the builder's plan approval) and crash-resume both need durable state keyed by tenant:ws:conversation, on the tenant's pinned RLS connection so a forged key reads nothing. Trade-off: a DB dependency on the hot path; a run degrades to in-memory (completes, but loses resume) if the store is down.

Model gateway (swappable providers)

Why: agents must not know which provider they're on. CortexaChatModel → AiProviderFactory centralizes failover, cache, rate-limit, and usage, so swapping OpenAI/Anthropic/Gemini/Kimi is config, not code. Trade-off: a LangChain adapter layer — small, and it keeps a single provider policy for every agent.

Policy/permission guard at the tool boundary

Why: agents are non-deterministic, so authorization cannot live in the prompt. Every tool call is checked: risk tier, A/G/M/D permissions, RLS row-scope, an argument allowlist, and the tenant pin — with idempotency keys so retried writes don't double-apply. Trade-off: a guard on every call; it's the security boundary, so it is mandatory, not optional.

Meter once per run

Why: a run is Σ LLM hops (supervisor + agent + tools); per-call billing over-rounds and double-counts. Accumulate the cost, round once, idempotent by runId. Trade-off: a settle seam at the end of the run.

Observability traces

Why: multi-agent runs are hard to debug and cost blind without per-hop records; ai_run_traces makes cost, latency, and approvals auditable. Trade-off: write volume — worth it for debuggability and cost control.

5. Front door & supervisor routing (shipped)

The single entry from §3 — now built. It lives in src/api/v1/ai/v2/graph/routing/. Every chat turn enters AiGraphService.run(), which decides WHICH agent handles the turn and delegates to that agent's existing facade.

Router, not a mega-graph

The supervisor is a classifier that picks an agent key; the front door then delegates to that agent's own compiled graph + facade. This is a deliberate deviation from the PoC's single graph with specialist nodes: the shipped facades each already own their graph, credit metering, guard wiring, and (the builder) a human-in-the-loop handshake, so folding them into one graph would rewrite working, tested code for no behavior gain. Trade-off: the supervisor is one extra hop rather than an in-graph node — paid down by a cheap model, external (free) metering, and being skipped entirely in Direct mode. Adding a specialist is "describe it in the roster + wire its facade", never a graph rewrite.

The three routes

• Onboarding gate — applied UPSTREAM by the onboarding channel (is_new_user → the onboarding WS flow). The chat front door is reached only after onboarding, so it does not re-check the gate.
• Direct mode — the client pins the agent (agent on the send-message body, or a single-purpose screen like the builder), skipping the supervisor. crm_records pinned is byte-for-byte the pre-router behavior.
• Supervisor — no pinned agent ⇒ SupervisorRouter classifies the message to one roster key. Fail-safe: an empty message, a timeout, a parse miss, or an unknown name all resolve to crm_records (the safe read agent), so routing never dead-ends a turn. The reply is only a key, never an instruction, so injected text cannot act.

Transport (REST + streaming)

The chat front door is reachable two ways, same engine, same supervisor routing:

• REST — POST /ai-v2/conversations/:slug/messages blocks and returns the full answer. Simplest for request/response.
• WebSocket /ai-v2/chat — streams the turn token-by-token (assistant:token) with live tool progress (assistant:tool_start/_result), a terminal assistant:complete, and assistant:handoff when the supervisor routes to the builder. It also emits assistant:agent { agent, routedBy } up front — the chosen agent, before any token — so the UI can switch mode (schema, builder) as the turn starts. Gated by AI_V2_ENABLED; event names mirror v1's assistant:*. See the AI v2 — Integration guide.

The Onboarding and Workspace Builder agents run on their own WebSocket namespaces (/ai-v2/onboarding, /ai-v2/builder) — separate flows, separate flags.

The roster

AGENT_ROUTES is the single source of the routable agents and their descriptions — the supervisor's prompt is generated FROM it, so there is no second place to update. Today: crm_records (read / Q&A, runs inline) and workspace_builder (schema build — a review-then-apply flow, so routing there returns a redirect to the Builder rather than running it on the chat surface). Onboarding is a gate, not a roster entry.

A resolved key is dispatched through an exhaustive Record<AgentKey, handler> table, never a switch/if (registry-first, like the tool + field-type registries). A new agent is a compile error until it is both described in the roster and given a handler, so an agent can never be routed to before it can be handled.

Metering

The supervisor hop is a classification, not the run's billable work, so it is metered externally (free, like the onboarding gates); the delegated specialist bills its own run once via the credit-gate. No double-count — each token has exactly one billing owner.

6. Policy guard & idempotency (shipped)

The authorization + safety boundary from §3–§4, now built. It lives in src/api/v1/ai/v2/policy/. Every real tool call — both graphs' tool nodes and the builder's out-of-graph completion pass — routes through one choke point, GuardedToolExecutor.execute(), which runs two checks, in order, before a tool touches data.

The guard — coarse authorization from the snapshot

PolicyGuard.evaluate(tool, args, ctx) answers purely from the auth snapshot already on the run (ToolRunContext.adminContext) — no DB round-trip, no v1 helper, so it is unaffected by v1's eventual removal. It is the coarse gate; the fine row-scoping stays in DataService.

• Object tools (query_data, seed_sample_data) — gated by the caller's A/G/M/D level for the target object, carried on the snapshot as a {verb}.{slug} route name (view→read, add→create). Coarse pass = that name is present, i.e. level ≠ D.
• Capability tools (schema + automation authoring — create_object, create_field, create_relation, create_view, create_folder, create_automation) — gated by a binary feature permission, not a per-object level (create_automation → workflow.create).
• Super-admin bypasses the permission check for a known tool.
• Fail-closed: a tool with no registry entry is denied at the strictest tier, so a new tool cannot ship an unguarded write path — it must be classified first. A denial becomes a ToolMessage, never a throw, so the model reads it and can recover.

Risk tiers

DANGER is defined but unused: classified now for the delete/bulk tools that do not ship yet, so the tier system is complete the day the first one lands.

The policy registry

TOOL_POLICIES is an exhaustive Record<toolName, ToolPolicy> — a discriminated union over object | capability | exempt that is the single source of a tool's tier + gate. Because a missing entry fails closed, "add a tool" and "classify a tool" are the same step.

Idempotency — a resume never double-writes

For WRITE / DANGER, ToolIdempotencyStore (Redis) keys the call by ai:v2:tool:{db}:{ws}:{runId}:{tool_call_id}; on success it remembers the result, so a graph resume or retry replays the prior success instead of applying the write twice. It is best-effort: only successes are remembered (a failed write stays retryable), and Redis being unreachable degrades to "no dedup" — it never fails the run, because losing dedup is strictly safer than dropping the user's work.

The builder's completion pass has no model-issued tool_call_id, so it derives a deterministic one — completion:{tool}:{sha1(args)} — giving its out-of-graph writes the identical guard + dedup as a graph tool call.

7. Isolation & tenancy

Isolation is enforced once, at the front door: AiGraphService enters tenant context and pins the RLS connection before the graph runs, and every checkpoint, trace, and tool read/write flows through that pin. thread_id = tenant:ws:conversation scopes state per workspace, so a cross-workspace thread resolves to nothing on hyper-tenant. The AI never opens its own unpinned connection — it follows the pinning contract in ISOLATION.md instead of re-implementing filters.

8. Cost model

cost = tokens_in/1e6 · rate_in + tokens_out/1e6 · rate_out, summed over every LLM hop in the run. Rates come from the effective-dated ai_model_pricing catalog (PricingResolver), kept real by a daily OpenRouter price sync (OpenRouter is the price source only; calls stay direct to Gemini/Kimi). Routing is cost-aware — supervisor → cheap model, planner/strong work → strong model — and the whole run is billed once (§4, meter once per run) into the immutable ai_usage_event ledger (one real-USD row per run, idempotent on run_id).

A super-admin cost dashboard (admin/ai-analytics) aggregates that ledger — spend per workspace, by model / channel, and over time — so cost is auditable without new per-run capture. Details: Billing & Subscriptions → AI usage cost & pricing.

9. The agents

Each agent is a graph + a transport + its own guarded tools. Live agents have a detail page; planned agents are the diagram's target specialists. The live agents are reached in Direct mode (a single-purpose screen pins them) or through the shipped supervisor (§5) on the general chat surface; the remaining specialists are the planned expansion.

10. Further reading

The full product-level design narrative and the source diagram live in the CRM product docs (AI_MULTI_AGENT_ARCHITECTURE.md + .excalidraw). This page is the backend view; where the two differ, the shipping code and the linked model docs win.