medOS ultra

Unified Clinical Assistant

Generalize VoiceOrder into one chat/voice command surface with three tool classes (READ/ACTION/FORM-FILL) and distinct safety contracts.

15 min read diagramsUpdated 2026-05-30docs/architecture/unified-clinical-assistant.md

Status: Design / not yet implemented (2026-05-29). One-line: Generalize the shipped VoiceOrder feature from “order entry by voice” into a single chat/voice command surface where one LLM function-calling loop can drive any clinical action — place orders, record vitals, fill assessment forms — and answer read/analytics queries (“pull biomarkers A,B,C from date X–Y and correlate them”), all behind one registry of tools.

Read first: the engine substrate already exists and is ~70% generic. This doc is about consolidation + extension, not a rewrite. Companion shipped code: web/src/services/ai/{shared,voice-order,smart-diagnosis}/.

0. TL;DR — what exists vs. what’s new

Layer Status Evidence
Generic tool-calling loop ✅ exists, generic runAgentLoop() in runner-engine.ts — domain-agnostic, takes an injected ToolDispatcher
Shared wire types ✅ exists AgentLLM, ChatMessage, ToolCall, RunnerStep, CatalogMatch, SeenMatches in shared/types.ts
LLM adapter (Ollama/vLLM) ✅ exists OpenAICompatibleLLM in shared/client.ts
Hallucination guardrail ✅ exists SeenMatches “no catalogId the model didn’t first search for”
Confirm-before-commit gate ✅ exists (orders only) VoiceOrderDialogChrome diff-review
Two domains using the substrate ⚠️ duplicated voice-order + smart-diagnosis each reimplement the loop, slice, FAB
Generic session slice ❌ new today: voiceOrderSlice + smartDiagnosisSlice are near-duplicates
Agent/tool registry ❌ new no central map of domain → {tools, prompt, dispatcher, surface}
Unified surface (command bar) ❌ new today: per-domain FABs hand-mounted
Read/analytics tools ❌ new (data ready) observations/lab_results_ts/vital_signs_ts queryable; correlateSignals RPC is new
Form-fill-any-form ❌ new (schema type ready) FormFieldDefinition[] exists in form-registry.types.ts; forms_registry.schema_json mostly unpopulated

Bottom line: the hard mechanics (loop, guardrail, confirm gate, LLM adapter, audit timeline) are built and proven. The work is (a) de-duplicating the two existing domains behind one registry, and (b) adding three new tool classes with the right safety contract per class.

1. Vision & scope

Vision. A nurse or doctor opens one assistant (FAB or Cmd-K-style command bar), speaks or types a natural-language request, and the LLM either:

  • answers it (read/analytics — renders a chart/table inline), or
  • proposes an action (order / vitals / form) that the clinician confirms before it commits.

In scope (this doc):

  • A single, registry-driven assistant surface that hosts many tool sets.
  • Three tool classes (read / action / form-fill) with distinct safety contracts.
  • The forms_registry as the shared source of truth for both LLM form-targeting and human discoverability (“what can I say?”).
  • Phased rollout that ships value early without a big-bang refactor.

Out of scope (explicitly):

  • Replacing any existing form, order dialog, or write path. The assistant funnels into existing canonical paths (e.g. DialogOrder.prefill → Formik → useUniversalCart), exactly as VoiceOrder does today.
  • The LLM ever auto-committing a clinical write. Every write is confirmed by a human.
  • Backend model hosting / RAG corpus changes (covered by the LLM platform docs).
  • Cross-patient / cohort analytics (read tools are strictly single-patient-scoped for now).

2. The core reframe — three tool classes

The single most important design decision: do not treat “everything the assistant can do” as one undifferentiated tool list. Tools fall into three classes with very different risk and gating. Conflating them is how you get an unsafe system.

Class Example utterance Effect Gating Data/infra readiness
READ / analytics “pull HbA1c, fasting glucose & weight from Jan–Apr and show the trend / correlation” none (read-only) none — render result inline ✅ time-series ready; only correlateSignals RPC is new
ACTION “give ceftriaxone 2g IV q12h ×7d” · “record BP 130/80, HR 88” writes a clinical record mandatory confirm gate (diff-review) → existing write path ✅ orders shipped; vitals via writeThroughVitals
FORM-FILL “start a nutrition assessment: weight 60kg, no dysphagia, normal appetite” pre-fills a Formik form for review schema validation + confirm (form opens pre-filled, human submits) ⚠️ needs forms_registry.schema_json populated per form

Sequencing implication: ship READ first (safe, data ready, high-wow), then consolidate the engine, then ACTION (reuse the gate), then FORM-FILL (the real work — depends on the registry). See §9.

3. Current state (grounded)

3.1 What’s already generic ✅

runAgentLoop(args) (runner-engine.ts:107) is the canonical, domain-agnostic loop:

runAgentLoop({
  messages,            // ChatMessage[] — system prompt + user input
  tools,               // OpenAI-compatible tool schemas
  llm,                 // AgentLLM (Ollama/vLLM/mock)
  toolDispatcher,      // (call, args, seenMatches) => DispatchResult   ← the ONLY domain-specific part
  maxSteps, onStep, temperature,
}): Promise<{ finalPayload, clarification, steps }>

It already implements: the chat→tool-call→result→loop cycle, the SeenMatches hallucination guard (terminal payloads referencing un-searched catalogIds are rejected by the dispatcher), max-step runaway protection, and a streamed RunnerStep[] audit timeline. It knows nothing about orders or diagnoses. That is the whole substrate for a unified system.

3.2 What’s duplicated ⚠️

  • Two runners: voice-order/runner.ts (290 lines) and smart-diagnosis/runner.ts (435 lines) each reimplement the loop instead of calling runAgentLoop. They predate the extraction.
  • Two slices: voiceOrderSlice.ts and smartDiagnosisSlice.ts have near-identical phases (idle/capturing/running/review/dispatching/error) and reducer names (startSession, beginRunning, appendRunnerStep, setPayload, clearSession, setFeatureEnabled).
  • No registry: each domain’s FAB/panel is hand-mounted in its parent (VoiceOrderFAB ← PatientProfileVoiceOrderFAB ← PatientProfileDisplayRGL; DiagnosisModuleAI ← diagnosis workflow). Adding a domain = copy a whole directory.

3.3 The forms_registry linchpin

form-registry.types.ts already defines a rich, machine-readable field schema:

interface FormFieldDefinition {
  fieldName: string; fieldType: FormFieldType;     // text|number|date|select|...
  label: string; labelTh?: string; required: boolean;
  validation?: { min; max; pattern; custom };       // Yup-as-string
  options?: { value; label; labelTh }[];            // for select/radio
  nestedFields?: FormFieldDefinition[];             // nested objects
  conditionalDisplay?: { dependsOn; condition; value };
  hl7Mapping?: { segment; field; component };       // HL7 export
}
interface FormRegistryEntry { formId; fields: FormFieldDefinition[]; componentPath; category; requiredRoles; ... }

The forms_registry Supabase table (20260417_forms_registry.sql) has a schema_json JSONB column to hold exactly this — but today it’s only populated for kaigo compliance forms, not NutritionAssessment / nursing assessment / vitals. Clinical forms are hand-coded Formik with no enumerable field list.

This one table is the answer to the discoverability problem. One populated schema_json per form feeds two consumers: the LLM’s fillForm tool schema (so it knows what’s fillable) and a human “what you can say” palette (so the nurse sees which forms/fields exist). Single source of truth → both machine and human discoverability.

4. Target architecture

            ┌──────────────────────────────────────────────────────────┐
            │  AssistantCommandBar  (one surface: FAB + Cmd-K + voice)   │
            │  ── "what can I say?" palette (rendered from registry) ──  │
            └───────────────┬──────────────────────────────────────────┘
                            │ transcript / typed text + patient context
                            ▼
                  ┌───────────────────┐     reads
                  │  assistantSession │◀──────────────┐
                  │  slice (generic)  │               │
                  └─────────┬─────────┘               │
                            │ runAgentLoop(tools, dispatcher, llm)
                            ▼                          │
       ┌────────────────────────────────────┐         │
       │  runAgentLoop (shared, unchanged)   │         │
       └───────────────┬────────────────────┘         │
                       │ per-domain ToolDispatcher     │
        ┌──────────────┼───────────────┬───────────────┐
        ▼              ▼               ▼               ▼
   READ tools     ACTION tools    FORM-FILL tools   (future domains)
   getObservations proposeOrder   listForms          ...
   correlateSignals proposeVitals getFormSchema
        │              │            fillForm
        ▼              ▼               ▼
   inline render   CONFIRM GATE    CONFIRM GATE (form opens pre-filled)
   (chart/table)   (diff-review)   (Formik + prefill)
   no commit          │               │
                      ▼               ▼
                 existing write paths (DialogOrder.prefill, writeThroughVitals,
                 form submit) — UNCHANGED canonical truth

4.1 Agent / tool registry (new — P0)

A central, declarative map. Adding a domain = add one entry, no new plumbing.

// web/src/services/ai/registry.ts
interface AssistantDomain {
  id: string;                       // 'order' | 'vitals' | 'clinical-query' | 'form-fill'
  class: 'read' | 'action' | 'form-fill';
  tools: unknown[];                 // OpenAI tool schemas
  buildDispatcher: (ctx: AssistantContext) => ToolDispatcher;
  systemPrompt: (ctx: AssistantContext) => string;
  /** How the terminal payload is presented to the user. */
  render: 'inline' | 'confirm-gate' | 'form-prefill';
  featureFlag?: string;             // per-tenant kill switch
  requiredRoles?: string[];
}
export const ASSISTANT_REGISTRY: AssistantDomain[] = [ /* order, vitals, clinical-query, form-fill */ ];

AssistantContext = { patientId, encounterId, locale, user, catalogs }. The command bar can either (a) let the LLM pick the domain via a top-level router tool, or (b) merge all enabled domains’ tools into one list and let function-calling choose — start with (b), it’s simpler and the LLM is good at tool selection.

4.2 Generic session slice (new — P0)

Collapse voiceOrderSlice + smartDiagnosisSlice into one, with a discriminated-union payload:

// web/src/store/assistant/assistantSessionSlice.ts
type AssistantPhase = 'idle'|'capturing'|'running'|'review'|'dispatching'|'error';
interface AssistantSession {
  phase: AssistantPhase;
  domainId: string | null;          // which domain is active this turn
  transcript: string; interim: string;
  steps: RunnerStep[];              // reuse shared RunnerStep
  payload: unknown | null;          // domain payload (discriminated by domainId)
  result?: unknown;                 // for READ tools: the data to render inline
  error: string | null;
  featureEnabled: boolean;
}

Reducers mirror voiceOrderSlice verbatim (startSession, setTranscript, beginRunning, appendRunnerStep, setPayload, setError, submitForReview, clearSession). The existing voice-order slice becomes a thin compatibility shim or is migrated.

4.3 The shared runner (consolidate — P0)

Redirect runVoiceOrder() and runSmartDiagnosis() to call runAgentLoop() with their existing dispatchers. Net effect: delete ~200 lines of duplicated loop, zero behavior change (the loop logic is identical; it was extracted from voice-order). This is the safest, highest-leverage first step and de-risks everything after.

4.4 The unified surface (new — P1)

AssistantCommandBar replaces the per-domain FABs (which can keep working during migration). Responsibilities:

  • voice (Web Speech API, as VoiceOrderFAB does today) and a typed input;
  • show the live RunnerStep timeline (reuse the sandbox timeline component);
  • route the terminal payload by domain.render:
    • inline → render the READ result (chart/table) in a result panel;
    • confirm-gate → mount the generalized diff-review (§6);
    • form-prefill → open the target Formik form pre-filled (§5).

4.5 Tool-class contracts

READ (clinical-query) — getObservations(patientId, codes[], from, to, aggregate?) and correlateSignals(patientId, codeA, codeB, from, to, method?). Dispatcher calls Supabase RPCs (§7), returns rows/coefficients as finalPayload. Surface renders inline. No commit, no gate.

ACTION (order, vitals) — terminal tool proposeX(...) returns a structured payload validated against SeenMatches. Surface mounts the confirm gate. On accept → existing write path. Orders reuse proposeOrder as-is; vitals add proposeVitals → writeThroughVitals → observation pipeline (CDS auto-fire + EWS recompute happen downstream, unchanged).

FORM-FILL (form-fill) — three tools driven by the registry:

  • listForms(category?) → [{ formId, formName, category }]
  • getFormSchema(formId) → FormFieldDefinition[] (so the LLM knows the fields)
  • fillForm(formId, values: Record<fieldName, value>) → validated against the schema (required/min/max/options), returns a prefill payload. Surface opens the form pre-filled; human reviews + submits.

5. Form-fill: the registry-driven path (the real work)

Why it’s the hard part: today only DialogOrder accepts a prefill prop. Every other Formik form (NutritionDialog, IPD InitialIntakeForm/DailyAssessmentForm, …) has no enumerable schema and no prefill entry point. Generalizing requires a small, repeatable per-form contract — not bespoke wiring per form.

Per-form adoption contract (do once per form):

  1. Author its schema_json in forms_registry (FormFieldDefinition[]). This is the bulk of the effort; can be partially generated from the existing Formik initialValues + Yup schema.
  2. Accept a prefillData?: Record<string,unknown> prop and seed Formik initialValues from it (one-liner in most forms).
  3. Register the component path (componentPath/componentName) — already a field on FormRegistryEntry, and the module is already in the DynamicCoreApp enum.

Once schema_json is populated, both of these come for free:

  • LLM can getFormSchema + fillForm against any registered form;
  • the command bar renders a palette / collapsed field list (“Nutrition Assessment — weight, height, appetite, dysphagia, …”) so clinicians know what they can dictate — directly addressing the “list view so nurses know which form to say” idea.

Pilot: NutritionAssessment (modules.NutritionRequest) — one form, end-to-end, proves the contract before scaling to the other ~268 modules opportunistically (only forms that benefit need schemas).

6. The confirm-gate contract (safety invariant)

Generalize VoiceOrderDialogChrome into a domain-agnostic ``:

  • shows the transcript + the model’s rationale + overall confidence;
  • renders each proposed field; fields the model filled without explicit transcript evidence are flagged (unverifiedFields) and block submit until acknowledged (this is already how the order chrome works);
  • onAccept(verified) → hand to the canonical write path; onCancel → discard.

Invariant: no ACTION or FORM-FILL tool ever writes without passing through this gate. READ tools never reach it (nothing to commit).

7. Read/analytics: data layer is ready

Single-patient time-series is already clean and queryable:

  • observations (FHIR R4-aligned: patient_id, category, code LOINC, value_numeric, value_systolic/diastolic, effective_at, interpretation), projected to hypertables lab_results_ts and vital_signs_ts.
  • useObservations() already supports multi-code (.in('code', codes)) + date-range (.gte('effective_at', since)) fetches. So “pull signs A,B,C from X–Y” works today.

New compute (small):

  • RPC get_observations_series(patient_id, codes[], from, to, bucket?) — raw or time-bucketed (continuous aggregates vitals_hourly/vitals_daily already exist).
  • RPC correlate_signals(patient_id, code_a, code_b, from, to, method) — Pearson / Spearman / lagged correlation; returns coefficient + aligned series for plotting the “crossover.”

Guardrail: the LLM never writes SQL. It calls these typed, parameterized RPCs (codes validated against the observation catalog; patient scope enforced by RLS). This keeps “let the LLM query data” safe.

8. Safety & guardrails (consolidated)

  1. LLM is recommender-only for writes. It proposes; a human confirms via the gate. No tool commits autonomously.
  2. No hallucinated entities. ACTION tools reuse SeenMatches (catalogId must come from a prior search). FORM-FILL validates against schema_json. READ validates codes against the observation catalog.
  3. READ never gates / never writes. Read-only is the safe-by-construction class.
  4. No raw SQL / no free-text writes. All data access via typed RPCs; all writes via existing validated endpoints.
  5. Per-tenant feature flag per domain (mirror featureEnabled), so a site can enable read-only analytics without enabling form-fill.
  6. Full audit. Every turn already emits RunnerStep[] (user / tool_call / tool_result / final). Persist per session for audit; include patient/encounter/user.
  7. PHI posture. Vitals + labs are PHI. If using a hosted model, structured-only payloads, regional endpoint, BAA/DPA, per-inference audit — defer hosted models; default to the self-hosted Ollama already used by VoiceOrder.
  8. Role-gating. FormRegistryEntry.requiredRoles / domain requiredRoles filter which tools a given user’s assistant exposes.

9. Phased rollout

Phase Deliverable Risk New behavior?
P0 Route runVoiceOrder + runSmartDiagnosis through runAgentLoop; add assistantSessionSlice; add ASSISTANT_REGISTRY low (pure de-dup) none
P1 AssistantCommandBar surface + READ domain (getObservations, correlateSignals RPCs + inline chart). Biomarker crossover works end-to-end. low (read-only) yes — new, safe
P2 Extract `` from VoiceOrderDialogChrome; migrate orders onto it; add vitals ACTION tool medium yes — gated writes
P3 Populate forms_registry.schema_json for NutritionAssessment; add listForms/getFormSchema/fillForm; prefill-prop contract; “what can I say” palette medium-high yes — form-fill pilot
P4 Broaden form coverage opportunistically; multi-tool turns; domain router; per-tenant flags + audit persistence medium scale-out

P0+P1 are demo-ready and safe. P1 directly answers the original “biomarker crossover” ask.

10. Open questions (decide before P1)

  1. Domain selection: merge all tools into one list (LLM picks) vs. an explicit top-level router tool? Lean: merge for P1, add router only if tool count hurts accuracy.
  2. READ rendering: which chart component renders the inline series/correlation? (Reuse VitalsChart/SovereignBiomarkers sparklines, or a new component?)
  3. Correlation semantics: what does “crossover” mean to clinicians here — Pearson? threshold-crossing co-occurrence? time-lagged? Needs a clinician definition; ship Pearson + aligned plot first.
  4. Surface placement: keep the patient-profile FAB, or also a global Cmd-K bar? Patient context resolution differs.
  5. Slice migration: big-bang voiceOrderSlice → assistantSessionSlice, or run both and migrate VoiceOrder last? Lean: run both; migrate VoiceOrder in P2.
  6. Form schema authoring: hand-author schema_json, or build a one-time generator from Formik initialValues + Yup? Lean: generator for the first few, hand-tune.

11. Invariants (must always hold)

  1. The assistant never commits a clinical write without human confirmation.
  2. ACTION/FORM-FILL terminal payloads pass through ``; READ never does.
  3. No tool references an entity (catalogId, form field, observation code) it did not first discover via a search/schema/catalog tool.
  4. The LLM emits no SQL and no free-text into clinical records; only typed RPC calls and schema-validated field maps.
  5. Writes funnel into existing canonical paths — the assistant adds an input path, never a new source of truth.
  6. Every turn is fully audited via persisted RunnerStep[] with patient/encounter/user.
  7. Each domain is independently feature-flagged and role-gated per tenant.

12. File / module plan

New

  • web/src/services/ai/registry.ts — ASSISTANT_REGISTRY, AssistantDomain, AssistantContext
  • web/src/store/assistant/assistantSessionSlice.ts — generic session slice
  • web/src/common/components/medical/assistant/AssistantCommandBar.tsx — unified surface (FAB + typed + voice + timeline + result router)
  • web/src/common/components/medical/assistant/AssistantConfirmGate.tsx — generalized diff-review (extracted from VoiceOrderDialogChrome)
  • web/src/common/components/medical/assistant/AssistantPalette.tsx — “what can I say” list (rendered from forms_registry + registry)
  • web/src/services/ai/clinical-query/ — READ domain (tools.ts, dispatcher.ts, prompts.ts, types.ts, index.ts)
  • web/src/services/ai/form-fill/ — FORM-FILL domain (registry-driven tools + dispatcher)
  • Supabase RPCs: get_observations_series, correlate_signals
  • forms_registry seeds for NutritionAssessment (P3)

Touched (additive / delegation only)

  • web/src/services/ai/voice-order/runner.ts — delegate to runAgentLoop
  • web/src/services/ai/smart-diagnosis/runner.ts — delegate to runAgentLoop
  • Target Formik forms — accept optional prefillData prop (one per form, P3+)
  • web/src/App.tsx — mount AssistantCommandBar (alongside existing VoiceOrderBridge during migration)

Unchanged

  • runAgentLoop / shared/types.ts / shared/client.ts — already generic
  • All existing write paths, order dialog, observation pipeline, CDS/EWS

Appendix — worked example: “biomarker crossover” (P1, READ)

Doctor: “Pull HbA1c, fasting glucose and weight from January to April and show me how they move together.”

  1. clinical-query dispatcher; getObservations(pid, ['4548-4','1558-6','29463-7'], '2026-01-01','2026-04-30') → 3 aligned series.
  2. correlate_signals(pid, '4548-4','1558-6', …) (+ the third pair) → Pearson coefficients.
  3. finalPayload = { series, correlations }; render: 'inline' → command bar shows a multi-line trend chart + a small correlation matrix.
  4. Nothing written; RunnerStep[] audited. Total new code: 2 RPCs + 1 dispatcher + 1 chart panel.
Ask Anything