Multimodal Human Event Stream (Layer 0)

The one “Layer 0” artifact the Membrane strategy (sovereign multimodal-biological-AI platform) says to ship first: a precise spec for how a single snapshot of code + lab + image + voice + genome events is typed, time-stamped, encoder-versioned, consent-scoped, and tokenized into one time-ordered Human Event Stream — expressed as an additive extension of the existing observations read model, not a new system.

Status: design intent, not a shipped system. The companion migration infrastructure/medbase/migrations/20260603d_human_event_stream_layer0.sql is written but NOT applied to any environment. Nothing here commits to federation, on-device training, or a model. Biometric/genetic-data compliance and any medical-device classification of an on-device runtime must be confirmed with qualified counsel per jurisdiction. “Membrane”/“Osmosis” are placeholder names.

---

1. What this is, and why it’s the right first step

Membrane’s argument: the schema is the real product — it commits you to nothing expensive yet is the precondition for every layer above it (runtime → SDK → modules → coordination plane), and it’s the natural thing to hand a neutral standards body. Everything else is gated on a model worth building on and a fusion benefit worth federating; the schema is not.

The disciplined move here is therefore not to build the SDK or the federated protocol, but to define the event shape on top of what medOS-ultra already has. The repo is the institution-tier node in Membrane’s §4 tiering (clinic/desktop node, never the patient phone — that’s the external Ever HealthWallet). Its observations table is already a typed, time-stamped, subject-scoped clinical event stream (infrastructure/medbase/migrations/20260512_observation_unification.sql). Layer 0 = add the four fields it’s missing, and write down the contract.

2. The canonical HumanEvent

One row = one typed event in a subject’s time-ordered stream:

HumanEvent {
  subject_ref      // operational: real patient_id (RLS-protected). corpus: pseudonym. (§3)
  time             // clinical effective time
  modality         // CODE | LAB | VITAL | IMAGE | AUDIO | GENOME | TEXT | WEARABLE
  type             // coding system + code: LOINC | SNOMED | ICD-10 | HPO | …
  numeric_value    // for discrete/quantized signals
  value_*          // systolic/diastolic/text/codeable variants (already in observations)
  payload_ref      // heavy modalities: pointer to encrypted blob (IPFS/wallet) — never inlined (§6)
  embedding        // OR the already-encoded vector, if an encoder ran (§8)
  encoder_id+ver   // which encoder produced `embedding` (reproducibility) (§8)
  consent_scope    // signed tags: what this event may be used for (§ readers enforce)
}

Two non-negotiable design commitments, both already honored by the repo’s architecture:

1. Heavy modalities are referenced, not inlined — a genome or an image lives encrypted in the filestore/wallet; the event carries a pointer or an already-encoded vector, never the raw blob (§6).
2. Every embedding records its encoder + version — a “voice-fatigue” vector from one encoder is not interchangeable with another’s, and the repo already runs three encoder dims (§8).

3. Two profiles — and the pseudonymity rule (read this before touching subject_ref)

Membrane’s schema says subject_ref is “pseudonymous, never a raw identifier.” That is true of the corpus/federated profile, not the operational one. The HumanEvent has two materializations, and conflating them would either break the clinical record or leak identity:

Profile	Store	subject_ref	Consent	Who reads it
Operational	observations (this migration)	real patient_id — it IS the live record; RLS-protected	RLS + consents + per-event consent_scope	clinicians, orchestrator, read-model consumers
Corpus / federated	ml_session_trajectories (Silver)	HMAC pseudonym; direct IDs dropped; quasi-IDs generalized; k-anon K=20	ml_export_consent (default FALSE), in-region S3, named owner, audit	de-identified training/RAG only

The de-identification boundary already exists in docs/architecture/ai-training-corpus.md (Bronze→Silver→Gold). This migration deliberately changes nothing about identity in the operational store — pseudonymization happens at export, on the Silver layer, exactly as that doc specifies. Layer 0’s job is to make the operational event shaped so the Silver projection is clean, not to pseudonymize the live chart.

4. Field mapping onto observations

HumanEvent field	observations column	Status
subject_ref	patient_id (+ encounter_id, tenant_id)	exists
time	effective_at (clinical) / recorded_at (system)	exists
type	code + code_system (+ code_display)	exists
numeric_value	value_numeric (+ value_systolic/diastolic/text/codeable)	exists
provenance	source_kind, source_id, device_id, recorded_by, status	exists
payload_ref	source_form_ref + metadata JSONB (IPFS CID / wallet ref)	exists (convention)
modality	modality (CHECK: 8 values)	added by 20260603d
embedding	embedding vector(768) + partial HNSW	added by 20260603d
encoder_id / encoder_version	encoder_id, encoder_version	added by 20260603d
consent_scope	consent_scope TEXT[]	added by 20260603d

modality is a new column, not an overload of the FHIR category (vital-signs/biomarker/device/assessment/imaging): the two are orthogonal axes (a row can be category='imaging' AND modality='IMAGE'), and reusing category would break its existing CHECK and every existing row.

5. The additive migration

infrastructure/medbase/migrations/20260603d_human_event_stream_layer0.sql — additive, idempotent, schema-only:

• ADD COLUMN IF NOT EXISTS for all four fields (every column nullable / NULL-tolerant → existing rows stay valid).
• Guarded CHECK constraints (DO $$ … pg_constraint $$) so re-runs are safe.
• CHECK (embedding IS NULL OR encoder_id IS NOT NULL) — no anonymous embeddings.
• Partial HNSW index WHERE embedding IS NOT NULL — empty/cheap until the (separate, not-yet-built) producer fills it.
• No change to existing columns, constraints, indexes, or RLS. New columns inherit the table’s RLS automatically.
• Includes a commented manual-rollback block.

6. Heavy modalities by reference (already the pattern here)

Membrane: the stream carries a pointer or an encoded vector, never the raw genome/image. medOS-ultra already does exactly this:

• Filestore + IPFS (services/filestore) stores blobs by CID; the event carries the CID, not bytes.
• Patient-wallet Bucket B (docs/integrations/ever-patient-wallet.md): X-rays / MRIs / wound photos / FHIR bundles are AES-256-GCM-encrypted under a wallet-issued CEK, nacl.box-wrapped to the hospital pubkey — the hospital holds a revocable grant, not the key. Bucket A (facility ops) uses the facility DEK.
• On-device encoding precedent: bio-identity-kit already computes a face embedding in-browser (onnxruntime-web) where the raw biometric never leaves the device — a working instance of “encode locally, transmit only the vector.”

So payload_ref = an IPFS CID or a wallet Bucket-B reference; embedding = the encoded vector when an encoder (server or on-device) has run.

7. The align keystone is already solved

Membrane’s align module does two jobs with one objective: cross-modal alignment and mapping messy local free-text into clean coded space. The second — the keystone that makes a type mean the same thing across two hospitals — already exists: @medical-kit/code-systems (web/packages/medical-kit/src/code-systems/) resolves any concept to the best coding across SNOMED/LOINC/ICD/RxNorm/… via profile (availability) × domain (preferred order), with honest null fallback. The terminology tables (th/ja/fil), smart-diagnosis (free-text→ICD/SNOMED), and the coder’s historical RAG (bge-m3) sit on top. Without this discipline the schema is a container for mutually unintelligible local dialects; with it, the type field is interoperable on day one.

8. Multi-encoder / multi-dim reality

The repo already runs three embedding dims, which is exactly why encoder_id is mandatory:

Encoder	Dim	Store
nomic-embed-text (on-prem default, LLM_EMBEDDING_DIM)	768	llm_embeddings (20260514c_llm_platform.sql) — and observations.embedding here
bge-m3 (coding RAG)	1024	llm_embeddings_lg (20260601a_coding_historical_rag.sql)
OpenAI text-embedding-3-small	1536	conference_decisions.ai_embedding (005_conference_decisions.sql)

A pgvector column has a fixed dim, so observations.embedding is sized to the on-prem default (768) for the common case. Other-dim encoders do not widen this column — they fan out to a companion table following the existing precedent, e.g. human_event_embedding_lg (observation_id, encoder_id, encoder_version, embedding vector(1024), created_at). The encoder_id on the base row records which encoder is authoritative for that event. (The companion table is a documented follow-up, not part of 20260603d.)

9. What this commits to — and what it explicitly does NOT

Commits to: a typed, time-stamped, modality-tagged, encoder-versioned, consent-scoped event shape, additive over the live read model, safe to publish and hand to a standards body.

Explicitly does NOT (gated per Membrane’s sequencing — each unlocks only after the prior shows measurable benefit):

• ❌ No federation, secure aggregation, differential privacy, or LoRA delta sync (Osmosis — absent by design; gated on single-modality FL working at all).
• ❌ No on-device training or tiered placement runtime (the secure-custody half of Layer 1 exists; the federated-compute half does not).
• ❌ No embedding producer — embedding ships NULL; the async encoder that fills it is a separate, gated job.
• ❌ No survival/time-to-event module, and no append(snapshot)/predict_next_events() trajectory SDK.
• ❌ No pseudonymization of the operational store (§3).

This mirrors Membrane’s own discipline: the platform is the last organ to grow, not the first. Layer 0 is the only piece that is useful now, costs nothing, and is reversible.

10. Invariants

1. Additive-only. Layer-0 changes never alter or drop an existing column/constraint/index/RLS policy.
2. Operational store keeps the real subject_ref. Pseudonymization is Silver-layer only.
3. No anonymous embeddings. embedding IS NOT NULL ⇒ encoder_id IS NOT NULL (DB-enforced).
4. Heavy modalities by reference. Raw blobs never inline in an event; only a CID/wallet ref or an encoded vector.
5. consent_scope NULL = inherit. A NULL scope changes no current behavior (ambient RLS/consents govern); non-null is an explicit, reader-enforced gate.
6. One encoder dim per column. Mixed dims fan out to companion tables; never widen in place.

11. Follow-ups (each separately gated, none included here)

• Backfill modality from category/source_kind (one-shot UPDATE, separate migration): vital-signs→VITAL, biomarker→LAB, imaging→IMAGE, device→WEARABLE|VITAL, assessment→TEXT|CODE. Keep additive; don’t block the schema on it.
• Embedding producer: an async edge function/worker that encodes new observations rows (on-prem nomic-embed-text) and stamps encoder_id/encoder_version — mirrors the llm_embeddings upsert path.
• Companion table human_event_embedding_lg for non-768 encoders (§8).
• Silver projection: extend ai-training-corpus.md’s ml_session_trajectories to carry modality + (optionally) the embedding, with the existing HMAC-pseudonymize + k-anon de-id.
• Hand the schema to a neutral steward (Membrane §10): standards are won by consortia, not single projects — own the network/refinery, donate the spec.

12. Query & retrieval (built — see membrane-buildout)

Three retrieval modes sit on this schema; all are SHIPPED as additive code (not yet applied/deployed):

1. Structured cohort — find_patient_cohort RPC (20260603g) joins patient_clinical_cache (age/gender) + encounter_coded_diagnosis (ICD) + observations (lab thresholds). Answers “how many diabetic patients aged 40–60.” SECURITY INVOKER (RLS-enforced); aggregates by default, subject list opt-in + capped.
2. Per-patient time-series — the shipped clinical-query READ domain + the getHumanEvents accessor (web/src/services/human-event-stream/).
3. Semantic vector — search_observations_by_embedding RPC (20260603f) over observations.embedding, populated by the observation-embedder edge fn (nomic-embed-text, 768-dim).

The LLM never writes SQL. The cohort-query assistant domain (web/src/services/ai/cohort-query/, registered in ASSISTANT_REGISTRY, feature-flagged assistant.cohortQuery.enabled, admin-gated) calls these RPCs as allowlisted, parameterized tools; codes are grounded via the terminology server (no hallucinated codes); patient identities are never returned on the LLM path. Demo surface: CohortQueryConsole at /admin/intelligence/cohort-query. Full inventory + rollout: docs/architecture/membrane-buildout/00-MASTER.md.

---

Companion to the Membrane platform analysis. Design intent, not a build commitment.