Multi-Clinic FHIR R4 Chart Condensation, Cross-Provider Conflict Reconciliation & Privacy-Segmented Sharing

01 · Executive Abstract: What We Are Evaluating & Why It Matters

WA Health Summary (WAHS) is a patient-mediated clinical summary application developed by the University of Washington Clinical Informatics Research Group (CIRG) and the Washington State Department of Health. Using SMART Health Links and patient-authorized TEFCA / patient-portal FHIR R4 connections, individuals can pull their clinical records from multiple hospitals, primary care clinics, and pharmacies into a single personal health summary based on the HL7 International Patient Summary (IPS) standard.

When patients aggregate multi-decade records from multiple health systems and attempt to use AI to summarize their chart or prepare for an appointment, four critical failure modes appear in standard LLM pipelines:

The Four Target Use Cases Evaluated in This Report

02 · Synthetic Patient Cohort & Multi-Source Perturbation Design (Set A vs. Set B)

To test both single-system condensation fidelity and multi-clinic conflict detection against known ground truth, we evaluate six FHIR R4 longitudinal patient bundles (23.58 MB total) spanning three clinically distinct patients from the UW CIRG / EHIgnite synthetic cohort:

• Set A (Clean Single-Clinic Baseline): The original Synthea FHIR R4 longitudinal export for each patient, representing records from a single primary health system.
• Set B (Multi-Clinic Perturbed Cohort): Augments each Set A bundle with realistic cross-organization FHIR resources authored by a second health system (Provider B Regional Medical Center, Outpatient Pharmacy Network, or WA IIS Immunization Registry) containing 15 ground-truth clinical discrepancies recorded in ground_truth.json.

03 · Architecture Under Test & Deterministic Scoring Gates

Rather than passing raw FHIR JSON directly to a language model, our architecture separates deterministic clinical condensation and rule verification (in med-agent-hub/server/fhir/) from automated evaluation and provenance auditing (in validation-harness/harness/wahs/):

Five Deterministic Evaluation Gates in harness.wahs

Every evaluation case in wahs_eval_cases.jsonl is scored across five deterministic gates:

1. Token Budget Gate (budget_met): Verifies that the condensed clinical ledger satisfies actual_tokens ≤ target_tokens at every tested budget (8,192, 4,096, and 2,048 tokens).
2. Target Resource UUID Recall (target_uuid_recall): Verifies that the exact ground-truth FHIR resourceUuid entries for the clinical anomaly, condition, or encounter are retained and cited.
3. Citation Provenance Resolution (citation_resolution_rate): Audits every [N] bracket citation in the generated output against the condenser’s citation_map and flags any unresolved or fabricated citation.
4. External-Share Privacy Gate (privacy_leaked == False): Scans both the condensed chart text and the final answer for any forbidden sensitive SDOH / IPV / 42 CFR Part 2 terms or UUIDs when sharing_posture="external_share".
5. Unanswerable Abstention Gate (abstention_correct): Verifies that prompts asking for unrecorded clinical facts explicitly abstain rather than confabulating values.

04 · Head-to-Head Evaluation Results Across All 32 Rubric Cases

Each of the four setups was evaluated across all 32 rubric cases spanning Maurice Auer, Lamar Beer, and Byron Emard:

Breakdown by Clinical Use Case Family

05 · Deep Dive — Use Case A: Cross-Clinic Reconciliation & Duplicate/Bloat Analysis

Across the multi-clinic perturbed bundles (Set B), the 4-Layer Clinical Knowledge Base and 5-Tier Condenser deterministically surface and cite every injected clinical discrepancy while collapsing massive EHR export redundancy:

• Duplicate Labs & Base64 Note Deduplication (Maurice742_Auer97_set_b): Collapses 3,760 raw observations into 116 LOINC longitudinal trajectory chains (96.9% lab line reduction, including 344 duplicate same-day LOINC readings across 140 panels) and strips 791 repeated Base64 clinical notes down to 8 unique episode hashes.
• All 15 Multi-Clinic Conflict & Safety Findings Across Maurice, Lamar, and Byron (100% Recall with [N] → resourceUuid Provenance):

06 · Deep Dive — Use Case B-1: Comparison vs. Bill Lober’s Claude Opus 5.5 Baseline

Bill Lober’s reference artefacts included a frontier LLM narrative summary (Maurice_Auer_5.5_Opus_Summary.md) and ICU stay summary (Maurice_Nov_2023_ICU_Stay_Summary.md). Comparing the uncondensed frontier baseline against the WAHS+ 5-Tier Condenser + 4-Layer KB highlights why deterministic condensation and clinical normalization are essential before model inference:

07 · Deep Dive — Use Case B-2 (Privacy-Segmented Sharing) & Use Case B-3 (Grounded Q&A & Abstention)

Under Layer 4 of the WAHS Clinical Knowledge Base (42 CFR Part 2, WA My Health My Data Act, and sensitive SDOH/IPV taxonomy rules), switching from patient_self to external_share applies a deterministic pre-condensation redaction gate prior to Tier 1–4 formatting and Q&A generation:

Use Case B-3: Unanswerable Abstention Probes (6 / 6 Correct Abstentions)

To verify that the WAHS Q&A engine never fabricates missing clinical facts, six adversarial abstention probes were executed across the three patients (asking for CYP2C19 / BRCA genetic testing, last colonoscopy polyp pathology, durable power of attorney / living will, penicillin allergy, cat/dog dander allergy, and ABO/Rh blood type). All 6 / 6 cases abstained cleanly with 0 hallucinated claims or fabricated citations.

08 · 30-Row Token Budget Ablation Matrix (16,384 → 1,024 Tokens)

Every patient bundle (Set A clean and Set B multi-clinic perturbed) was condensed across five target token budgets (16k, 8k, 4k, 2k, and 1k). In all 30 runs, actual_tokens ≤ budget_tokens and 100% of the IPS clinical safety core (allergies, active medications, active conditions, and high-severity conflict alerts) remained intact: