Part 1. Fragmented Authority
Many systems have fragmented authority: no single component can provide an accurate and complete view of a domain’s state with the guarantees that domain requires.
For example, consider a system that integrates with dozens of external services by transforming data received from several upstream systems.
Several experienced colleagues proposed connecting each adapter directly to the input sources it needed as a seemingly pragmatic approach.
That pragmatism would have been unsustainable. It would produce local consistency but global inconsistency, as data silently drifted across integrations. Also:
- Architectural changes – replacing RabbitMQ with Kafka topics, bumping a topic version – would require coordinated work across multiple adapters instead of remaining local to a single component.
- Unpredictable changes, such as an upstream source silently changing its semantics, would increase the blast radius and demand careful fixes across every integration that depended on the old semantics.
The problem would compound with every new integration.
Instead, we built a platform that ingests upstream data, normalises it, filters it, aggregates it and exposes it to adapters through a unified, versioned API. Crucially, it establishes where the knowledge about a business concept is defined and enforced.
By becoming the source of truth for our subdomain, it defined a clear place to observe incoming data, diagnose invalid metadata, and present consistent data with a uniform model to downstream services.
It also made the system easier to reason about, shielding developers working on specific downstream integrations from upstream complexity. We kept finding capabilities it enabled for years after its first release in production.
Because this approach worked so well, I kept wondering how systems in other domains could benefit from it.
The Fragmentation Problem
Fragmented authority is an epistemological problem: it concerns what a system can know and expose about a business concept, not where data lives.
Data can be distributed, and the system can still provide an accurate view of the business concept’s state. Conversely, a system can centralise everything in a single store and still fail to provide the view and guarantees the business requires.
Fragmented authority is also not the same as weak consistency. Some domains tolerate eventual consistency in exchange for availability and still meet their requirements.
The guarantees an authoritative core must provide are not universal. They are domain-dependent.
The Authoritative Core
The Authoritative Core is not defined by topology, a specific component, or a technology choice. It defines a logical architectural boundary where correctness is guaranteed. It is responsible for enforcing business invariants, for maintaining the canonical state of a business concept, and for defining the authoritative API for accessing or modifying state.
It may be implemented by distributed components, and it defines what correctness means according to the domain requirements: some domains demand strong consistency, while others benefit from eventual consistency in exchange for availability.
Part 2. Authority in Action
A Simplified Clinical Care System
Consider a fictionalised network of hospitals and clinics building a clinical care system. It must be the authoritative source for a patient’s evolving clinical state, support safe clinical treatment, and enable analysis of historical data.
It must support point-in-time queries of a patient’s current and historical state; trigger alerts when it detects risks and anomalies; supply data for population-level analytics and clinical studies; and support auditability, traceability and incident investigation.
The system must provide accurate information at all times and with no data loss. Effective treatment does not permit eventually consistent medication dosage. It must be fault-tolerant and highly available – downtime directly degrades the quality and safety of patient care.
This system cannot tolerate fragmented authority. A system that cannot provide data with the guarantees engineers need will fail to provide it with the guarantees patients need — accurate medication dosage and deterioration alerts cannot be best-effort.
▶ Sources and Inputs
| Input | Source | Type |
|---|---|---|
| Admissions, referrals, transfers, discharges | Administrative or partner systems | Clinical Fact |
| Diagnoses and clinical observations | Clinicians, EHRs, or partner systems | Clinical Fact |
| Medication administration records | Clinicians, EHRs, or partner systems | Clinical Fact |
| Device telemetry and physiological signals | Medical devices | Clinical Fact |
| Contraindications, dosage guidance and allergies | Clinical references sources | Clinical Reference (used to interpret and validate facts, or guide decisions) |
▶ Core Capabilities
| Capability | Example | Purpose |
|---|---|---|
| Point-in-time queries | Patient condition, administered medication at time t | Safe clinical treatment |
| Alerts | Arrhythmia detection and SpO₂ below safe threshold | Risk mitigation |
| Population-level insights | Clinical studies, trend analyses, cohort discoveries | Research and clinical improvements |
| Historical analyses | Retrospective audits, outcome analyses | Quality control and incident analyses |
The Clinical Care System Components
Trust Boundaries (How Data Enters the System)
Components at the system’s boundary secure the system and ensure that only authenticated, well-structured, normalised messages are accepted.
Authentication is performed at this stage. High-volume, heterogeneous, asynchronous events enter the system via message bus and the ingestion service. The API Gateway is the entry point for synchronous requests.
Interactive Path
Clinical observations and medication administration are examples of events entered through the Clinical UI. The UI submits directly to the Clinical Service through the API Gateway.
Ingestion Path
The Ingestion Service receives a high-volume stream of heterogeneous, unreliable data – diagnoses, clinical observations, medication administrations, physiological telemetry – from sources that don’t speak the clinical domain: medical devices, external systems.
It converts payloads from other systems into canonical messages and handles unreliable input: duplicate, inconsistent, conflicting, malformed, and delayed events. It produces a stream of telemetry data and clinical events downstream.
The Ingestion Service stamps patientId onto the event sent downstream to bridge device and patient identities. Every event carries an eventAt timestamp that positions it in the patient’s clinical timeline and preserves ordering guarantees during replays. It stamps ingestionAt on each event for operational monitoring. The two timestamps distinguish when something happens from when it enters the system.
Asynchronous events that fail to be processed cannot inform a patient’s state and are routed to a dead-letter queue: events that silently disappear would leave the system in an inconsistent state without any indication of the underlying reason.
How Truth is Established
The core components receive structurally valid inputs from the layer above. Different events may follow different processing paths and live in different stores if different kinds of canonical state are required.
The frequency and the meaning of the normalised input about a patient’s state vary enormously. Events such as diagnoses are modelled as commands sent to the Clinical Service. Physiological signals arrive continuously and at high volume.
Observed Vital Signs (Telemetry)
Medical device streams are processed and stored in a time-series database.
Raw telemetry arrives as a live stream from devices, thus it’s often impossible to replay it. Downsampled telemetry is derived from raw – since raw events have short retention, downsampled telemetry is also not reconstructable in practice. Therefore, they are both canonical.
Clinical Observations
Clinical observations are handled by the Clinical Service and stored as an immutable, timestamped log. The event log provides a complete canonical clinical record, but reading from it is slow.
Query-tables are updated within the same transaction as the event append, keeping reads immediately consistent with writes. This allows clinical updates to be immediately visible, which is necessary for safe treatment.
The authoritative core enforces correctness before accepting new events: before administering a medication, the system can check if the dosage is appropriate, which may include consulting the Clinical Reference component.
The Clinical Reference provides information the core needs to make decisions, but which it has no authority over.
Data Held, But Not Owned
The data that the system has no authority over is clearly demarcated: patients’ demographics, clinical references. A local copy of non-canonical data can be stored, but the authority over it is external.
Other non-canonical state is required by the care system besides clinical references. Events signalling admission, transfer, and discharge are consumed from external systems and used to initialise a patient record and update operational state. They are operationally important, but not owned.
The non-canonical data should be reconstructable from its source; the core data must be safe against disaster.
What Derives from Truth
Once truth is established, canonical facts are available to downstream services, each with their particular need, external or internal to the clinical care system. The Alert Engine, the analytics layer and external systems consume an unbounded stream of events from the core.
The Alert Engine processes data in near real-time and triggers alerts prompting clinical interventions, such as arrhythmia detection or SpO₂ below a safe threshold.
The analytics layer processes large volumes of clinical facts without affecting the system’s operational path, enabling operational and analytical dashboards, population-level insights and historical analyses.
When a patient is transferred, the clinical care system sends a snapshot of the required data to external systems to allow effective care. How they use that data is their concern; the authority over it is not transferred.
The same principles apply to the integration system discussed in the introduction: adapters consume and store a snapshot of the platform‘s canonical data. They can store additional information to support a particular integration, making them the canonical reference regarding the state of that integration within the organisation. The integration system is itself a federation of authoritative cores.
Emergent Capabilities
When systems have clear boundaries where truth is established, security, auditability, data governance and compliance follow. Access controls have well-defined boundaries to secure. Event logs provide auditability and traceability of clinical history by design. Data governance and compliance have a clear place without needing to be retrofitted.
Conclusion
Each business concept should have a clear authority. In the opening example, the platform provides the authoritative view of the business for each external system integration; each of its sources is the authority for their own business concept. The Clinical Care System is the authority on clinical care.
Regardless of the complexity of an organisation, the goal is logical boundaries where information about each business concept is guaranteed to be correct, with a clear separation of canonical and non-canonical state. A company may need a single authoritative core for a single primary business concept; another may require a federation of authoritative cores, one per concept.
When a business concept has a clear home, effective security, monitoring and data governance are tractable. When it doesn’t, extending a production system may require collating information from different sources, creating new views without the necessary correctness guarantees the business requires. These different views multiply, further fragmenting authority. This is not a static problem; it compounds. When we cannot fully trust the data our components process, how can we provide guarantees on what they produce?