Architecture Overview

RelataDB is a governed temporal knowledge database. It collapses the multi-database stack — relational, graph, vector, full-text search, and audit — into a single Rust binary. Policy, provenance, and bi-temporal history are in the query path, not bolted on after the fact.

The trade-off is deliberate: the engineering cost of building all planes in-process is accepted in exchange for eliminating the consistency, governance, and operational seams that fragment a polyglot stack.

Deployment profiles

One binary (relata), three profiles. The profile changes capacity knobs and which subsystems boot. The wire protocol, on-disk format, SQL dialect, and ACL semantics are identical across all three.

Profile	Topology	Headroom	Status
`free` (`lite` alias removed — rejected at startup)	Embedded, single binary, local disk	~1 B entities	Stable
`server`	Single node + S3-compatible object store	~10 B entities	Stable
`cluster`	Coordinator + reader(s) + writer + indexer	100 B – 1 T+ entities	Alpha — not recommended for production

Set with RELATA_PROFILE=free|server|cluster.

The five planes

Every feature belongs to exactly one plane. Cross-plane work goes through typed interfaces; no shared mutable globals exist between planes.

Plane	Owns
Data plane	Ontology, typed tables, link tables, IdentityIndex, bitmaps, vectors, full-text postings
Compute plane	Planner, executor, cost-based optimizer, MV refresh, DAG job engine
Trust plane	Cedar-inspired ABAC, classification labels, audit log, PROV-O provenance, hash-chained manifests
Identity plane	`Identity` umbrella type, IdentityIndex MV, `RESOLVE_IDENTITY`, `IDENTITY_CLUSTER`, `SAME_IDENTITY`
Distribution plane	Coordinator/reader/writer/indexer roles, write leases, hash partitioning, multi-region replication

Data flow

Ingest

Writes are split into an eager phase (synchronous, in the writer's commit critical path) and a lazy phase (asynchronous, re-runnable from the WAL).

SOURCE         Connector         Writer (eager)                 Storage             Indexer (lazy)
──────         ─────────         ──────────────                 ───────             ──────────────
CSV  ─┐                          validate + canonicalize ──►  Parquet segment
JSON  ├─► normalize ────────►    ACL + tenant stamp       ──►  CSR adjacency    ──► IdentityIndex delta
API  ─┤                          WAL append (fsync/async)  ──►  blob (SHA-256)       embeddings (HNSW)
Kafka ┘                          row write (&self)          ──►  PROV-O assertion     MV refresh
                                                            └──► commit manifest      SmartIngest

The eager phase does only declared-column work: parse, canonicalize, stamp principal/agency/classification, append WAL, write row. It does not run embeddings, refresh the IdentityIndex, or extract PostMentions. Writer p99 stays in single-digit milliseconds.

The lazy phase is driven by the indexer role. It reads the WAL delta since the last committed offset and runs MV refresh, embedding generation, SmartIngest identity extraction, and PostMention inference. Every lazy step is re-runnable: a crashed indexer resumes at the last committed WAL offset. All lazy output is itself bi-temporal — late corrections are first-class, never in-place overwrites.

Query

Client                                              Result
──────                                              ──────
psql / gRPC / HTTP / MCP / Arrow Flight             Arrow batches
SPARQL                                              + optional PROV-O columns
compat: S3 / pgvector / ClickHouse            + audit log entry
               / Neo4j / Redis / MongoDB / Bolt          ▲
        │                                                │
        ▼                                                │
  Wire auth ─► principal (OIDC JWT / mTLS cert)          │
        │                                                │
        ▼                                                │
  Coordinator ─► parse + PURPOSE check                   │
               ─► ACL bitmap compilation                 │
               ─► cost-based optimize + MV rewrite       │
        │                                                │
        ▼                                                │
  Execution (streaming, no full materialization)         │
    ─► columnar scan (Parquet + mutable tail)            │
    ─► CSR graph traversal (BFS / Pregel)                │
    ─► vector ANN (custom HNSW / DiskANN warm tier)      │
    ─► BM25 full-text (custom, not Tantivy)              │
    ─► bitmap intersection (ACL + filter, fused)         │
        │                                                │
        ▼                                                │
  Cache ─► L1 foyer (RAM/NVMe) → L2 ring → L3 S3 ───────┘

ACL predicates are compiled into bitmap scan predicates before execution begins, never applied as a post-filter.

Design invariants

These are non-negotiable. A change that breaks any of them produces a different database.

Bi-temporal everywhere. Every row carries valid_from, valid_to, system_from, system_to as i64 nanoseconds UTC. No opt-out. A table without bi-temporality is not a RelataDB table.

Time is i64 nanoseconds UTC. No timezone is stored. Wire format is RFC 3339 with a Z suffix. Non-UTC offsets are rejected at parse time.

unsafe is forbidden in library crates. Every lib.rs compiles with unsafe_code forbidden or denied (relata-query and relata-storage use #![deny]; the rest use #![forbid]). The only workspace unsafe is the relata CLI binary's libc::kill FFI shim for the test harness. Vector hot loops rely on the compiler's auto-vectorizer rather than hand-written SIMD.

missing_docs is denied. Every public item — function, struct, enum variant, trait, module — has a doc comment. Builds fail otherwise.

Locking via parking_lot. PlMutex and PlRwLock type aliases are the only mutexes in the workspace. parking_lot does not poison on panic, eliminating the recover_poison() pattern that pollutes std::sync::Mutex codebases.

Per-type interior locking. Writes take &self, not &mut self. Each object type owns its own RwLock<TableState>, so a write to Profile does not block a write to Post.

Hash-chained commit manifests. Every commit references the SHA-256 of the previous manifest. Tampering with any historical file breaks the chain, detectable offline via relata doctor or online via GET /audit/count.

Object-store native. Durable bytes are Parquet, Arrow IPC, and content-addressed blobs. Readable by any S3 client, any Parquet reader, any Arrow consumer.

Single-writer-per-branch strong consistency. At most one active writer lease per branch at a time, enforced by the coordinator.

PURPOSE is optional. Declared PURPOSE '<id>' is recorded in audit and may drive policy, but the engine runs without one.

Cell-level ACL is pushed into scans. ACL decisions compile to bitmap predicates evaluated at vectorized loop level alongside the filter. Cost is bounded under 2.5× raw-scan p50.

One ontology is the schema. No CREATE TABLE. Declare ObjectType, EventType, LinkType, ActionType; the planner, storage layout, and SDKs derive from those declarations.

Every background work item is a typed Job. No opaque daemons. Indexing, MV refresh, compaction, and pattern detection all surface in system.jobs with state, owner, and progress.

Workspace invariants

Workspace resolver "3", edition 2024, MSRV 1.85
All workspace dependency versions are pinned in the root [workspace.dependencies] table. Use dep.workspace = true, not inline version strings.
cargo deny check enforces a license allowlist and bans unknown registries and git dependencies.

Architecture Overview#

Deployment profiles#

The five planes#

Data flow#

Ingest#

Query#

Design invariants#

Workspace invariants#

See also#