Architecture Overview

Relata DB 3.0
Parts of this page still describe the 2.x line, which is kept in the 2.5.8 documentation snapshot. In 3.0 HTTP routes are under /v1/, the compatibility protocol doors (Postgres, Mongo, Redis, Neo4j, ClickHouse, S3, Flight, SPARQL) are retired, and clustering and multi-tenancy are Enterprise. See What is new in 3.0 and HTTP route restructure.

RelataDB is a governed temporal knowledge database. It collapses the multi-database stack — relational, graph, vector, full-text search, and audit — into a single Rust binary. Policy, provenance, and bi-temporal history are in the query path, not bolted on after the fact.

The trade-off is deliberate: the engineering cost of building all planes in-process is accepted in exchange for eliminating the consistency, governance, and operational seams that fragment a polyglot stack.

Deployment shape

Relata DB 3.0 is one binary on one machine, serving one tenant. There are no deployment profiles to choose between: RELATA_PROFILE accepts only free, and the security posture follows configuration (bind address, TLS, tokens). Clustering, replication and several tenants belong to Relata Enterprise DB. See Deployment and Editions and limits.

The five planes

Every feature belongs to exactly one plane. Cross-plane work goes through typed interfaces; no shared mutable globals exist between planes.

PlaneOwns
Data planeOntology, typed tables, link tables, IdentityIndex, bitmaps, vectors, full-text postings
Compute planePlanner, executor, cost-based optimizer, MV refresh, DAG job engine
Trust planeCedar-inspired ABAC, classification labels, audit log, PROV-O provenance, hash-chained manifests
Identity planeIdentity umbrella type, IdentityIndex MV, RESOLVE_IDENTITY, IDENTITY_CLUSTER, SAME_IDENTITY
Distribution planeEnterprise only: coordinator/reader/writer/indexer roles, write leases, hash partitioning, replication. Not part of the single-machine 3.0 build.

Data flow

RelataDB request pipelineHorizontal flow: clients reach the gateway, which authenticates and stamps tenant and purpose; the planner compiles policy into bitmap predicates; execution engines scan relationally, graph-wise, by vector, and by full-text; results land in the bi-temporal storage layer.REQUEST FLOWClientspsql · gRPC · HTTPMCP · Arrow FlightSPARQL+ 8 compat doorsGatewayauthenticate(OIDC / mTLS)extract PURPOSEtenant + egressPlanner + Policyparse + optimizeACL bitmap compilecell maskingtemporal planningEnginesrelational scangraph (CSR / BFS)vector HNSW / DiskANNBM25 + hybridStoragebi-temporal rowsWAL + manifestsParquet / ArrowPROV-O provenance

Ingest

Writes are split into an eager phase (synchronous, in the writer's commit critical path) and a lazy phase (asynchronous, re-runnable from the WAL).

SOURCE         OpenETL / client  Writer (eager)                 Storage             Indexer (lazy)
──────         ─────────         ──────────────                 ───────             ──────────────
CSV  ─┐                          validate + canonicalize ──►  Parquet segment
JSON  ├─► normalize ────────►    ACL + tenant stamp       ──►  CSR adjacency    ──► IdentityIndex delta
API  ─┤                          WAL append (fsync/async)  ──►  blob (SHA-256)       embeddings (HNSW)
      ┘                          row write (&self)          ──►  PROV-O assertion     MV refresh
                                                            └──► commit manifest      SmartIngest

The eager phase does only declared-column work: parse, canonicalize, stamp principal/agency/classification, append WAL, write row. It does not run embeddings, refresh the IdentityIndex, or extract PostMentions. Writer p99 stays in single-digit milliseconds.

The lazy phase is driven by the background indexer. It reads the WAL delta since the last committed offset and runs MV refresh, embedding generation, SmartIngest identity extraction, and PostMention inference. Every lazy step is re-runnable: a crashed indexer resumes at the last committed WAL offset. All lazy output is itself bi-temporal — late corrections are first-class, never in-place overwrites.

Query

Client                                              Result
──────                                              ──────
HTTP/JSON · GraphQL (read-only) · MCP · A2A            Arrow batches
SDKs: Python · TypeScript · Go                      + optional PROV-O columns
                                                    + audit log entry
        │                                                ▲
        ▼                                                │
  Auth ─► principal (registry token / mTLS cert)         │
        │                                                │
        ▼                                                │
  Coordinator ─► parse + PURPOSE check                   │
               ─► ACL bitmap compilation                 │
               ─► cost-based optimize + MV rewrite       │
        │                                                │
        ▼                                                │
  Execution (streaming, no full materialization)         │
    ─► columnar scan (Parquet + mutable tail)            │
    ─► CSR graph traversal (BFS / Pregel)                │
    ─► vector ANN (custom HNSW / DiskANN warm tier)      │
    ─► BM25 full-text (custom, not Tantivy)              │
    ─► bitmap intersection (ACL + filter, fused)         │
        │                                                │
        ▼                                                │
  Cache ─► L1 foyer (RAM/NVMe) → L2 ring → L3 S3 ───────┘

ACL predicates are compiled into bitmap scan predicates before execution begins, never applied as a post-filter.

Design invariants

These are non-negotiable. A change that breaks any of them produces a different database.

Bi-temporal everywhere. Every row carries valid_from, valid_to, system_from, system_to as i64 nanoseconds UTC. No opt-out. A table without bi-temporality is not a RelataDB table.

Time is i64 nanoseconds UTC. No timezone is stored. Wire format is RFC 3339 with a Z suffix. Non-UTC offsets are rejected at parse time.

unsafe is forbidden in library crates. Every lib.rs compiles with unsafe_code forbidden or denied (relata-query and relata-storage use #![deny]; the rest use #![forbid]). The only workspace unsafe is the relata CLI binary's libc::kill FFI shim for the test harness. Vector hot loops rely on the compiler's auto-vectorizer rather than hand-written SIMD.

missing_docs is denied. Every public item — function, struct, enum variant, trait, module — has a doc comment. Builds fail otherwise.

Locking via parking_lot. PlMutex and PlRwLock type aliases are the only mutexes in the workspace. parking_lot does not poison on panic, eliminating the recover_poison() pattern that pollutes std::sync::Mutex codebases.

Per-type interior locking. Writes take &self, not &mut self. Each object type owns its own RwLock<TableState>, so a write to Profile does not block a write to Post.

Hash-chained commit manifests. Every commit references the SHA-256 of the previous manifest. Tampering with any historical file breaks the chain, detectable offline via relata doctor or online via GET /audit/count.

Object-store native. Durable bytes are Parquet, Arrow IPC, and content-addressed blobs. Readable by any S3 client, any Parquet reader, any Arrow consumer.

Single-writer-per-branch strong consistency. At most one active writer lease per branch at a time, enforced by the coordinator.

PURPOSE is optional. Declared PURPOSE '<id>' is recorded in audit and may drive policy, but the engine runs without one.

Cell-level ACL is pushed into scans. ACL decisions compile to bitmap predicates evaluated at vectorized loop level alongside the filter. Cost is bounded under 2.5× raw-scan p50.

One ontology is the schema. No CREATE TABLE. Declare ObjectType, EventType, LinkType, ActionType; the planner, storage layout, and SDKs derive from those declarations.

Every background work item is a typed Job. No opaque daemons. Indexing, MV refresh, compaction, and pattern detection all surface in system.jobs with state, owner, and progress.

Workspace invariants

  • Workspace resolver "3", edition 2024, MSRV 1.85
  • All workspace dependency versions are pinned in the root [workspace.dependencies] table. Use dep.workspace = true, not inline version strings.
  • cargo deny check enforces a license allowlist and bans unknown registries and git dependencies.

See also

  • Data Model — ontology primitives, row shape, canonical types
  • Storage Engine — WAL, Parquet segments, three-tier cache, manifests
  • Query Engine — parse → plan → optimize → execute
  • Crate Map — workspace crate graph