Jobs, Workflows & Detection

Removed in 3.0
import_sigma, relata import-sigma and POST /rules/sigma are removed from Relata DB 3.0 core; Sigma conversion is optional OpenETL work and native rules at /v1/rules stay. The examples below apply to the released 2.5.8 line only. See Detection rules.
Relata DB 3.0
Cluster-category jobs and multi-tenant statements on this page describe 2.5.8 and the Enterprise edition; the 3.0 open build is a single node with one tenant. Connectors and Sigma conversion are outside the core (OpenETL). The 2.5.8 line, kept in the 2.5.8 documentation snapshot, is described by the older material on this page.

Relata ships a unified extension + job + report framework — six extension kinds (Connector, Detector, Enricher, Scorer, Job, Report) sharing one signed Manifest, one ACL/principal/audit contract, and one scheduler plane. This page covers the detection/analysis surface: pattern-detection jobs, the detection-rules engine, governance-aware workflows, intelligence feeds, and the domain packs that bundle them.

Pattern detection

Four typed continuous Job extensions run on a schedule, scan ingested data for investigation-significant patterns, and emit typed AlertEvent records with full PROV-O provenance. Each runs under the BATCH scheduler class (yields to interactive), writes alerts that inherit the strictest classification of the data that triggered them, and is validated by ExtensionShakedown against a GoldenDataset before promotion from shadow to live.

JobScheduleScansDetects
C2BeaconDetectJobhourlyNetFlowEvent, DnsQueryEventPeriodic outbound traffic (coefficient-of-variation < 0.15, small dest-IP set, consistent payload) → C2_BEACON
ConvoyDetectJobevery 30 minMovementEvent, CellAttachEvent≥3 entities within 500 m moving together ≥15 min → CONVOY
TransactionRingDetectJobhourlytransaction graph (WireTransferHop, UpiTxn, ImpsTxn, CryptoTxn)Circular flow A→B→C→A (depth ≤6, ≥ tenant threshold) → TRANSACTION_RING
ContradictionDetectJobdailyentity/link/event assertionsSame (entity, property) with conflicting values from reliable sources, not explained by bi-temporal succession → CONTRADICTION

Jobs are discoverable via SUGGEST_EXTENSIONS() and surface in the MCP list_jobs / job_status / schedule_job tools. The detection algorithms live in crates/relata-jobs/src/pattern_detection/.

Detection rules engine

A lightweight detection-rules engine complements the typed pattern-detection jobs. Rules are continuously evaluated against new data; each is defined in SQL or Sigma-compatible YAML.

# Register a detection rule (SQL condition against a target type)
curl -X POST http://localhost:9090/rules \
  -H "Authorization: Bearer $RELATA_BEARER_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "large-transfer-flag",
    "trigger": {
      "type": "Transaction",
      "condition": "amount > 100000 AND currency = '\''USD'\''"
    },
    "action": "flag"
  }'

Sigma rule import

Sigma is the vendor-neutral signature format for SIEM detection rules. Import Sigma YAML directly:

relata import-sigma rules/suspicious_dns.yaml

Rules must pass a precision/recall gate against a golden dataset before promotion from shadow to live.

Rule lifecycle

EndpointMethodDescription
/rulesGET / POSTList / create rules
/rules/:idDELETEDisable/delete rule
/rules/:id/snoozePOSTTemporarily disable
/rules/:id/suppressPOSTSuppress alerts
/rules/:id/exceptionsPOSTAdd exception
/rules/:id/tuningGETTuning suggestions
/alerts/listGETList fired alerts
/alerts/update/:idPATCHUpdate alert status

Rule conditions are validated at create + eval time (rejects ;, --, /* */; rejects UNION/INTERSECT/EXCEPT adjacent to punctuation; 2048-byte length cap) to prevent SQL injection through rule text.

Detection modes

  • live — alerts fire immediately when data matches
  • shadow — alerts are logged but not surfaced (for validation before promotion)
  • disabled — rule is inactive

Workflows (governance-aware DAGs)

Workflows are DAGs of steps that automate multi-stage analysis:

curl -X POST http://localhost:9090/workflows \
  -H "Authorization: Bearer $RELATA_BEARER_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "fraud-investigation",
    "steps": [
      {"name": "detect", "type": "rule", "rule": "large-transfer-flag"},
      {"name": "enrich", "type": "lookup", "table": "sanctions_list"},
      {"name": "report", "type": "report", "template": "sars-template"}
    ]
  }'
EndpointMethodDescription
/workflowsGET / POSTList / register workflows
/workflows/:nameGETGet definition
/workflows/:name/runPOSTTrigger a run
/workflows/runs/:run_idGETCheck run status

Governance: every workflow step inherits the tenant context, PURPOSE, and ACL of the triggering request. Steps that would violate governance are blocked — workflows cannot bypass policy.

The crates behind it

CrateRole
relata-jobsThe Job/Report extension framework + 42 built-in jobs across 7 categories (storage hygiene, correctness, retention/compliance, audit/integrity, performance, egress) + the typed pattern-detection jobs
relata-intelligenceIncident clustering, anomaly detection, LLM interpretation of alert clusters, detection-rule tuning suggestions

The MCP surface (job_status, list_jobs, schedule_job, list_workflows, run_workflow, workflow_status, list_rules, create_rule, import_sigma) mirrors these endpoints for tool-calling agents.

RIFN (not part of the core)

RIFN, the Relata Intelligence Feed Network, is a separate service. Its crates (relata-feed, relata-feed-broker) and the FeedSyncJob are not part of Relata DB 3.0; feeds reach the database through the ingest contract (POST /v1/ingest/bulk), the same way any other source does. See Connectors & Extensions.

Domain packs

Relata DB 3.0
Packs are documentation-only in the source tree: relata pack install <name> is a stub that does not register anything, and pack jobs are proposed specifications, not live detections. The packs in the repository are agent-memory, cyber, finint, healthcare, knowledge, lea and maritime; relata-pack-stub shows the layout for authoring new ones. Earlier text here listed about 20 domain packs and 25 jurisdiction packs, which is not the 3.0 tree. See the use cases for what was exercised.

Jobs

Background jobs (storage hygiene, correctness, retention/compliance, audit/integrity, performance, egress) run maintenance tasks:

relata jobs                  # list all jobs
relata jobs status indexer   # check a specific job
EndpointMethodDescription
/jobsGETList jobs
/jobs/:nameGETJob status

See also