Ingestion

SOC pipeline

How an event becomes a case with evidence behind it.

Everything a collector sends runs the same path. The stages are separate on purpose: each one is recoverable, and each one keeps what it received rather than replacing it.

#The stages

  1. Receive — the event lands against a registered source, inside that source’s workspace.
  2. Normalize — a native adapter maps the vendor payload to the shared model, preserving the original.
  3. Correlate — related signals are grouped rather than counted separately.
  4. Durable job — work is queued so a restart resumes instead of losing the batch.
  5. Case and evidence — the correlation becomes a case that carries what produced it.

#Why it is durable

Bounded batches, backpressure, deduplication, leases, retries, dead-letter handling, and replay keep the pipeline recoverable rather than process-local. A burst does not cost you ordering, ownership, or evidence — it costs you latency.

Deduplication is not deletion

Correlating four alerts into one case does not discard three of them. The case cites all four, and each one keeps its source and arrival time.

#Reading the result

EndpointReturns
GET /api/soc/casesThe workspace case queue.
GET /api/soc/cases/{id}One case with the signals it cites.
GET /api/soc/cases/{id}/investigationsWhat was examined, and what was concluded.
GET /api/alerts/correlationsHow raw alerts were grouped.
GET /api/soc/investigations/metricsInvestigation throughput.

#Response stays behind a boundary

Investigation produces a proposal, not an action. A response intent carries the plan, the evidence, and the reasoning, and waits for a named person.

EndpointReturns
GET /api/soc/response/intentsProposed responses and their state.
GET /api/soc/response/policiesWhat the deployment permits.
GET /api/soc/response/catalogActions available to propose.
GET /api/soc/response/controlThe approval boundary’s current configuration.
Nothing consequential runs unattended

Propose, approve, activate — with the actor recorded at the approval step. An unapproved intent is never shown as a completed action.