feat(flow): anycast telemetry ingest — POP identity, tagging, liveness, sampling sanity #165

Merged

vxfemboy merged 10 commits from sp-telemetry-spec into main

2026-07-11 13:01:30 +00:00

vxfemboy commented

2026-07-11 12:54:40 +00:00

(Migrated from github.com)

What

M0 telemetry foundation for the AS214806 deployment (backlog #1 POP sensor + #2 anycast-aware aggregation). Lets the central flow daemon treat the sFlow feeds from multiple anycast POPs as one logical view.

Key insight

The detector already keys per-victim and scales by per-sample rate, so cross-POP volume aggregation already works. This change threads the POP identity that's already in every sFlow datagram (the agent address, previously parsed-then-discarded) and builds per-agent robustness on top — it is not a detector rebuild.

Changes

Agent identity: FlowObservation.agent extracted from the sFlow agent address (v4 + v6).
POP map: pop <name> agent=<ip> sampling=<n> config directive → Policy.pops → blackwall_flow::AgentRegistry (rejects duplicate/malformed entries).
Per-POP tagging: Detection.pops (which POPs contributed, with per-POP est pps/bps).
Robustness: per-agent liveness (agent_last_seen, bounded to known-registry agents — closes a spoofed-agent memory-DoS), and a sampling-sanity clamp (a rogue agent's rate → its configured expected rate).
Attribution: Detection.top_source_blocks (top-5 attacker /24 v4 / /48 v6 blocks).
Metrics: blackwall_flow_pop_last_seen_seconds{pop}, blackwall_flow_agent_sampling_mismatch_total{pop}, blackwall_flow_unknown_agent_observations_total.
Deploy: shipped render_hsflowd_conf + blackwalld sensor render-hsflowd (generates each POP's hsflowd.conf from the POP-map) + a docs/deployment.md POP-sensor section.

Safety

Non-breaking: with no pop block, behavior equals today except agent is populated and all traffic tags unknown (no clamp, aggregation unchanged). The mitigation controllers are untouched.

Quality

Subagent-driven: 6 tasks each spec+quality reviewed + a final whole-branch review (opus). The final review's one Important finding (missing duplicate-pop rejection — which would corrupt a Prometheus scrape) was fixed. clippy --workspace --all-targets --deny warnings clean; fmt clean; coverage 95.33%. No as casts, #[expect] not #[allow], exact pins.

🤖 Generated with Claude Code

## What M0 telemetry foundation for the AS214806 deployment (backlog #1 POP sensor + #2 anycast-aware aggregation). Lets the central `flow` daemon treat the sFlow feeds from multiple anycast POPs as one logical view. ## Key insight The detector already keys **per-victim** and scales by per-sample rate, so cross-POP volume aggregation already works. This change threads the **POP identity that's already in every sFlow datagram** (the agent address, previously parsed-then-discarded) and builds per-agent robustness on top — it is *not* a detector rebuild. ## Changes - **Agent identity**: `FlowObservation.agent` extracted from the sFlow agent address (v4 + v6). - **POP map**: `pop <name> agent=<ip> sampling=<n>` config directive → `Policy.pops` → `blackwall_flow::AgentRegistry` (rejects duplicate/malformed entries). - **Per-POP tagging**: `Detection.pops` (which POPs contributed, with per-POP est pps/bps). - **Robustness**: per-agent liveness (`agent_last_seen`, **bounded to known-registry agents** — closes a spoofed-agent memory-DoS), and a sampling-sanity **clamp** (a rogue agent's rate → its configured expected rate). - **Attribution**: `Detection.top_source_blocks` (top-5 attacker /24 v4 / /48 v6 blocks). - **Metrics**: `blackwall_flow_pop_last_seen_seconds{pop}`, `blackwall_flow_agent_sampling_mismatch_total{pop}`, `blackwall_flow_unknown_agent_observations_total`. - **Deploy**: shipped `render_hsflowd_conf` + `blackwalld sensor render-hsflowd` (generates each POP's hsflowd.conf from the POP-map) + a `docs/deployment.md` POP-sensor section. ## Safety Non-breaking: with no `pop` block, behavior equals today except `agent` is populated and all traffic tags `unknown` (no clamp, aggregation unchanged). The mitigation controllers are untouched. ## Quality Subagent-driven: 6 tasks each spec+quality reviewed + a final whole-branch review (opus). The final review's one Important finding (missing duplicate-`pop` rejection — which would corrupt a Prometheus scrape) was fixed. clippy `--workspace --all-targets --deny warnings` clean; fmt clean; coverage 95.33%. No `as` casts, `#[expect]` not `#[allow]`, exact pins. 🤖 Generated with [Claude Code](https://claude.com/claude-code)