docs: multi-core sharding (#10) + FEC object batching design specs #32

Merged

vxfemboy merged 1 commit from docs/throughput-design-specs into main

2026-07-03 19:35:27 +00:00

vxfemboy commented

2026-07-03 19:11:44 +00:00

(Migrated from github.com)

Two design specs answering "how do we reach WireGuard throughput?" The io_uring investigation already proved the answer isn't I/O — throughput is CPU/single-thread-bound (RaptorQ FEC encode ~24 µs/pkt + everything on one core). So:

`2026-07-03-multicore-sharding-design.md` (#10) — the primary lever

Per-peer engine sharding: N independent single-thread DataPlane engines (one pinned per core), peers assigned by hash, each with its own SO_REUSEPORT UDP socket + its own IFF_MULTI_QUEUE TUN queue. No shared state, no locks — preserves the Phase A per-flow latency win and reuses the existing run_poll/run_uring loops verbatim (engines=1 is byte-identical to today). Scales aggregate throughput across peers.

Key honesty: yipd is a single connected peer today (sock.connect), so SO_REUSEPORT needs the multi-peer data plane (sub-project #2) — this ships the seams and is a deliberate no-op until then. Single-flow multi-Gbit (one fat link) needs the deferred crypto worker-pool (regime B, #28).

`2026-07-03-fec-object-batching-design.md` — compounds it on lossy links

Batch N sealed packets into one RaptorQ object → amortize the ~24 µs Encoder::new to ~24/N. Container framing + derived AEAD counters (no wire change); the accumulator buffers plaintext and seals consecutively at flush so the LossDetector's counters stay contiguous. Bulk-only, N=8 / 1 ms. Honest scope: clean links already bypass the encoder, so this helps only the repair>0 (lossy) path — not clean-link throughput.

Follow-ups filed

#27 (batching impl), #28 (single-flow worker-pool, deferred), #29 (extend zero-repair bypass to non-Bulk), #30 (GSO for ARQ retx), #31 (RaptorQ encode SIMD/systematic). The specs' own follow-up sections capture smaller refinements (eBPF steering, session migration, NUMA, adaptive symbol_size, indexed RetxBuffer).

Design only — no code. Both want your sign-off on their open questions before implementation.

🤖 Generated with Claude Code

Two design specs answering **"how do we reach WireGuard throughput?"** The io_uring investigation already proved the answer isn't I/O — throughput is **CPU/single-thread-bound** (RaptorQ FEC encode ~24 µs/pkt + everything on one core). So: ## `2026-07-03-multicore-sharding-design.md` (#10) — the primary lever **Per-peer engine sharding:** N independent single-thread `DataPlane` engines (one pinned per core), peers assigned by hash, each with its own `SO_REUSEPORT` UDP socket + its own `IFF_MULTI_QUEUE` TUN queue. No shared state, no locks — preserves the Phase A per-flow latency win and reuses the existing `run_poll`/`run_uring` loops verbatim (`engines=1` is byte-identical to today). Scales **aggregate** throughput across peers. Key honesty: yipd is a **single connected peer today** (`sock.connect`), so `SO_REUSEPORT` needs the multi-peer data plane (sub-project #2) — this ships the *seams* and is a deliberate **no-op** until then. Single-flow multi-Gbit (one fat link) needs the deferred **crypto worker-pool** (regime B, #28). ## `2026-07-03-fec-object-batching-design.md` — compounds it on lossy links Batch N sealed packets into one RaptorQ object → amortize the ~24 µs `Encoder::new` to ~24/N. Container framing + **derived** AEAD counters (no wire change); the accumulator buffers **plaintext** and seals consecutively at flush so the `LossDetector`'s counters stay contiguous. Bulk-only, **N=8 / 1 ms**. Honest scope: clean links already bypass the encoder, so this helps **only the repair>0 (lossy) path** — not clean-link throughput. ## Follow-ups filed #27 (batching impl), #28 (single-flow worker-pool, deferred), #29 (extend zero-repair bypass to non-Bulk), #30 (GSO for ARQ retx), #31 (RaptorQ encode SIMD/systematic). The specs' own follow-up sections capture smaller refinements (eBPF steering, session migration, NUMA, adaptive symbol_size, indexed RetxBuffer). Design only — no code. Both want your sign-off on their open questions before implementation. 🤖 Generated with [Claude Code](https://claude.com/claude-code)