throughput: P+Q fast-path FEC — cheap proactive repair for R≤2 #51

Merged

vxfemboy merged 6 commits from feat/throughput-pq-fec into main

2026-07-11 09:21:29 +00:00

vxfemboy commented

2026-07-11 09:17:58 +00:00

(Migrated from github.com)

What

Adds a generator scheme to the RS FEC codec so proactive repair for the common low-redundancy cases (R=1, R=2) is cheap: RAID-6 P+Q (P = XOR all-ones, Q = 2^i syndrome, MDS for R≤2) for non-ARQ classes, with the existing Cauchy generator for ARQ classes and R≥3. R=1 repair becomes a pure XOR (no GF multiply). Lever 1 of 3 toward single-core-10-Gbit (cheap FEC → fast AEAD → AF_XDP I/O).

Why

On a 1-core box the throughput ceiling is single-core CPU. Post-4a the per-packet budget for ~10 Gbit is ~1.2 µs; the general Cauchy repair (~1.3 µs) alone busts it, so keeping proactive repair on a lossy link forced throughput down. P+Q makes R≤2 repair sub-µs, so loss protection stays on within budget — cheap-and-on when there's loss, ~free when clean (the AdaptiveController already picks R). Bandwidth is free on the target servers, so running generous cheap repair costs nothing that matters.

This finishes the "XOR fast path" 4a deliberately deferred — deferred because mixing XOR-for-R=1 with Cauchy-for-R≥2 made a repair symbol ambiguous to the decoder. Resolved here by signaling the generator scheme in the reserved payload_id[3] byte 4a set aside.

How

crates/yip-transport/src/rs.rs — Scheme {Cauchy, Pq}, repair_row(scheme,k,m), scheme args on encode_repair/decode_source (with the PQ m≥2 reject), R=1 pure-XOR fast path. gf256 untouched (2^i built incrementally).
crates/yip-transport/src/fec.rs — scheme selection keyed on params.arq (see below), payload_id[3] pack/parse, reassembler scheme threading + ingest guard (rejects out-of-range P+Q rows, unknown scheme ids, per-block scheme mismatch).
payload_id layout [codec_tag=0x01][symbol_index:u16 BE][scheme:u8]. yip-wire::Frame, wire_glue.rs, Symbol unchanged.

Scheme is keyed on the stable params.arq flag, not per-call R. This is load-bearing: the ARQ retransmit path re-encodes the same object_id in a separate call with a different repair count (RETX_EXTRA_REPAIR=4), and all symbols of one object must use the same generator or the reassembler rejects the mismatched batch. ARQ classes always use Cauchy (retransmit-compatible); non-ARQ classes (encoded once, never retransmitted) get the P+Q fast path — exactly where proactive repair is always on. (A review caught this as a Critical bug in the first pass and it was fixed before merge — see the ARQ regression test.)

Results

Per-packet R=1 FEC encode: ~1.3 µs → ~0.32 µs (~4×) via transport_encode_1300.
Correctness rests on the exhaustive K-of-(K+R) MDS proof for both schemes + a RAID-6 two-erasure test.
yip-transport 69/69, workspace 346/346, clippy clean.
netns end-to-end (real sudo run): tunnel, 10%-loss recovery, and ARQ integrity (99.3% delivery, 134 retransmits fired) all pass — the ARQ fix holds in a real tunnel.

Final whole-branch review (opus): READY TO MERGE, no Critical/Important — single-scheme-per-object invariant verified airtight against the dataplane.rs ARQ caller.

Wire compatibility

The scheme byte makes the RS-v1 wire self-describing; a 4a peer (payload_id[3]=0) reads as Cauchy. A P+Q sender's R≤2 repair needs a P+Q-aware receiver. Fine under the pre-release "peers rebuild together" posture.

Deferred follow-ups (non-blocking)

rs::repair_row returns an all-zero row for an invalid SCHEME_PQ m≥2 (currently guarded by both callers; a debug_assert!/Option would harden it).
No apples-to-apples R=2 Cauchy-vs-P+Q benchmark (honestly disclosed in RESULTS.md; the R=1 win is the measured, targeted one).

Design docs

Spec: docs/superpowers/specs/2026-07-11-throughput-pq-fec-design.md
Plan: docs/superpowers/plans/2026-07-11-throughput-pq-fec.md

🤖 Generated with Claude Code

https://claude.ai/code/session_01RVP6NnbDMAg1iTsMMTfL86

## What Adds a generator **scheme** to the RS FEC codec so proactive repair for the common low-redundancy cases (R=1, R=2) is *cheap*: **RAID-6 P+Q** (P = XOR all-ones, Q = 2^i syndrome, MDS for R≤2) for non-ARQ classes, with the existing **Cauchy** generator for ARQ classes and R≥3. R=1 repair becomes a **pure XOR** (no GF multiply). Lever 1 of 3 toward single-core-10-Gbit (cheap FEC → fast AEAD → AF_XDP I/O). ## Why On a 1-core box the throughput ceiling is single-core CPU. Post-4a the per-packet budget for ~10 Gbit is ~1.2 µs; the general Cauchy repair (~1.3 µs) alone busts it, so keeping proactive repair on a lossy link forced throughput down. P+Q makes R≤2 repair sub-µs, so loss protection stays *on* within budget — cheap-and-on when there's loss, ~free when clean (the `AdaptiveController` already picks R). Bandwidth is free on the target servers, so running generous cheap repair costs nothing that matters. This finishes the "XOR fast path" 4a deliberately deferred — deferred because mixing XOR-for-R=1 with Cauchy-for-R≥2 made a repair symbol ambiguous to the decoder. Resolved here by **signaling the generator scheme in the reserved `payload_id[3]` byte** 4a set aside. ## How - `crates/yip-transport/src/rs.rs` — `Scheme {Cauchy, Pq}`, `repair_row(scheme,k,m)`, scheme args on `encode_repair`/`decode_source` (with the PQ `m≥2` reject), R=1 pure-XOR fast path. `gf256` untouched (2^i built incrementally). - `crates/yip-transport/src/fec.rs` — scheme selection keyed on **`params.arq`** (see below), `payload_id[3]` pack/parse, reassembler scheme threading + ingest guard (rejects out-of-range P+Q rows, unknown scheme ids, per-block scheme mismatch). - `payload_id` layout `[codec_tag=0x01][symbol_index:u16 BE][scheme:u8]`. `yip-wire::Frame`, `wire_glue.rs`, `Symbol` unchanged. **Scheme is keyed on the stable `params.arq` flag, not per-call R.** This is load-bearing: the ARQ retransmit path re-encodes the same `object_id` in a separate call with a different repair count (`RETX_EXTRA_REPAIR=4`), and all symbols of one object must use the same generator or the reassembler rejects the mismatched batch. ARQ classes always use Cauchy (retransmit-compatible); non-ARQ classes (encoded once, never retransmitted) get the P+Q fast path — exactly where proactive repair is always on. *(A review caught this as a Critical bug in the first pass and it was fixed before merge — see the ARQ regression test.)* ## Results - **Per-packet R=1 FEC encode: ~1.3 µs → ~0.32 µs (~4×)** via `transport_encode_1300`. - Correctness rests on the **exhaustive K-of-(K+R) MDS proof for both schemes** + a RAID-6 two-erasure test. - `yip-transport` 69/69, workspace 346/346, clippy clean. - netns end-to-end (real sudo run): tunnel, 10%-loss recovery, and **ARQ integrity (99.3% delivery, 134 retransmits fired)** all pass — the ARQ fix holds in a real tunnel. Final whole-branch review (opus): **READY TO MERGE**, no Critical/Important — single-scheme-per-object invariant verified airtight against the `dataplane.rs` ARQ caller. ## Wire compatibility The scheme byte makes the RS-v1 wire self-describing; a 4a peer (`payload_id[3]=0`) reads as Cauchy. A P+Q sender's R≤2 repair needs a P+Q-aware receiver. Fine under the pre-release "peers rebuild together" posture. ## Deferred follow-ups (non-blocking) - `rs::repair_row` returns an all-zero row for an invalid `SCHEME_PQ` `m≥2` (currently guarded by both callers; a `debug_assert!`/`Option` would harden it). - No apples-to-apples R=2 Cauchy-vs-P+Q benchmark (honestly disclosed in `RESULTS.md`; the R=1 win is the measured, targeted one). ## Design docs - Spec: `docs/superpowers/specs/2026-07-11-throughput-pq-fec-design.md` - Plan: `docs/superpowers/plans/2026-07-11-throughput-pq-fec.md` 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01RVP6NnbDMAg1iTsMMTfL86