One Connection, Many Paths

Two Components, One Extension

Multipath is implemented on both ends of the connection, in two separate Rust workspaces:

Both consume the same QUIC stack. Both keep every other feature they already had — the proxy's post-quantum TLS, Encrypted Client Hello, ACK-frequency extension, JA3/JA4 fingerprinting, WAF, and rate limiting; the scanner's RFC 9218 priority probe, MASQUE CONNECT-UDP test, and certificate analysis. The migration had to be invisible to all of it.

Build vs. Adopt: the noq Decision

The adopt decision was gated on a real question: does noq actually fit? A time-boxed viability spike answered it before any production code changed. Four findings mattered.

One risk was flagged honestly and turned out to be the real work: noq ships no HTTP/3 or WebTransport binding of its own. PQ Crypta's proxy is nothing but HTTP/3 and WebTransport. Closing that gap — not the multipath transport itself — was the bulk of the effort.

The Binding Layer

Because noq keeps quinn's public API surface, most consuming code compiles against it with a dependency alias rather than a rewrite. The proxy and scanner both alias noq as quinn (and noq-proto as quinn_proto) in their manifests, so the dozens of quinn:: references resolve unchanged; only genuine API divergences are patched at the call site.

# pqcrypta-proxy/Cargo.toml — the QUIC stack is noq, aliased as quinn
quinn = { package = "noq", path = "vendor/noq/noq",
          default-features = false,
          features = ["runtime-tokio", "rustls", "aws-lc-rs", "bloom", "platform-verifier"] }
quinn-proto = { package = "noq-proto", path = "vendor/noq/noq-proto" }
h3-quinn    = { package = "h3-noq",   path = "vendor/h3-noq" }

Two small binding crates make noq serve HTTP/3 and WebTransport:

A handful of stack-specific renames were also needed — noq split quinn-proto's single Codec trait into Encodable/Decodable, and its Endpoint::handle takes a FourTuple (remote + optional local IP folded together) rather than separate arguments — but these are the exception, not the rule. The aliasing strategy is what kept a QUIC-stack swap from becoming a full rewrite of two large codebases.

Vendoring and workspace isolation

noq is vendored into both projects rather than pulled from crates.io, for the same reason PQ Crypta vendors its rustls fork: the diagnostic accessors the scanner needs (covered under Implementation) are local additions, so the source has to be ours to edit. noq is itself a multi-crate workspace (noq, noq-proto, noq-udp) that uses workspace dependency inheritance, so it can't simply be dropped inside another workspace's tree — it is kept as its own workspace and excluded from the parent, then referenced by path:

# ent/Cargo.toml — keep the vendored noq workspace out of the parent
[workspace]
exclude = ["vendor/noq", "vendor/wtransport-noq"]

A subtle version detail had to be reconciled. noq depends on rustls 0.23.33; PQ Crypta's vendored ECH fork is 0.23.36. Because those are semver-compatible, a single [patch.crates-io] pointing rustls at the fork satisfies noq, the HTTP TCP listeners, and the QUIC stack all at once — one rustls for the whole process, carrying both server-side ECH and the per-path multipath nonce derivation. The old narrow-probe quinn/quinn-proto patches were removed; nothing depends on crates.io quinn any more.

Server: pqcrypta-proxy

On the server, enabling full multipath is one line of transport configuration. Everything the earlier narrow probe wired by hand — the per-connection path-1 CID request — is deleted, because noq manages the path lifecycle itself.

// pqcrypta-proxy/src/quic_listener.rs — enable full multipath
//
// Allow up to 4 concurrent, data-carrying paths per connection when the
// peer negotiates multipath. noq implements the complete extension
// (per-path packet-number spaces, PATH_ACK/ABANDON/STATUS lifecycle
// frames, per-path loss recovery, and a scheduler) and manages path
// creation / validation / teardown automatically once negotiated.
transport_config.max_concurrent_multipath_paths(4);

There is deliberately no per-connection multipath code in the accept loop anymore. Once max_concurrent_multipath_paths is set, a peer that also negotiates the extension can open additional paths; noq validates them with PATH_CHALLENGE / PATH_RESPONSE, schedules traffic across the validated set, and tears paths down on PATH_ABANDON — all beneath the unchanged HTTP/3 and WebTransport request handling. The value 4 is a cap, not a promise: paths only ever open if the client asks for them.

What noq Does Automatically

The reason one configuration line suffices is that the hard machinery lives in the transport. When multipath is negotiated, noq maintains, per path:

• an independent 1-RTT packet-number space and its own receive/dedup state;
• a dedicated congestion controller, RTT estimator, pacer, and MTU discovery — a fast Wi-Fi path and a lossy cellular path are controlled separately;
• its own loss and PTO timers, so a stall on one path does not distort recovery on another;
• a connection-ID sequence space, issued and retired via the path-scoped CID frames and gated by MAX_PATH_ID / PATHS_BLOCKED / PATH_CIDS_BLOCKED flow control;
• a status (available or backup) that the scheduler reads to decide where each packet goes.

Acknowledgements are carried per path in ACK_MP frames and attributed to the correct path's recovery state. None of this is visible to the proxy's application code — it sees the same Connection, streams, and datagrams it always did.

Client: the Scanner Probe

The scanner's job is to test whether an arbitrary target supports multipath. With noq that test is no longer a hand-built packet — it is a real request to open a second path. The client advertises multipath in its transport config, and after the normal HTTP/3 scan completes it calls open_path to the same server over a fresh path:

// http3_scanner.rs — genuine second-path open (replaces the narrow probe)
//
//   Some(true)  = a real, data-carrying second path opened and validated
//   Some(false) = multipath negotiated, but the path failed to open/validate
//   None        = the peer does not negotiate multipath at all
let open = connection.open_path(remote, quinn::PathStatus::Available);
match timeout(Duration::from_secs(5), open).await {
    Ok(Ok(_path))                                      => Some(true),
    Ok(Err(quinn::PathError::MultipathNotNegotiated))  => None,
    Ok(Err(quinn::PathError::ServerSideNotAllowed))    => None,
    Ok(Err(_))                                         => Some(false),
    Err(_)                                             => Some(false),
}

The returned future resolves only once the path is fully open and ready to carry application data, driving noq's complete path-establishment machinery: path-CID issuance, PATH_CHALLENGE / PATH_RESPONSE validation, and per-path packet-number spaces. A distinct MultipathNotNegotiated error cleanly separates “the peer doesn't support this” (reported as not supported) from “negotiated but the path didn't come up” (reported as negotiated, not validated). The scanner enables the extension client-side with the mirror of the server's knob:

// scanner client transport config
transport_config.max_concurrent_multipath_paths(2);

One further piece of work was needed here. The scanner reads a few diagnostic facts noq does not expose out of the box — the peer's transport parameters, retry / stateless-reset / new-token signals, and the negotiated cipher suite. Because the vendored noq copy is ours to maintain, these were added back as small, honest accessors (mirroring what the old quinn fork carried): Connection::peer_transport_params(), Connection::security_info(), and a negotiated_cipher_suite field on noq's rustls HandshakeData (which already exposed the negotiated key-exchange group). No fabricated values, no fallbacks — real data or an honest “unknown.”

Crypto: Per-Path AEAD, PQC, ECH

Multipath changes packet protection in exactly one way: the AEAD nonce must incorporate the Path ID, so that identical packet numbers on different paths never produce the same nonce (a fatal AEAD violation). noq derives the per-path nonce through rustls's native multipath primitives — the same rustls crate PQ Crypta already vendors for server-side ECH. That vendored rustls is a 0.23.36 fork carrying both the ECH server support and the per-path nonce derivation; noq requests 0.23.33 and accepts it under semantic-versioning compatibility, so a single rustls serves the whole process.

Because everything routes through that one rustls ServerConfig, the post-quantum and privacy features are untouched by the transport swap:

• Post-quantum key exchange. X25519MLKEM768 hybrid, provided by the rustls post-quantum provider — the same provider noq exercises in its own PQC test.
• Encrypted Client Hello. Server-side ECH lives entirely in the rustls ServerConfig; noq consumes that config through the identical with_crypto / QuicServerConfig::try_from path, so ECH rides along with no multipath-specific code.

A multipath connection to pqcrypta.com therefore still negotiates a quantum-resistant hybrid key exchange and can hide its SNI — on every path.

The Multipath Frame Set

Single-path QUIC never needed to name a path, so its frames don't. Multipath adds a family of path-scoped frames on top of RFC 9000. noq implements the full set; the proxy and scanner never touch them directly — the transport emits and consumes them beneath the application — but they are what makes concurrent paths possible. These are the frames noq carries:

noq also enforces a draft safety rule that single-path stacks have no reason to: a multipath-specific frame received in the wrong packet type is a connection error. Correctness here is not optional — getting per-path acknowledgement or CID accounting subtly wrong silently corrupts data, which is precisely why adopting a stack that already tests this behaviour was the sound choice.

Per-Path Packet Protection

QUIC's AEAD guarantees rest on never reusing a nonce with the same key. Single-path QUIC gets this for free: packet numbers are unique within a space, so the nonce (the static IV XOR'd with the packet number) is unique. Multipath breaks that assumption — two paths each number their packets from zero, so packet number 42 exists on both. If the nonce derivation ignored the path, the two “packet 42”s would share a nonce and shatter the AEAD guarantee.

The draft's fix, which noq derives through rustls's native multipath primitive (the same vendored rustls that carries our ECH support), folds the Path ID into the nonce:

// vendor rustls — per-path AEAD nonce (draft-ietf-quic-multipath §2.4)
// nonce = IV  XOR  ( path_id : u32 big-endian  ||  packet_number : u64 )
pub fn for_path(path_id: u32, iv: &Iv, pn: u64) -> Self {
    let mut seq_bytes = [0u8; NONCE_LEN];        // 12 bytes
    seq_bytes[0..4].copy_from_slice(&path_id.to_be_bytes());
    codec::put_u64(pn, &mut seq_bytes[4..]);
    Self::new_from_seq(iv, seq_bytes)            // XOR with the traffic-secret IV
}

The Path ID occupies the top four bytes of the twelve-byte nonce input and the packet number the lower eight, so distinct paths can never collide even at identical packet numbers. A consequence worth stating: this construction needs a full twelve-byte nonce, so the draft (and noq) exclude any cipher suite with a shorter nonce from multipath connections. The 1-RTT traffic key itself is shared across paths — only the nonce differs — so no extra key schedule is introduced.

Path Establishment Sequence

When the scanner calls open_path (or, symmetrically, when a real client opens a path to the proxy), the following happens inside the transport before the returned future resolves. Nothing below is application code — it is the machinery a single configuration line switches on.

1. Negotiation. Both endpoints advertised initial_max_path_id in their transport parameters during the handshake. Without that, open_path fails immediately with MultipathNotNegotiated and the scanner reports not supported.
2. Path-CID provisioning. The initiator obtains a connection ID scoped to the new Path ID (via the path-scoped NEW_CONNECTION_ID exchange). A path cannot be addressed without a CID that belongs to it.
3. Validation. A PATH_CHALLENGE carrying a random token is sent over the new 4-tuple; the peer echoes it in a PATH_RESPONSE. This proves the path is real and reachable and defends against off-path spoofing — exactly the RFC 9000 migration check, now per path.
4. Activation. On successful validation the path becomes available, gets its own packet-number space, congestion controller, RTT estimator and loss timers, and the scheduler starts placing packets on it. The open_path future resolves with the live Path.
5. Teardown. Either endpoint may later send PATH_ABANDON to retire the path; traffic continues on the surviving paths, and the connection stays up as long as at least one validated path remains.

Because validation is a real cryptographic round trip — not a parameter check — the scanner's path1_validated: true is a strong claim: a second path was provisioned, challenged, and confirmed working, from a genuinely separate UDP 4-tuple.

The Scheduler & Path Status

Once more than one path is available, something must decide where each outgoing packet goes. noq keeps the paths in an ordered map so that packet transmission can pick the next Path ID deterministically, and it honours the per-path status the peer advertises:

Status is not fixed: PATH_STATUS_AVAILABLE / PATH_STATUS_BACKUP frames let either endpoint re-classify a path at runtime, and the local scheduler updates accordingly. This is the seam where a future application-level policy (redundant duplication for latency-critical traffic, or aggregation for bulk transfer) would plug in; the current deployment uses noq's default capacity-based scheduling across available paths.

Per-Path State

The reason multipath is a deep change — and the reason hand-writing it was rejected — is that most of what a QUIC connection tracks becomes per path. In noq each path owns a full slice of connection state; the connection is the coordinator, not the holder:

This is exactly the state the single-path engine holds once, keyed only by packet-number space. Making it per-path across acknowledgement, loss recovery, congestion control, CID accounting and the timer subsystem is the “thousands of lines in the most safety-critical code” the Overview referred to — and the reason a tested implementation was worth adopting whole.

Design Decisions & Tradeoffs

Four choices shaped this implementation. Each was made deliberately, and each has a cost.

Verification & Deployment

Nothing here is asserted without a live check. The claim “pqcrypta.com does full multipath” is verified by the scanner opening a real second path to it. A live scan of the deployed stack returns:

// GET the http3-quic scan of pqcrypta.com (abridged, real output)
"grade":           "A++",
"http3_supported": true,
"multipath_probe": {
    "negotiated":      true,
    "path1_validated": true,
    "draft_version":   "draft-ietf-quic-multipath-21",
    "path1_id":        1,          // the second path's own Path ID
    "path1_rtt_ms":    0,          // its own RTT estimate (see note below)
    "path1_cwnd_bytes": 12000,     // its own congestion window (10-packet IW)
    "path1_mtu":       1200        // its own discovered MTU
},
"webtransport_supported": true,
"tls_extensions":  { "pqc_key_exchange": "X25519MLKEM768" }

path1_validated: true here means the scanner — a separate process, over a fresh UDP 4-tuple — actually opened and validated a second data-carrying path, not merely that the server advertised a transport parameter. The path1_* fields are the payoff of genuine multipath: the second path is assigned Path ID 1 and carries its own independently-tracked state — a congestion window of 12,000 bytes (QUIC's ten-packet initial window at a 1,200-byte MTU) and its own MTU of 1,200 bytes, read straight from noq's per-path PathStats. These are distinct objects from the primary path's, exactly as the per-path-state model requires.

WebTransport was confirmed by a live session established through the noq-ported wtransport crate, and the handshake negotiated the post-quantum hybrid group.

The proxy binary was then rolled to all three edge nodes (US East, US Midwest, US East 2) with identical hashes and clean startup, verified one node at a time; ordinary HTTP/3 and WebTransport traffic on every hosted domain was regression-checked before and after. Both Rust workspaces build and pass their existing test suites against noq.

Conclusion

Full multipath QUIC is now live on both sides of PQ Crypta's stack: a client that opens a genuine second path to test any target, and a production server that runs concurrent, scheduled, per-path QUIC beneath HTTP/3, WebTransport, post-quantum TLS, and Encrypted Client Hello. The engineering lesson generalises past multipath: when a subtle protocol already has a real, tested implementation, the disciplined move is to adopt it and invest the effort in the integration seams — the HTTP/3 and WebTransport bindings, the crypto carry-over, the honest diagnostic accessors — rather than re-implement the hard core and hope. Multipath remains an active IETF draft; this is one working deployment of it, described exactly as it was built.

References

1. IETF QUIC WG. draft-ietf-quic-multipath — Multipath Extension for QUIC.
2. Iyengar, J., Thomson, M. (eds.). RFC 9000 — QUIC: A UDP-Based Multiplexed and Secure Transport.
3. Iyengar, J., Swett, I. (eds.). RFC 9002 — QUIC Loss Detection and Congestion Control.
4. n0-computer. noq — a multipath-capable QUIC implementation in Rust (fork of quinn).
5. quinn-rs. quinn — async QUIC implementation in Rust.
6. hyperium. h3 — HTTP/3 implementation; the h3-quinn binding this work ported to noq.
7. PQ Crypta. HTTP/3 QUIC & WebTransport Scanner and pqcrypta-proxy.