Every result here was produced by pointing a real client binary at the adversarial conformance suite and recording what the server observed. Nothing is hand-entered, and no client is asked to report on itself.
This is not a ranking. Several tests are discretionary — the specification permits more than one answer — and a client can be entirely conformant and still never reach a test, which is recorded as inconclusive rather than counted against it. The interesting cells are the ones where implementations disagree.
The implementations
| Client | Stack | Language | Version | Result |
|---|---|---|---|---|
curl |
ngtcp2 + nghttp3 | C | ngtcp2/1.11.0 | 30 pass 0 fail 22 incon |
quinn |
quinn + h3 (our fork) | Rust | noq fork | 38 pass 3 fail 16 incon |
aioquic |
aioquic | Python | 1.3.0 | 38 pass 3 fail 11 incon |
chromium |
Chromium QUICHE | C++ | build 1223 | 43 pass 5 fail 9 incon |
quic-go |
quic-go | Go | quic-go v0.61.0 | 42 pass 2 fail 13 incon |
quiche |
Cloudflare quiche | Rust | 0.30.0 | 41 pass 3 fail 13 incon |
neqo |
neqo (Firefox) | Rust | unknown | 42 pass 4 fail 11 incon |
Per-test results
Rows where the implementations disagree are marked ◆. Hover a cell for what the server actually observed.
Verdict and implementation work together: choosing a verdict alone shows every test where any client scored it, and adding an implementation narrows that to one column — “every failure in Chromium”, or “every RFC 9114 correctness test where implementations disagree”. Column headings sort.
q-version-negotiation
|
Pass | Pass | Pass | Pass | Pass | Pass | Pass |
q-retry
|
Pass | Pass | Pass | Pass | Pass | Pass | Pass |
q-reserved-transport-param
|
Pass | Pass | Pass | Pass | Pass | Pass | Pass |
q-reserved-frame
◆
|
Pass | Pass | Pass | Pass | Pass | Pass | Fail |
q-cid-rotation
◆
|
Inconclusive | Inconclusive | Inconclusive | Inconclusive | Pass | Inconclusive | Inconclusive |
q-stateless-reset
◆
|
Pass | Pass | Fail | Pass | Pass | Pass | Pass |
q-flow-control
|
Pass | Pass | Pass | Pass | Pass | Pass | Pass |
q-ack-frequency
|
Pass | Pass | Pass | Pass | Pass | Pass | Pass |
q-ecn
◆
|
Inconclusive | Inconclusive | Inconclusive | Pass | Pass | Inconclusive | Inconclusive |
q-pmtu-blackhole
|
Pass | Pass | Pass | Pass | Pass | Pass | Pass |
q-path-challenge
|
Pass | Pass | Pass | Pass | Pass | Pass | Pass |
q-zero-rtt-reject
|
Inconclusive | Inconclusive | Inconclusive | Inconclusive | Inconclusive | Inconclusive | Inconclusive |
q-multipath
|
Pass | Pass | Pass | Pass | Pass | Pass | Pass |
q-key-update
|
Pass | Pass | Pass | Pass | Pass | Pass | Pass |
q-stream-limit
|
Pass | Pass | Pass | Pass | Pass | Pass | Pass |
q-loss-recovery
|
Pass | Pass | Pass | Pass | Pass | Pass | Pass |
q-connection-migration
|
Pass | Pass | Pass | Pass | Pass | Pass | Pass |
q-invalid-transport-param
◆
|
Pass | Inconclusive | Pass | Fail | Fail | Pass | Fail |
q-zero-rtt-replay
|
Inconclusive | Inconclusive | Inconclusive | Inconclusive | Inconclusive | Inconclusive | Inconclusive |
q-ecn-congestion
◆
|
Inconclusive | Inconclusive | Inconclusive | Pass | Pass | Inconclusive | Inconclusive |
q-key-update-repeated
|
Pass | Pass | Pass | Pass | Pass | Pass | Pass |
q-max-streams-credit
◆
|
Inconclusive | Pass | Pass | Pass | Pass | Pass | Pass |
h-grease-settings
|
Pass | Pass | Pass | Pass | Pass | Pass | Pass |
h-grease-frame
|
Pass | Pass | Pass | Pass | Pass | Pass | Pass |
h-reserved-uni-stream
|
Pass | Pass | Pass | Pass | Pass | Pass | Pass |
h-duplicate-setting
|
Pass | Pass | Pass | Pass | Pass | Pass | Pass |
h-control-frame-unexpected
◆
|
Inconclusive | Pass | Pass | Pass | Pass | Pass | Pass |
h-missing-settings
◆
|
Inconclusive | Pass | Pass | Pass | Pass | Pass | Pass |
h-second-control-stream
◆
|
Inconclusive | Pass | Pass | Pass | Pass | Pass | Pass |
h-qpack-dynamic-table
|
Inconclusive | Inconclusive | Inconclusive | Inconclusive | Inconclusive | Inconclusive | Inconclusive |
h-qpack-huffman
|
Pass | Pass | Pass | Pass | Pass | Pass | Pass |
h-oversized-field-section
|
Pass | Pass | Pass | Pass | Pass | Pass | Pass |
h-trailers
|
Pass | Pass | Pass | Pass | Pass | Pass | Pass |
h-early-hints
◆
|
Pass | Fail | Fail | Pass | Pass | Pass | Pass |
h-goaway
◆
|
Pass | Pass | Pass | Fail | Pass | Pass | Pass |
h-max-push-id
◆
|
Inconclusive | Pass | Pass | Pass | Inconclusive | Pass | Pass |
h-settings-on-request-stream
◆
|
Inconclusive | Pass | Pass | Pass | Pass | Pass | Pass |
h-data-before-headers
◆
|
Inconclusive | Pass | Pass | Pass | Pass | Fail | Pass |
h-cancel-push-unsolicited
◆
|
Inconclusive | Fail | Inconclusive | Fail | Inconclusive | Inconclusive | Pass |
h-qpack-blocked-stream
|
Inconclusive | Inconclusive | Inconclusive | Inconclusive | Inconclusive | Inconclusive | Inconclusive |
h-response-stream-reset
|
Pass | Pass | Pass | Pass | Pass | Pass | Pass |
h-priority-update
|
Pass | Pass | Pass | Pass | Pass | Pass | Pass |
h-extended-connect
|
Pass | Pass | Pass | Pass | Pass | Pass | Pass |
h-push-promise-unsolicited
◆
|
Inconclusive | Fail | Fail | Fail | Inconclusive | Fail | Fail |
h-datagram-setting-invalid
◆
|
Inconclusive | Inconclusive | Pass | Pass | Fail | Pass | Pass |
h-qpack-encoder-overflow
|
Inconclusive | Inconclusive | Inconclusive | Inconclusive | Inconclusive | Inconclusive | Inconclusive |
h-goaway-increasing
◆
|
Inconclusive | Pass | Inconclusive | Pass | Pass | Pass | Fail |
h-push-stream-unpromised
◆
|
Inconclusive | Inconclusive | Inconclusive | Fail | Pass | Fail | Pass |
h-qpack-static-index-invalid
◆
|
Inconclusive | Pass | Pass | Pass | Inconclusive | Pass | Pass |
q-packet-reordering
|
Pass | Pass | Pass | Pass | Pass | Pass | Pass |
h-qpack-encoder-bad-name-index
◆
|
Inconclusive | Inconclusive | Pass | Pass | Inconclusive | Inconclusive | Pass |
t-hybrid-only
◆
|
Not run | Pass | Not run | Pass | Pass | Pass | Pass |
t-classical-only
◆
|
Pass | Inconclusive | Pass | Pass | Pass | Pass | Pass |
t-hybrid-large-hello
◆
|
Not run | Pass | Not run | Pass | Pass | Pass | Pass |
t-group-not-offered
◆
|
Not run | Inconclusive | Not run | Inconclusive | Inconclusive | Inconclusive | Inconclusive |
t-corrupt-hybrid-share
◆
|
Not run | Inconclusive | Not run | Inconclusive | Inconclusive | Inconclusive | Inconclusive |
t-grease-group
◆
|
Not run | Inconclusive | Not run | Inconclusive | Inconclusive | Inconclusive | Inconclusive |
Reading this honestly
- A fail is a specification violation, not a security vulnerability. Some may have security relevance; most are simply behaviour a specification prohibits. Treating every failure as a vulnerability would make the whole dataset less credible.
-
Inconclusive is not a soft fail. Most often it means
the run never put that client in the situation the test is about
— no session ticket to attempt 0-RTT with, a network that
stripped ECN, a zero QPACK table capacity. It also covers the case
where the client's answer could not be observed at all: an anomaly
written to the control stream that a one-shot request closed before
reading, or a rejection whose
CONNECTION_CLOSEwas lost on the way back. It counts neither way. -
Rejecting at the QUIC layer is not a pass either.
Some clients answer an HTTP/3 violation by closing the QUIC connection
rather than sending an HTTP/3 error code —
H3_MISSING_SETTINGSarrives as a transportINTERNAL_ERROR. The client plainly objected, but the code the clause names never reaches the wire, so those cells read inconclusive. It is the single biggest reason a column here is full of them, and it is not a criticism: it means the one thing the test asks about could not be observed. - A fail is never inferred from silence. Every failure in this table rests on something the server watched the client do — a rejection carrying the wrong error code, datagrams that kept arriving after a Stateless Reset, a valid response abandoned. Absence of a rejection is consistent with a client that accepted the violation and with one that never saw it, so it is not reported as either.
- Client wrappers are deliberately thin. Each is a few lines that make one request and exit. Everything under test belongs to the library, not the wrapper; anything clever in a wrapper would be measuring the wrapper.
-
One of these is ours. The
quinnrow runs the same forked stack that serves the suite. It is included because excluding it would be hiding a result, but it is the one row to read with the most suspicion.
Generated 2026-09-17T20:03:02Z. Run the suite against your own client at conformance.pqcrypta.com.