Every result here was produced by pointing a real client binary at the adversarial conformance suite and recording what the server observed. Nothing is hand-entered, and no client is asked to report on itself.
This is not a ranking. Several tests are discretionary — the specification permits more than one answer — and a client can be entirely conformant and still never reach a test, which is recorded as inconclusive rather than counted against it. The interesting cells are the ones where implementations disagree.
The implementations
| Client | Stack | Language | Version | Result |
|---|---|---|---|---|
curl |
ngtcp2 + nghttp3 | C | ngtcp2/1.11.0 | 22 pass 0 fail 5 incon |
quinn |
quinn + h3 (our fork) | Rust | noq fork | 20 pass 1 fail 5 incon |
aioquic |
aioquic | Python | 1.3.0 | 19 pass 2 fail 6 incon |
chromium |
Chromium QUICHE | C++ | build 1223 | 23 pass 0 fail 4 incon |
quic-go |
quic-go | Go | quic-go v0.61.0 | 23 pass 1 fail 3 incon |
Per-test results
Rows where the implementations disagree are marked ◆. Hover a cell for what the server actually observed.
| Test | curl | quinn | aioquic | chromium | quic-go |
|---|---|---|---|---|---|
q-version-negotiation
|
Pass | Pass | Pass | Pass | Pass |
q-retry
|
Pass | Pass | Pass | Pass | Pass |
q-reserved-transport-param
|
Pass | Pass | Pass | Pass | Pass |
q-reserved-frame
◆ |
Pass | Not run | Inconclusive | Pass | Pass |
q-cid-rotation
◆ |
Inconclusive | Inconclusive | Inconclusive | Inconclusive | Pass |
q-stateless-reset
◆ |
Pass | Pass | Fail | Pass | Pass |
q-flow-control
|
Pass | Pass | Pass | Pass | Pass |
q-ack-frequency
|
Pass | Pass | Pass | Pass | Pass |
q-ecn
◆ |
Inconclusive | Inconclusive | Inconclusive | Pass | Pass |
q-pmtu-blackhole
|
Pass | Pass | Pass | Pass | Pass |
q-path-challenge
|
Inconclusive | Inconclusive | Inconclusive | Inconclusive | Inconclusive |
q-zero-rtt-reject
|
Inconclusive | Inconclusive | Inconclusive | Inconclusive | Inconclusive |
q-multipath
|
Pass | Pass | Pass | Pass | Pass |
h-grease-settings
|
Pass | Pass | Pass | Pass | Pass |
h-grease-frame
|
Pass | Pass | Pass | Pass | Pass |
h-reserved-uni-stream
|
Pass | Pass | Pass | Pass | Pass |
h-duplicate-setting
|
Pass | Pass | Pass | Pass | Pass |
h-control-frame-unexpected
|
Pass | Pass | Pass | Pass | Pass |
h-missing-settings
|
Pass | Pass | Pass | Pass | Pass |
h-second-control-stream
|
Pass | Pass | Pass | Pass | Pass |
h-qpack-dynamic-table
|
Inconclusive | Inconclusive | Inconclusive | Inconclusive | Inconclusive |
h-qpack-huffman
|
Pass | Pass | Pass | Pass | Pass |
h-oversized-field-section
|
Pass | Pass | Pass | Pass | Pass |
h-trailers
|
Pass | Pass | Pass | Pass | Pass |
h-early-hints
◆ |
Pass | Fail | Fail | Pass | Pass |
h-goaway
|
Pass | Pass | Pass | Pass | Pass |
h-max-push-id
◆ |
Pass | Pass | Pass | Pass | Fail |
Reading this honestly
- A fail is a specification violation, not a security vulnerability. Some may have security relevance; most are simply behaviour a specification prohibits. Treating every failure as a vulnerability would make the whole dataset less credible.
- Inconclusive is not a soft fail. It means the run never put that client in the situation the test is about — no session ticket to attempt 0-RTT with, a network that stripped ECN, a zero QPACK table capacity. It counts neither way.
- Client wrappers are deliberately thin. Each is a few lines that make one request and exit. Everything under test belongs to the library, not the wrapper; anything clever in a wrapper would be measuring the wrapper.
-
One of these is ours. The
quinnrow runs the same forked stack that serves the suite. It is included because excluding it would be hiding a result, but it is the one row to read with the most suspicion.
Generated 2026-08-30T12:40:46Z. Run the suite against your own client at conformance.pqcrypta.com.