curl
30 Pass0 Fail22 Inconclusive5 Not run
100% of 30 conclusive
Each implementation below was driven through every test by the runner, which spawns the client's own binary. The suite is the judge: it can see what the client did on the wire, which the client cannot honestly report about itself.
A pass rate is over conclusive results only — pass plus fail. Inconclusive means the run never put the client in the situation the test is about, which is a fact about the run and not about the client.
quinn
38 Pass3 Fail16 Inconclusive
93% of 41 conclusive
aioquic
38 Pass3 Fail11 Inconclusive5 Not run
93% of 41 conclusive
chromium
43 Pass5 Fail9 Inconclusive
90% of 48 conclusive
quic-go
42 Pass2 Fail13 Inconclusive
95% of 44 conclusive
quiche
41 Pass3 Fail13 Inconclusive
93% of 44 conclusive
neqo
42 Pass4 Fail11 Inconclusive
91% of 46 conclusive
Where they disagree
28 of 57 tests drew more than one answer. Several are discretionary — the RFC permits either behaviour and the disagreement is not a defect in anybody. The rest are the rows worth reading.
- Unknown frame type in a 1-RTT packet
- NEW_CONNECTION_ID followed by RETIRE_CONNECTION_ID
- Stateless reset
- Packets marked ECT(0)
- Transport parameter carrying a value the specification forbids
- Path marking packets CE, not just ECT(0)
- No stream credit at first, then MAX_STREAMS mid-connection
- A DATA frame on the control stream
- Control stream whose first frame is not SETTINGS
- A second control stream opened by the server
- 103 Early Hints before the final response
- GOAWAY sent mid-connection
- MAX_PUSH_ID sent by the server
- SETTINGS frame on a request stream
- DATA frame before any HEADERS on the response stream
- CANCEL_PUSH for a push ID that was never promised
- PUSH_PROMISE for a push the client never allowed
- SETTINGS_H3_DATAGRAM with a value that is neither 0 nor 1
- A second GOAWAY naming a larger identifier than the first
- Push stream opened for a push nobody allowed
- Field line indexing a static table entry that does not exist
- Encoder instruction naming a static index that does not exist
- Server negotiates only the post-quantum hybrid X25519MLKEM768
- Server negotiates only classical X25519 against a hybrid offer
- Hybrid key share large enough to split the Initial across packets
- Server selects a key exchange group the client never offered
- Hybrid key share with an intact X25519 half and a corrupt ML-KEM half
- GREASE named group a client must tolerate
What this suite is · The full grid · All clients · All tests · Findings