This page consolidates the status of the official Jakarta Servlet 6.1 TCK runs against Foy. It is regenerated on every release. The day-to-day source of truth remains TCK.md at the repo root.
Summary
This is a measurement, not a conformance claim. At the latest full run (jakarta.tck:servlet-tck-runtime:6.1.0, Phase 6 exit, 2026-10-11):
-
1714 tests run: 1692 passing, 7 skipped, 0 failures / 15 errors — 98.7 % of the tests run pass (Phase 5 exit: 1659; Phase 4 exit: 1649). A skipped test is never counted as passing;
-
Phase 6 added security enforcement:
<security-constraint>and@ServletSecurityper url-pattern,deny-uncovered-http-methods, the transport guarantee, BASIC and FORM login,security-role-ref, programmaticlogin/logout/authenticate,setServletSecurityanddeclareRoles; no class regressed and 5 improved (33 tests). Thespec.security.*family passes 55 of its 59 tests (Phase 5 exit: 22); -
the 15 remaining errors are multipart (8), JSP-dependent tests (3 + 2 FORM tests whose error page is a JSP) and CLIENT-CERT (2, deferred); see Security status;
-
the 7 skipped tests are HTTP/2 server push, an accepted gap (see What is skipped).
Per-family breakdown (passing / run):
| Family | Run | Pass | Skipped | Score | Reading |
|---|---|---|---|---|---|
|
859 |
848 |
0 |
98.7 % |
✅ The Servlet engine itself is solid; the rest is multipart (8) and JSP (3). Family mode also runs |
|
646 |
646 |
0 |
100 % |
✅ Fragments, ordering and initializers |
|
207 |
196 |
7 |
94.7 % |
✅ Security 55/59 (CLIENT-CERT 2 deferred, 2 FORM tests need JSP); server push skipped (7) |
|
2 |
2 |
0 |
100 % |
✅ Leading-slash dispatch compat (legacy 2.2/2.3 descriptors) |
|
Exact figures live in |
Security status
Class (spec.security.*) |
Run | Pass | Note |
|---|---|---|---|
|
7 |
7 |
|
|
5 |
5 |
|
|
6 |
6 |
|
|
14 |
14 |
|
|
25 |
23 |
|
|
1 |
0 |
accepted gap: CLIENT-CERT deferred |
|
1 |
0 |
accepted gap: CLIENT-CERT deferred |
CLIENT-CERT is deferred: it needs TLS client authentication and the peer certificates from Chappe, and an https Arquillian container in foy-tck. Until then a CLIENT-CERT application fails closed (403). The TCK users (user/password and authuser/authpassword system properties of the tck-official profile in foy-tck/pom.xml) and their roles are declared in an in-memory identity store by the harness (TckIdentities).
What is skipped
The full run reports Skipped: 7, all in spec.serverpush.ServerPushTests:
-
HTTP/2 server push is an accepted gap: chappe has no h2c Upgrade nor PUSH_PROMISE writer, and push is deprecated in Servlet 6.1 in favour of 103 Early Hints.
HttpServletRequest.newPushBuilder()returnsnull, and the TCK’s own switchservlet.tck.support.http2Push=false(a surefire system property of thetckprofile infoy-tck/pom.xml) skips the 7 tests that need push. The eighth test of the class runs and passes.
No other test battery is excluded. Two more product decisions show up as errors rather than skips:
-
No JSP engine in Foy itself — the 3 JSP-dependent
ServletContext40Testsfail. A JSP-engine SPI in Foy, plugged by Ibarra (Jakarta Pages 4.0), is the planned way to run them. For dynamic pages today, use Cassini (REST) or pure Servlets. -
Hang safety net — some TCK clients read raw sockets without
SO_TIMEOUTand would hang forever on an unanswered request;foy-tcksets a global JUnit 120 s timeout (SEPARATE_THREADmode). No test hits it at the Phase 6 exit.
Reading the logs
The run log carries expected WARNING lines that are not failures. servlet.tck.api.jakarta_servlet.asynccontext.ACListener2 throws an IOException from onComplete on purpose, and Foy logs every listener failure at WARNING with its stack trace (AsyncListener.onComplete threw).
Per-class figures come from foy-tck/tck-tally.sh <log>, which prints <class> <run> <bad> <skipped>, where bad (the third column) is failures + errors; a class passes run - bad - skipped tests. Comparing two tallies is a LC_ALL=C join (both files sorted with LC_ALL=C); the script header gives the exact commands.
Runner architecture
foy-tck is an in-reactor module gated behind the tck Maven profile (TCK harmonisation, same pattern as the vidocq-runtime-tck-* runners): a plain mvn install neither downloads nor runs anything TCK-related, and the release reactor never sees the module. Invoke through ./run-official-tck-servlet6.1.sh, or directly via ./mvnw -Ptck,tck-official -pl foy-tck test.
Historical note. The module used to be excluded from the reactor as a standalone Model 4.0.0 POM: ShrinkWrap Maven Resolver 3.3 (transitive dependency of the official Jakarta TCK) could not parse the Model 4.1.0 POMs the workspace used at the time (Bad artifact coordinates … jar:). That constraint disappeared with the workspace-wide migration to Maven 3.9.16 / Model 4.0.0.
Reproduce locally
Prerequisites
Install the official Jakarta Servlet 6.1 TCK artefacts (non-public) into your local M2:
-
jakarta.tck:servlet-tck-runtime:6.1.0 -
jakarta.tck:servlet-tck-util:6.1.0 -
jakarta.tck:servlet-tck:6.1.0(pom)
Detailed procedure in foy-tck/README.md.
Run the TCK
cd foy
./run-official-tck-servlet6.1.sh # smoke test (fast)
./run-official-tck-servlet6.1.sh --all # full suite (long)
./run-official-tck-servlet6.1.sh -Dtest=ServletTests # one class
./run-official-tck-servlet6.1.sh --family compat # one family: api | spec | pluggability | compat
./run-official-tck-servlet6.1.sh --failing # only the classes failing in tck-baseline.txt
./run-official-tck-servlet6.1.sh --no-install --family api # skip the reactor install
./run-official-tck-servlet6.1.sh --dry-run --failing # print the Maven commands only--no-install skips the reactor build (use it when the code is unchanged),
--family and --failing select tests (--failing reads the bad count — failures + errors, the third column — of
foy-tck/tck-baseline.txt, or the tally file given as argument), --dry-run
prints the Maven commands, and --help lists every mode.