Abstract
--------

Most `javac` tool calls are short-lived, and for them the cost of the default 
tiered compilation can outweigh what it gives back. This PR adds a simple 
heuristic to the `javac` launcher: when the **total size** of the Java source 
files to be compiled is below a 2 MiB threshold, the launcher adds 
`-XX:TieredStopAtLevel=1` to the JVM options. On the 375 real-world projects we 
measured, all 148 projects that trigger the heuristic (under 2 MiB, compiled 
with `javac` directly) compile at least 5% faster, with speedups up to 56%, and 
no project regresses by 5% or more.

Background
----------

Tiered compilation is costly for short-lived processes and can make up a large 
part of their wall time. For example, a build of tablesaw [1] (398 `.java` 
files, 1.74 MiB of sources) on an AMD Turin machine, with different JVM options:

| JIT mode                                       | 1 CPU  | 16 CPUs |
|------------------------------------------------|-------:|--------:|
| default (C1 with profiling, then C2)           | 23.41s |   8.33s |
| default, speculative traps disabled (*)        | 23.35s |   8.24s |
| C1 with profiling, no C2 (TieredStopAtLevel=3) | 17.78s |  15.71s |
| C1 only, no profiling (TieredStopAtLevel=1)    | 12.07s |   7.42s |

(*) -XX:PerMethodTrapLimit=0 -XX:PerBytecodeTrapLimit=0
    -XX:PerMethodSpecTrapLimit=0; deoptimizations drop from 593 to 24
    at 1 CPU and from 1686 to 59 at 16 CPUs.

We can see that tiered compilation accounts for a large share of the cost. It 
first runs slower, profiled code to collect profiles, then spends CPU time on 
C2, and this pays off only over a long run. A short-lived process exits before 
it does.

`javac` is usually such a process: it compiles a set of sources once and exits; 
it is not a long-running service. And while each `javac` run is short, frequent 
invocations in scripts, CI pipelines, and automated workflows (such as AI 
coding agents working on tasks like SWE-bench [2]) make the cumulative 
wall-time overhead significant.

How to fix it
-------------

Users expect a fix that works out of the box. The mechanisms that exist today 
do not provide one for `javac`:

- The AOT cache of Project Leyden [3] can hold loaded classes, profiles and 
compiled code, but it has to be created by a training run, and the JDK does not 
ship one for `javac` currently.
- The default CDS archive that ships with the JDK contains no `javac` classes, 
and CDS only saves class loading and linking time; it does not change JIT 
compilation.

The key is to reduce the cost of tiered compilation for short-lived runs. 
`-XX:TieredStopAtLevel=1` does that: it compiles hot methods with C1 only, 
without profiling, so a method gets its final code at its first compilation, 
and C2 is never involved. The JDK build itself already uses this option (see 
`JAVA_FLAGS_SMALL` in `make/autoconf/boot-jdk.m4`). This PR makes the option 
the default for `javac` runs that a simple heuristic predicts to be short-lived.

This change
-----------

1. Adds a heuristic in the `javac` launcher: before the JVM is created, it sums 
the sizes of the `.java` source files given on the command line and in 
`@argfiles`. If the total is **below 2 MiB**, it adds 
`-XX:TieredStopAtLevel=1`; otherwise the JVM options are unchanged.

2. The option is added before the user's options, so 
`-J-XX:TieredStopAtLevel=<level>` on the command line overrides it. 
`JAVA_TOOL_OPTIONS` does not, because the VM applies it before the launcher's 
options.

3. Documents `-XX:TieredStopAtLevel=` in the `java` man page and documents the 
automatic heuristic and `-J-XX:TieredStopAtLevel=` override under `-J` in the 
`javac` man page (without exposing the internal threshold).

4. New test `test/jdk/tools/launcher/JavacTieredStopHeuristicTest.java`.

Test results
------------

The patch was tested on 375 real projects: 213 from the 10,000 most popular 
artifacts on Maven Central, 159 from the 2,000 most-starred Java repositories 
on GitHub, and 3 SWE-bench workloads (189 compiled directly with `javac`, 186 
built with Maven). Each was compiled on Arm Neoverse V2, AMD Turin and NVIDIA 
Vera machines with 1 to 64 CPUs. `ratio = patched_wall_time / base_wall_time` 
is computed on each machine, and `mean_ratio` is the mean over the three 
machines; smaller is better. Median wall time stops improving beyond 4 CPUs, so 
testing up to 64 CPUs.

| mean_ratio     | 1 CPU | 4 CPUs | 8 CPUs | 32 CPUs | 64 CPUs |
|----------------|------:|-------:|-------:|--------:|--------:|
| <= 0.95        | 39.5% |  39.5% |  39.5% |   39.5% |   40.0% |
| 0.95-1.05      | 60.3% |  60.5% |  60.3% |   60.5% |   59.7% |
| >= 1.05        |  0.3% |   0.0% |   0.3% |    0.0% |    0.3% |
| average        |  0.84 |   0.92 |   0.93 |    0.93 |    0.93 |


- Heuristic hit rate: out of the 189 projects compiled directly with `javac`, 
148 (78.3%, or 39.5% of all 375 projects) are below 2 MiB and trigger the 
heuristic; the remaining 41 `javac` projects are >= 2 MiB, and the 186 Maven 
projects do not trigger it (see Limitation 5).
- 100% of the 148 projects that trigger the heuristic compile 5% or more faster 
at every CPU count: average `mean_ratio` is 0.58 (up to 56% faster) at 1 CPU, 
0.81 (up to 29% faster) at 4-8 CPUs, and 0.84 (up to 26% faster) at 32-64 CPUs. 
The gain is largest with few CPUs, where C2 competes most with the compiler 
threads.
- Nearly all of the remaining ~60% (0.95-1.05) did not trigger the heuristic 
(either >= 2 MiB or built by Maven); the single case at 1, 8 and 64 CPUs with 
>= 1.05 is also a project the heuristic did not apply to, so it is measurement 
noise.
- The OpenJDK build time is essentially unchanged, as expected: the build 
already compiles with `-XX:TieredStopAtLevel=1`.

Please see the attachments in JDK-8396147 for detailed test results.

Limitations
-----------

1. It applies only to the `javac` launcher, not to the compiler API 
(`javax.tools`) or any other JDK tool.

2. It does not apply when an option that makes `javac` compile sources it 
cannot count, run plugins, or use an AOT cache is given, on the command line or 
in an `@argfile`:

-sourcepath,
--source-path,
--module-source-path,
-m,
--module,
--patch-module,
-processor,
-processorpath,
--processor-path,
--processor-module-path,
-proc:full,
-proc:only,
-Xprint,
-A<key>,
-Xplugin:<name>,
-J-XX:AOTCache=<file>,
-J-XX:AOTCacheOutput=<file>,
-J-XX:AOTConfiguration=<file>,
-J-XX:AOTMode=<mode>

Nor does it apply when `JDK_JAVAC_OPTIONS` is set, when no `.java` source is 
given, or when a `.java` argument cannot be found or read.

3. Any argument that does not start with `-` and ends in `.java` is counted, 
including one that is not a source file. This can only make the heuristic more 
conservative.

4. Sources that `javac` finds on the class path, without `-sourcepath`, are not 
counted.

5. Many real projects are compiled by build tools such as Maven and Gradle, 
which may or may not run the `javac` launcher (for example, Maven compiles 
in-process by default, and tablesaw above runs via Maven unless compiled 
directly). When they do not, the heuristic does not apply. The `javac` and 
`java` man pages document `-J-XX:TieredStopAtLevel=<level>` and 
`-XX:TieredStopAtLevel=<level>`, so users, and AI agents that drive the build, 
can apply them where it pays.

6. The heuristic and its 2 MiB threshold are derived empirically from tests on 
189 real projects across current JDKs. As HotSpot evolves (for example, lower 
C2 startup overhead or out-of-the-box Project Leyden AOT caching for `javac`), 
the optimal crossover may shift and the fixed 2 MiB threshold may become 
sub-optimal near the boundary. Even so, near the threshold the difference 
between using `-XX:TieredStopAtLevel=1` or not is small.

7. Because the heuristic lives in `java.base` (`libjli`) and mirrors a subset 
of `javac`'s CLI and `@argfile` syntax, future `javac` options that introduce 
implicit sources or heavy compilation work must be kept in sync here; otherwise 
the heuristic may apply where it should have declined.

<details>
<summary> How is this heuristic derived?</summary> 
------------------------------

Of the factors we examined, total source size is the main factor in whether 
`-XX:TieredStopAtLevel=1` pays. The JDK version and the machine shift the size 
of the gain, but not where it ends.

1. Goal: find when `-XX:TieredStopAtLevel=1` (Tier1) gives a shorter wall time 
than the default JVM options, that is, when `ratio = tier1_wall_time / 
default_wall_time < 1.0`.

2. Candidate factors: the `javac` implementation (JDK version), the execution 
environment (machine and CPU count), the total source size, the number of 
files, and the code density (code bytes / total bytes).

3. Machines: Arm Neoverse V2 (NVIDIA Grace and AWS Graviton4), AMD Turin and 
NVIDIA Vera, with 1, 4, 8, 32 and 64 CPUs.

4. Test sets:
   - 189 real projects from Maven Central whose published sources can be 
compiled by running `javac` directly. Each was compiled with Tier1 always on 
and with the default options; `mean_ratio` is the mean of the three machines' 
ratios.
   - Generated sources, to vary the total size, the number of files and the 
code density independently of each other.

5. JDK version: on JDK 25 the mean ratio over the 189 projects is 0.02 (1 CPU) 
to 0.07 (64 CPUs) higher than on mainline, so Tier1 gains a little less. But 
the smallest project that does not gain, `mean_ratio >= 0.995`, is the same 
2.25 MiB project on both. The JDK version shifts the gain, not the threshold.

6. Execution environment: the mean ratio differs between the three machines by 
up to 0.07, but on each of them the smallest project that does not gain is the 
same 2.25 MiB project.

7. Total source size: with generated sources (JDK 28, 8 CPUs on Grace), the 
median ratio is about 0.74-0.85 below 1 MiB; above 1 MiB, it grows with the 
total size and crosses 1.0 at about 8 MiB.

8. Number of files: with generated sources of the same total size, going from 1 
file to 20000 files moves the ratio by only 0.016-0.056.

9. Code density: `javac` does less work on whitespace and comments than on 
code, so denser sources cost more. Even so, with generated sources of up to 
about 2.8 MiB, Tier1 pays even at nearly 100% code density.

10. So the total source size is the deciding factor, and the heuristic uses 
only it. The threshold is then chosen from the real-project results. Across all 
tested CPU counts, the smallest project that does not benefit from Tier1 is 
`2.25 MiB`, so the threshold is set conservatively at `2 MiB`. </details>

[1] https://github.com/jtablesaw/tablesaw
[2] https://github.com/SWE-bench/SWE-bench
[3] https://openjdk.org/projects/leyden/




---------
- [x] I confirm that I make this contribution in accordance with the [OpenJDK 
Interim AI Policy](https://openjdk.org/legal/ai).

-------------

Commit messages:
 - 8396147: javac launcher: set -XX:TieredStopAtLevel=1 for small source sets

Changes: https://git.openjdk.org/jdk/pull/33991/files
  Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=33991&range=00
  Issue: https://bugs.openjdk.org/browse/JDK-8396147
  Stats: 820 lines in 7 files changed: 818 ins; 0 del; 2 mod
  Patch: https://git.openjdk.org/jdk/pull/33991.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/33991/head:pull/33991

PR: https://git.openjdk.org/jdk/pull/33991

Reply via email to