[
https://issues.apache.org/jira/browse/SPARK-56908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gengliang Wang updated SPARK-56908:
-----------------------------------
Description:
Whole-stage codegen generates a fresh Java class per stage. Across many
operators the generated code carries (a) type-independent boilerplate that is
re-emitted into every stage, and (b) branches/variables that are statically
dead at codegen time. This costs us in three places: the JVM 64KB method-size
and constant-pool limits (which force interpreted fallback on deep plans),
Janino compile time per stage, and JIT work.
This umbrella tracks small, behavior-preserving cleanups to shrink the
generated code. Each subtask is independently PR-able and verified by the
relevant operator's existing suite with {{spark.sql.codegen.wholeStage}} forced
both on and off.
h3. How we do it
Two techniques:
- *Deduplicate boilerplate into compiled helpers* -- move type-independent
generated logic into a hand-written Java/Scala method (or base class) that is
compiled once and called from the generated code, instead of re-emitting it per
stage. E.g. SPARK-57907 (fast-hash-map base class), SPARK-57908 (close-hook),
SPARK-57909 (ColumnarToRow advance), SPARK-57905
({{UnsafeRowWriter.writeNullable}}).
- *Skip statically-dead code* -- don't emit a branch/guard the plan proves
unreachable. E.g. SPARK-57198 (divide-by-zero guard for a non-zero literal),
SPARK-57201 (null check for non-null values).
Size is tracked with a planning-only benchmark added here (SPARK-57915),
{{WholeStageCodegenSizeBenchmark}}: it plans every TPC-DS query to its
{{executedPlan}} (empty tables, SF=100 stats, no execution), walks the
{{WholeStageCodegenExec}} subtrees via {{debug.codegenStringSeq}}, and reports
grand totals -- source size, summed/max method bytecode, inner-class count,
constant-pool entries, and codegen fallbacks. Run: {{build/sbt
"sql/Test/runMain
org.apache.spark.sql.execution.benchmark.WholeStageCodegenSizeBenchmark"}}.
h3. Measured impact
{{branch-4.2}} (cut 2026-05-01, before most of this umbrella) vs. current
{{master}} (~40 subtasks merged since), over all 135 TPC-DS queries:
|| metric || branch-4.2 || master || delta ||
| source code size | 24,190,570 | 21,845,997 | -9.7% |
| max method bytecode (summed over stages) | 951,309 | 841,185 | -11.6% |
| max method bytecode (largest single method) | 6,527 | 4,962 | -24.0% |
| generated inner classes | 786 | 258 | -67.2% |
| constant pool (summed) | 454,006 | 432,875 | -4.7% |
| codegen fallbacks | 0 | 0 | unchanged |
Biggest wins are where it gates execution: the largest single method shrank
~24% (the metric behind the 64KB method limit and HotSpot 8KB JIT threshold)
and inner classes dropped ~67%. Zero regressions, zero fallbacks. This is the
umbrella's cumulative effect, not any single subtask -- per-subtask numbers are
in each linked PR.
h3. Out of scope
- *Source-only shrinkage.* Janino constant-folds {{if (true)}}/{{if (false)}},
so removing such a branch changes no bytecode and gives no JIT/runtime benefit.
Even the most frequent source-only patterns (~445 occurrences) moved compile
time below the run-to-run noise floor (~0.7%). A dead-branch subtask is only
worth it when it removes *more than source text* -- e.g. an unreachable
constant-pool/{{references[]}} entry Janino can't fold, or keeping a method
under the 64KB limit.
- *Behavior changes.* Every subtask is result-preserving (verified codegen-on
and off).
- *Raw frequency as justification.* Numerous-but-source-only patterns are
dropped, not merged; frequency matters only when the dead code is a large
fraction of a single large method.
was:
Whole-stage codegen generates a fresh Java class per stage. Across many
operators the generated source contains (a) boilerplate that is
type-independent across stages and can be deduplicated into static Java
helpers, and (b) branches or variables that are statically dead at codegen time
but emitted anyway.
These patterns cost us in three places:
- JVM 64KB method-size and constant-pool limits, which force interpreted
fallback on deep query plans.
- Janino compile time per stage.
- JIT compile work (each stage class has its own bodies).
This umbrella tracks small, behavior-preserving cleanups across the generated
Java to address these issues. Each subtask is independently PR-able; behavior
is preserved end-to-end and verified by the relevant operator's existing test
suite with {{spark.sql.codegen.wholeStage}} forced both on and off.
h3. Scope guidance (when is a dead-branch / simplification subtask worth it?)
Not every statically-dead branch is worth eliminating. We measured the payoff
on real generated code (TPC-DS whole-stage codegen): Janino constant-folds {{if
(true)}} / {{if (false)}}, so skipping such a branch produces *no bytecode
change* and no JIT/runtime benefit -- only smaller source. Janino compile time
is ~linear in source (~0.36 ms/KB), and even the most frequent source-only
patterns we measured ({{if (true)}} x445, a dead null-write branch x358 --
together only ~0.4% of generated source) saved compile time below the
run-to-run noise floor (~0.7%).
Therefore a subtask that adds branching / complexity to the codegen logic to
skip a dead branch is only justified when:
- *(b) it removes more than source text -- this is the real bar.* For example
SPARK-57198: skipping the divide-by-zero guard for a non-zero literal also
stops registering the unreachable {{errCtx}} entry in the {{references[]}} /
constant pool, which Janino cannot fold away (unlike the {{if (false)}} text
itself). Likewise SPARK-57199 moved the {{AGGREGATE_OUT_OF_MEMORY}} string +
map constructor out of all 142 generated aggregate classes' constant pools into
one compiled method. Keeping a large method under the 64KB / huge-method limit
also qualifies.
- *(a) raw frequency rarely suffices on its own.* As measured above, even
patterns occurring 358-445 times stayed below the compile-time noise floor with
no bytecode change. Frequency only matters when the dead code is a large
fraction of a _single_ method (approaching the 64KB limit), not merely numerous
across the corpus.
Trivial, infrequent (and even frequent-but-source-only) dead-branch removals
add codegen-logic complexity for negligible benefit and should be dropped
rather than merged.
h3. Measured cumulative impact (as of master, 2026-07)
We now measure generated codegen size across all TPC-DS queries with a
dedicated, planning-only benchmark added under this umbrella (SPARK-57915):
{{org.apache.spark.sql.execution.benchmark.WholeStageCodegenSizeBenchmark}}. It
creates the TPC-DS tables empty (SF=100 stats injected so plan shapes match
production), plans every query to its {{executedPlan}} with no data or
execution, walks the {{WholeStageCodegenExec}} subtrees via
{{debug.codegenStringSeq}}, and reports grand totals: generated source size,
summed/max compiled method bytecode, generated inner-class count, constant-pool
entries, cold Janino compile time, and codegen fallbacks. Run it with
{{build/sbt "sql/Test/runMain
org.apache.spark.sql.execution.benchmark.WholeStageCodegenSizeBenchmark"}}.
Comparing {{branch-4.2}} (the 4.2 cut on 2026-05-01, before the bulk of this
umbrella's subtasks) against current {{master}} (which carries ~40 SPARK-56908
subtasks merged since) over all 135 measured TPC-DS queries:
|| metric || branch-4.2 || master || delta ||
| source code size (chars) | 24,190,570 | 21,845,997 | -9.7% |
| max method bytecode, summed over stages | 951,309 | 841,185 | -11.6% |
| max method bytecode, single largest method | 6,527 | 4,962 | -24.0% |
| generated inner classes | 786 | 258 | -67.2% |
| constant pool, summed over stages | 454,006 | 432,875 | -4.7% |
| Janino compile time (ms) | 10,784 | 10,362 | -3.9% |
| codegen fallbacks | 0 | 0 | unchanged |
The largest wins are in the metrics that actually gate execution and cost: the
single largest generated method shrank ~24% (the metric behind the HotSpot 8KB
JIT-compilation threshold and the 64KB method limit), and generated inner
classes dropped ~67% (fewer classes for Janino to compile, load, and verify,
and less metaspace per query per executor). No query regressed on any metric
and codegen fallbacks stayed at zero, confirming the changes are
behavior-preserving.
This is the cumulative effect of the whole umbrella's work over that window
(dead-check elimination such as SPARK-57198/SPARK-57201, static-helper
extraction across many expressions, and the recent fast-hash-map /
UnsafeRowWriter / ColumnarToRow / Coalesce reductions), not of any single
subtask; each subtask's own before/after numbers are in its linked PR.
> Reduce the size of code generated by whole-stage codegen
> --------------------------------------------------------
>
> Key: SPARK-56908
> URL: https://issues.apache.org/jira/browse/SPARK-56908
> Project: Spark
> Issue Type: Umbrella
> Components: SQL
> Affects Versions: 4.3.0
> Reporter: Gengliang Wang
> Priority: Major
> Labels: pull-request-available
>
> Whole-stage codegen generates a fresh Java class per stage. Across many
> operators the generated code carries (a) type-independent boilerplate that is
> re-emitted into every stage, and (b) branches/variables that are statically
> dead at codegen time. This costs us in three places: the JVM 64KB method-size
> and constant-pool limits (which force interpreted fallback on deep plans),
> Janino compile time per stage, and JIT work.
> This umbrella tracks small, behavior-preserving cleanups to shrink the
> generated code. Each subtask is independently PR-able and verified by the
> relevant operator's existing suite with {{spark.sql.codegen.wholeStage}}
> forced both on and off.
> h3. How we do it
> Two techniques:
> - *Deduplicate boilerplate into compiled helpers* -- move type-independent
> generated logic into a hand-written Java/Scala method (or base class) that is
> compiled once and called from the generated code, instead of re-emitting it
> per stage. E.g. SPARK-57907 (fast-hash-map base class), SPARK-57908
> (close-hook), SPARK-57909 (ColumnarToRow advance), SPARK-57905
> ({{UnsafeRowWriter.writeNullable}}).
> - *Skip statically-dead code* -- don't emit a branch/guard the plan proves
> unreachable. E.g. SPARK-57198 (divide-by-zero guard for a non-zero literal),
> SPARK-57201 (null check for non-null values).
> Size is tracked with a planning-only benchmark added here (SPARK-57915),
> {{WholeStageCodegenSizeBenchmark}}: it plans every TPC-DS query to its
> {{executedPlan}} (empty tables, SF=100 stats, no execution), walks the
> {{WholeStageCodegenExec}} subtrees via {{debug.codegenStringSeq}}, and
> reports grand totals -- source size, summed/max method bytecode, inner-class
> count, constant-pool entries, and codegen fallbacks. Run: {{build/sbt
> "sql/Test/runMain
> org.apache.spark.sql.execution.benchmark.WholeStageCodegenSizeBenchmark"}}.
> h3. Measured impact
> {{branch-4.2}} (cut 2026-05-01, before most of this umbrella) vs. current
> {{master}} (~40 subtasks merged since), over all 135 TPC-DS queries:
> || metric || branch-4.2 || master || delta ||
> | source code size | 24,190,570 | 21,845,997 | -9.7% |
> | max method bytecode (summed over stages) | 951,309 | 841,185 | -11.6% |
> | max method bytecode (largest single method) | 6,527 | 4,962 | -24.0% |
> | generated inner classes | 786 | 258 | -67.2% |
> | constant pool (summed) | 454,006 | 432,875 | -4.7% |
> | codegen fallbacks | 0 | 0 | unchanged |
> Biggest wins are where it gates execution: the largest single method shrank
> ~24% (the metric behind the 64KB method limit and HotSpot 8KB JIT threshold)
> and inner classes dropped ~67%. Zero regressions, zero fallbacks. This is the
> umbrella's cumulative effect, not any single subtask -- per-subtask numbers
> are in each linked PR.
> h3. Out of scope
> - *Source-only shrinkage.* Janino constant-folds {{if (true)}}/{{if
> (false)}}, so removing such a branch changes no bytecode and gives no
> JIT/runtime benefit. Even the most frequent source-only patterns (~445
> occurrences) moved compile time below the run-to-run noise floor (~0.7%). A
> dead-branch subtask is only worth it when it removes *more than source text*
> -- e.g. an unreachable constant-pool/{{references[]}} entry Janino can't
> fold, or keeping a method under the 64KB limit.
> - *Behavior changes.* Every subtask is result-preserving (verified
> codegen-on and off).
> - *Raw frequency as justification.* Numerous-but-source-only patterns are
> dropped, not merged; frequency matters only when the dead code is a large
> fraction of a single large method.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
