Max Gekk created SPARK-59764:
--------------------------------
Summary: BenchmarkQueryTest.checkGeneratedCode checks nothing
under adaptive query execution
Key: SPARK-59764
URL: https://issues.apache.org/jira/browse/SPARK-59764
Project: Spark
Issue Type: Test
Components: SQL
Affects Versions: 5.0.0
Reporter: Max Gekk
{{BenchmarkQueryTest.checkGeneratedCode}} is meant to check that every
whole-stage codegen subtree of a benchmark query compiles and that its largest
generated method is within {{CodeGenerator.DEFAULT_JVM_HUGE_METHOD_LIMIT}}
(8000 bytes, HotSpot's {{HugeMethodLimit}}). {{TPCDSQuerySuite}} (and
{{TPCDSQueryWithStatsSuite}}, {{TPCDSQueryANSISuite}}), {{TPCHQuerySuite}} and
{{SSBQuerySuite}} call it on {{sql(query).queryExecution.executedPlan}}.
Since adaptive query execution became the default (SPARK-33679, 3.2.0), that
plan is an {{AdaptiveSparkPlanExec}}, a leaf node whose query stages, and so
its {{WholeStageCodegenExec}} nodes, are created only when the query runs. The
suites never run the queries, so the {{plan foreach}} walk in
{{checkGeneratedCode}} finds no {{WholeStageCodegenExec}} and the method
returns without compiling or checking anything. The tests pass vacuously.
Evidence: walking the same way as {{checkGeneratedCode}} (the plan plus its
subqueries) over all TPC-DS v1.4 (103), v2.7 (32) and modified (21) queries and
all 22 TPC-H queries finds 0 whole-stage stages with
{{spark.sql.adaptive.enabled=true}}. With {{spark.sql.adaptive.enabled=false}}
the same walk finds about 1,700 TPC-DS stages (without injected statistics) and
126 TPC-H stages, and exactly one of them is past 8000 bytes: {{modified-q3}}'s
stage (12214 bytes; 12167 with injected statistics), which the suite already
excludes as SPARK-29128.
Two consequences:
* The "can be properly compiled" check and the method size check have not run
since 3.2.0, so a codegen regression in these queries would not be caught here.
* The workarounds in {{TPCDSQuerySuite}} guard an assertion that does not run.
{{spark.sql.readSideCharPadding=false}} ("so that the generated code is less
than 8000") is not needed with AQE off either: padding changes the largest
method of some queries by at most about 500 bytes, and none crosses 8000.
Proposed fix: set {{spark.sql.adaptive.enabled=false}} in
{{BenchmarkQueryTest}}'s {{sparkConf}}, since these suites only compile plans,
and assert that at least one {{WholeStageCodegenExec}} was found, so the check
cannot become vacuous again. Keep the {{modified-q3}} exclusion, and consider
dropping the char padding override.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
