txwyy123 commented on PR #59841:
URL: https://github.com/apache/spark/pull/59841#issuecomment-6596858690
@Raviston6296 Thanks for the excellent catch! I've reproduced the issue and
pushed a fix.
**Root cause**: The field-based ExprCodes used `code""` (which creates a
`CodeBlock("")`) instead of `EmptyBlock` (the singleton object). These are NOT
reference-equal, so `collectInputs` in `CodeGenerator.scala` treats the code as
still pending:
```scala
// Line 1375 in CodeGenerator.scala:
if (input.code != EmptyBlock) { // CodeBlock("") != EmptyBlock → true
possible = false // ← blocks the whole-stage expression split
}
```
With `possible = false`, `wholeStageSplitArguments` returns `None`, and
`splitExpressionsWithSources` falls back to inlining the entire CASE WHEN
inside `processNext()`. A 2000-branch CASE WHEN inline is >64KB bytecode →
Janino compilation failure.
**Fix**: Changed `code""` → `EmptyBlock` in both
`genOneSideJoinVarsAsFields` (JoinCodegenSupport.scala) and
`createStreamedVarsAsFields` (SortMergeJoinExec.scala). This is the same
sentinel the existing `genOneSideJoinVars` uses after `evaluateVariables()`
consumes the ExprCodes.
**Verified**: Your exact scenario (inner join + 2000-branch CASE WHEN +
`methodLimit=0`) now passes. Full WholeStageCodegenSuite (106 tests) also
passes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
