XiDuo You created SPARK-61232:
---------------------------------
Summary: UnionExec whole-stage codegen fails when a child output
repeats an ExprId
Key: SPARK-61232
URL: https://issues.apache.org/jira/browse/SPARK-61232
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 5.0.0
Reporter: XiDuo You
UnionExec's whole-stage codegen path projects each child output column to the
union's output, but it binds that projection by ExprId through BindReferences.
When a child output repeats an ExprId (e.g. SELECT p, p, w), every reference to
that ExprId resolves to the first occurrence, so:
- the second union output column reads the first column's value (wrong
results), and
- the generated code emits that input variable's declaration twice, which
Janino rejects with "CompileException: Redefinition of local variable
inputadapter_value_3".
The repeated-ExprId shape is normally removed by the analyzer's union-child
deduplication. It reaches a physical UnionExec when
spark.sql.analyzer.unionIsResolvedWhenDuplicatesPerChildResolved and
spark.sql.enforceTypeCoercionBeforeUnionDeduplication.enabled are disabled, or
through a path that skips that deduplication.
The codegen path must read each union output position from its own child
position, which is what doExecute does when it concatenates child rows. The fix
binds the per-child projection to BoundReference(i) instead of the child's
Attribute and drops the BindReferences call in UnionExec.doConsume.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
