Anish Mahto created SPARK-60406:
-----------------------------------
Summary: Count CollapseProject's column references by expression ID
Key: SPARK-60406
URL: https://issues.apache.org/jira/browse/SPARK-60406
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 4.4.0
Reporter: Anish Mahto
Count CollapseProject's references to a column by expression ID instead of by
attribute equality.
When two projections are adjacent, CollapseProject inlines each expression the
lower one defines into the upper one's references to it. It does this only if
the expression is referenced once, or is cheap enough to evaluate at every
reference. Otherwise it keeps both projections, so the value is evaluated once
and shared. References are counted by attribute equality, which includes the
qualifier. So when one column is referenced both directly and through a
subquery alias, its references are counted apart, and an expression referenced
twice can count as once and be evaluated at each reference.
Inlining itself finds references by expression ID, which identifies a column
whatever its qualifier. Counting by expression ID therefore counts exactly the
copies inlining will make. Only plans that reference one column under different
qualifiers change.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
