cloud-fan commented on PR #44688: URL: https://github.com/apache/spark/pull/44688#issuecomment-1895721464
There are two kinds of expr IDs: 1) to reference attributes from children. 2) to indicate the output of the plan node. After canonicalization, the output expr IDs for each plan node (if they have output columns, such as Project) will be normalized to 0, 1, 2, .... The reference expr IDs will be normalized to the ordinal of the matching column from children output columns. So ideally they can match, as the ordinal 0 means the first output column, whose expr ID should also be 0. However, it's possible that an attribute is both a reference and a output, e.g. `Project(a#1 AS aa#3, b#2)`. So it seems we can't find the event column on a canonicalized plan. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
