sarutak commented on PR #59940: URL: https://github.com/apache/spark/pull/59940#issuecomment-6535573927
Thank you for the second round, @dongjoon-hyun. I ran every example, and each one behaves as you described (for item 4 I checked the plan). I pushed dbda81fef60 for them; replies are inline. In short: an inferred `IS NOT NULL` over a cheap alias that a held condition reads now stays above the projection (items 1 and 6); `StartsWith`, `EndsWith`, `Contains`, `Length`, `OctetLength` and a field of a struct built from plain values are in the allowlist (item 8); the new tests no longer depend on the regex heuristic (item 7); the work in item 9 is skipped when nothing needs it. Items 2 to 5 are now in the description and the migration guide, without code changes here; I would like to handle the `FilterExec` part of item 2 separately (see there). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
