sarutak commented on PR #59940: URL: https://github.com/apache/spark/pull/59940#issuecomment-6521593524
Thank you for the careful review, @dongjoon-hyun. I pushed 142312384bd for the comments; replies are inline. In short: the projection split no longer holds conditions back across passes, but keeps the filter as written when a condition held back is rejected by the check, i.e. can fail or is non-deterministic (items 1, 3, 5, 8); conditions are no longer held back behind a scalar Python UDF (item 2); `Size` and numeric widening casts are in the allowlist (item 4); and the description now says this targets 4.4 first (item 6). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
