peter-toth commented on issue #27518: [SPARK-30768][SQL] Constraints should be inferred from inequality attributes URL: https://github.com/apache/spark/pull/27518#issuecomment-584839346 Is the title correct? Isn't what you do here is constant propagation among constraints? Obviously it helps with inequalities of attributes, but also helps where you have complex equalities like `select t1.* from SPARK_30768_1 t1 join SPARK_30768_2 t2 on f(t1.c1) = t2.c1 where t1.c1 = 3` (where `f` is a deterministic function). BTW, I have a PR open to enhance `ConstantPropagation`: https://github.com/apache/spark/pull/24553 where among other enhancements I try to extend propagation of attribute => constant mapping to propagation of deterministic expression => constant mapping. Maybe you could do expression => constant propagation here as well.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
