Peter Toth created SPARK-27604:
----------------------------------
Summary: Enhance constant and constraint propagation
Key: SPARK-27604
URL: https://issues.apache.org/jira/browse/SPARK-27604
Project: Spark
Issue Type: Improvement
Components: SQL
Affects Versions: 3.0.0
Reporter: Peter Toth
There is some room for improvement in terms of how we can use constraints to
simplify expressions and maybe what new expressions we can infer.
Here are some examples what the current implementation can do in terms of
expression simplification:
{code:java}
SELECT * FROM table WHERE i = 5 AND j = i + 3 => SELECT * FROM table
WHERE i = 5 AND j = 8
{code}
And here are some that it could with some improvement:
{code:java}
SELECT * FROM table WHERE i <= 5 AND i = 5 => SELECT * FROM table
WHERE i = 5
SELECT * FROM table WHERE i < j AND ... AND i = j => SELECT * FROM table
WHERE false{code}
I also think that `ConstantPropagation` and `InferFiltersFromConstraints`
functionality somewhat overlaps and with some refactoring we could come up with
a better solution.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
