Gengliang Wang created SPARK-21979:
--------------------------------------
Summary: Improve QueryPlanConstraints framework
Key: SPARK-21979
URL: https://issues.apache.org/jira/browse/SPARK-21979
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 2.2.0
Reporter: Gengliang Wang
Priority: Critical
Improve QueryPlanConstraints framework, make it robust and simple.
In apache/spark#15319, constraints for expressions like a = f(b, c) is resolved.
However, for expressions like
a = f(b, c) && c = g(a, b)
The current QueryPlanConstraints framework will produce non-converging
constraints.
Essentially, the problem is caused by having both the name and child of aliases
in the same constraint set. We infer constraints, and push down constraints as
predicates in filters, later on these predicates are propagated as constraints,
etc..
Simply using the alias names only can resolve these problems. The size of
constraints is reduced without losing any information. We can always get these
inferred constraints on child of aliases when pushing down filters.
Also, the EqualNullSafe between name and child in propagating alias is
meaningless
allConstraints += EqualNullSafe(e, a.toAttribute)
It just produce redundant constraints.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
