GitHub user gengliangwang opened a pull request:
https://github.com/apache/spark/pull/19201
[SPARK-21979][SQL]Improve QueryPlanConstraints framework
## What changes were proposed in this pull request?
Improve QueryPlanConstraints framework, make it robust and simple.
In https://github.com/apache/spark/pull/15319, constraints for expressions
like `a = f(b, c)` is resolved.
However, for expressions like
```scala
a = f(b, c) && c = g(a, b)
```
The current QueryPlanConstraints framework will produce non-converging
constraints.
Essentially, the problem is caused by having both the name and child of
aliases in the same constraint set. We infer constraints, and push down
constraints as predicates in filters, later on these predicates are propagated
as constraints, etc..
Simply using the alias names only can resolve these problems. The size of
constraints is reduced without losing any information. We can always get these
inferred constraints on child of aliases when pushing down filters.
Also, the EqualNullSafe between name and child in propagating alias is
meaningless
```scala
allConstraints += EqualNullSafe(e, a.toAttribute)
```
It just produces redundant constraints.
## How was this patch tested?
Unit test
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/gengliangwang/spark QueryPlanConstraints
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/19201.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #19201
----
commit 036e846a571f7aea3ad28b875afd5f9d714c25a5
Author: Wang Gengliang <[email protected]>
Date: 2017-09-12T09:06:09Z
improve QueryPlanConstraints
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
