GitHub user bogdanrdc opened a pull request:
https://github.com/apache/spark/pull/22205
[SPARK-25212][SQL] Support Filter in ConvertToLocalRelation
## What changes were proposed in this pull request?
Support Filter in ConvertToLocalRelation, similar to how Project works.
Additionally, in Optimizer, run ConvertToLocalRelation earlier to simplify
the plan. This is good for very short queries which often are queries on local
relations.
## How was this patch tested?
New test. Manual benchmark.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/bogdanrdc/spark local-relation-filter
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/22205.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #22205
----
commit 2a1cd27d7f188b51429553718cac91d87d667bbd
Author: Bogdan Raducanu <bogdan@...>
Date: 2018-08-23T13:17:11Z
optimization + test
commit 421ee20e27c6d1ba9384528c765690103f407d34
Author: Bogdan Raducanu <bogdan@...>
Date: 2018-08-23T15:07:18Z
debug benchmark + early batch
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
