GitHub user viirya opened a pull request:
https://github.com/apache/spark/pull/19569
[SPARK-22348][SQL] The table cache providing ColumnarBatch should also do
partition batch pruning
## What changes were proposed in this pull request?
We enable table cache `InMemoryTableScanExec` to provide `ColumnarBatch`
now. But the cached batches are retrieved without pruning. In this case, we
still need to do partition batch pruning.
## How was this patch tested?
Existing tests.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/viirya/spark-1 SPARK-22348
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/19569.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #19569
----
commit f2c8266b4074bc387c4b46bb5b46cca82ff3929d
Author: Liang-Chi Hsieh <[email protected]>
Date: 2017-10-25T02:11:09Z
Do partition batch pruning for ColumnarBatch path too.
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
