GitHub user jinfengni opened a pull request:
https://github.com/apache/drill/pull/328
Drill 2517: Apply directory-based partition pruning before reading files in
planning.
1. Run the pre-commit tests and unit tests. Some queries in pre-commit
suites have changed plan. Most of the changed plan looks better than before.
The only exception is for the cases caused by one existing issue (DRILL-4279),
when * column is used together with SKIP_ALL mode. That happens when the filter
is applied and then removed, for the following query:
SELECT count(*) from T1 where dir0 = 1990 and dir1 = 'Q1'.
2. I need figure out how to cherry-pick Adam's patch in DRILL-2517, since
that's the initial work on this issue, although there is quite big change from
that patch.
@amansinha100 , could you please take a look and give some initial review
comments? Thanks!
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/jinfengni/incubator-drill DRILL-2517
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/drill/pull/328.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #328
----
commit bc5e13972d116f41ed9441b49a6781b2b602c2fd
Author: Mehant Baid <[email protected]>
Date: 2015-11-11T06:26:26Z
DRILL-2571: (Prototype from Mehant) Move directory based partition pruning
to logical phase.
commit 1b05e372ee7193308bea420302bdd0e259193e3a
Author: Jinfeng Ni <[email protected]>
Date: 2016-01-08T18:28:53Z
DRILL-2517: Move directory-based partition pruning to Calcite logical
planning phase.
1) Make directory-based pruning rule both work in calcite logical and drill
logical planning phase.
2) Only apply directory-based pruning in logical phase when there is no
metadata cache.
3) Make FileSelection constructor public, since FileSelection.create()
would modify selectionRoot.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
