This is an automated email from the ASF dual-hosted git repository.
bridgetb pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/drill-site.git
The following commit(s) were added to refs/heads/asf-site by this push:
new 43cca72 updates for drill 1.14
43cca72 is described below
commit 43cca72f5b285bdfba2a58d6bf01b9fdcfc6fc58
Author: Bridget Bevens <[email protected]>
AuthorDate: Thu Jun 14 15:29:55 2018 -0700
updates for drill 1.14
---
docs/configuration-options-introduction/index.html | 4 +-
docs/parquet-filter-pushdown/index.html | 53 ++++++++++++++++------
feed.xml | 4 +-
3 files changed, 42 insertions(+), 19 deletions(-)
diff --git a/docs/configuration-options-introduction/index.html
b/docs/configuration-options-introduction/index.html
index be7bebf..e6a6f10 100644
--- a/docs/configuration-options-introduction/index.html
+++ b/docs/configuration-options-introduction/index.html
@@ -1228,7 +1228,7 @@
</div>
- Jun 8, 2018
+ Jun 14, 2018
<link href="/css/docpage.css" rel="stylesheet" type="text/css">
@@ -1624,7 +1624,7 @@
<tr>
<td>store.hive.optimize_scan_with_native_readers</td>
<td>FALSE</td>
-<td>By default, Drill reads Hive tables using the native Hive reader. When
you enable this option, Drill reads Hive tables using Drill native readers,
which enables faster reads and enforces direct memory usage. Starting in
Drill 1.14, the option also enables Drill to apply filter push down and to
query Parquet data (created by Drill) with decimal values.</td>
+<td>By default, Drill reads Hive tables using the native Hive reader. When
you enable this option, Drill reads Hive tables using Drill native readers,
which enables faster reads and enforces direct memory usage. Starting in
Drill 1.14, this option also enables Drill to apply filter push down
optimizations.</td>
</tr>
<tr>
<td>store.json.all_text_mode</td>
diff --git a/docs/parquet-filter-pushdown/index.html
b/docs/parquet-filter-pushdown/index.html
index bd4bc87..7da9874 100644
--- a/docs/parquet-filter-pushdown/index.html
+++ b/docs/parquet-filter-pushdown/index.html
@@ -1226,7 +1226,7 @@
</div>
- Apr 3, 2018
+ Jun 14, 2018
<link href="/css/docpage.css" rel="stylesheet" type="text/css">
@@ -1284,15 +1284,29 @@
<h3 id="viewing-the-query-plan">Viewing the Query Plan</h3>
-<p>Because Drill applies Parquet filter pushdown during the query planning
phase, you can view the query execution plan to see if Drill pushes down the
filter when a query on a Parquet file contains a filter expression.</p>
+<p>Because Drill applies Parquet filter pushdown during the query planning
phase, you can view the query execution plan to see if Drill pushes down the
filter when a query on a Parquet file contains a filter expression. You can run
the <a href="/docs/explain-commands/">EXPLAIN PLAN command</a> to see the
execution plan for the query, as shown in the following example.</p>
-<p>Run the <a href="/docs/explain-commands/">EXPLAIN PLAN command</a> to see
the execution plan for the query. See <a href="/docs/query-plans/">Query
Plans</a> for more information. </p>
+<p><strong>Example</strong>
+Starting in Drill 1.14, Drill supports the planner rule,
JoinPushTransitivePredicatesRule, which enables Drill to infer filter
conditions for join queries and push the filter conditions down to the data
source. </p>
+
+<p>This example shows a query plan where the JoinPushTransitivePredicatesRule
is used to push the filter down to each table referenced in the following
query: </p>
+<div class="highlight"><pre><code class="language-text" data-lang="text">
SELECT * FROM dfs.`/tmp/first` t1 JOIN dfs.`/tmp/second` t2 ON t1.`month` =
t2.`month` WHERE t2.`month` = 4
+</code></pre></div>
+<p>This query performs a join on two tables partitioned by the “month” column.
The “first” table has 16 Parquet files, and the “second” table has 7. Issuing
the <code>EXPLAIN PLAN FOR</code> command, you can see that the query planner
applies the filter to both tables, significantly reducing the number of files
read by the Scan operator in each table. </p>
+<div class="highlight"><pre><code class="language-text" data-lang="text">
EXPLAIN PLAN FOR SELECT * FROM dfs.`/tmp/first` t1 JOIN dfs.`/tmp/second` t2
ON t1.`month` = t2.`month` WHERE t2.`month` = 4
+
+ DrillProjectRel(**=[$0], **0=[$2])
+ DrillJoinRel(condition=[=($1, $3)], joinType=[inner])
+ DrillScanRel(table=[[dfs, first]], groupscan=[ParquetGroupScan
[entries=[ReadEntryWithPath [path=/tmp/first/0_0_9.parquet], ReadEntryWithPath
[path=/tmp/first/0_0_10.parquet]], selectionRoot=file:/tmp/first, numFiles=2,
numRowGroups=2, usedMetadataFile=false, columns=[`**`]]])
+ DrillScanRel(table=[[dfs, second]], groupscan=[ParquetGroupScan
[entries=[ReadEntryWithPath [path=/tmp/second/0_0_5.parquet]],
selectionRoot=file:/tmp/second, numFiles=1, numRowGroups=1,
usedMetadataFile=false, columns=[`**`]]])
+</code></pre></div>
+<p>See <a href="/docs/query-plans/">Query Plans</a> for more information. </p>
<h2 id="support">Support</h2>
<p>The following table lists the supported and unsupported clauses, operators,
data types, function, and scenarios for Parquet filter pushdown: </p>
-<p><strong>Note:</strong> An asterisk (*) indicates support as of Drill 1.13.
</p>
+<p><strong>Note:</strong> ^1^ indicates support as of Drill 1.13. ^2^
indicates support as of Drill 1.14. </p>
<table><thead>
<tr>
@@ -1303,36 +1317,45 @@
</thead><tbody>
<tr>
<td>Clauses</td>
-<td>WHERE, *WITH, HAVING (HAVING is supported if Drill can pass the filter
through GROUP BY.)</td>
+<td>WHERE, ^1^WITH, HAVING (HAVING is supported if Drill can pass the filter
through GROUP BY.)</td>
<td>-</td>
</tr>
<tr>
<td>Operators</td>
-<td>AND, OR, NOT, *IS [NOT] NULL, *IS [NOT] TRUE, *IS [NOT] FALSE, IN (An IN
list is converted to OR if the number in the IN list is within a certain
threshold, for example 20. If greater than the threshold, pruning cannot
occur.)</td>
-<td>ITEM (Drill does not push the filter past the ITEM operator, which is
used for complex fields.)</td>
+<td>^2^BETWEEN, ^2^ITEM, AND, OR, NOT, ^1^IS [NOT] NULL, ^1^IS [NOT] TRUE,
^1^IS [NOT] FALSE, IN (An IN list is converted to OR if the number in the IN
list is within a certain threshold, for example 20. If greater than the
threshold, pruning cannot occur.)</td>
+<td>-</td>
</tr>
<tr>
-<td>Comparison Operators</td>
-<td><>, <, >, <=, >=, =</td>
+<td>Comparison Operators</td>
+<td><>, <, >, <=, >=, =</td>
<td>-</td>
</tr>
<tr>
-<td>Data Types</td>
-<td>INT, BIGINT, FLOAT, DOUBLE, DATE, TIMESTAMP, TIME, *BOOLEAN (true,
false)</td>
-<td>CHAR, VARCHAR columns, Hive TIMESTAMP</td>
+<td>Data Types</td>
+<td>INT, BIGINT, FLOAT, DOUBLE, DATE, TIMESTAMP, TIME, *BOOLEAN (true,
false)</td>
+<td>CHAR, VARCHAR columns, Hive TIMESTAMP</td>
</tr>
<tr>
<td>Function</td>
-<td>CAST is supported among the following types only: int, bigint, float,
double, *date, *timestamp, and *time</td>
+<td>CAST is supported among the following types only: int, bigint, float,
double, ^1^date, ^1^timestamp, and ^1^time</td>
<td>-</td>
</tr>
<tr>
<td>Other</td>
-<td>Files with multiple row groups</td>
-<td>Joins, Enabled Native Hive reader</td>
+<td>^2^Enabled ^2^native Hive reader, Files with multiple row groups,
Joins</td>
+<td>-</td>
</tr>
</tbody></table>
+<p><strong>Note:</strong> Drill cannot infer filter conditions for join
queries that have: </p>
+
+<ul>
+<li>a dynamic star in the sub-query or queries that include the WITH
statement.<br></li>
+<li>several filter predicates with the OR logical operator.<br></li>
+<li>more than one EXISTS operator (instead of JOIN operators).<br></li>
+<li>INNER JOIN and local filtering with a several conditions.
|<br></li>
+</ul>
+
<div class="doc-nav">
diff --git a/feed.xml b/feed.xml
index d35c9f5..8a40edb 100644
--- a/feed.xml
+++ b/feed.xml
@@ -6,8 +6,8 @@
</description>
<link>/</link>
<atom:link href="/feed.xml" rel="self" type="application/rss+xml"/>
- <pubDate>Wed, 13 Jun 2018 11:33:06 -0700</pubDate>
- <lastBuildDate>Wed, 13 Jun 2018 11:33:06 -0700</lastBuildDate>
+ <pubDate>Thu, 14 Jun 2018 15:27:53 -0700</pubDate>
+ <lastBuildDate>Thu, 14 Jun 2018 15:27:53 -0700</lastBuildDate>
<generator>Jekyll v2.5.2</generator>
<item>
--
To stop receiving notification emails like this one, please contact
[email protected].
