[jira] [Commented] (DRILL-6174) Parquet pushdown planning improvements

[Bridget Bevens (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-6174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16440036#comment-16440036
 ] 

Bridget Bevens commented on DRILL-6174:
---

Documentation updated to reflect changes described in this JIRA: 
[https://drill.apache.org/docs/parquet-filter-pushdown/#support] 

> Parquet pushdown planning improvements
> --
>
> Key: DRILL-6174
> URL: https://issues.apache.org/jira/browse/DRILL-6174
> Project: Apache Drill
>  Issue Type: Improvement
>Affects Versions: 1.12.0
>Reporter: Arina Ielchiieva
>Assignee: Roman Kulyk
>Priority: Major
>  Labels: doc-impacting, ready-to-commit
> Fix For: 1.13.0
>
>
> Currently parquet pushdown planning has certain limitations 
> (https://drill.apache.org/docs/parquet-filter-pushdown/). This Jira aims to 
> fix some of them. List of improvements can be find below:
> 1. IS [NOT] NULL / TRUE / FALSE
> 2. Timestamp / date / time implicit / explicit casts
> {noformat}
> timestamp -> date
> timestamp -> varchar
> date -> timestamp
> date -> varchar
> time -> timestamp
> time -> date
> time -> varchar
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (DRILL-6174) Parquet pushdown planning improvements

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-6174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16384793#comment-16384793
 ] 

ASF GitHub Bot commented on DRILL-6174:
---

Github user asfgit closed the pull request at:

https://github.com/apache/drill/pull/1131


> Parquet pushdown planning improvements
> --
>
> Key: DRILL-6174
> URL: https://issues.apache.org/jira/browse/DRILL-6174
> Project: Apache Drill
>  Issue Type: Improvement
>Affects Versions: 1.12.0
>Reporter: Arina Ielchiieva
>Assignee: Roman Kulyk
>Priority: Major
>  Labels: doc-impacting, ready-to-commit
> Fix For: 1.13.0
>
>
> Currently parquet pushdown planning has certain limitations 
> (https://drill.apache.org/docs/parquet-filter-pushdown/). This Jira aims to 
> fix some of them. List of improvements can be find below:
> 1. IS [NOT] NULL / TRUE / FALSE
> 2. Timestamp / date / time implicit / explicit casts
> {noformat}
> timestamp -> date
> timestamp -> varchar
> date -> timestamp
> date -> varchar
> time -> timestamp
> time -> date
> time -> varchar
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (DRILL-6174) Parquet pushdown planning improvements

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-6174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16378887#comment-16378887
 ] 

ASF GitHub Bot commented on DRILL-6174:
---

Github user arina-ielchiieva commented on the issue:

https://github.com/apache/drill/pull/1131
  
+1, LGTM.


> Parquet pushdown planning improvements
> --
>
> Key: DRILL-6174
> URL: https://issues.apache.org/jira/browse/DRILL-6174
> Project: Apache Drill
>  Issue Type: Improvement
>Affects Versions: 1.12.0
>Reporter: Arina Ielchiieva
>Assignee: Roman Kulyk
>Priority: Major
>  Labels: doc-impacting, ready-to-commit
> Fix For: 1.13.0
>
>
> Currently parquet pushdown planning has certain limitations 
> (https://drill.apache.org/docs/parquet-filter-pushdown/). This Jira aims to 
> fix some of them. List of improvements can be find below:
> 1. IS [NOT] NULL / TRUE / FALSE
> 2. Timestamp / date / time implicit / explicit casts
> {noformat}
> timestamp -> date
> timestamp -> varchar
> date -> timestamp
> date -> varchar
> time -> timestamp
> time -> date
> time -> varchar
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (DRILL-6174) Parquet pushdown planning improvements

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-6174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16378832#comment-16378832
 ] 

ASF GitHub Bot commented on DRILL-6174:
---

Github user KulykRoman commented on a diff in the pull request:

https://github.com/apache/drill/pull/1131#discussion_r170971218
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/expr/stat/ParquetPredicatesHelper.java
 ---
@@ -0,0 +1,41 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ * 
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * 
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.drill.exec.expr.stat;
+
+import org.apache.parquet.column.statistics.Statistics;
+
+/**
+ * Parquet predicates class helper for filter pushdown.
+ */
+public class ParquetPredicatesHelper {
+
+  /**
+   * Check does the column chunk's statistics have all nulls
+   */
+  static boolean isAllNulls(Statistics stat, long rowCount) {
--- End diff --

@arina-ielchiieva,
Done. 


> Parquet pushdown planning improvements
> --
>
> Key: DRILL-6174
> URL: https://issues.apache.org/jira/browse/DRILL-6174
> Project: Apache Drill
>  Issue Type: Improvement
>Affects Versions: 1.12.0
>Reporter: Arina Ielchiieva
>Assignee: Roman Kulyk
>Priority: Major
>  Labels: doc-impacting
> Fix For: 1.13.0
>
>
> Currently parquet pushdown planning has certain limitations 
> (https://drill.apache.org/docs/parquet-filter-pushdown/). This Jira aims to 
> fix some of them. List of improvements can be find below:
> 1. IS [NOT] NULL / TRUE / FALSE
> 2. Timestamp / date / time implicit / explicit casts
> {noformat}
> timestamp -> date
> timestamp -> varchar
> date -> timestamp
> date -> varchar
> time -> timestamp
> time -> date
> time -> varchar
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (DRILL-6174) Parquet pushdown planning improvements

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-6174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16378794#comment-16378794
 ] 

ASF GitHub Bot commented on DRILL-6174:
---

Github user arina-ielchiieva commented on a diff in the pull request:

https://github.com/apache/drill/pull/1131#discussion_r170964343
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/expr/stat/ParquetPredicatesHelper.java
 ---
@@ -0,0 +1,41 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ * 
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * 
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.drill.exec.expr.stat;
+
+import org.apache.parquet.column.statistics.Statistics;
+
+/**
+ * Parquet predicates class helper for filter pushdown.
+ */
+public class ParquetPredicatesHelper {
+
+  /**
+   * Check does the column chunk's statistics have all nulls
+   */
+  static boolean isAllNulls(Statistics stat, long rowCount) {
--- End diff --

1. Please make helper method public as well. 
2. Please add `@param` and `@return` with description to method javadoc.
Please fix here and in method below.


> Parquet pushdown planning improvements
> --
>
> Key: DRILL-6174
> URL: https://issues.apache.org/jira/browse/DRILL-6174
> Project: Apache Drill
>  Issue Type: Improvement
>Affects Versions: 1.12.0
>Reporter: Arina Ielchiieva
>Assignee: Roman Kulyk
>Priority: Major
>  Labels: doc-impacting
> Fix For: 1.13.0
>
>
> Currently parquet pushdown planning has certain limitations 
> (https://drill.apache.org/docs/parquet-filter-pushdown/). This Jira aims to 
> fix some of them. List of improvements can be find below:
> 1. IS [NOT] NULL / TRUE / FALSE
> 2. Timestamp / date / time implicit / explicit casts
> {noformat}
> timestamp -> date
> timestamp -> varchar
> date -> timestamp
> date -> varchar
> time -> timestamp
> time -> date
> time -> varchar
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (DRILL-6174) Parquet pushdown planning improvements

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-6174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16378607#comment-16378607
 ] 

ASF GitHub Bot commented on DRILL-6174:
---

Github user KulykRoman commented on the issue:

https://github.com/apache/drill/pull/1131
  
All changes were applied and all tests passed.


> Parquet pushdown planning improvements
> --
>
> Key: DRILL-6174
> URL: https://issues.apache.org/jira/browse/DRILL-6174
> Project: Apache Drill
>  Issue Type: Improvement
>Affects Versions: 1.12.0
>Reporter: Arina Ielchiieva
>Assignee: Roman Kulyk
>Priority: Major
>  Labels: doc-impacting
> Fix For: 1.13.0
>
>
> Currently parquet pushdown planning has certain limitations 
> (https://drill.apache.org/docs/parquet-filter-pushdown/). This Jira aims to 
> fix some of them. List of improvements can be find below:
> 1. IS [NOT] NULL / TRUE / FALSE
> 2. Timestamp / date / time implicit / explicit casts
> {noformat}
> timestamp -> date
> timestamp -> varchar
> date -> timestamp
> date -> varchar
> time -> timestamp
> time -> date
> time -> varchar
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (DRILL-6174) Parquet pushdown planning improvements

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-6174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16378493#comment-16378493
 ] 

ASF GitHub Bot commented on DRILL-6174:
---

Github user arina-ielchiieva commented on a diff in the pull request:

https://github.com/apache/drill/pull/1131#discussion_r170902942
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/expr/stat/ParquetPredicatesHelper.java
 ---
@@ -0,0 +1,36 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ * 
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * 
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.drill.exec.expr.stat;
+
+import org.apache.parquet.column.statistics.Statistics;
+
+class ParquetPredicatesHelper {
--- End diff --

Please make public and add java doc for each helper method.


> Parquet pushdown planning improvements
> --
>
> Key: DRILL-6174
> URL: https://issues.apache.org/jira/browse/DRILL-6174
> Project: Apache Drill
>  Issue Type: Improvement
>Affects Versions: 1.12.0
>Reporter: Arina Ielchiieva
>Assignee: Roman Kulyk
>Priority: Major
>  Labels: doc-impacting
> Fix For: 1.13.0
>
>
> Currently parquet pushdown planning has certain limitations 
> (https://drill.apache.org/docs/parquet-filter-pushdown/). This Jira aims to 
> fix some of them. List of improvements can be find below:
> 1. IS [NOT] NULL / TRUE / FALSE
> 2. Timestamp / date / time implicit / explicit casts
> {noformat}
> timestamp -> date
> timestamp -> varchar
> date -> timestamp
> date -> varchar
> time -> timestamp
> time -> date
> time -> varchar
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (DRILL-6174) Parquet pushdown planning improvements

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-6174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16378492#comment-16378492
 ] 

ASF GitHub Bot commented on DRILL-6174:
---

Github user arina-ielchiieva commented on a diff in the pull request:

https://github.com/apache/drill/pull/1131#discussion_r170902502
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/expr/stat/ParquetBooleanPredicates.java
 ---
@@ -0,0 +1,72 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.drill.exec.expr.stat;
+
+import org.apache.drill.common.expression.BooleanOperator;
+import org.apache.drill.common.expression.ExpressionPosition;
+import org.apache.drill.common.expression.LogicalExpression;
+import org.apache.drill.common.expression.visitors.ExprVisitor;
+
+import java.util.List;
+
+public abstract class ParquetBooleanPredicates {
--- End diff --

1. Please remove abstract.
2. Add javadoc.
Here and in other two similar classes.


> Parquet pushdown planning improvements
> --
>
> Key: DRILL-6174
> URL: https://issues.apache.org/jira/browse/DRILL-6174
> Project: Apache Drill
>  Issue Type: Improvement
>Affects Versions: 1.12.0
>Reporter: Arina Ielchiieva
>Assignee: Roman Kulyk
>Priority: Major
>  Labels: doc-impacting
> Fix For: 1.13.0
>
>
> Currently parquet pushdown planning has certain limitations 
> (https://drill.apache.org/docs/parquet-filter-pushdown/). This Jira aims to 
> fix some of them. List of improvements can be find below:
> 1. IS [NOT] NULL / TRUE / FALSE
> 2. Timestamp / date / time implicit / explicit casts
> {noformat}
> timestamp -> date
> timestamp -> varchar
> date -> timestamp
> date -> varchar
> time -> timestamp
> time -> date
> time -> varchar
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (DRILL-6174) Parquet pushdown planning improvements

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-6174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16377058#comment-16377058
 ] 

ASF GitHub Bot commented on DRILL-6174:
---

Github user arina-ielchiieva commented on a diff in the pull request:

https://github.com/apache/drill/pull/1131#discussion_r170633434
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/expr/stat/ParquetPredicates.java
 ---
@@ -66,6 +66,27 @@ public ParquetBooleanPredicate(String name, 
List args, Expres
 }
   }
 
+  public static abstract class ParquetIsPredicate extends 
LogicalExpressionBase implements ParquetFilterPredicate {
--- End diff --

`ParquetPredicates` class becomes too overloaded. Please split it into two: 
`ParquetComparisionPredicates` and `ParquetIsPredicate`.


> Parquet pushdown planning improvements
> --
>
> Key: DRILL-6174
> URL: https://issues.apache.org/jira/browse/DRILL-6174
> Project: Apache Drill
>  Issue Type: Improvement
>Affects Versions: 1.12.0
>Reporter: Arina Ielchiieva
>Assignee: Roman Kulyk
>Priority: Major
>  Labels: doc-impacting
> Fix For: 1.13.0
>
>
> Currently parquet pushdown planning has certain limitations 
> (https://drill.apache.org/docs/parquet-filter-pushdown/). This Jira aims to 
> fix some of them. List of improvements can be find below:
> 1. IS [NOT] NULL / TRUE / FALSE
> 2. Timestamp / date / time implicit / explicit casts
> {noformat}
> timestamp -> date
> timestamp -> varchar
> date -> timestamp
> date -> varchar
> time -> timestamp
> time -> date
> time -> varchar
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (DRILL-6174) Parquet pushdown planning improvements

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-6174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16377059#comment-16377059
 ] 

ASF GitHub Bot commented on DRILL-6174:
---

Github user arina-ielchiieva commented on a diff in the pull request:

https://github.com/apache/drill/pull/1131#discussion_r170630734
  
--- Diff: 
exec/java-exec/src/test/java/org/apache/drill/exec/store/parquet/TestParquetFilterPushDownForDateTimeCasts.java
 ---
@@ -0,0 +1,127 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.drill.exec.store.parquet;
+
+import org.apache.drill.PlanTestBase;
+import org.junit.AfterClass;
+import org.junit.BeforeClass;
+import org.junit.Test;
+
+import static org.junit.Assert.assertEquals;
+
+public class TestParquetFilterPushDownForDateTimeCasts extends 
PlanTestBase {
+
+  private static final String TABLE_NAME = "dateTimeCasts";
+
+  @BeforeClass
+  public static void init() throws Exception {
+test("use dfs.tmp");
+test("create table `%s/p1` as\n" +
+"select timestamp '2017-01-01 00:00:00' as col_timestamp, date 
'2017-01-01' as col_date, time '00:00:00' as col_time from (values(1)) union 
all\n" +
+"select timestamp '2017-01-02 00:00:00' as col_timestamp, date 
'2017-01-02' as col_date, time '00:00:00' as col_time from (values(1)) union 
all\n" +
+"select timestamp '2017-01-02 21:01:15' as col_timestamp, date 
'2017-01-02' as col_date, time '21:01:15' as col_time from (values(1))", 
TABLE_NAME);
+
+test("create table `%s/p2` as\n" +
+"select timestamp '2017-01-03 08:50:00' as col_timestamp, date 
'2017-01-03' as col_date, time '08:50:00' as col_time from (values(1)) union 
all\n" +
+"select timestamp '2017-01-04 15:25:00' as col_timestamp, date 
'2017-01-04' as col_date, time '15:25:00' as col_time from (values(1)) union 
all\n" +
+"select timestamp '2017-01-04 22:14:29' as col_timestamp, date 
'2017-01-04' as col_date, time '22:14:29' as col_time from (values(1))", 
TABLE_NAME);
+
+test("create table `%s/p3` as\n" +
+"select timestamp '2017-01-05 05:46:11' as col_timestamp, date 
'2017-01-05' as col_date, time '05:46:11' as col_time from (values(1)) union 
all\n" +
+"select timestamp '2017-01-06 06:17:59' as col_timestamp, date 
'2017-01-06' as col_date, time '06:17:59' as col_time from (values(1)) union 
all\n" +
+"select timestamp '2017-01-06 06:17:59' as col_timestamp, date 
'2017-01-06' as col_date, time '06:17:59' as col_time from (values(1)) union 
all\n" +
+"select cast(null as timestamp) as col_timestamp, cast(null as 
date) as col_date, cast(null as time) as col_time from (values(1))", 
TABLE_NAME);
+  }
+
+  @AfterClass
+  public static void tearDown() throws Exception {
+test("drop table if exists `%s`", TABLE_NAME);
+  }
+
+  @Test
+  public void testCastTimestampVarchar() throws Exception {
+testParquetFilterPushDown("col_timestamp = '2017-01-05 05:46:11'", 1, 
1);
+testParquetFilterPushDown("col_timestamp = cast('2017-01-05 05:46:11' 
as varchar)", 1, 1);
+testParquetFilterPushDown("col_timestamp = cast('2017-01-05 05:46:11' 
as timestamp)", 1, 1);
+testParquetFilterPushDown("col_timestamp > '2017-01-02 00:00:00'", 7, 
3);
+testParquetFilterPushDown("col_timestamp between '2017-01-03 21:01:15' 
and '2017-01-06 05:46:11'", 3, 2);
+testParquetFilterPushDown("col_timestamp between '2017-01-03' and 
'2017-01-06'", 4, 2);
+  }
+
+  @Test
+  public void testCastTimestampDate() throws Exception {
+testParquetFilterPushDown("col_timestamp = date '2017-01-02'", 1, 1);
+testParquetFilterPushDown("col_timestamp = cast(date '2017-01-02' as 
timestamp)", 1, 1);
+testParquetFilterPushDown("col_timestamp > date '2017-01-02'", 7, 3);
+testParquetFilterPushDown("col_timestamp between date '2017-01-03' and 
date '2017-01-06'", 4, 

[jira] [Commented] (DRILL-6174) Parquet pushdown planning improvements

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-6174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16377057#comment-16377057
 ] 

ASF GitHub Bot commented on DRILL-6174:
---

Github user arina-ielchiieva commented on a diff in the pull request:

https://github.com/apache/drill/pull/1131#discussion_r170629893
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/ParquetFilterBuilder.java
 ---
@@ -261,6 +273,37 @@ private LogicalExpression 
handleCompareFunction(FunctionHolderExpression functio
 }
   }
 
+  private LogicalExpression handleIsFunction(FunctionHolderExpression 
functionHolderExpression, Set value) {
+String funcName;
+
+if (functionHolderExpression.getHolder() instanceof 
DrillSimpleFuncHolder) {
+  funcName = ((DrillSimpleFuncHolder) 
functionHolderExpression.getHolder()).getRegisteredNames()[0];
+} else {
+  logger.warn("Can not cast {} to DrillSimpleFuncHolder. Parquet 
filter pushdown can not handle function.",
+  functionHolderExpression.getHolder());
+  return null;
+}
+LogicalExpression arg = functionHolderExpression.args.get(0);
+
+switch (funcName) {
--- End diff --

Please use proper formatting for switch case.


> Parquet pushdown planning improvements
> --
>
> Key: DRILL-6174
> URL: https://issues.apache.org/jira/browse/DRILL-6174
> Project: Apache Drill
>  Issue Type: Improvement
>Affects Versions: 1.12.0
>Reporter: Arina Ielchiieva
>Assignee: Roman Kulyk
>Priority: Major
>  Labels: doc-impacting
> Fix For: 1.13.0
>
>
> Currently parquet pushdown planning has certain limitations 
> (https://drill.apache.org/docs/parquet-filter-pushdown/). This Jira aims to 
> fix some of them. List of improvements can be find below:
> 1. IS [NOT] NULL / TRUE / FALSE
> 2. Timestamp / date / time implicit / explicit casts
> {noformat}
> timestamp -> date
> timestamp -> varchar
> date -> timestamp
> date -> varchar
> time -> timestamp
> time -> date
> time -> varchar
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (DRILL-6174) Parquet pushdown planning improvements

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-6174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16377056#comment-16377056
 ] 

ASF GitHub Bot commented on DRILL-6174:
---

Github user arina-ielchiieva commented on a diff in the pull request:

https://github.com/apache/drill/pull/1131#discussion_r170630466
  
--- Diff: 
exec/java-exec/src/test/java/org/apache/drill/exec/store/parquet/TestParquetFilterPushDown.java
 ---
@@ -388,6 +388,45 @@ public void testTimeStampPredicate() throws Exception {
 testParquetFilterPD(query3, 49, 2, false);
   }
 
+  @Test
+  public void testBooleanPredicate() throws Exception {
+// Table blnTbl was created by CTAS in drill 1.12.0 and consist of 4 
files withe the next data:
--- End diff --

Can we generate files on the fly rather then storing them? This will help 
to prevent any issues, for example, if files generation will change in the next 
version.


> Parquet pushdown planning improvements
> --
>
> Key: DRILL-6174
> URL: https://issues.apache.org/jira/browse/DRILL-6174
> Project: Apache Drill
>  Issue Type: Improvement
>Affects Versions: 1.12.0
>Reporter: Arina Ielchiieva
>Assignee: Roman Kulyk
>Priority: Major
>  Labels: doc-impacting
> Fix For: 1.13.0
>
>
> Currently parquet pushdown planning has certain limitations 
> (https://drill.apache.org/docs/parquet-filter-pushdown/). This Jira aims to 
> fix some of them. List of improvements can be find below:
> 1. IS [NOT] NULL / TRUE / FALSE
> 2. Timestamp / date / time implicit / explicit casts
> {noformat}
> timestamp -> date
> timestamp -> varchar
> date -> timestamp
> date -> varchar
> time -> timestamp
> time -> date
> time -> varchar
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (DRILL-6174) Parquet pushdown planning improvements

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-6174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16377013#comment-16377013
 ] 

ASF GitHub Bot commented on DRILL-6174:
---

GitHub user KulykRoman opened a pull request:

https://github.com/apache/drill/pull/1131

DRILL-6174: Parquet filter pushdown improvements.

Added support IS [NOT] NULL/TRUE/FALSE operator for the parquet filter 
pushdown.
Added timestamp/date/time implicit/explicit casts.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/KulykRoman/drill DRILL-6174

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/drill/pull/1131.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1131


commit c77fe23f7742325f534f45df816d80230fe45e75
Author: Roman Kulyk 
Date:   2018-02-15T15:37:43Z

DRILL-6174: Parquet filter pushdown improvements.

Added support IS [NOT] NULL/TRUE/FALSE operator for the parquet filter 
pushdown.
Added timestamp/date/time implicit/explicit casts.




> Parquet pushdown planning improvements
> --
>
> Key: DRILL-6174
> URL: https://issues.apache.org/jira/browse/DRILL-6174
> Project: Apache Drill
>  Issue Type: Improvement
>Affects Versions: 1.12.0
>Reporter: Arina Ielchiieva
>Assignee: Roman Kulyk
>Priority: Major
>  Labels: doc-impacting
> Fix For: 1.13.0
>
>
> Currently parquet pushdown planning has certain limitations 
> (https://drill.apache.org/docs/parquet-filter-pushdown/). This Jira aims to 
> fix some of them. List of improvements can be find below:
> 1. IS [NOT] NULL / TRUE / FALSE
> 2. Timestamp / date / time implicit / explicit casts
> {noformat}
> timestamp -> date
> timestamp -> varchar
> date -> timestamp
> date -> varchar
> time -> timestamp
> time -> date
> time -> varchar
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)