[GitHub] vrozov commented on a change in pull request #1360: DRILL-6578: Handle query cancellation in Parquet reader

vrozov commented on a change in pull request #1360: DRILL-6578: Handle query cancellation in Parquet reader URL: https://github.com/apache/drill/pull/1360#discussion_r199688079 ## File path:

Need help in Apache Drill

Hi there, Hope you're doing good. I need suggestion on performance issues which I'm facing in Apache Drill. Apache Drill version - 1.12.0 DB - MongoDB Details: 1. For us, it took 9.530 seconds for 1 M records within Apache Drill. 2. Hence, the response time to load

[GitHub] vrozov commented on a change in pull request #1360: DRILL-6578: Handle query cancellation in Parquet reader

vrozov commented on a change in pull request #1360: DRILL-6578: Handle query cancellation in Parquet reader URL: https://github.com/apache/drill/pull/1360#discussion_r199686189 ## File path:

[GitHub] vrozov commented on a change in pull request #1360: DRILL-6578: Handle query cancellation in Parquet reader

vrozov commented on a change in pull request #1360: DRILL-6578: Handle query cancellation in Parquet reader URL: https://github.com/apache/drill/pull/1360#discussion_r199686154 ## File path:

[GitHub] sachouche commented on issue #1361: DRILL-6579: Added sanity checks to the Parquet reader to avoid infini…

sachouche commented on issue #1361: DRILL-6579: Added sanity checks to the Parquet reader to avoid infini… URL: https://github.com/apache/drill/pull/1361#issuecomment-402014849 @Ben-Zvi Can you please review this PR? Thanks!

[GitHub] sachouche opened a new pull request #1361: DRILL-6579: Added sanity checks to the Parquet reader to avoid infini…

sachouche opened a new pull request #1361: DRILL-6579: Added sanity checks to the Parquet reader to avoid infini… URL: https://github.com/apache/drill/pull/1361 …te loops Added sanity checks to avoid infinite loops.

Re: [DISCUSS] 1.14.0 release

Do we plan to fix this one too, because this is a regression from Apache Drill 1.13.0. https://issues.apache.org/jira/browse/DRILL-6453 On Mon, Jul 2, 2018 at 9:33 PM, Kunal Khatua wrote: > DRILL-6570 seems like a must-have (release blocker, IMHO). > On 7/2/2018 8:02:00 PM, Boaz Ben-Zvi wrote:

Re: [DISCUSS] 1.14.0 release

DRILL-6570 seems like a must-have (release blocker, IMHO). On 7/2/2018 8:02:00 PM, Boaz Ben-Zvi wrote: Let's try to make progress on the 1.14 release, aiming for a Release Candidate towards the end of this week (a little ambitious, with the July 4th and people on vacations). Current Status of

[jira] [Created] (DRILL-6579) Sanity checks to avoid infinite loops

salim achouche created DRILL-6579: - Summary: Sanity checks to avoid infinite loops Key: DRILL-6579 URL: https://issues.apache.org/jira/browse/DRILL-6579 Project: Apache Drill Issue Type:

[GitHub] sachouche commented on a change in pull request #1360: DRILL-6578: Handle query cancellation in Parquet reader

sachouche commented on a change in pull request #1360: DRILL-6578: Handle query cancellation in Parquet reader URL: https://github.com/apache/drill/pull/1360#discussion_r199678649 ## File path:

[GitHub] sachouche commented on a change in pull request #1360: DRILL-6578: Handle query cancellation in Parquet reader

sachouche commented on a change in pull request #1360: DRILL-6578: Handle query cancellation in Parquet reader URL: https://github.com/apache/drill/pull/1360#discussion_r199674082 ## File path:

[GitHub] vrozov commented on a change in pull request #1360: DRILL-6578: Handle query cancellation in Parquet reader

vrozov commented on a change in pull request #1360: DRILL-6578: Handle query cancellation in Parquet reader URL: https://github.com/apache/drill/pull/1360#discussion_r199673424 ## File path:

Re: [DISCUSS] 1.14.0 release

  Let's try to make progress on the 1.14 release, aiming for a Release Candidate towards the end of this week (a little ambitious, with the July 4th and people on vacations). Current Status of the previously requested Jiras: == In Progress - DRILL-6104: Generic

[GitHub] sachouche commented on issue #1360: DRILL-6578: Handle query cancellation in Parquet reader

sachouche commented on issue #1360: DRILL-6578: Handle query cancellation in Parquet reader URL: https://github.com/apache/drill/pull/1360#issuecomment-401993011 @vrozov, can you please review this fix? Thanks! This

[GitHub] sachouche opened a new pull request #1360: DRILL-6578: Handle query cancellation in Parquet reader

sachouche opened a new pull request #1360: DRILL-6578: Handle query cancellation in Parquet reader URL: https://github.com/apache/drill/pull/1360 Goal - - The optimized Parquet reader uses an iterator style to load column data - We need to ensure the code can properly handle query

[jira] [Created] (DRILL-6578) Ensure the Flat Parquet Reader can handle query cancellation

salim achouche created DRILL-6578: - Summary: Ensure the Flat Parquet Reader can handle query cancellation Key: DRILL-6578 URL: https://issues.apache.org/jira/browse/DRILL-6578 Project: Apache Drill

[GitHub] Ben-Zvi opened a new pull request #1359: DRILL-6577: Change Hash-Join fallback default to false

Ben-Zvi opened a new pull request #1359: DRILL-6577: Change Hash-Join fallback default to false URL: https://github.com/apache/drill/pull/1359 Option's default setting changed to *false*. This is an automated message

[jira] [Created] (DRILL-6577) Change Hash-Join default to not fallback (into pre-1.14 unlimited memory)

Boaz Ben-Zvi created DRILL-6577: --- Summary: Change Hash-Join default to not fallback (into pre-1.14 unlimited memory) Key: DRILL-6577 URL: https://issues.apache.org/jira/browse/DRILL-6577 Project:

[GitHub] vdiravka commented on issue #1345: DRILL-6494: Drill Plugins Handler

vdiravka commented on issue #1345: DRILL-6494: Drill Plugins Handler URL: https://github.com/apache/drill/pull/1345#issuecomment-401977745 @sohami @arina-ielchiieva The commit with new BOOT option for controlling the file after it's use is added. It works as expected. The branch is

[GitHub] vdiravka commented on a change in pull request #1345: DRILL-6494: Drill Plugins Handler

vdiravka commented on a change in pull request #1345: DRILL-6494: Drill Plugins Handler URL: https://github.com/apache/drill/pull/1345#discussion_r199358015 ## File path: contrib/storage-kafka/src/main/resources/bootstrap-storage-plugins.json ## @@ -2,8 +2,8 @@

[GitHub] vrozov edited a comment on issue #1333: DRILL-6410: Memory leak in Parquet Reader during cancellation

vrozov edited a comment on issue #1333: DRILL-6410: Memory leak in Parquet Reader during cancellation URL: https://github.com/apache/drill/pull/1333#issuecomment-401862300 @ilooner I guess by "last discussion" you refer to the discussion between you, me and @sachouche, where "majority"

[GitHub] parthchandra commented on issue #1356: DRILL-6561: Lateral excluding the columns from output container provided by projection push into rules

parthchandra commented on issue #1356: DRILL-6561: Lateral excluding the columns from output container provided by projection push into rules URL: https://github.com/apache/drill/pull/1356#issuecomment-401961420 +1 . I also took care of the rebase and merge

[GitHub] parthchandra closed pull request #1343: DRILL-6530: JVM crash with a query involving multiple json files with…

parthchandra closed pull request #1343: DRILL-6530: JVM crash with a query involving multiple json files with… URL: https://github.com/apache/drill/pull/1343 This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for the sake of

[GitHub] parthchandra closed pull request #1356: DRILL-6561: Lateral excluding the columns from output container provided by projection push into rules

parthchandra closed pull request #1356: DRILL-6561: Lateral excluding the columns from output container provided by projection push into rules URL: https://github.com/apache/drill/pull/1356 This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is

[GitHub] parthchandra closed pull request #1339: DRILL-6535: ClassCastException in Lateral Unnest queries when dealing…

parthchandra closed pull request #1339: DRILL-6535: ClassCastException in Lateral Unnest queries when dealing… URL: https://github.com/apache/drill/pull/1339 This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for the sake of

[GitHub] priteshm commented on issue #1348: DRILL-6346: Create an Official Drill Docker Container

priteshm commented on issue #1348: DRILL-6346: Create an Official Drill Docker Container URL: https://github.com/apache/drill/pull/1348#issuecomment-401960422 Since @arina-ielchiieva is the batch committer, I marked the JIRA as ready-to-commit, so she can review it as well.

[jira] [Created] (DRILL-6576) Unnest reports incoming record counts incorrectly

Parth Chandra created DRILL-6576: Summary: Unnest reports incoming record counts incorrectly Key: DRILL-6576 URL: https://issues.apache.org/jira/browse/DRILL-6576 Project: Apache Drill Issue

[GitHub] parthchandra opened a new pull request #1358: DRILL-6516: EMIT support in streaming agg

parthchandra opened a new pull request #1358: DRILL-6516: EMIT support in streaming agg URL: https://github.com/apache/drill/pull/1358 Support for EMIT in the streaming aggregator. Also includes a fix from @sohami in the external sort memory management (since streaming agg requires

[GitHub] priteshm commented on issue #1355: DRILL-6560: Enhanced the batch statistics logging enablement

priteshm commented on issue #1355: DRILL-6560: Enhanced the batch statistics logging enablement URL: https://github.com/apache/drill/pull/1355#issuecomment-401940066 @bitblender did you get a chance to review this? This is

[GitHub] ilooner commented on issue #1336: DRILL-6496: Added missing logging statement in VectorUtil.showVectorAccessibleContent(VectorAccessible va, int[] columnWidths)

ilooner commented on issue #1336: DRILL-6496: Added missing logging statement in VectorUtil.showVectorAccessibleContent(VectorAccessible va, int[] columnWidths) URL: https://github.com/apache/drill/pull/1336#issuecomment-401925379 @arina-ielchiieva We can already do that by skipping

[GitHub] jacques-n closed pull request #272: Add additional configuration information for specific databases

jacques-n closed pull request #272: Add additional configuration information for specific databases URL: https://github.com/apache/drill/pull/272 This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for the sake of provenance:

[jira] [Created] (DRILL-6575) Add store.hive.conf.properties option to allow set Hive properties at session level

Arina Ielchiieva created DRILL-6575: --- Summary: Add store.hive.conf.properties option to allow set Hive properties at session level Key: DRILL-6575 URL: https://issues.apache.org/jira/browse/DRILL-6575

[GitHub] vrozov commented on issue #1333: DRILL-6410: Memory leak in Parquet Reader during cancellation

vrozov commented on issue #1333: DRILL-6410: Memory leak in Parquet Reader during cancellation URL: https://github.com/apache/drill/pull/1333#issuecomment-401862300 @ilooner I guess by "last discussion" you refer to the discussion between you, me and @sachouche, where "majority" does not

[GitHub] vvysotskyi edited a comment on issue #1346: DRILL-6546: Allow unnest function with nested columns and complex expressions

vvysotskyi edited a comment on issue #1346: DRILL-6546: Allow unnest function with nested columns and complex expressions URL: https://github.com/apache/drill/pull/1346#issuecomment-401854121 After the rebase on current master, unit tests failed. Fixing now.

[GitHub] vvysotskyi commented on issue #1346: DRILL-6546: Allow unnest function with nested columns and complex expressions

vvysotskyi commented on issue #1346: DRILL-6546: Allow unnest function with nested columns and complex expressions URL: https://github.com/apache/drill/pull/1346#issuecomment-401854121 After the rebase on current master, unit tests are failed. Fixing now.

[GitHub] arina-ielchiieva commented on issue #1331: DRILL-6519: Add String Distance and Phonetic Functions

arina-ielchiieva commented on issue #1331: DRILL-6519: Add String Distance and Phonetic Functions URL: https://github.com/apache/drill/pull/1331#issuecomment-401837372 +1, LGTM. This is an automated message from the Apache

[GitHub] cgivre commented on issue #1331: DRILL-6519: Add String Distance and Phonetic Functions

cgivre commented on issue #1331: DRILL-6519: Add String Distance and Phonetic Functions URL: https://github.com/apache/drill/pull/1331#issuecomment-401831328 @arina-ielchiieva Should be ready to go... This is an automated

[GitHub] jbimbert commented on a change in pull request #1298: DRILL-5796: Filter pruning for multi rowgroup parquet file

jbimbert commented on a change in pull request #1298: DRILL-5796: Filter pruning for multi rowgroup parquet file URL: https://github.com/apache/drill/pull/1298#discussion_r199489637 ## File path: exec/java-exec/src/main/java/org/apache/drill/exec/expr/stat/ParquetIsPredicate.java

[GitHub] jbimbert commented on a change in pull request #1298: DRILL-5796: Filter pruning for multi rowgroup parquet file

jbimbert commented on a change in pull request #1298: DRILL-5796: Filter pruning for multi rowgroup parquet file URL: https://github.com/apache/drill/pull/1298#discussion_r199486245 ## File path: exec/java-exec/src/main/java/org/apache/drill/exec/expr/stat/ParquetIsPredicate.java

[GitHub] jbimbert commented on a change in pull request #1298: DRILL-5796: Filter pruning for multi rowgroup parquet file

jbimbert commented on a change in pull request #1298: DRILL-5796: Filter pruning for multi rowgroup parquet file URL: https://github.com/apache/drill/pull/1298#discussion_r199483001 ## File path:

[GitHub] jbimbert commented on a change in pull request #1298: DRILL-5796: Filter pruning for multi rowgroup parquet file

jbimbert commented on a change in pull request #1298: DRILL-5796: Filter pruning for multi rowgroup parquet file URL: https://github.com/apache/drill/pull/1298#discussion_r199482330 ## File path:

[GitHub] jbimbert commented on a change in pull request #1298: DRILL-5796: Filter pruning for multi rowgroup parquet file

jbimbert commented on a change in pull request #1298: DRILL-5796: Filter pruning for multi rowgroup parquet file URL: https://github.com/apache/drill/pull/1298#discussion_r199480607 ## File path: exec/java-exec/src/main/java/org/apache/drill/exec/expr/stat/ParquetIsPredicate.java

[GitHub] jbimbert commented on a change in pull request #1298: DRILL-5796: Filter pruning for multi rowgroup parquet file

jbimbert commented on a change in pull request #1298: DRILL-5796: Filter pruning for multi rowgroup parquet file URL: https://github.com/apache/drill/pull/1298#discussion_r199479547 ## File path: exec/java-exec/src/main/java/org/apache/drill/exec/expr/stat/ParquetIsPredicate.java

[GitHub] jbimbert commented on a change in pull request #1298: DRILL-5796: Filter pruning for multi rowgroup parquet file

jbimbert commented on a change in pull request #1298: DRILL-5796: Filter pruning for multi rowgroup parquet file URL: https://github.com/apache/drill/pull/1298#discussion_r199478733 ## File path:

[GitHub] vdiravka commented on a change in pull request #1337: Upgrade ZooKeeper patch version to 3.4.12 and add Apache Curator to dependencyManagement

vdiravka commented on a change in pull request #1337: Upgrade ZooKeeper patch version to 3.4.12 and add Apache Curator to dependencyManagement URL: https://github.com/apache/drill/pull/1337#discussion_r199453686 ## File path: exec/jdbc-all/pom.xml ## @@ -636,6 +636,7 @@

[GitHub] vdiravka commented on a change in pull request #1337: Upgrade ZooKeeper patch version to 3.4.12 and add Apache Curator to dependencyManagement

vdiravka commented on a change in pull request #1337: Upgrade ZooKeeper patch version to 3.4.12 and add Apache Curator to dependencyManagement URL: https://github.com/apache/drill/pull/1337#discussion_r199467902 ## File path: exec/java-exec/pom.xml ## @@ -309,7 +309,7 @@

[GitHub] vdiravka commented on a change in pull request #1337: Upgrade ZooKeeper patch version to 3.4.12 and add Apache Curator to dependencyManagement

vdiravka commented on a change in pull request #1337: Upgrade ZooKeeper patch version to 3.4.12 and add Apache Curator to dependencyManagement URL: https://github.com/apache/drill/pull/1337#discussion_r199470655 ## File path: pom.xml ## @@ -1552,6 +1553,11 @@

[GitHub] vdiravka commented on a change in pull request #1337: Upgrade ZooKeeper patch version to 3.4.12 and add Apache Curator to dependencyManagement

vdiravka commented on a change in pull request #1337: Upgrade ZooKeeper patch version to 3.4.12 and add Apache Curator to dependencyManagement URL: https://github.com/apache/drill/pull/1337#discussion_r199454793 ## File path: pom.xml ## @@ -60,6 +61,7 @@ 4.11

[GitHub] vdiravka commented on a change in pull request #1337: Upgrade ZooKeeper patch version to 3.4.12 and add Apache Curator to dependencyManagement

vdiravka commented on a change in pull request #1337: Upgrade ZooKeeper patch version to 3.4.12 and add Apache Curator to dependencyManagement URL: https://github.com/apache/drill/pull/1337#discussion_r199451218 ## File path: pom.xml ## @@ -1552,6 +1553,11 @@

[GitHub] vdiravka commented on a change in pull request #1337: Upgrade ZooKeeper patch version to 3.4.12 and add Apache Curator to dependencyManagement

vdiravka commented on a change in pull request #1337: Upgrade ZooKeeper patch version to 3.4.12 and add Apache Curator to dependencyManagement URL: https://github.com/apache/drill/pull/1337#discussion_r199457978 ## File path: pom.xml ## @@ -257,6 +259,16 @@

[GitHub] vdiravka commented on a change in pull request #1337: Upgrade ZooKeeper patch version to 3.4.12 and add Apache Curator to dependencyManagement

vdiravka commented on a change in pull request #1337: Upgrade ZooKeeper patch version to 3.4.12 and add Apache Curator to dependencyManagement URL: https://github.com/apache/drill/pull/1337#discussion_r199452917 ## File path: pom.xml ## @@ -2356,6 +2417,7 @@

[jira] [Created] (DRILL-6574) Add LIMIT(0) on top of SCAN for a prepare statement

Bohdan Kazydub created DRILL-6574: - Summary: Add LIMIT(0) on top of SCAN for a prepare statement Key: DRILL-6574 URL: https://issues.apache.org/jira/browse/DRILL-6574 Project: Apache Drill

[GitHub] KazydubB edited a comment on issue #1337: Upgrade ZooKeeper patch version to 3.4.12 and add Apache Curator to dependencyManagement

KazydubB edited a comment on issue #1337: Upgrade ZooKeeper patch version to 3.4.12 and add Apache Curator to dependencyManagement URL: https://github.com/apache/drill/pull/1337#issuecomment-401707337 @vdiravka, I've addressed code review comments, please review.

[GitHub] KazydubB commented on issue #1337: Upgrade ZooKeeper patch version to 3.4.12 and add Apache Curator to dependencyManagement

KazydubB commented on issue #1337: Upgrade ZooKeeper patch version to 3.4.12 and add Apache Curator to dependencyManagement URL: https://github.com/apache/drill/pull/1337#issuecomment-401707337 Done. This is an automated

[GitHub] asfgit closed pull request #1324: DRILL-6310: limit batch size for hash aggregate

asfgit closed pull request #1324: DRILL-6310: limit batch size for hash aggregate URL: https://github.com/apache/drill/pull/1324 This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for the sake of provenance: As this is a

[GitHub] kkhatua commented on issue #1354: DRILL-6570: Fixed IndexOutofBoundException in Parquet Reader

kkhatua commented on issue #1354: DRILL-6570: Fixed IndexOutofBoundException in Parquet Reader URL: https://github.com/apache/drill/pull/1354#issuecomment-401693445 The IDE (or Maven, itself) might report this as unused. It might be worth mentioning as a that this is a placeholder for the

[GitHub] arina-ielchiieva commented on issue #1357: DRILL-6557: Use size in bytes during Hive statistics calculation if present

arina-ielchiieva commented on issue #1357: DRILL-6557: Use size in bytes during Hive statistics calculation if present URL: https://github.com/apache/drill/pull/1357#issuecomment-401691697 @vvysotskyi please review. This is

[GitHub] arina-ielchiieva opened a new pull request #1357: DRILL-6557: Use size in bytes during Hive statistics calculation if present

arina-ielchiieva opened a new pull request #1357: DRILL-6557: Use size in bytes during Hive statistics calculation if present URL: https://github.com/apache/drill/pull/1357 1. Check size in bytes presence in stats before fetching input splits and use it if present. 2. Add log trace

[jira] [Created] (DRILL-6573) Enhance JPPD with NDV

weijie.tong created DRILL-6573: -- Summary: Enhance JPPD with NDV Key: DRILL-6573 URL: https://issues.apache.org/jira/browse/DRILL-6573 Project: Apache Drill Issue Type: Improvement Affects

[jira] [Created] (DRILL-6572) Add memory calculattion of JPPD BloomFilter

weijie.tong created DRILL-6572: -- Summary: Add memory calculattion of JPPD BloomFilter Key: DRILL-6572 URL: https://issues.apache.org/jira/browse/DRILL-6572 Project: Apache Drill Issue Type:

[GitHub] weijietong commented on issue #1334: DRILL-6385: Support JPPD feature

weijietong commented on issue #1334: DRILL-6385: Support JPPD feature URL: https://github.com/apache/drill/pull/1334#issuecomment-401686350 @amansinha100 The scan node's memory copy logic has removed. Thanks for the knowledge of

[GitHub] weijietong commented on a change in pull request #1334: DRILL-6385: Support JPPD feature

weijietong commented on a change in pull request #1334: DRILL-6385: Support JPPD feature URL: https://github.com/apache/drill/pull/1334#discussion_r199395144 ## File path: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/join/HashJoinBatch.java ## @@

[GitHub] asfgit closed pull request #1342: DRILL-6537:Limit the batch size for buffering operators based on how …

asfgit closed pull request #1342: DRILL-6537:Limit the batch size for buffering operators based on how … URL: https://github.com/apache/drill/pull/1342 This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for the sake of

[GitHub] asfgit closed pull request #309: DRILL-4020: The not-equal operator returns incorrect results when used on the HBase row key

asfgit closed pull request #309: DRILL-4020: The not-equal operator returns incorrect results when used on the HBase row key URL: https://github.com/apache/drill/pull/309 This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for

[GitHub] asfgit closed pull request #1352: DRILL-6548: IllegalStateException: Unexpected EMIT outcome received i…

asfgit closed pull request #1352: DRILL-6548: IllegalStateException: Unexpected EMIT outcome received i… URL: https://github.com/apache/drill/pull/1352 This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for the sake of

[GitHub] asfgit closed pull request #1349: DRILL-6554: Minor code improvements in parquet statistics handling

asfgit closed pull request #1349: DRILL-6554: Minor code improvements in parquet statistics handling URL: https://github.com/apache/drill/pull/1349 This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for the sake of provenance: