RE: Switch between new parquet reader and old one

+1 on this, though in the long term, we ought to be able to handle even complex types within the Drill native reader. -Original Message- From: Jinfeng Ni [mailto:j...@apache.org] Sent: Wednesday, September 13, 2017 4:29 PM To: dev Subject: Re: Switch between new

[GitHub] drill issue #926: DRILL-5269 Make DirectSubScan Jackson JSON deserializable

Github user vrozov commented on the issue: https://github.com/apache/drill/pull/926 rebased and squashed ---

[GitHub] drill issue #921: DRILL-4286 Graceful shutdown of drillbit

Github user dvjyothsna commented on the issue: https://github.com/apache/drill/pull/921 @weijietong Please refer to comments under JIRA - https://issues.apache.org/jira/browse/DRILL-4286 ---

[GitHub] drill pull request #:

Github user dvjyothsna commented on the pull request: https://github.com/apache/drill/commit/f5b975adf26622451885149acf26ce4c2c2e0a93#commitcomment-24291455 @Vlad-Storona Hi, Vlad I think you might have added PCAP_SUB_SCAN in UserBitShared.proto. But forgot to commit

[GitHub] drill issue #926: DRILL-5269 Make DirectSubScan Jackson JSON deserializable

Github user vrozov commented on the issue: https://github.com/apache/drill/pull/926 @paul-rogers Added unit test and adopted query to an existing parquet file. ---

Re: Switch between new parquet reader and old one

That makes sense. We should only check the requested columns, not every column in parquet file, to decide which parquet reader to use. On Wed, Sep 13, 2017 at 4:17 PM, Damien Profeta wrote: > Hi, > > I was looking at the code that read the parquet file and noticed

Switch between new parquet reader and old one

Hi, I was looking at the code that read the parquet file and noticed there is a switch 'isComplex' to choose if it is possible to use the new reader or if we have to use the old one. The switch is based on the columns of the files (complex type or repetition level) but it doesn't care about

[GitHub] drill issue #919: DRILL-5721: Query with only root fragment and no non-root ...

Github user sohami commented on the issue: https://github.com/apache/drill/pull/919 I rebased this PR to latest master and [re-ran functional test](http://10.10.104.91:8080/job/Functional/2447/console) and seeing only 2 random failures now. As per @agirish recommendation I have

[jira] [Created] (DRILL-5789) AlreadyClosedSqlException: ResultSet is already closed.

Sorabh Hamirwasia created DRILL-5789: Summary: AlreadyClosedSqlException: ResultSet is already closed. Key: DRILL-5789 URL: https://issues.apache.org/jira/browse/DRILL-5789 Project: Apache Drill

[jira] [Created] (DRILL-5788) java.sql.SQLException: RESOURCE ERROR: Waited for 15000ms, but tasks for 'Fetch parquet metadata' are not complete

Sorabh Hamirwasia created DRILL-5788: Summary: java.sql.SQLException: RESOURCE ERROR: Waited for 15000ms, but tasks for 'Fetch parquet metadata' are not complete Key: DRILL-5788 URL:

[GitHub] drill issue #930: DRILL-5761: Disable Lilith ClassicMultiplexSocketAppender ...

Github user vvysotskyi commented on the issue: https://github.com/apache/drill/pull/930 @paul-rogers, @vrozov, thanks for the review. Squashed commits and rebased onto master. ---

[GitHub] drill issue #940: DRILL-5752 Speed Up Unit Tests add Test Categories

Github user ilooner commented on the issue: https://github.com/apache/drill/pull/940 Please merge my other PR https://github.com/apache/drill/pull/923 before this one. Thanks! ---

[GitHub] drill issue #923: DRILL-5723: Added System Internal Options That can be Modi...

Github user ilooner commented on the issue: https://github.com/apache/drill/pull/923 Thanks @paul-rogers squashed and ready to commit. Please merge this change before my other PR https://github.com/apache/drill/pull/940 ---

[GitHub] drill pull request #930: DRILL-5761: Disable Lilith ClassicMultiplexSocketAp...

Github user vrozov commented on a diff in the pull request: https://github.com/apache/drill/pull/930#discussion_r138722693 --- Diff: common/src/test/resources/logback-test.xml --- @@ -0,0 +1,92 @@ + + + + + + + +true +

[GitHub] drill issue #919: DRILL-5721: Query with only root fragment and no non-root ...

Github user sohami commented on the issue: https://github.com/apache/drill/pull/919 @paul-rogers - Last week @amansinha100 also reported 5 Random failures happening with this commit. But when ran in isolation for multiple iterations those tests were passing all the time. Can you

[GitHub] drill issue #936: DRILL-5772: Add unit tests to indicate how utf-8 support c...

Github user paul-rogers commented on the issue: https://github.com/apache/drill/pull/936 @arina-ielchiieva, my comments boil down to two points: 1. What happens if UTF-8 support in Drill is disabled? Do we have consistent handling of character data? Do we revert to ASCII? The

[GitHub] drill issue #940: DRILL-5752 Speed Up Unit Tests add Test Categories

Github user ilooner commented on the issue: https://github.com/apache/drill/pull/940 Thanks @paul-rogers squashed and ready to commit ---

[GitHub] drill issue #903: DRILL-5712: Update the pom files with dependency exclusion...

Github user paul-rogers commented on the issue: https://github.com/apache/drill/pull/903 @sindhurirayavaram, will you be able to rebase this & squash commits? Then I can merge it into master. Thanks! ---

[GitHub] drill issue #919: DRILL-5721: Query with only root fragment and no non-root ...

Github user paul-rogers commented on the issue: https://github.com/apache/drill/pull/919 Tried to commit this one to master. The functional test run failed with 20+ "random failures." That is too many to comfortably allow. So, @sohami, please rerun this, determine if the 20+ failures

[jira] [Created] (DRILL-5787) Add Code Coverage Reporting

Timothy Farkas created DRILL-5787: - Summary: Add Code Coverage Reporting Key: DRILL-5787 URL: https://issues.apache.org/jira/browse/DRILL-5787 Project: Apache Drill Issue Type: Improvement

[GitHub] drill issue #937: DRILL-5002: Using hive's date functions on top of date col...

Github user paul-rogers commented on the issue: https://github.com/apache/drill/pull/937 The original description talks about data with local times. The TPC-H data has no TZ. Now, maybe we made one up in creating the Parquet files, but the original date just has dates without a tz.

[GitHub] drill pull request #941: DRILL-5775: Expand * into list of column/column fam...

GitHub user jinfengni opened a pull request: https://github.com/apache/drill/pull/941 DRILL-5775: Expand * into list of column/column family for MapR-DB b… …inary table groupscan. Move common code to HBaseUtils, so it's shared by both HBaseGroupScan and

[GitHub] drill issue #916: DRILL-5377: Five-digit year dates are displayed incorrectl...

Github user paul-rogers commented on the issue: https://github.com/apache/drill/pull/916 Other than dealing with corrupted dates from Parquet, what other uses of five digit years are expected? Few business processes project 8,000 years into the future. Scientific projects are likely

[GitHub] drill pull request #923: DRILL-5723: Added System Internal Options That can ...

Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/923#discussion_r138689935 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/server/options/OptionManager.java --- @@ -17,49 +17,97 @@ */ package

[GitHub] drill pull request #923: DRILL-5723: Added System Internal Options That can ...

Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/923#discussion_r138694085 --- Diff: exec/java-exec/src/test/java/org/apache/drill/exec/server/TestOptions.java --- @@ -56,7 +56,7 @@ public void checkChangedColumn() throws

[GitHub] drill pull request #923: DRILL-5723: Added System Internal Options That can ...

Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/923#discussion_r138694479 --- Diff: exec/java-exec/src/test/java/org/apache/drill/exec/server/rest/StatusResourcesTest.java --- @@ -0,0 +1,80 @@ +/* + * Licensed to the

[GitHub] drill pull request #939: DRILL-5550: Missing CSV column value set to null

Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/939#discussion_r138685942 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/easy/text/compliant/FieldVarCharOutput.java --- @@ -45,11 +45,11 @@ static

[GitHub] drill pull request #939: DRILL-5550: Missing CSV column value set to null

Github user paul-rogers commented on a diff in the pull request: https://github.com/apache/drill/pull/939#discussion_r138687170 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/easy/text/compliant/FieldVarCharOutput.java --- @@ -211,10 +210,10 @@ public void

[GitHub] drill issue #940: DRILL-5752 Speed Up Unit Tests add Test Categories

Github user paul-rogers commented on the issue: https://github.com/apache/drill/pull/940 Thanks for the explanations and corrections. Please squash your commits to prepare for commit to master. +1 ---

[GitHub] drill pull request #940: DRILL-5752 Speed Up Unit Tests add Test Categories

Github user ilooner commented on a diff in the pull request: https://github.com/apache/drill/pull/940#discussion_r138683227 --- Diff: exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/writer/TestCorruptParquetDateCorrection.java --- @@ -59,6 +61,7 @@ * Use of

[GitHub] drill issue #930: DRILL-5761: Disable Lilith ClassicMultiplexSocketAppender ...

Github user paul-rogers commented on the issue: https://github.com/apache/drill/pull/930 Holding of on the merge waiting for review by @vrozov. ---

[GitHub] drill issue #925: DRILL-5749: solve foreman and netty threads deadlock

Github user paul-rogers commented on the issue: https://github.com/apache/drill/pull/925 Turns out that I cannot cleanly do the rebase. @weijietong, please rebase this branch onto the latest master, and squash your commits into a single commit. ---

[GitHub] drill pull request #907: DRILL-5697: Improve performance of filter operator ...

Github user asfgit closed the pull request at: https://github.com/apache/drill/pull/907 ---

[GitHub] drill pull request #930: DRILL-5761: Disable Lilith ClassicMultiplexSocketAp...

Github user vvysotskyi commented on a diff in the pull request: https://github.com/apache/drill/pull/930#discussion_r138620064 --- Diff: common/src/test/resources/logback-test.xml --- @@ -0,0 +1,92 @@ + + + + + + + +true +

[GitHub] drill pull request #930: DRILL-5761: Disable Lilith ClassicMultiplexSocketAp...

Github user vvysotskyi commented on a diff in the pull request: https://github.com/apache/drill/pull/930#discussion_r138622881 --- Diff: common/src/test/resources/logback-test.xml --- @@ -0,0 +1,111 @@ + + + + + + + +true +

[GitHub] drill issue #905: DRILL-1162: Fix OOM for hash join operator when the right ...

Github user vvysotskyi commented on the issue: https://github.com/apache/drill/pull/905 @jinfengni, the idea is very interesting for me. More appropriate memory cost estimation will cause to choosing better plans, that will cause the less time execution and memory costs.

[jira] [Created] (DRILL-5786) Query enters Exception in RPC communication

Robert Hou created DRILL-5786: - Summary: Query enters Exception in RPC communication Key: DRILL-5786 URL: https://issues.apache.org/jira/browse/DRILL-5786 Project: Apache Drill Issue Type: Bug