[GitHub] drill issue #518: DRILL-4653.json - Malformed JSON should not stop the entir...

[ssriniva123]

Github user ssriniva123 commented on the issue:

https://github.com/apache/drill/pull/518
  
I updated the pull request with more changes requested by the reviewers.

On Wed, Jun 15, 2016 at 5:28 PM, Aman Sinha 
wrote:

> Yes, it does in fact have a conflict with DRILL-3764 which has changes to
> the JsonRecordReader, although this issue is still in progress. I noticed
> that @adeneche  mentioned this in the JIRA.
> @ssriniva123  did you get a chance to
> look at DRILL-3764 ?
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> , or mute
> the thread
> 

> .
>



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] drill pull request #519: DRILL-4530: Optimize partition pruning with metadat...

[amansinha100]

Github user amansinha100 commented on a diff in the pull request:

https://github.com/apache/drill/pull/519#discussion_r67289117
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/ParquetFormatPlugin.java
 ---
@@ -208,8 +209,18 @@ public DrillTable isReadable(DrillFileSystem fs, 
FileSelection selection,
 FileSystemPlugin fsPlugin, String storageEngineName, String 
userName)
 throws IOException {
   // TODO: we only check the first file for directory reading.
-  if(selection.containsDirectories(fs)){
-if(isDirReadable(fs, selection.getFirstPath(fs))){
+  if(selection.containsDirectories(fs)) {
+Path dirMetaPath = new Path(selection.getSelectionRoot(), 
Metadata.METADATA_DIRECTORIES_FILENAME);
+if (fs.exists(dirMetaPath)) {
+  ParquetTableMetadataDirs mDirs = Metadata.readMetadataDirs(fs, 
dirMetaPath.toString());
+  if (mDirs.getDirectories().size() > 0) {
+FileSelection dirSelection = 
FileSelection.createFromDirectories(mDirs.getDirectories(), selection);
+dirSelection.setExpandedPartial();
+return new DynamicDrillTable(fsPlugin, storageEngineName, 
userName,
--- End diff --

I intentionally don't call isDirReadable() here because that method returns 
true if a metadata cache file exists and I am doing a similar check for the 
directories file here with fs.exists(dirMetaPath).If this check fails,  we 
will fall through to the old code path (line 223) which does check 
isDirReadable().   


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] drill issue #518: DRILL-4653.json - Malformed JSON should not stop the entir...

[ssriniva123]

Github user ssriniva123 commented on the issue:

https://github.com/apache/drill/pull/518
  
For some reason the build terminated- What needs to be done to fire it 
again?



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: DRILL-4199: Add Support for HBase 1.X - planning to merge

[qiang li]

Also when I query hbase, its throw exception :
2016-06-16 10:29:51,788 [289df0e3-05dc-f882-8c51-20dd0f98235b:frag:0:0]
ERROR o.a.drill.exec.compile.MergeAdapter - Failure while merging classes.
java.lang.RuntimeException: Error at instruction 370: Expected an object
reference, but found . doEval(II)V
at
org.objectweb.asm.util.CheckMethodAdapter$1.visitEnd(CheckMethodAdapter.java:463)
~[asm-debug-all-5.0.3.jar:5.0.3]

But the query is success. Does this exception will affect the query
process,  for example will it affect the query speed or data integrity?

2016-06-16 10:27 GMT+08:00 qiang li :

> Hi Aditya,
>
> I tested the latest version and got this exception and the drillbit fail
> to startup .
>
> Exception in thread "main" java.lang.NoSuchMethodError:
> io.netty.util.UniqueName.(Ljava/lang/String;)V
> at io.netty.channel.ChannelOption.(ChannelOption.java:136)
> at io.netty.channel.ChannelOption.valueOf(ChannelOption.java:99)
> at io.netty.channel.ChannelOption.(ChannelOption.java:42)
> at
> org.apache.drill.exec.rpc.BasicServer.(BasicServer.java:63)
> at
> org.apache.drill.exec.rpc.user.UserServer.(UserServer.java:74)
> at
> org.apache.drill.exec.service.ServiceEngine.(ServiceEngine.java:78)
> at org.apache.drill.exec.server.Drillbit.(Drillbit.java:108)
> at org.apache.drill.exec.server.Drillbit.start(Drillbit.java:285)
> at org.apache.drill.exec.server.Drillbit.start(Drillbit.java:271)
> at org.apache.drill.exec.server.Drillbit.main(Drillbit.java:267)
>
> It will working if I remove jars/3rdparty/netty-all-4.0.23.Final.jar, the
> drill can startup. I think there have some package dependency version
> issue, do you think so ?
>
>
>
> 2016-06-15 8:14 GMT+08:00 Aditya :
>
>> HBase 1.x support has been merged and is available in latest
>> 1.7.0-SNAPSHOT
>> builds.
>>
>> On Wed, Jun 1, 2016 at 1:23 PM, Aditya  wrote:
>>
>> > Thanks Jacques for promptly reviewing my long series of patches!
>> >
>> > I'm planning to merge the HBase 1.x support some time in next 48 hours.
>> >
>> > If anyone else is interested and willing, please review the latest patch
>> > here[1].
>> >
>> > aditya...
>> >
>> > [1] https://github.com/apache/drill/pull/443/files
>> >
>>
>
>

[GitHub] drill pull request #519: DRILL-4530: Optimize partition pruning with metadat...

[parthchandra]

Github user parthchandra commented on a diff in the pull request:

https://github.com/apache/drill/pull/519#discussion_r67272839
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/ParquetFormatPlugin.java
 ---
@@ -208,8 +209,18 @@ public DrillTable isReadable(DrillFileSystem fs, 
FileSelection selection,
 FileSystemPlugin fsPlugin, String storageEngineName, String 
userName)
 throws IOException {
   // TODO: we only check the first file for directory reading.
-  if(selection.containsDirectories(fs)){
-if(isDirReadable(fs, selection.getFirstPath(fs))){
+  if(selection.containsDirectories(fs)) {
+Path dirMetaPath = new Path(selection.getSelectionRoot(), 
Metadata.METADATA_DIRECTORIES_FILENAME);
+if (fs.exists(dirMetaPath)) {
+  ParquetTableMetadataDirs mDirs = Metadata.readMetadataDirs(fs, 
dirMetaPath.toString());
+  if (mDirs.getDirectories().size() > 0) {
+FileSelection dirSelection = 
FileSelection.createFromDirectories(mDirs.getDirectories(), selection);
+dirSelection.setExpandedPartial();
+return new DynamicDrillTable(fsPlugin, storageEngineName, 
userName,
--- End diff --

Are you missing the isDirReadable() check here?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] drill issue #518: DRILL-4653.json - Malformed JSON should not stop the entir...

[ssriniva123]

Github user ssriniva123 commented on the issue:

https://github.com/apache/drill/pull/518
  
I have added support so that it prints the offending file/line nos.

Error parsing JSON in DRILL-4653.json : line nos :13
Error parsing JSON in DRILL-4653.json : line nos :14


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] drill issue #512: Drill 4573 fix issue with unicode chars

[jinfengni]

Github user jinfengni commented on the issue:

https://github.com/apache/drill/pull/512
  
I did the following small changes to this PR, and I'm going to rebase and 
re-run the regression suite before merge this PR.

1) fix one bug which causes regression failure (IOBE).
2) add unit test with regexp function over non-ascii input
3) code clean up.

+1



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] drill issue #518: DRILL-4653.json - Malformed JSON should not stop the entir...

[ssriniva123]

Github user ssriniva123 commented on the issue:

https://github.com/apache/drill/pull/518
  
Yes, I looked at DRILL-3764. The changes made to JSONProcessing do not 
conflict with my changes.
The only conflict I see is these two constants, I have defined my own.

 String ENABLE_SKIP_INVALID_RECORD_KEY = "exec.enable_skip_invalid_record";
 +  BooleanValidator ENABLE_SKIP_INVALID_RECORD = new 
BooleanValidator(ENABLE_SKIP_INVALID_RECORD_KEY, false);




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] drill issue #518: DRILL-4653.json - Malformed JSON should not stop the entir...

[amansinha100]

Github user amansinha100 commented on the issue:

https://github.com/apache/drill/pull/518
  
Yes, it does in fact have a conflict with DRILL-3764 which has changes to 
the JsonRecordReader, although this issue is still in progress.   I noticed 
that @adeneche  mentioned this in the JIRA. @ssriniva123 did you get a chance 
to look at DRILL-3764 ?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Manta Object Store Support

[Elijah Zupancic]

Please disregard this. I’ve figured out how to get it working. Once I realized 
that error logging for Hadoop Filesystem drivers wasn’t enabled by logback by 
default, then everything clicked into place. I’ll be back to ask more questions.

Thank you for your time,
Elijah Zupancic

> On May 6, 2016, at 9:43 AM, Elijah Zupancic  wrote:
> 
> I'm trying to get started contributing to Apache Drill. I've got the project 
> checked out and it is building to my satisfaction. Right now, I'm trying to 
> add support for the open source object store Manta 
> (https://github.com/joyent/manta ). I 
> thought that this would be a good learning project.
> 
> Initially, I want to add support in the same way that S3 has support. 
> However, I can't seem to find a reference to the S3 storage driver in the 
> code base. Is the s3 storage driver part of a different project? How would 
> you suggest that I get started?
> 
> Thank you,
> Elijah Zupancic

[GitHub] drill pull request #518: DRILL-4653.json - Malformed JSON should not stop th...

[amansinha100]

Github user amansinha100 commented on a diff in the pull request:

https://github.com/apache/drill/pull/518#discussion_r67267511
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/store/easy/json/JSONRecordReader.java
 ---
@@ -217,11 +228,11 @@ public int next() {
   updateRunningCount();
   return recordCount;
 
-} catch (final Exception e) {
-  handleAndRaise("Error parsing JSON", e);
-}
+   // } catch (final Exception e) {
+   //   handleAndRaise("Error parsing JSON", e);
+   // }
 // this is never reached
-return 0;
+//return 0;
--- End diff --

Uncomment this (best practice since function has a return type)


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] drill issue #518: DRILL-4653.json - Malformed JSON should not stop the entir...

[parthchandra]

Github user parthchandra commented on the issue:

https://github.com/apache/drill/pull/518
  
Does this conflict with Drill 3764 (see 
https://github.com/apache/drill/pull/497)?. I think there was a broader 
discussion around this as part of enabling the functionality of 
skipping/logging erroneous records. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] drill pull request #518: DRILL-4653.json - Malformed JSON should not stop th...

[amansinha100]

Github user amansinha100 commented on a diff in the pull request:

https://github.com/apache/drill/pull/518#discussion_r67267273
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/store/easy/json/JSONRecordReader.java
 ---
@@ -64,6 +64,8 @@
   private final boolean enableAllTextMode;
   private final boolean readNumbersAsDouble;
   private final boolean unionEnabled;
+  private int parseErrorCount;
--- End diff --

This should be long instead of int since parseErrorCount is cumulative..so 
in the worst case it could be as large as runningRecordCount.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] drill issue #518: DRILL-4653.json - Malformed JSON should not stop the entir...

[ssriniva123]

Github user ssriniva123 commented on the issue:

https://github.com/apache/drill/pull/518
  
Do u anticipate this to be logged using log4j?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] drill issue #518: DRILL-4653.json - Malformed JSON should not stop the entir...

[amansinha100]

Github user amansinha100 commented on the issue:

https://github.com/apache/drill/pull/518
  
A high level comment I have about this PR is that keeping the number of 
'malformed' records in the json file may not be sufficient..I would expect the 
user would want to know the file name and the location within the file for such 
record.  


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] drill issue #522: DRILL-4716: status.json doesn't work in drill ui

[parthchandra]

Github user parthchandra commented on the issue:

https://github.com/apache/drill/pull/522
  
+1 Looks good.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Time for a 1.7 release

[Aman Sinha]

I can take a look at DRILL-4653.

Could someone familiar with the Decimal type take a look at DRILL-4704 ?
Agree with Dave that it is a simple case that should be fixed (note,
however, that decimal is disabled by default currently).


On Wed, Jun 15, 2016 at 3:12 PM, Subbu Srinivasan 
wrote:

> Who can  review https://issues.apache.org/jira/browse/DRILL-4653 ?
>
> On Wed, Jun 15, 2016 at 1:37 PM, Parth Chandra 
> wrote:
>
> > +1 on the 1.7 release
> >
> > I'm reviewing the following and hope to get them in the release before
> > cutoff:
> > https://issues.apache.org/jira/browse/DRILL-2593
> > https://issues.apache.org/jira/browse/DRILL-4309
> >
> >
> >
> > On Wed, Jun 15, 2016 at 1:20 PM, Jinfeng Ni 
> wrote:
> >
> > > I'm reviewing a follow-up PR [1] for DRILL-4573. I think we need get
> > > it merged in, since it's a regression in terms of query correctness
> > > from release 1.6.
> > >
> > > [1] https://github.com/apache/drill/pull/512
> > >
> > > On Wed, Jun 15, 2016 at 12:21 PM, Dave Oshinsky <
> doshin...@commvault.com
> > >
> > > wrote:
> > > > This is a pretty basic bug affecting decimal values, with a simple
> fix:
> > > > https://issues.apache.org/jira/browse/DRILL-4704
> > > >
> > > > It would be great if it could be reviewed.
> > > >
> > > > -Original Message-
> > > > From: Aman Sinha [mailto:amansi...@apache.org]
> > > > Sent: Wednesday, June 15, 2016 3:15 PM
> > > > To: dev
> > > > Subject: Time for a 1.7 release
> > > >
> > > > Hello everyone,
> > > >
> > > >   A 1.7 release is quite overdue !  I would like to get the ball
> > > rolling...
> > > >
> > > >   In preparation, could developers please close the pull requests
> that
> > > have already been merged into master ?  This will give me a better
> sense
> > of
> > > what is in progress.
> > > >
> > > >   Can all the folks working on open issues let me know if there are
> any
> > > JIRAs you would like to get into the release?
> > > >
> > > >Tentatively, I am thinking beginning of next week to get a release
> > > candidate.
> > > >
> > > > thanks,
> > > > -Aman
> > > >
> > > >
> > > >
> > > > ***Legal
> Disclaimer***
> > > > "This communication may contain confidential and privileged material
> > for
> > > the
> > > > sole use of the intended recipient. Any unauthorized review, use or
> > > distribution
> > > > by others is strictly prohibited. If you have received the message by
> > > mistake,
> > > > please advise the sender by reply email and delete the message. Thank
> > > you."
> > > >
> **
> > >
> >
>

Re: Time for a 1.7 release

[Subbu Srinivasan]

Who can  review https://issues.apache.org/jira/browse/DRILL-4653 ?

On Wed, Jun 15, 2016 at 1:37 PM, Parth Chandra 
wrote:

> +1 on the 1.7 release
>
> I'm reviewing the following and hope to get them in the release before
> cutoff:
> https://issues.apache.org/jira/browse/DRILL-2593
> https://issues.apache.org/jira/browse/DRILL-4309
>
>
>
> On Wed, Jun 15, 2016 at 1:20 PM, Jinfeng Ni  wrote:
>
> > I'm reviewing a follow-up PR [1] for DRILL-4573. I think we need get
> > it merged in, since it's a regression in terms of query correctness
> > from release 1.6.
> >
> > [1] https://github.com/apache/drill/pull/512
> >
> > On Wed, Jun 15, 2016 at 12:21 PM, Dave Oshinsky  >
> > wrote:
> > > This is a pretty basic bug affecting decimal values, with a simple fix:
> > > https://issues.apache.org/jira/browse/DRILL-4704
> > >
> > > It would be great if it could be reviewed.
> > >
> > > -Original Message-
> > > From: Aman Sinha [mailto:amansi...@apache.org]
> > > Sent: Wednesday, June 15, 2016 3:15 PM
> > > To: dev
> > > Subject: Time for a 1.7 release
> > >
> > > Hello everyone,
> > >
> > >   A 1.7 release is quite overdue !  I would like to get the ball
> > rolling...
> > >
> > >   In preparation, could developers please close the pull requests that
> > have already been merged into master ?  This will give me a better sense
> of
> > what is in progress.
> > >
> > >   Can all the folks working on open issues let me know if there are any
> > JIRAs you would like to get into the release?
> > >
> > >Tentatively, I am thinking beginning of next week to get a release
> > candidate.
> > >
> > > thanks,
> > > -Aman
> > >
> > >
> > >
> > > ***Legal Disclaimer***
> > > "This communication may contain confidential and privileged material
> for
> > the
> > > sole use of the intended recipient. Any unauthorized review, use or
> > distribution
> > > by others is strictly prohibited. If you have received the message by
> > mistake,
> > > please advise the sender by reply email and delete the message. Thank
> > you."
> > > **
> >
>

[GitHub] drill issue #501: DRILL-2385: Count on complex objects failed with missing f...

[parthchandra]

Github user parthchandra commented on the issue:

https://github.com/apache/drill/pull/501
  
+1 LGTM


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Time for a 1.7 release

[Parth Chandra]

+1 on the 1.7 release

I'm reviewing the following and hope to get them in the release before
cutoff:
https://issues.apache.org/jira/browse/DRILL-2593
https://issues.apache.org/jira/browse/DRILL-4309



On Wed, Jun 15, 2016 at 1:20 PM, Jinfeng Ni  wrote:

> I'm reviewing a follow-up PR [1] for DRILL-4573. I think we need get
> it merged in, since it's a regression in terms of query correctness
> from release 1.6.
>
> [1] https://github.com/apache/drill/pull/512
>
> On Wed, Jun 15, 2016 at 12:21 PM, Dave Oshinsky 
> wrote:
> > This is a pretty basic bug affecting decimal values, with a simple fix:
> > https://issues.apache.org/jira/browse/DRILL-4704
> >
> > It would be great if it could be reviewed.
> >
> > -Original Message-
> > From: Aman Sinha [mailto:amansi...@apache.org]
> > Sent: Wednesday, June 15, 2016 3:15 PM
> > To: dev
> > Subject: Time for a 1.7 release
> >
> > Hello everyone,
> >
> >   A 1.7 release is quite overdue !  I would like to get the ball
> rolling...
> >
> >   In preparation, could developers please close the pull requests that
> have already been merged into master ?  This will give me a better sense of
> what is in progress.
> >
> >   Can all the folks working on open issues let me know if there are any
> JIRAs you would like to get into the release?
> >
> >Tentatively, I am thinking beginning of next week to get a release
> candidate.
> >
> > thanks,
> > -Aman
> >
> >
> >
> > ***Legal Disclaimer***
> > "This communication may contain confidential and privileged material for
> the
> > sole use of the intended recipient. Any unauthorized review, use or
> distribution
> > by others is strictly prohibited. If you have received the message by
> mistake,
> > please advise the sender by reply email and delete the message. Thank
> you."
> > **
>

Re: Time for a 1.7 release

[Jinfeng Ni]

I'm reviewing a follow-up PR [1] for DRILL-4573. I think we need get
it merged in, since it's a regression in terms of query correctness
from release 1.6.

[1] https://github.com/apache/drill/pull/512

On Wed, Jun 15, 2016 at 12:21 PM, Dave Oshinsky  wrote:
> This is a pretty basic bug affecting decimal values, with a simple fix:
> https://issues.apache.org/jira/browse/DRILL-4704
>
> It would be great if it could be reviewed.
>
> -Original Message-
> From: Aman Sinha [mailto:amansi...@apache.org]
> Sent: Wednesday, June 15, 2016 3:15 PM
> To: dev
> Subject: Time for a 1.7 release
>
> Hello everyone,
>
>   A 1.7 release is quite overdue !  I would like to get the ball rolling...
>
>   In preparation, could developers please close the pull requests that have 
> already been merged into master ?  This will give me a better sense of what 
> is in progress.
>
>   Can all the folks working on open issues let me know if there are any JIRAs 
> you would like to get into the release?
>
>Tentatively, I am thinking beginning of next week to get a release 
> candidate.
>
> thanks,
> -Aman
>
>
>
> ***Legal Disclaimer***
> "This communication may contain confidential and privileged material for the
> sole use of the intended recipient. Any unauthorized review, use or 
> distribution
> by others is strictly prohibited. If you have received the message by mistake,
> please advise the sender by reply email and delete the message. Thank you."
> **

RE: Time for a 1.7 release

[Dave Oshinsky]

This is a pretty basic bug affecting decimal values, with a simple fix:
https://issues.apache.org/jira/browse/DRILL-4704

It would be great if it could be reviewed.

-Original Message-
From: Aman Sinha [mailto:amansi...@apache.org] 
Sent: Wednesday, June 15, 2016 3:15 PM
To: dev
Subject: Time for a 1.7 release

Hello everyone,

  A 1.7 release is quite overdue !  I would like to get the ball rolling...

  In preparation, could developers please close the pull requests that have 
already been merged into master ?  This will give me a better sense of what is 
in progress.

  Can all the folks working on open issues let me know if there are any JIRAs 
you would like to get into the release?

   Tentatively, I am thinking beginning of next week to get a release candidate.

thanks,
-Aman



***Legal Disclaimer***
"This communication may contain confidential and privileged material for the
sole use of the intended recipient. Any unauthorized review, use or distribution
by others is strictly prohibited. If you have received the message by mistake,
please advise the sender by reply email and delete the message. Thank you."
**

Re: Time for a 1.7 release

[Subbu Srinivasan]

Can https://issues.apache.org/jira/browse/DRILL-4653 be reviewed?



On Wed, Jun 15, 2016 at 12:14 PM, Aman Sinha  wrote:

> Hello everyone,
>
>   A 1.7 release is quite overdue !  I would like to get the ball rolling...
>
>   In preparation, could developers please close the pull requests that have
> already been merged into master ?  This will give me a better sense of what
> is in progress.
>
>   Can all the folks working on open issues let me know if there are any
> JIRAs you would like to get into the release?
>
>Tentatively, I am thinking beginning of next week to get a release
> candidate.
>
> thanks,
> -Aman
>

Time for a 1.7 release

[Aman Sinha]

Hello everyone,

  A 1.7 release is quite overdue !  I would like to get the ball rolling...

  In preparation, could developers please close the pull requests that have
already been merged into master ?  This will give me a better sense of what
is in progress.

  Can all the folks working on open issues let me know if there are any
JIRAs you would like to get into the release?

   Tentatively, I am thinking beginning of next week to get a release
candidate.

thanks,
-Aman

[GitHub] drill pull request #524: DRILL-4724: Improvements to InfoSchema RecordGenera...

[vkorukanti]

GitHub user vkorukanti opened a pull request:

https://github.com/apache/drill/pull/524

DRILL-4724: Improvements to InfoSchema RecordGenerator needed for DRILL-4714

1. Add support for pushing the filter on following fields into 
InfoSchemaRecordGenerator:
   - CATALOG_NAME
   - COLUMN_NAME

2. Pushdown LIKE with ESCAPE. Add test 
TestInfoSchemaFilterPushDown#testFilterPushdown_LikeWithEscape

3. Add a method visitCatalog() to InfoSchemaRecordGenerator to decide 
whether to explore the catalog or not

4. Refactor CATALOG_DESCRIPTION and CATALOG_CONNECT as constant strings in 
InfoSchemaConstants.java

5. Update TestInfoSchemaFilterPushDown#testPartialFilterPushDownWithProject 
as
   we are now pushing the filter on COLUMN_NAME field

6. Cleanup:
   Rename RecordGenerator -> InfoSchemaRecordGenerator
   Add comments in RecordGenerator
   Rename SelectedTable -> InfoSchemaTableType

@parthchandra Can you please review this change?

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/vkorukanti/drill DRILL-4725

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/drill/pull/524.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #524


commit f331507af044d42b74b1fedbf7b9de8adc861334
Author: vkorukanti 
Date:   2016-06-09T07:00:31Z

DRILL-4724: Improvements to InfoSchema RecordGenerator needed for DRILL-4714

1. Add support for pushing the filter on following fields into 
InfoSchemaRecordGenerator:
   - CATALOG_NAME
   - COLUMN_NAME

2. Pushdown LIKE with ESCAPE. Add test 
TestInfoSchemaFilterPushDown#testFilterPushdown_LikeWithEscape

3. Add a method visitCatalog() to InfoSchemaRecordGenerator to decide 
whether to explore the catalog or not

4. Refactor CATALOG_DESCRIPTION and CATALOG_CONNECT as constant strings in 
InfoSchemaConstants.java

5. Update TestInfoSchemaFilterPushDown#testPartialFilterPushDownWithProject 
as
   we are now pushing the filter on COLUMN_NAME field

6. Cleanup:
   Rename RecordGenerator -> InfoSchemaRecordGenerator
   Add comments in RecordGenerator
   Rename SelectedTable -> InfoSchemaTableType

Change-Id: I0b2e16d04cb72fe3ce5961f5f357a00655f1cb05




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (DRILL-4725) Improvements to InfoSchema RecordGenerator needed for DRILL-4714

[Venki Korukanti (JIRA)]

Venki Korukanti created DRILL-4725:
--

 Summary: Improvements to InfoSchema RecordGenerator needed for 
DRILL-4714
 Key: DRILL-4725
 URL: https://issues.apache.org/jira/browse/DRILL-4725
 Project: Apache Drill
  Issue Type: Sub-task
  Components: Metadata
Reporter: Venki Korukanti
Assignee: Venki Korukanti


1. Add support for pushing the filter on following fields into 
InfoSchemaRecordGenerator:
   - CATALOG_NAME
   - COLUMN_NAME

2. Pushdown LIKE with ESCAPE. Add test.

3. Add a method visitCatalog() to InfoSchemaRecordGenerator to decide whether 
to explore the catalog or not





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] drill issue #500: DRILL-3149: TextReader should support multibyte line delim...

[parthchandra]

Github user parthchandra commented on the issue:

https://github.com/apache/drill/pull/500
  
+1 looks good. Can you squash and rebase ?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (DRILL-4724) convert_from(binary_string(expression),'INT_BE') results in Exception

[Khurram Faraaz (JIRA)]

Khurram Faraaz created DRILL-4724:
-

 Summary: convert_from(binary_string(expression),'INT_BE') results 
in Exception
 Key: DRILL-4724
 URL: https://issues.apache.org/jira/browse/DRILL-4724
 Project: Apache Drill
  Issue Type: Bug
  Components: Execution - Data Types
Affects Versions: 1.7.0
Reporter: Khurram Faraaz


The below query that uses binary_string function results in Exception

Drill git commit ID : 6286c0a4

{noformat}
2016-06-15 09:20:43,623 [289ee213-8ada-808f-e59d-5a6b67c53732:foreman] INFO  
o.a.drill.exec.work.foreman.Foreman - Query text for query id 
289ee213-8ada-808f-e59d-5a6b67c53732: 
values(convert_from(binary_string('\\x99\\x8c\\x2f\\x77'),'INT_BE'))
2016-06-15 09:20:43,666 [289ee213-8ada-808f-e59d-5a6b67c53732:foreman] ERROR 
o.a.drill.exec.work.foreman.Foreman - SYSTEM ERROR: IllegalArgumentException: 
Wrong length 8(8-0) in the buffer '\x5C\x99\x5C\x8C\x5C/\x5Cw', expected 4.


[Error Id: bb6968cd-44c2-4c48-bb12-865f8709167e on centos-01.qa.lab:31010]
org.apache.drill.common.exceptions.UserException: SYSTEM ERROR: 
IllegalArgumentException: Wrong length 8(8-0) in the buffer 
'\x5C\x99\x5C\x8C\x5C/\x5Cw', expected 4.


[Error Id: bb6968cd-44c2-4c48-bb12-865f8709167e on centos-01.qa.lab:31010]
at 
org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:543)
 ~[drill-common-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
at 
org.apache.drill.exec.work.foreman.Foreman$ForemanResult.close(Foreman.java:791)
 [drill-java-exec-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
at 
org.apache.drill.exec.work.foreman.Foreman.moveToState(Foreman.java:901) 
[drill-java-exec-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:271) 
[drill-java-exec-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) 
[na:1.7.0_101]
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) 
[na:1.7.0_101]
at java.lang.Thread.run(Thread.java:745) [na:1.7.0_101]
Caused by: org.apache.drill.exec.work.foreman.ForemanException: Unexpected 
exception during fragment initialization: Internal error: Error while applying 
rule ReduceExpressionsRule_Project, args 
[rel#1460:LogicalProject.NONE.ANY([]).[](input=rel#1459:Subset#0.NONE.ANY([]).[0],EXPR$0=CONVERT_FROMINT_BE(BINARY_STRING('\\x99\\x8c\\x2f\\x77')))]
... 4 common frames omitted
Caused by: java.lang.AssertionError: Internal error: Error while applying rule 
ReduceExpressionsRule_Project, args 
[rel#1460:LogicalProject.NONE.ANY([]).[](input=rel#1459:Subset#0.NONE.ANY([]).[0],EXPR$0=CONVERT_FROMINT_BE(BINARY_STRING('\\x99\\x8c\\x2f\\x77')))]
at org.apache.calcite.util.Util.newInternal(Util.java:792) 
~[calcite-core-1.4.0-drill-r11.jar:1.4.0-drill-r11]
at 
org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:251)
 ~[calcite-core-1.4.0-drill-r11.jar:1.4.0-drill-r11]
at 
org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp(VolcanoPlanner.java:808)
 ~[calcite-core-1.4.0-drill-r11.jar:1.4.0-drill-r11]
at 
org.apache.calcite.tools.Programs$RuleSetProgram.run(Programs.java:303) 
~[calcite-core-1.4.0-drill-r11.jar:1.4.0-drill-r11]
at 
org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.transform(DefaultSqlHandler.java:400)
 ~[drill-java-exec-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
at 
org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.transform(DefaultSqlHandler.java:339)
 ~[drill-java-exec-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
at 
org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToDrel(DefaultSqlHandler.java:237)
 ~[drill-java-exec-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
at 
org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToDrel(DefaultSqlHandler.java:286)
 ~[drill-java-exec-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
at 
org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.getPlan(DefaultSqlHandler.java:168)
 ~[drill-java-exec-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
at 
org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan(DrillSqlWorker.java:94)
 ~[drill-java-exec-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
at org.apache.drill.exec.work.foreman.Foreman.runSQL(Foreman.java:978) 
[drill-java-exec-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:257) 
[drill-java-exec-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
... 3 common frames omitted
Caused by: java.lang.RuntimeException: Error in evaluating function of 
convert_fromint_be
at 
org.apache.drill.exec.expr.fn.interpreter.InterpreterEvaluator$EvalVisitor.visitFunctionHolderExpression(InterpreterEvaluator.java:323)
 ~[drill-java-exec-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
at 

[GitHub] drill issue #436: DRILL-4514 : Add describe schema command

[arina-ielchiieva]

Github user arina-ielchiieva commented on the issue:

https://github.com/apache/drill/pull/436
  
@jinfengni 
PR is updated, physical_location is removed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---