Re: [Drill 1.13.0] : org.apache.thrift.TApplicationException: Invalid method name: 'get_table_req'

[Anup Tiwari]

Please find JIRA Link :-
https://issues.apache.org/jira/browse/DRILL-6293






On Sat, Mar 24, 2018 7:01 PM, Vitalii Diravka vitalii.dira...@gmail.com  wrote:
Hi Anup.




The API of Hive2.3 is changed, but not documented fully yet.

So the difference should be found and resolved in Drill.




Please go ahead and create Jira [1] with description of your environment,

settings, CTAS and query, which doesn't work.




Thanks




[1] https://issues.apache.org/jira/projects/DRILL/




Kind regards

Vitalii




On Sat, Mar 24, 2018 at 12:50 PM, Anup Tiwari <anup.tiw...@games24x7.com>

wrote:





I have not upgraded hive version but installed hive 2.3.2 on a server and



tried to read data and its working.Can we have any workaround to run drill



1.13



with hive 2.1 or up-gradation is the only option?























On Sat, Mar 24, 2018 3:52 PM, Anup Tiwari anup.tiw...@games24x7.com



wrote:



Sorry for delayed response as i didn't got time to test this.



@Vitalii, I have tried setting hive.metastore.client.capability.check=false



in



both ways which are :-



1. "hive.metastore.uris":



"thrift://prod-hadoop-107.bom-prod.aws.games24x7.com:9083?



hive.metastore.client.capability.check=false",



in drill hive plugin and restarted metastore and tried to access hive



tables



via drill.







2. Added capability property in hive-site.xml and restarted metastore and



tried



to access hive tables via drill.







Both ways didn't work. So does that mean Drill 1.13.0 version is



compatible with



Hive 2.3 and above?























On Tue, Mar 20, 2018 6:28 PM, Vitalii Diravka vitalii.dira...@gmail.com



wrote:



@Anup, it should. If it isn't back compatible, it is a Hive issue.



















Hive Trift Metastore API was changed, but still isn't documented in







cwiki.apache.org [1]







*hive.metastore.client.capability.check *[2] property is true by default.







Possibly changing this could help you.







You can change it in Drill Hive plugin or in hive-site.xml







Looks like, the issue will be the same for using hive-server2 2.3 version







and hive-metastore 2.1 version. If so it is a Hive issue.



















So you can try to change the property before updating to Hive 2.3 version.



















[1] https://issues.apache.org/jira/browse/HIVE-15062







[2]







https://issues.apache.org/jira/browse/HIVE-15062?



focusedCommentId=15659298=com.atlassian.jira.



plugin.system.issuetabpanels:comment-tabpanel#comment-15659298



















Kind regards







Vitalii



















On Tue, Mar 20, 2018 at 1:54 PM, Anup Tiwari <anup.tiw...@games24x7.com>







wrote:



















> Please find below information :-







> Apache Hadoop 2.7.3Apache Hive 2.1.1







> @Vitalii, For testing i can setup upgrade hive but upgrading hive will







> take time







> on our production server. Don't you think it should be back compatible?







>







>







>







>







> On Tue, Mar 20, 2018 4:33 PM, Vitalii Diravka vitalii.dira...@gmail.com







> wrote:







> Anup,







>







>







>







>







> "get_table_req" method is present in ThriftHiveMetastore header of Apache







>







> Hive 2.3 version.







>







> I believe Hive upgrade will help you. Probably it is Hive's back







>







> compatibility issue.







>







> Please let us know, whether upgrade helps.







>







>







>







>







> Kind regards







>







> Vitalii







>







>







>







>







> On Tue, Mar 20, 2018 at 12:56 PM, Vitalii Diravka <







> vitalii.dira...@gmail.com







>







> wrote:







>>







>







>







>







>







> Hi Anup,







>>







>







>







>>







> Maybe we missed something after updating onto hive-2.3 client versions.







>>







>







> Could you provide the following info:







>>







>







> * What is your hive-server and metastore versions? If it is not 2.3







>>







>







> version could you update onto this?







>>







>







> * What is your hadoop distribution?







>>







>







>







>>







> Kind regards







>>







>







> Vitalii







>>







>







>







>>







> On Tue, Mar 20, 2018 at 12:31 PM, Abhishek Girish <agir...@apache.org>







>>







>







> wrote:







>>







>







>







>>







> Okay, that confirms that the Hive storage plugin is not configured







>>>







>>







> correctly - you are unable to access any Hive table. What

Re: [Drill 1.13.0] : org.apache.thrift.TApplicationException: Invalid method name: 'get_table_req'

[Anup Tiwari]

I have not upgraded hive version but installed hive 2.3.2 on a server and
tried to read data and its working.Can we have any workaround to run drill 1.13
with hive 2.1 or up-gradation is the only option?  





On Sat, Mar 24, 2018 3:52 PM, Anup Tiwari anup.tiw...@games24x7.com  wrote:
Sorry for delayed response as i didn't got time to test this.
@Vitalii, I have tried setting hive.metastore.client.capability.check=false in
both ways which are :-
 1.  "hive.metastore.uris":

"thrift://prod-hadoop-107.bom-prod.aws.games24x7.com:9083?hive.metastore.client.capability.check=false",
in drill hive plugin and restarted metastore and tried to access hive tables
via drill.

 2. Added capability property in hive-site.xml and restarted metastore and tried
to access hive tables via drill.

Both ways didn't work. So does that mean Drill 1.13.0 version is compatible with
Hive 2.3 and above?  





On Tue, Mar 20, 2018 6:28 PM, Vitalii Diravka vitalii.dira...@gmail.com  wrote:
@Anup, it should. If it isn't back compatible, it is a Hive issue.




Hive Trift Metastore API was changed, but still isn't documented in

cwiki.apache.org [1]

*hive.metastore.client.capability.check *[2] property is true by default.

Possibly changing this could help you.

You can change it in Drill Hive plugin or in hive-site.xml

Looks like, the issue will be the same for using hive-server2 2.3 version

and hive-metastore 2.1 version. If so it is a Hive issue.




So you can try to change the property before updating to Hive 2.3 version.




[1] https://issues.apache.org/jira/browse/HIVE-15062

[2]

https://issues.apache.org/jira/browse/HIVE-15062?focusedCommentId=15659298=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15659298




Kind regards

Vitalii




On Tue, Mar 20, 2018 at 1:54 PM, Anup Tiwari <anup.tiw...@games24x7.com>

wrote:




> Please find below information :-

> Apache Hadoop 2.7.3Apache Hive 2.1.1

> @Vitalii, For testing i can setup upgrade hive but upgrading hive will

> take time

> on our production server. Don't you think it should be back compatible?

>

>

>

>

> On Tue, Mar 20, 2018 4:33 PM, Vitalii Diravka vitalii.dira...@gmail.com

> wrote:

> Anup,

>

>

>

>

> "get_table_req" method is present in ThriftHiveMetastore header of Apache

>

> Hive 2.3 version.

>

> I believe Hive upgrade will help you. Probably it is Hive's back

>

> compatibility issue.

>

> Please let us know, whether upgrade helps.

>

>

>

>

> Kind regards

>

> Vitalii

>

>

>

>

> On Tue, Mar 20, 2018 at 12:56 PM, Vitalii Diravka <

> vitalii.dira...@gmail.com

>

> wrote:

>>

>

>

>

>

> Hi Anup,

>>

>

>

>>

> Maybe we missed something after updating onto hive-2.3 client versions.

>>

>

> Could you provide the following info:

>>

>

> * What is your hive-server and metastore versions? If it is not 2.3

>>

>

> version could you update onto this?

>>

>

> * What is your hadoop distribution?

>>

>

>

>>

> Kind regards

>>

>

> Vitalii

>>

>

>

>>

> On Tue, Mar 20, 2018 at 12:31 PM, Abhishek Girish <agir...@apache.org>

>>

>

> wrote:

>>

>

>

>>

> Okay, that confirms that the Hive storage plugin is not configured

>>>

>>

> correctly - you are unable to access any Hive table. What's your Hive

>>>

>>

> server version?

>>>

>>

>

>>>

> On Tue, Mar 20, 2018 at 3:39 PM, Anup Tiwari <anup.tiw...@games24x7.com>

>>>

>>

> wrote:

>>>

>>

>

>>>

> > Hi,

>>>

>>

> > Please find my reply :-

>>>

>>

> > Can you do a 'use hive;` followed by 'show tables;' and see if table

>>>

>>

> > 'cad' is listed? : Did and got empty set(No rows selected).

>>>

>>

> >

>>>

>>

> > If you try via hive shell, do you see it? : Yes

>>>

>>

> >

>>>

>>

> > can you check if this is impacting accessing all hive tables (may be

>>>

>>

> > create a new one and try) or if this is specific to a certain table /

>>>

>>

> > database in Hive? : Tried 2 tables but getting same error. I have not

>>>

>>

> tried

>>>

>>

> > creating anew one, will try that and let you know.

>>>

>>

> >

>>>

>>

> >

>>>

>>

> >

>>>

>>

> >

>>>

>>

> > On Tue, Mar 20, 2018 3:19 PM, Abhishek G

Re: [Drill 1.13.0] : org.apache.thrift.TApplicationException: Invalid method name: 'get_table_req'

[Anup Tiwari]

Sorry for delayed response as i didn't got time to test this.
@Vitalii, I have tried setting hive.metastore.client.capability.check=false in
both ways which are :-
 1.  "hive.metastore.uris":

"thrift://prod-hadoop-107.bom-prod.aws.games24x7.com:9083?hive.metastore.client.capability.check=false",
in drill hive plugin and restarted metastore and tried to access hive tables
via drill.

 2. Added capability property in hive-site.xml and restarted metastore and tried
to access hive tables via drill.

Both ways didn't work. So does that mean Drill 1.13.0 version is compatible with
Hive 2.3 and above?  





On Tue, Mar 20, 2018 6:28 PM, Vitalii Diravka vitalii.dira...@gmail.com  wrote:
@Anup, it should. If it isn't back compatible, it is a Hive issue.




Hive Trift Metastore API was changed, but still isn't documented in

cwiki.apache.org [1]

*hive.metastore.client.capability.check *[2] property is true by default.

Possibly changing this could help you.

You can change it in Drill Hive plugin or in hive-site.xml

Looks like, the issue will be the same for using hive-server2 2.3 version

and hive-metastore 2.1 version. If so it is a Hive issue.




So you can try to change the property before updating to Hive 2.3 version.




[1] https://issues.apache.org/jira/browse/HIVE-15062

[2]

https://issues.apache.org/jira/browse/HIVE-15062?focusedCommentId=15659298=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15659298




Kind regards

Vitalii




On Tue, Mar 20, 2018 at 1:54 PM, Anup Tiwari <anup.tiw...@games24x7.com>

wrote:




> Please find below information :-

> Apache Hadoop 2.7.3Apache Hive 2.1.1

> @Vitalii, For testing i can setup upgrade hive but upgrading hive will

> take time

> on our production server. Don't you think it should be back compatible?

>

>

>

>

> On Tue, Mar 20, 2018 4:33 PM, Vitalii Diravka vitalii.dira...@gmail.com

> wrote:

> Anup,

>

>

>

>

> "get_table_req" method is present in ThriftHiveMetastore header of Apache

>

> Hive 2.3 version.

>

> I believe Hive upgrade will help you. Probably it is Hive's back

>

> compatibility issue.

>

> Please let us know, whether upgrade helps.

>

>

>

>

> Kind regards

>

> Vitalii

>

>

>

>

> On Tue, Mar 20, 2018 at 12:56 PM, Vitalii Diravka <

> vitalii.dira...@gmail.com

>

> wrote:

>>

>

>

>

>

> Hi Anup,

>>

>

>

>>

> Maybe we missed something after updating onto hive-2.3 client versions.

>>

>

> Could you provide the following info:

>>

>

> * What is your hive-server and metastore versions? If it is not 2.3

>>

>

> version could you update onto this?

>>

>

> * What is your hadoop distribution?

>>

>

>

>>

> Kind regards

>>

>

> Vitalii

>>

>

>

>>

> On Tue, Mar 20, 2018 at 12:31 PM, Abhishek Girish <agir...@apache.org>

>>

>

> wrote:

>>

>

>

>>

> Okay, that confirms that the Hive storage plugin is not configured

>>>

>>

> correctly - you are unable to access any Hive table. What's your Hive

>>>

>>

> server version?

>>>

>>

>

>>>

> On Tue, Mar 20, 2018 at 3:39 PM, Anup Tiwari <anup.tiw...@games24x7.com>

>>>

>>

> wrote:

>>>

>>

>

>>>

> > Hi,

>>>

>>

> > Please find my reply :-

>>>

>>

> > Can you do a 'use hive;` followed by 'show tables;' and see if table

>>>

>>

> > 'cad' is listed? : Did and got empty set(No rows selected).

>>>

>>

> >

>>>

>>

> > If you try via hive shell, do you see it? : Yes

>>>

>>

> >

>>>

>>

> > can you check if this is impacting accessing all hive tables (may be

>>>

>>

> > create a new one and try) or if this is specific to a certain table /

>>>

>>

> > database in Hive? : Tried 2 tables but getting same error. I have not

>>>

>>

> tried

>>>

>>

> > creating anew one, will try that and let you know.

>>>

>>

> >

>>>

>>

> >

>>>

>>

> >

>>>

>>

> >

>>>

>>

> > On Tue, Mar 20, 2018 3:19 PM, Abhishek Girish agir...@apache.org

>>>

>>

> wrote:

>>>

>>

> > Down in the stack trace it's complaining that the table name 'cad' was

>>>

>>

> not

>>>

>>

> >

>>>

>>

> > found; Can you do a 'use hive;` followed by 'show tables;' and see if

>>>

>>

> table

>>>

>>

> >

>>>

>>

> > 'cad' is listed?

>>>

>>

> >

>>>

>>

> >

>>>

>>

> >

>>>

>>

> >

>>>

>>

> > If you try via hive shell, do you see it?

>>>

>>

> >

>>>

>>

> >

>>>

>>

> >

>>>

>>

> >

>>>

>>

> > Also, can you check if this is impacting accessing all hive tables (may

>>>

>>

> be

>>>

>>

> >

>>>

>>

> > create a new one and try) or if this is specific to a certain table /

>>>

>>

> >

>>>

>>

> > database in Hive?

>>>

>>

> >

>>>

>>

> >

>>>

>>

> >

>>>

>>

> >

>>>

>>

> > -Abhishek

>>>

>>

> >

>>>

>>

> >

>>>

>>

>

>>>

>

>>

>

>>

>

>

>

>

>

> Regards,

> Anup Tiwari






Regards,
Anup Tiwari

Re: [Drill 1.13.0] : org.apache.thrift.TApplicationException: Invalid method name: 'get_table_req'

[Anup Tiwari]

Please find below information :-
Apache Hadoop 2.7.3Apache Hive 2.1.1
@Vitalii, For testing i can setup upgrade hive but upgrading hive will take time
on our production server. Don't you think it should be back compatible?  






On Tue, Mar 20, 2018 4:33 PM, Vitalii Diravka vitalii.dira...@gmail.com  wrote:
Anup,




"get_table_req" method is present in ThriftHiveMetastore header of Apache

Hive 2.3 version.

I believe Hive upgrade will help you. Probably it is Hive's back

compatibility issue.

Please let us know, whether upgrade helps.




Kind regards

Vitalii




On Tue, Mar 20, 2018 at 12:56 PM, Vitalii Diravka <vitalii.dira...@gmail.com


wrote:






Hi Anup,







Maybe we missed something after updating onto hive-2.3 client versions.



Could you provide the following info:



* What is your hive-server and metastore versions? If it is not 2.3



version could you update onto this?



* What is your hadoop distribution?







Kind regards



Vitalii







On Tue, Mar 20, 2018 at 12:31 PM, Abhishek Girish <agir...@apache.org>



wrote:







Okay, that confirms that the Hive storage plugin is not configured



correctly - you are unable to access any Hive table. What's your Hive



server version?







On Tue, Mar 20, 2018 at 3:39 PM, Anup Tiwari <anup.tiw...@games24x7.com>



wrote:







> Hi,



> Please find my reply :-



> Can you do a 'use hive;` followed by 'show tables;' and see if table



> 'cad' is listed? : Did and got empty set(No rows selected).



>



> If you try via hive shell, do you see it? : Yes



>



> can you check if this is impacting accessing all hive tables (may be



> create a new one and try) or if this is specific to a certain table /



> database in Hive? : Tried 2 tables but getting same error. I have not



tried



> creating anew one, will try that and let you know.



>



>



>



>



> On Tue, Mar 20, 2018 3:19 PM, Abhishek Girish agir...@apache.org



wrote:



> Down in the stack trace it's complaining that the table name 'cad' was



not



>



> found; Can you do a 'use hive;` followed by 'show tables;' and see if



table



>



> 'cad' is listed?



>



>



>



>



> If you try via hive shell, do you see it?



>



>



>



>



> Also, can you check if this is impacting accessing all hive tables (may



be



>



> create a new one and try) or if this is specific to a certain table /



>



> database in Hive?



>



>



>



>



> -Abhishek



>



>



















Regards,
Anup Tiwari

Re: [Drill 1.13.0] : org.apache.thrift.TApplicationException: Invalid method name: 'get_table_req'

[Anup Tiwari]

Hi,
Please find my reply :-
Can you do a 'use hive;` followed by 'show tables;' and see if table
'cad' is listed? : Did and got empty set(No rows selected).

If you try via hive shell, do you see it? : Yes

can you check if this is impacting accessing all hive tables (may be
create a new one and try) or if this is specific to a certain table /
database in Hive? : Tried 2 tables but getting same error. I have not tried
creating anew one, will try that and let you know.  






On Tue, Mar 20, 2018 3:19 PM, Abhishek Girish agir...@apache.org  wrote:
Down in the stack trace it's complaining that the table name 'cad' was not

found; Can you do a 'use hive;` followed by 'show tables;' and see if table

'cad' is listed?




If you try via hive shell, do you see it?




Also, can you check if this is impacting accessing all hive tables (may be

create a new one and try) or if this is specific to a certain table /

database in Hive?




-Abhishek




On Tue, Mar 20, 2018 at 2:37 PM, Anup Tiwari <anup.tiw...@games24x7.com>

wrote:





Note : Using Show databases, i can see hive schemas.























On Tue, Mar 20, 2018 2:36 PM, Anup Tiwari anup.tiw...@games24x7.com



wrote:



Hi,



I am not able to read my hive tables in drill 1.13.0 and with same plugin



conf



it was working in Drill 1.12.0 and 1.10.0. Please look into it asap and



let me



know if i have missed anything.



Hive Plugin :-



{ "type": "hive", "enabled": true, "configProps":



{ "hive.metastore.uris":



"thrift://prod-hadoop-1xx.com:9083", "hive.metastore.sasl.enabled":



"false",



"fs.default.name": "hdfs://prod-hadoop-1xx.com:9000" }}



Query :-



select id from hive.cad where log_date = '2018-03-18' limit 3



Error :-



2018-03-20 14:25:27,351 [254f337f-9ac3-b66f-ed17-1de459da3283:foreman]



INFO



o.a.drill.exec.work.foreman.Foreman - Query text for query id



254f337f-9ac3-b66f-ed17-1de459da3283: select id from hive.cad where



log_date =



'2018-03-18' limit 32018-03-20 14:25:27,354



[254f337f-9ac3-b66f-ed17-1de459da3283:foreman] WARN



o.a.d.e.s.h.DrillHiveMetaStoreClient - Failure while attempting to get



hive



table. Retries once.org.apache.thrift.TApplicationException: Invalid



method



name: 'get_table_req' at



org.apache.thrift.TApplicationException.read(TApplicationExc



eption.java:111)



~[drill-hive-exec-shaded-1.13.0.jar:1.13.0] at



org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:79)



~[drill-hive-exec-shaded-1.13.0.jar:1.13.0] at



org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$



Client.recv_get_table_req(ThriftHiveMetastore.java:1563)



~[drill-hive-exec-shaded-1.13.0.jar:1.13.0] at



org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$



Client.get_table_req(ThriftHiveMetastore.java:1550)



~[drill-hive-exec-shaded-1.13.0.jar:1.13.0] at



org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getTabl



e(HiveMetaStoreClient.java:1344)



~[drill-hive-exec-shaded-1.13.0.jar:1.13.0] at



org.apache.drill.exec.store.hive.DrillHiveMetaStoreClient.ge



tHiveReadEntryHelper(DrillHiveMetaStoreClient.java:285)



~[drill-storage-hive-core-1.13.0.jar:1.13.0] at



org.apache.drill.exec.store.hive.DrillHiveMetaStoreClient$Ta



bleLoader.load(DrillHiveMetaStoreClient.java:535)



[drill-storage-hive-core-1.13.0.jar:1.13.0] at



org.apache.drill.exec.store.hive.DrillHiveMetaStoreClient$Ta



bleLoader.load(DrillHiveMetaStoreClient.java:531)



[drill-storage-hive-core-1.13.0.jar:1.13.0] at



com.google.common.cache.LocalCache$LoadingValueReference.loa



dFuture(LocalCache.java:3527)



[guava-18.0.jar:na] at



com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2319)



[guava-18.0.jar:na] at



com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(



LocalCache.java:2282)



[guava-18.0.jar:na] at



com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2197)



[guava-18.0.jar:na] at



com.google.common.cache.LocalCache.get(LocalCache.java:3937)



[guava-18.0.jar:na]



at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3941)



[guava-18.0.jar:na] at



com.google.common.cache.LocalCache$LocalLoadingCache.get(



LocalCache.java:4824)



[guava-18.0.jar:na] at



org.apache.drill.exec.store.hive.DrillHiveMetaStoreClient$Hi



veClientWithCaching.getHiveReadEntry(DrillHiveMetaStoreClient.java:495)



[drill-storage-hive-core-1.13.0.jar:1.13.0] at



org.apache.drill.exec.store.hive.schema.HiveSchemaFactory$Hi



veSchema.getSelectionBaseOnName(HiveSchemaFactory.java:233)



[drill-storage-hive-core-1.13.0.jar:1.13.0] at



org.apache.drill.exec.store.hive.schema.HiveSchemaFactory$Hi



veSchema.getDrillTable(HiveSchemaFactory.java:213)



[drill-storage-hive-core-1.13.0.jar:1.13.0] at



org.apache.drill.exec.store.hive.schema

Re: [Drill 1.13.0] : org.apache.thrift.TApplicationException: Invalid method name: 'get_table_req'

[Anup Tiwari]

Note :  Using Show databases, i can see hive schemas.  





On Tue, Mar 20, 2018 2:36 PM, Anup Tiwari anup.tiw...@games24x7.com  wrote:
Hi,
I am not able to read my hive tables in drill 1.13.0 and with same plugin conf
it was working in Drill 1.12.0 and 1.10.0. Please look into it asap and let me
know if i have missed anything.
Hive Plugin :-
{  "type": "hive",  "enabled": true,  "configProps": {"hive.metastore.uris":
"thrift://prod-hadoop-1xx.com:9083","hive.metastore.sasl.enabled": "false",
"fs.default.name": "hdfs://prod-hadoop-1xx.com:9000"  }}
Query :-
select id from hive.cad where log_date = '2018-03-18' limit 3
Error :-
2018-03-20 14:25:27,351 [254f337f-9ac3-b66f-ed17-1de459da3283:foreman] INFO
o.a.drill.exec.work.foreman.Foreman - Query text for query id
254f337f-9ac3-b66f-ed17-1de459da3283: select id from hive.cad where log_date =
'2018-03-18' limit 32018-03-20 14:25:27,354
[254f337f-9ac3-b66f-ed17-1de459da3283:foreman] WARN
o.a.d.e.s.h.DrillHiveMetaStoreClient - Failure while attempting to get hive
table. Retries once.org.apache.thrift.TApplicationException: Invalid method
name: 'get_table_req' at
org.apache.thrift.TApplicationException.read(TApplicationException.java:111)
~[drill-hive-exec-shaded-1.13.0.jar:1.13.0] at
org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:79)
~[drill-hive-exec-shaded-1.13.0.jar:1.13.0] at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_table_req(ThriftHiveMetastore.java:1563)
~[drill-hive-exec-shaded-1.13.0.jar:1.13.0] at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_table_req(ThriftHiveMetastore.java:1550)
~[drill-hive-exec-shaded-1.13.0.jar:1.13.0] at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getTable(HiveMetaStoreClient.java:1344)
~[drill-hive-exec-shaded-1.13.0.jar:1.13.0] at
org.apache.drill.exec.store.hive.DrillHiveMetaStoreClient.getHiveReadEntryHelper(DrillHiveMetaStoreClient.java:285)
~[drill-storage-hive-core-1.13.0.jar:1.13.0] at
org.apache.drill.exec.store.hive.DrillHiveMetaStoreClient$TableLoader.load(DrillHiveMetaStoreClient.java:535)
[drill-storage-hive-core-1.13.0.jar:1.13.0] at
org.apache.drill.exec.store.hive.DrillHiveMetaStoreClient$TableLoader.load(DrillHiveMetaStoreClient.java:531)
[drill-storage-hive-core-1.13.0.jar:1.13.0] at
com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3527)
[guava-18.0.jar:na] at
com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2319)
[guava-18.0.jar:na] at
com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2282)
[guava-18.0.jar:na] at
com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2197)
[guava-18.0.jar:na] at
com.google.common.cache.LocalCache.get(LocalCache.java:3937) [guava-18.0.jar:na]
 at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3941)
[guava-18.0.jar:na] at
com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4824)
[guava-18.0.jar:na] at
org.apache.drill.exec.store.hive.DrillHiveMetaStoreClient$HiveClientWithCaching.getHiveReadEntry(DrillHiveMetaStoreClient.java:495)
[drill-storage-hive-core-1.13.0.jar:1.13.0] at
org.apache.drill.exec.store.hive.schema.HiveSchemaFactory$HiveSchema.getSelectionBaseOnName(HiveSchemaFactory.java:233)
[drill-storage-hive-core-1.13.0.jar:1.13.0] at
org.apache.drill.exec.store.hive.schema.HiveSchemaFactory$HiveSchema.getDrillTable(HiveSchemaFactory.java:213)
[drill-storage-hive-core-1.13.0.jar:1.13.0] at
org.apache.drill.exec.store.hive.schema.HiveDatabaseSchema.getTable(HiveDatabaseSchema.java:62)
[drill-storage-hive-core-1.13.0.jar:1.13.0] at
org.apache.drill.exec.store.hive.schema.HiveSchemaFactory$HiveSchema.getTable(HiveSchemaFactory.java:201)
[drill-storage-hive-core-1.13.0.jar:1.13.0] at
org.apache.calcite.jdbc.SimpleCalciteSchema.getImplicitTable(SimpleCalciteSchema.java:82)
[calcite-core-1.15.0-drill-r0.jar:1.15.0-drill-r0] at
org.apache.calcite.jdbc.CalciteSchema.getTable(CalciteSchema.java:257)
[calcite-core-1.15.0-drill-r0.jar:1.15.0-drill-r0] at
org.apache.calcite.sql.validate.SqlValidatorUtil.getTableEntryFrom(SqlValidatorUtil.java:1003)
[calcite-core-1.15.0-drill-r0.jar:1.15.0-drill-r0] at
org.apache.calcite.sql.validate.SqlValidatorUtil.getTableEntry(SqlValidatorUtil.java:960)
[calcite-core-1.15.0-drill-r0.jar:1.15.0-drill-r0] at
org.apache.calcite.prepare.CalciteCatalogReader.getTable(CalciteCatalogReader.java:117)
[calcite-core-1.15.0-drill-r0.jar:1.15.0-drill-r0] at
org.apache.drill.exec.planner.sql.SqlConverter$DrillCalciteCatalogReader.getTable(SqlConverter.java:633)
[drill-java-exec-1.13.0.jar:1.13.0] at
org.apache.drill.exec.planner.sql.SqlConverter$DrillValidator.validateFrom(SqlConverter.java:261)
[drill-java-exec-1.13.0.jar:1.13.0] at
org.apache.calcite.sql.validate.SqlValidatorImpl.validateSelect(SqlValidatorImpl.java:3216)
[calcite-core-

[Drill 1.13.0] : org.apache.thrift.TApplicationException: Invalid method name: 'get_table_req'

[Anup Tiwari]

lidate(SqlConverter.java:190)
[drill-java-exec-1.13.0.jar:1.13.0] ... 10 common frames omittedCaused by:
org.apache.calcite.sql.validate.SqlValidatorException: Object 'cad' not found
within 'hive' at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
Method) ~[na:1.8.0_72] at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
~[na:1.8.0_72] at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
~[na:1.8.0_72] at
java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[na:1.8.0_72]
 at org.apache.calcite.runtime.Resources$ExInstWithCause.ex(Resources.java:463)
~[calcite-core-1.15.0-drill-r0.jar:1.15.0-drill-r0] at
org.apache.calcite.runtime.Resources$ExInst.ex(Resources.java:572)
~[calcite-core-1.15.0-drill-r0.jar:1.15.0-drill-r0] ... 31 common frames omitted
2018-03-20 14:25:27,375 [254f337f-9ac3-b66f-ed17-1de459da3283:foreman] INFO
o.apache.drill.exec.work.WorkManager - Waiting for 0 queries to complete before
shutting down2018-03-20 14:25:27,375
[254f337f-9ac3-b66f-ed17-1de459da3283:foreman] INFO
o.apache.drill.exec.work.WorkManager - Waiting for 0 running fragments to
complete before shutting down
Regards,
Anup Tiwari

Re: [Drill 1.9.0] : [CONNECTION ERROR] :- (user client) closed unexpectedly. Drillbit down?

[Anup Tiwari]

Thanks.. will upgrade to 1.13.0 and let you know.  





On Tue, Mar 20, 2018 11:08 AM, Parth Chandra par...@apache.org  wrote:
Hi Anup,




  I don't have full context for the proposed hack, and it might have worked,

but looks like Vlad has addressed the issue in the right place. Perhaps you

can try out 1.13.0 and let us all know.




Thanks




Parth




On Sat, Mar 17, 2018 at 11:43 AM, Anup Tiwari <anup.tiw...@games24x7.com>

wrote:




> Thanks Parth for Info. I am really looking forward to it.

> But can you tell me if the second part(about hack) was right or not?

> Because i

> really want to test it as we got this issue several time in last 2-3 days

> post

> upgrading to 1.12.0.

> Also i have seen sometimes after lost connection , drillbit gets killed on

> few/all nodes and i am not getting any logs in drillbit.out/drillbit.log.

>

>

>

>

> On Fri, Mar 16, 2018 11:07 PM, Parth Chandra par...@apache.org wrote:

> On Fri, Mar 16, 2018 at 8:10 PM, Anup Tiwari <anup.tiw...@games24x7.com>

>

> wrote:

>

>

>

>

> Hi All,

>>

>

> I was just going through this post and found very good suggestions.

>>

>

> But this issue is still there in Drill 1.12.0 and i can see

>>

>

> https://issues.apache.org/jira/browse/DRILL-4708 is now marked as

>>

>

> resolved in

>>

>

> "1.13.0" so i am hoping that this will be fixed in drill 1.13.0.

>>

>

> Few things i want to ask :-

>>

>

> - Any Planned date for Drill 1.13.0 release?

>>

>

>

>>

>

>

>

>

>

>

> Real Soon Now. :)

>

> The release will be out in a couple of days. Watch this list for an

>

> announcement.

>

>

>

>

>

>

> Regards,

> Anup Tiwari






Regards,
Anup Tiwari

Re: [Drill 1.9.0] : [CONNECTION ERROR] :- (user client) closed unexpectedly. Drillbit down?

[Anup Tiwari]

Thanks Parth for Info. I am really looking forward to it.
But can you tell me if the second part(about hack) was right or not? Because i
really want to test it as we got this issue several time in last 2-3 days post
upgrading to 1.12.0.
Also i have seen sometimes after lost connection , drillbit gets killed on
few/all nodes and i am not getting any logs in drillbit.out/drillbit.log.  






On Fri, Mar 16, 2018 11:07 PM, Parth Chandra par...@apache.org  wrote:
On Fri, Mar 16, 2018 at 8:10 PM, Anup Tiwari <anup.tiw...@games24x7.com>

wrote:





Hi All,



I was just going through this post and found very good suggestions.



But this issue is still there in Drill 1.12.0 and i can see



https://issues.apache.org/jira/browse/DRILL-4708 is now marked as



resolved in



"1.13.0" so i am hoping that this will be fixed in drill 1.13.0.



Few things i want to ask :-



- Any Planned date for Drill 1.13.0 release?












Real Soon Now. :)

The release will be out in a couple of days. Watch this list for an

announcement.






Regards,
Anup Tiwari

Re: [Drill 1.9.0] : [CONNECTION ERROR] :- (user client) closed unexpectedly. Drillbit down?

[Anup Tiwari]

Hi All,
I was just going through this post and found very good suggestions.
But this issue is still there in Drill 1.12.0 and i can see
https://issues.apache.org/jira/browse/DRILL-4708 is now marked as resolved in
"1.13.0" so i am hoping that this will be fixed in drill 1.13.0.
Few things i want to ask :-
- Any Planned date for Drill 1.13.0 release?
- If i have to apply hack suggested by Francois(till Drill-4708 gets released)
which is
org.apache.drill.exec.work.foreman
QueryManager.java
private void drillbitUnregistered(.)

if (atLeastOneFailure)
-> just log the error, do not cancel query.

Then should i have to just comment below line from code and rebuild drill from
source :-
if (atLeastOneFailure) {logger.warn("Drillbits [{}] no longer registered
in cluster. Canceling query {}",failedNodeList,
QueryIdHelper.getQueryId(queryId));foreman.addToEventQueue(QueryState.FAILED,
new ForemanException(String.format("One more more nodes lost
connectivity during query. Identified nodes were [%s].",
failedNodeList)));
I haven't done something like above before so i might not be making sense also
it might impact somewhere else so please suggest a path forward.  





On Tue, Mar 21, 2017 8:15 PM, François Méthot fmetho...@gmail.com  wrote:
Hi,




  We have been having client-foreman connection and ZkConnection issue few

months ago. It went from annoying to a show stopper when we moved from a 12

nodes cluster to a 220 nodes cluster.




Nodes specs

- 8 cores total (2 x E5620)

- 72 GB RAM Total

- Other applications share the same hardware.




~ 100 TB parquet data on hdfs.













Based on our observation we have done few months ago, we ended up with

those setting/guideline/changes:




- Memory Setting

  DRILL_MAX_DIRECT_MEMORY="20G"

  DRILL_HEAP="8G"




  Remaining RAM is for other applications







- Threading

  planner.width.max_per_node = 4




  We think that higher number of threads will generate network traffic or

more context switch on each node, leading to more chances of getting Zk

disconnection.

  But we observed that even with max_per_node of 1, we would still get

disconnection. We had no clear indication from Cloudera Manager that

Mem/CPU/Network is overloaded on faulty node. Although on very rare

occasion we would get no stats data at all from certain node.




- Affinity Factor

  We change the affinity factor from default to a big value.

  planner.affinity_factor = 1000.0




  This improved issue with some drillbit of our cluster scanning data

stored on remote nodes. It somehow maximizes the chances of a drillbit

reading local data. When drillbits only scan local data, it reduces the

amount of network traffic, It accelerate queries and reduce the chance of

ZkDisconnect.




- If using hdfs, make sure each data file is stored on 1 block




- Try more recent 1.8 JVM or switch to JVM 1.7

  We have had CLIENT to FOREMAN disconnection issue with certain version

of JVM (linux, windows, mac). (we sent an email about this to the dev

mailing list in the past)




- Query Pattern

  The more fields are getting selected (select * vs select few specific

field) the more chance we will get the error. More data selected means more

cpu/network activity leading to more chances of Zookeeper skipping a

heartbeat.







- Foreman QueryManager Resilience Hack

When a query would fail, our log indicated that a drillbit was getting

unregistered and then get registed again a short time after (few ms to few

seconds), but the foreman QueryManager would catch the

"drillbitUnregistered" event and fail the queries right away. As a test, we

changed the QueryManager to not fail queries when a drillbit is getting

unregistered. We have put this change in place in 1.8 and our log now

indicates Zk Disconnect-Reconnect while query keeps running, so we kept

that test code in. A query will now fail only if the drillbit lose

connection with other drillbit (through the RPC bus) at some point. We have

since move to 1.9 with that change as well. I haven't had chance to try

back without the hack in 1.9.




org.apache.drill.exec.work.foreman

  QueryManager.java

  private void drillbitUnregistered(.)

  

  if (atLeastOneFailure)

  -> just log the error, do not cancel query.




our query success rate went from <50% to >95% with all the changes above.

We hope to get rid of the hack when an official fix is available.










To cover the missing 5% error (any other type of errors), we advise users

to try again. We also have built-in retry strategy implemented in our

hourly python scripts that aggregates data.




Hope it helps




Francois




























On Thu, Mar 9, 2017 at 2:31 PM, Anup Tiwari <anup.tiw...@games24x7.com>

wrote:




> Hi John,

>

> First of all sorry for delayed response and thanks for your suggestion,

Re: [Drill 1.12.0] : RESOURCE ERROR: Not enough memory for internal partitioning and fallback mechanism for HashAgg to use unbounded memory is disabled

[Anup Tiwari]

Hi Kunal,
First of all, thanks for such a good explanation, it really helped me
understanding few things.But as you have mentioned that in case of failure
"Drillbits capped at around 1.2GB" and suggested to "increase the
memory-per-query-per-node from the current 2GB to a higher level".
Are you saying to increase this "planner.memory.max_query_memory_per_node" from
2GB? If yes then just wanted to mention that i have already set
planner.memory.max_query_memory_per_node  = 4G(mentioned in trail mail).
Let me know if i have misinterpreted anything.  





On Fri, Mar 16, 2018 3:08 AM, Kunal Khatua ku...@apache.org  wrote:
Anup




If you look at the successful run's major fragment overview, you can see

the amount of memory consumed across the 5 nodes and the average per node

(shown in the table):

++-+-+-+-++-+---+

| Major Fragment | Minor Fragments | Min Runtime | Avg Runtime | Max

Runtime | % Busy | Max Peak Memory | Total Peak Memory |

++-+-+-+-++-+---+

| 00-xx-xx | 1 / 1 | 2m18s | 2m18s | 2m18s

  | 0.28% | 3MB | 3MB |

| 01-xx-xx | 30 / 30 | 2m17s | 2m17s | 2m18s

  | 0.19% | 55MB | 1650MB |

| 02-xx-xx | 5 / 5 | 2m4s | 2m11s | 2m17s

  | 0.07% | 9MB | 45MB |

| 03-xx-xx | 30 / 30 | 1m18s | 2m1s | 2m17s

  | 70.89% | 200MB | 6000MB |

| 04-xx-xx | 2 / 2 | 2.608s | 2.627s | 2.646s

  | 5.52% | 10MB | 20MB |

| | | | |

  | | Avg/Node => | 1544MB |

++-+-+-+-++-+---+




So, we're looking at 1.5GB average.




The failed run was

++-+-+-+-++-++

| Major Fragment | Minor Fragments | Min Runtime | Avg Runtime | Max

Runtime | % Busy | Max Peak Memory | Total Peak |

++-+-+-+-++-++

| 00-xx-xx | 0 / 1 | | |

  | | | #N/A |

| 01-xx-xx | 30 / 30 | 1m30s | 1m33s | 1m34s

  | 0.00% | 8MB | 240MB |

| 02-xx-xx | 5-May | 1m29s | 1m31s | 1m33s

  | 0.03% | 6MB | 30MB |

| 03-xx-xx | 30 / 30 | 1m17s | 1m30s | 1m34s

  | 31.25% | 195MB | 5850MB |

| 04-xx-xx | 2-Feb | 2.673s | 2.674s | 2.676s

  | 2.39% | 10MB | 20MB |

| | | | |

  | | Avg/Node => | 1228MB |

++-+-+-+-++-++




You can see that the Drillbits here capped at around 1.2GB before failing.




Drill 1.12 currently does not have the Spill-to-disk for the HashJoin

operator, but there is memory usage cap being enforced for HashAgg.

Enabling the `fallback` option removes this cap.




The option here would be to increase the memory-per-query-per-node from the

current 2GB to a higher level.




The option you're looking for is:

planner.memory.max_query_memory_per_node

and/or

planner.memory.percent_per_query







Read this for a better understanding of all your options:

https://drill.apache.org/docs/configuring-drill-memory/




~ Kunal













On Tue, Mar 13, 2018 at 11:41 PM, Anup Tiwari <anup.tiw...@games24x7.com>

wrote:




> Hi Kunal,

> Please find below cluster/platform details :-

> Number of Nodes : 5

> RAM/Node : 32GBCore/Node : 8DRILL_MAX_DIRECT_MEMORY="20G"

> DRILL_HEAP="8G"DRILL

> VERSION = 1.12.0HADOOP VERSION = 2.7.3ZOOKEEPER VERSION = 3.4.8(Installed

> in

> Distributed Mode on 3 nodes)planner.memory.max_query_memory_per_node = 4G

> Please find below query which was giving error before setting "alter

> session set

> `drill.exec.hashagg.fallback.enabled`=TRUE;" and its json profile for

> success(variable is set) and failure(variable is not set). Please note this

> query was working fine in Drill 1.10.0 in same environment.

> Query :-

> create table a_games_log_visit_utm asselectdistinctglv.sessionid,case when

> (UFG('utms=', glv.url, '&') <> 'null') then UFG('utms=', glv.url, '&') else

> UFG('utms=',glv.hcookie, '&') end as utms,

> case when (UFG('utms=', glv.url, '&') <> 'null') then UFG('utmm=',

> glv.url, '&')

> else UFG('utmm=',glv.hcookie, '&') end as utmm,

> case when (UFG('utms=', glv.url, '&') <> 'null') then UFG('utmc=',

> glv.url, '&')

> else UFG('utmc=',glv.hcookie, '&') end as utmc,

> case when (UFG('utms=', glv.url, '&') <> 'null') then UFG('utmt=',

> glv.url, '&')

> else UFG('utmt=',glv.hcookie, '&') end as utmt,

> case when (UFG('utms=', glv.url, '&') <> 'null') then UFG('utmcmpn=',

> glv.url,

> '&') else UFG('utmcmp

Re: [Drill 1.12.0] : Suggestions on Downgrade to 1.11.0 & com.mysql.jdbc.exceptions.jdbc4.CommunicationsException

[Anup Tiwari]

Hi All,
We checked our MySQL max number of connections which is set to 200 and i think
this might be due to exceeding max number of connections only as right now i can
see 89 connections to MySQL.
I want to know community's thoughts on this whether i am heading in right
direction or not.  





On Fri, Mar 16, 2018 1:03 PM, Anup Tiwari anup.tiw...@games24x7.com  wrote:
Hi All,
We are getting a lot of different type of issues/error post upgrading from Drill
1.10.0 to 1.12.0 which i am asking on forum as well so just wanted to know
whether downgrading to Drill 1.11.0 will help or not?
This time we got exception related to mysql connection storage and please note
that this issue is not consistent i.e. if i execute this query after some time
then it works. Please find below query are error logs.
Query :
create table dfs.tmp.table_info as select * from mysql.test.table_info;
Error :
WARN o.a.d.e.store.jdbc.JdbcStoragePlugin - Failure while attempting to load
JDBC schema.com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: The last
packet successfully received from the server was 49,949,177 milliseconds ago. 
The last packet sent successfully to the server was 49,949,196 milliseconds ago.
is longer than the server configured value of 'wait_timeout'. You should
consider either expiring and/or testing connection validity before use in your
application, increasing the server configured values for client timeouts, or
using the Connector/J connection property 'autoReconnect=true' to avoid this
problem.at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
Method) ~[na:1.8.0_72]at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
~[na:1.8.0_72]at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
~[na:1.8.0_72]at
java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[na:1.8.0_72]
at com.mysql.jdbc.Util.handleNewInstance(Util.java:389)
~[mysql-connector-java-5.1.35-bin.jar:5.1.35]at
com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:1038)
~[mysql-connector-java-5.1.35-bin.jar:5.1.35]at
com.mysql.jdbc.MysqlIO.send(MysqlIO.java:3609)
~[mysql-connector-java-5.1.35-bin.jar:5.1.35]at
com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2417)
~[mysql-connector-java-5.1.35-bin.jar:5.1.35]at
com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2582)
~[mysql-connector-java-5.1.35-bin.jar:5.1.35]at
com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2531)
~[mysql-connector-java-5.1.35-bin.jar:5.1.35]at
com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2489)
~[mysql-connector-java-5.1.35-bin.jar:5.1.35]at
com.mysql.jdbc.StatementImpl.executeQuery(StatementImpl.java:1446)
~[mysql-connector-java-5.1.35-bin.jar:5.1.35]at
com.mysql.jdbc.DatabaseMetaData.getCatalogs(DatabaseMetaData.java:2025)
~[mysql-connector-java-5.1.35-bin.jar:5.1.35]at
org.apache.commons.dbcp.DelegatingDatabaseMetaData.getCatalogs(DelegatingDatabaseMetaData.java:190)
~[commons-dbcp-1.4.jar:1.4]at
org.apache.drill.exec.store.jdbc.JdbcStoragePlugin$JdbcCatalogSchema.(JdbcStoragePlugin.java:309)
~[drill-jdbc-storage-1.12.0.jar:1.12.0]at
org.apache.drill.exec.store.jdbc.JdbcStoragePlugin.registerSchemas(JdbcStoragePlugin.java:430)
[drill-jdbc-storage-1.12.0.jar:1.12.0]at
org.apache.drill.exec.planner.sql.DynamicRootSchema.loadSchemaFactory(DynamicRootSchema.java:94)
[drill-java-exec-1.12.0.jar:1.12.0]at
org.apache.drill.exec.planner.sql.DynamicRootSchema.getSubSchema(DynamicRootSchema.java:74)
[drill-java-exec-1.12.0.jar:1.12.0]at
org.apache.calcite.prepare.CalciteCatalogReader.getSchema(CalciteCatalogReader.java:160)
[calcite-core-1.4.0-drill-r23.jar:1.4.0-drill-r23]at
org.apache.calcite.prepare.CalciteCatalogReader.getTableFrom(CalciteCatalogReader.java:114)
[calcite-core-1.4.0-drill-r23.jar:1.4.0-drill-r23]at
org.apache.calcite.prepare.CalciteCatalogReader.getTable(CalciteCatalogReader.java:108)
[calcite-core-1.4.0-drill-r23.jar:1.4.0-drill-r23]at
org.apache.drill.exec.planner.sql.SqlConverter$DrillCalciteCatalogReader.getTable(SqlConverter.java:493)
[drill-java-exec-1.12.0.jar:1.12.0]at
org.apache.drill.exec.planner.sql.SqlConverter$DrillCalciteCatalogReader.getTable(SqlConverter.java:434)
[drill-java-exec-1.12.0.jar:1.12.0]at
org.apache.calcite.sql.validate.EmptyScope.getTableNamespace(EmptyScope.java:75)
[calcite-core-1.4.0-drill-r23.jar:1.4.0-drill-r23]at
org.apache.calcite.sql.validate.DelegatingScope.getTableNamespace(DelegatingScope.java:124)
[calcite-core-1.4.0-drill-r23.jar:1.4.0-drill-r23]at
org.apache.calcite.sql.validate.IdentifierNamespace.validateImpl(IdentifierNamespace.java:104)
[calcite-core-1.4.0-drill-r23.jar:1.4.0-drill-r23]at
org.apache.calcite.sql.validate.AbstractNamespace.validate

[Drill 1.12.0] : Suggestions on Downgrade to 1.11.0 & com.mysql.jdbc.exceptions.jdbc4.CommunicationsException

[Anup Tiwari]

]at
org.apache.calcite.sql.validate.IdentifierNamespace.validateImpl(IdentifierNamespace.java:104)
[calcite-core-1.4.0-drill-r23.jar:1.4.0-drill-r23]at
org.apache.calcite.sql.validate.AbstractNamespace.validate(AbstractNamespace.java:86)
[calcite-core-1.4.0-drill-r23.jar:1.4.0-drill-r23]at
org.apache.calcite.sql.validate.SqlValidatorImpl.validateNamespace(SqlValidatorImpl.java:886)
[calcite-core-1.4.0-drill-r23.jar:1.4.0-drill-r23]at
org.apache.calcite.sql.validate.SqlValidatorImpl.validateQuery(SqlValidatorImpl.java:872)
[calcite-core-1.4.0-drill-r23.jar:1.4.0-drill-r23]at
org.apache.calcite.sql.validate.SqlValidatorImpl.validateFrom(SqlValidatorImpl.java:2817)
[calcite-core-1.4.0-drill-r23.jar:1.4.0-drill-r23]at
org.apache.calcite.sql.validate.SqlValidatorImpl.validateFrom(SqlValidatorImpl.java:2802)
[calcite-core-1.4.0-drill-r23.jar:1.4.0-drill-r23]at
org.apache.calcite.sql.validate.SqlValidatorImpl.validateSelect(SqlValidatorImpl.java:3025)
[calcite-core-1.4.0-drill-r23.jar:1.4.0-drill-r23]at
org.apache.calcite.sql.validate.SelectNamespace.validateImpl(SelectNamespace.java:60)
[calcite-core-1.4.0-drill-r23.jar:1.4.0-drill-r23]at
org.apache.calcite.sql.validate.AbstractNamespace.validate(AbstractNamespace.java:86)
[calcite-core-1.4.0-drill-r23.jar:1.4.0-drill-r23]at
org.apache.calcite.sql.validate.SqlValidatorImpl.validateNamespace(SqlValidatorImpl.java:886)
[calcite-core-1.4.0-drill-r23.jar:1.4.0-drill-r23]at
org.apache.calcite.sql.validate.SqlValidatorImpl.validateQuery(SqlValidatorImpl.java:872)
[calcite-core-1.4.0-drill-r23.jar:1.4.0-drill-r23]at
org.apache.calcite.sql.SqlSelect.validate(SqlSelect.java:210)
[calcite-core-1.4.0-drill-r23.jar:1.4.0-drill-r23]at
org.apache.calcite.sql.validate.SqlValidatorImpl.validateScopedExpression(SqlValidatorImpl.java:846)
[calcite-core-1.4.0-drill-r23.jar:1.4.0-drill-r23]at
org.apache.calcite.sql.validate.SqlValidatorImpl.validate(SqlValidatorImpl.java:560)
[calcite-core-1.4.0-drill-r23.jar:1.4.0-drill-r23]at
org.apache.drill.exec.planner.sql.SqlConverter.validate(SqlConverter.java:172)
[drill-java-exec-1.12.0.jar:1.12.0]at
org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.validateNode(DefaultSqlHandler.java:617)
[drill-java-exec-1.12.0.jar:1.12.0]at
org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.validateAndConvert(DefaultSqlHandler.java:192)
[drill-java-exec-1.12.0.jar:1.12.0]at
org.apache.drill.exec.planner.sql.handlers.CreateTableHandler.getPlan(CreateTableHandler.java:77)
[drill-java-exec-1.12.0.jar:1.12.0]at
org.apache.drill.exec.planner.sql.DrillSqlWorker.getQueryPlan(DrillSqlWorker.java:131)
[drill-java-exec-1.12.0.jar:1.12.0]at
org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan(DrillSqlWorker.java:79)
[drill-java-exec-1.12.0.jar:1.12.0]at
org.apache.drill.exec.work.foreman.Foreman.runSQL(Foreman.java:1017)
[drill-java-exec-1.12.0.jar:1.12.0]at
org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:289)
[drill-java-exec-1.12.0.jar:1.12.0]at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[na:1.8.0_72]at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[na:1.8.0_72]at java.lang.Thread.run(Thread.java:745) [na:1.8.0_72]
Let me know what to do here.
Regards,
Anup Tiwari

Re: [Drill 1.10.0/1.12.0] Query Started Taking Time + frequent one or more node lost connectivity error

[Anup Tiwari]

Hi Kunal,
Please find my answers(highlighted) w.r.t. your questions below :-
it seems like you are trying to read from a source and write to a destination
with partitioning (or a
HashJoin/HashAgg prior to writing) : No partitions ; It is simple left join
query with CTAS and all tables of join are created in drill(parquet).
Is the data highly skewed on such a column? : We are not partitioning data on
any column however join condition is based some string column which should not
be skewed but i will check this from my end.

I will try to share json profile asap.  





On Wed, Mar 14, 2018 7:43 PM, Kunal Khatua ku...@apache.org  wrote:
Hi Anup




It helps if you can share the profile (*.sys.drill / *.json files) to help

explain. I don't think the user mailing list allows attachments, so you

could use an online document sharing service (e.g. Google Drive, etc) to do

the same.




Coming back to your description, it seems like you are trying to read from

a source and write to a destination with partitioning (or a

HashJoin/HashAgg prior to writing). If that is the case, the records are

all getting into 1 fragment most likely because of skew in the data's

unique values on which you are doing a partition.




Is the data highly skewed on such a column?










On Wed, Mar 14, 2018 at 1:16 AM, Anup Tiwari <anup.tiw...@games24x7.com>

wrote:




> Also i have observed one thing, the query which is taking time is creating

> ~30-40 fragments and 99.9% of record is getting written into only one

> fragment.

>

>

>

>

> On Wed, Mar 14, 2018 1:37 PM, Anup Tiwari anup.tiw...@games24x7.com

> wrote:

> Hi Padma,

> Please find my highlighted answer w.r.t. your question :-

> Connection loss error can happen when zookeeper thinks that a node is dead

> becauseit did not get heartbeat from the node. It can be because the node

> is

> busy or you havenetwork problems. Q) Did anything changed in your network

> ? Answer : No. Also we cross verify Intra communication within nodes and its

> working fine.

>

> Q) Is the data static or are you adding new data ? Answer : Data is static.

> Q) Do you have metadata caching enabled ?Answer : No.

> PARQUET_WRITER seem to be indicate you are doing some kind of CTAS. : This

> is

> correct, we are doing CTAS.

> The block missing exception could possibly mean some problem with name

> node or

> bad diskson one of the node. : There is no bad disk also when i checked

> that

> file from hadoop ls command and it is present so can you tell me why here

> drill

> is showing block missing? Also you have mentioned "it could possibly mean

> problem with name node"; i have checked namenode is running fine. Also we

> are

> executing some hive queries on same cluster those are running fine so if

> it is

> namenode issue then i think it should affect all queries.

>

>

>

>

> On Mon, Mar 12, 2018 11:24 PM, Padma Penumarthy ppenumar...@mapr.com

> wrote:

> There can be lot of issues here.

>

> Connection loss error can happen when zookeeper thinks that a node is dead

> because

>

> it did not get heartbeat from the node. It can be because the node is busy

> or

> you have

>

> network problems. Did anything changed in your network ?

> Is the data static or are you adding new data ? Do you have metadata

> caching

> enabled ?

>

> PARQUET_WRITER seem to be indicate you are doing some kind of CTAS.

>

> The block missing exception could possibly mean some problem with name

> node or

> bad disks

>

> on one of the node.

>

>

>

> Thanks

>

> Padma

>

>

>

>

> On Mar 12, 2018, at 1:27 AM, Anup Tiwari <anup.tiw...@games24x7.com>

>> wrote:

>>

>

>

>>

> Hi All,

>>

>

> From last couple of days i am stuck in a problem. I have a query which left

>>

>

> joins 3 drill tables(parquet), everyday it is used to take around 15-20

>> mins

>>

> but

>

> from last couple of days it is taking more than 45 mins and when i tried to

>>

>

> drill down i can see in operator profile that 40% query time is going to

>>

>

> PARQUET_WRITER and 28% time in PARQUET_ROW_GROUP_SCAN. I am not sure if

>> before

>>

>

> this issue the stats were same or not as earlier it gets executed in 15-20

>> min

>>

>

> max.Also on top of this a table, we used to create a table which is now

>>

> showing

>

> below error :-

>>

>

> SYSTEM ERROR: BlockMissingException: Could not obtain block:

>>

>

> BP-1083556055-10.51.2.101-148327179:blk_1094763477_21022752

>>

>

> Also in last few days i am getting frequent one or more node lost

>> connectivity

>>

>

> error.

>>

>

> I just upgraded to Drill 1.12.0 from 1.10.0 but above issues are still

>> there.

>>

>

> Any help will be appreciated.

>>

>

> Regards,

>>

>

> Anup Tiwari

>>

>

>

>

>

>

>

>

>

>

> Regards,

> Anup Tiwari

>

>

> Regards,

> Anup Tiwari






Regards,
Anup Tiwari

Re: [Drill 1.12.0] : RESOURCE ERROR: Not enough memory for internal partitioning and fallback mechanism for HashAgg to use unbounded memory is disabled

[Anup Tiwari]

Hi Kunal,
Please find below link :-
https://drive.google.com/open?id=13NVDqSgDD-Pe6H0smAkvzqktgXURgZF4
SQL File contains platform details and log files contains success/failure logs
of query.  






On Wed, Mar 14, 2018 7:51 PM, Kunal Khatua ku...@apache.org  wrote:
Hi Anup




Can you share this as a file ? There seems to be some truncation of the

contents.




Share it using some online service like Google Drive or Dropbox, since the

mailing list might not allow for attachments.




Thanks




~ Kunal




On Tue, Mar 13, 2018 at 11:44 PM, Anup Tiwari <anup.tiw...@games24x7.com>

wrote:





JSON Profile when Succeeded :-







{"id":{"part1":2690693429455769721,"part2":65093823787227620



87},"type":1,"start":1521007764471,"end":1521007906770,"query":"create



table a_games_log_visit_utm as\nselect\ndistinct\nglv.sessionid,\ncase



when



(UFG('utms=', glv.url, '&') <> 'null') then UFG('utms=', glv.url, '&') else



UFG('utms=',glv.hcookie, '&') end as utms,\n\ncase when (UFG('utms=',



glv.url,



'&') <> 'null') then UFG('utmm=', glv.url, '&') else



UFG('utmm=',glv.hcookie,



'&') end as utmm,\n\ncase when (UFG('utms=', glv.url, '&') <> 'null') then



UFG('utmc=', glv.url, '&') else UFG('utmc=',glv.hcookie, '&') end as



utmc,\n\ncase when (UFG('utms=', glv.url, '&') <> 'null') then UFG('utmt=',



glv.url, '&') else UFG('utmt=',glv.hcookie, '&') end as utmt,\n\ncase when



(UFG('utms=', glv.url, '&') <> 'null') then UFG('utmcmpn=', glv.url, '&')



else



UFG('utmcmpn=',glv.hcookie, '&') end as utmcmpn,\n\ncase when (UFG('utms=',



glv.url, '&') <> 'null') then UFG('utmplnt=', glv.url, '&') else



UFG('utmplnt=',glv.hcookie, '&') end as utmplnt,\n\ncase when (UFG('utms=',



glv.url, '&') <> 'null') then UFG('uid=', glv.url, '&') else UFG('uid=',



glv.hcookie, '&') end as uid,\n\ncase when (UFG('utms=', glv.url, '&') <>



'null') then UFG('cid=', glv.url, '&') else UFG('cid=', glv.hcookie, '&')



end as



cid,\n\ncase when (UFG('utms=', glv.url, '&') <> 'null') then UFG('gclid=',



glv.url, '&') else UFG('gclid=', glv.hcookie, '&') end as



gclid\n\nfrom\nglv_utm_1 a,\na_games_log_visit_base



glv\nwhere\na.sessionid =



glv.sessionid\nand a.max_serverTime = glv.serverTime\nand



(STRPOS(glv.url,'utms=') > 0 OR STRPOS(glv.hcookie,'utms=') > 0) \nand



glv.url



is not null ","plan":"00-00 Screen : rowType = RecordType(VARCHAR(255)



Fragment,



BIGINT Number of records written): rowcount = 402043.18275, cumulative



cost =



{5.354375750937748E8 rows, 7.821369551257525E9 cpu, 0.0 io,



1.88842215292928E11



network, 4.283794390722E9 memory}, id = 7873881\n00-01



Project(Fragment=[$0],



Number of records written=[$1]) : rowType = RecordType(VARCHAR(255)



Fragment,



BIGINT Number of records written): rowcount = 402043.18275, cumulative



cost =



{5.353973707754998E8 rows, 7.82132934693925E9 cpu, 0.0 io,



1.88842215292928E11



network, 4.283794390722E9 memory}, id = 7873880\n00-02 UnionExchange :



rowType =



RecordType(VARCHAR(255) Fragment, BIGINT Number of records written):



rowcount =



402043.18275, cumulative cost = {5.353973707754998E8 rows,



7.82132934693925E9



cpu, 0.0 io, 1.88842215292928E11 network, 4.283794390722E9 memory}, id =



7873879\n01-01 Writer : rowType = RecordType(VARCHAR(255) Fragment, BIGINT



Number of records written): rowcount = 402043.18275, cumulative cost =



{5.3499532759274983E8 rows, 7.81811300147725E9 cpu, 0.0 io,



1.8554867753984E11



network, 4.283794390722E9 memory}, id = 7873878\n01-02



ProjectAllowDup(sessionid=[$0], utms=[$1], utmm=[$2], utmc=[$3],



utmt=[$4],



utmcmpn=[$5], utmplnt=[$6], uid=[$7], cid=[$8], gclid=[$9]) : rowType =



RecordType(ANY sessionid, ANY utms, ANY utmm, ANY utmc, ANY utmt, ANY



utmcmpn,



ANY utmplnt, ANY uid, ANY cid, ANY gclid): rowcount = 402043.18275,



cumulative



cost = {5.345932844085E8 rows, 7.8177109582945E9 cpu, 0.0 io,



1.8554867753984E11 network, 4.283794390722E9 memory}, id = 7873877\n01-03



Project(sessionid=[$0], utms=[$1], utmm=[$2], utmc=[$3], utmt=[$4],



utmcmpn=[$5], utmplnt=[$6], uid=[$7], cid=[$8], gclid=[$9]) : rowType =



RecordType(ANY sessionid, ANY utms, ANY utmm, ANY utmc, ANY utmt, ANY



utmcmpn,



ANY utmplnt, ANY uid, ANY cid, ANY gclid): rowcount = 402043.18275,



cumulative



cost = {5.345932844085E8 rows, 7.8177109582945E9 cpu, 0.0 io,



1.8554867753984E11 network, 4.283794390722E9 memory}, id = 7873876\n01-04



HashAgg(group=[{0, 1, 2, 3, 4, 5, 6, 7, 8, 9}]) : rowType = RecordType(ANY



sessionid, ANY utms, ANY utmm, ANY utmc, ANY utmt, ANY utmcmpn, ANY



utmplnt, ANY



uid, ANY cid, ANY gclid): rowcount = 402043.18275, cumulative co

Re: [Drill 1.10.0/1.12.0] Query Started Taking Time + frequent one or more node lost connectivity error

[Anup Tiwari]

Also i have observed one thing, the query which is taking time is creating
~30-40 fragments and 99.9% of record is getting written into only one
fragment.  






On Wed, Mar 14, 2018 1:37 PM, Anup Tiwari anup.tiw...@games24x7.com  wrote:
Hi Padma,
Please find my highlighted answer w.r.t. your question :-
Connection loss error can happen when zookeeper thinks that a node is dead
becauseit did not get heartbeat from the node. It can be because the node is
busy or you havenetwork problems. Q) Did anything changed in your network ? 
Answer : No. Also we cross verify Intra communication within nodes and its

working fine.

Q) Is the data static or are you adding new data ? Answer : Data is static.
Q) Do you have metadata caching enabled ?Answer : No.
PARQUET_WRITER seem to be indicate you are doing some kind of CTAS. : This is
correct, we are doing CTAS.
The block missing exception could possibly mean some problem with name node or
bad diskson one of the node. : There is no bad disk also when i checked that
file from hadoop ls command and it is present so can you tell me why here drill
is showing block missing? Also you have mentioned "it could possibly mean
problem with name node"; i have checked namenode is running fine. Also we are
executing some hive queries on same cluster those are running fine so if it is
namenode issue then i think it should affect all queries.  






On Mon, Mar 12, 2018 11:24 PM, Padma Penumarthy ppenumar...@mapr.com  wrote:
There can be lot of issues here.

Connection loss error can happen when zookeeper thinks that a node is dead
because

it did not get heartbeat from the node. It can be because the node is busy or
you have

network problems. Did anything changed in your network ? 


Is the data static or are you adding new data ? Do you have metadata caching
enabled ?

PARQUET_WRITER seem to be indicate you are doing some kind of CTAS.

The block missing exception could possibly mean some problem with name node or
bad disks

on one of the node. 





Thanks

Padma





On Mar 12, 2018, at 1:27 AM, Anup Tiwari <anup.tiw...@games24x7.com> wrote:







Hi All,



From last couple of days i am stuck in a problem. I have a query which left



joins 3 drill tables(parquet), everyday it is used to take around 15-20 mins

but


from last couple of days it is taking more than 45 mins and when i tried to



drill down i can see in operator profile that 40% query time is going to



PARQUET_WRITER and 28% time in PARQUET_ROW_GROUP_SCAN. I am not sure if before



this issue the stats were same or not as earlier it gets executed in 15-20 min



max.Also on top of this a table, we used to create a table which is now

showing


below error :-



SYSTEM ERROR: BlockMissingException: Could not obtain block:



BP-1083556055-10.51.2.101-148327179:blk_1094763477_21022752



Also in last few days i am getting frequent one or more node lost connectivity



error.



I just upgraded to Drill 1.12.0 from 1.10.0 but above issues are still there.



Any help will be appreciated.



Regards,



Anup Tiwari










Regards,
Anup Tiwari


Regards,
Anup Tiwari

Re: [Drill 1.10.0/1.12.0] Query Started Taking Time + frequent one or more node lost connectivity error

[Anup Tiwari]

Hi Padma,
Please find my highlighted answer w.r.t. your question :-
Connection loss error can happen when zookeeper thinks that a node is dead
becauseit did not get heartbeat from the node. It can be because the node is
busy or you havenetwork problems. Q) Did anything changed in your network ? 
Answer : No. Also we cross verify Intra communication within nodes and its

working fine.

Q) Is the data static or are you adding new data ? Answer : Data is static.
Q) Do you have metadata caching enabled ?Answer : No.
PARQUET_WRITER seem to be indicate you are doing some kind of CTAS. : This is
correct, we are doing CTAS.
The block missing exception could possibly mean some problem with name node or
bad diskson one of the node. : There is no bad disk also when i checked that
file from hadoop ls command and it is present so can you tell me why here drill
is showing block missing? Also you have mentioned "it could possibly mean
problem with name node"; i have checked namenode is running fine. Also we are
executing some hive queries on same cluster those are running fine so if it is
namenode issue then i think it should affect all queries.  






On Mon, Mar 12, 2018 11:24 PM, Padma Penumarthy ppenumar...@mapr.com  wrote:
There can be lot of issues here.

Connection loss error can happen when zookeeper thinks that a node is dead
because

it did not get heartbeat from the node. It can be because the node is busy or
you have

network problems. Did anything changed in your network ? 


Is the data static or are you adding new data ? Do you have metadata caching
enabled ?

PARQUET_WRITER seem to be indicate you are doing some kind of CTAS.

The block missing exception could possibly mean some problem with name node or
bad disks

on one of the node. 





Thanks

Padma





On Mar 12, 2018, at 1:27 AM, Anup Tiwari <anup.tiw...@games24x7.com> wrote:







Hi All,



From last couple of days i am stuck in a problem. I have a query which left



joins 3 drill tables(parquet), everyday it is used to take around 15-20 mins

but


from last couple of days it is taking more than 45 mins and when i tried to



drill down i can see in operator profile that 40% query time is going to



PARQUET_WRITER and 28% time in PARQUET_ROW_GROUP_SCAN. I am not sure if before



this issue the stats were same or not as earlier it gets executed in 15-20 min



max.Also on top of this a table, we used to create a table which is now

showing


below error :-



SYSTEM ERROR: BlockMissingException: Could not obtain block:



BP-1083556055-10.51.2.101-148327179:blk_1094763477_21022752



Also in last few days i am getting frequent one or more node lost connectivity



error.



I just upgraded to Drill 1.12.0 from 1.10.0 but above issues are still there.



Any help will be appreciated.



Regards,



Anup Tiwari










Regards,
Anup Tiwari

Re: [Drill 1.12.0] : RESOURCE ERROR: Not enough memory for internal partitioning and fallback mechanism for HashAgg to use unbounded memory is disabled

[Anup Tiwari]

JSON Profile when Succeeded :-

{"id":{"part1":2690693429455769721,"part2":6509382378722762087},"type":1,"start":1521007764471,"end":1521007906770,"query":"create
table a_games_log_visit_utm as\nselect\ndistinct\nglv.sessionid,\ncase when
(UFG('utms=', glv.url, '&') <> 'null') then UFG('utms=', glv.url, '&') else
UFG('utms=',glv.hcookie, '&') end as utms,\n\ncase when (UFG('utms=', glv.url,
'&') <> 'null') then UFG('utmm=', glv.url, '&') else UFG('utmm=',glv.hcookie,
'&') end as utmm,\n\ncase when (UFG('utms=', glv.url, '&') <> 'null') then
UFG('utmc=', glv.url, '&') else UFG('utmc=',glv.hcookie, '&') end as
utmc,\n\ncase when (UFG('utms=', glv.url, '&') <> 'null') then UFG('utmt=',
glv.url, '&') else UFG('utmt=',glv.hcookie, '&') end as utmt,\n\ncase when
(UFG('utms=', glv.url, '&') <> 'null') then UFG('utmcmpn=', glv.url, '&') else
UFG('utmcmpn=',glv.hcookie, '&') end as utmcmpn,\n\ncase when (UFG('utms=',
glv.url, '&') <> 'null') then UFG('utmplnt=', glv.url, '&') else
UFG('utmplnt=',glv.hcookie, '&') end as utmplnt,\n\ncase when (UFG('utms=',
glv.url, '&') <> 'null') then UFG('uid=', glv.url, '&') else UFG('uid=',
glv.hcookie, '&') end as uid,\n\ncase when (UFG('utms=', glv.url, '&') <>
'null') then UFG('cid=', glv.url, '&') else UFG('cid=', glv.hcookie, '&') end as
cid,\n\ncase when (UFG('utms=', glv.url, '&') <> 'null') then UFG('gclid=',
glv.url, '&') else UFG('gclid=', glv.hcookie, '&') end as
gclid\n\nfrom\nglv_utm_1 a,\na_games_log_visit_base glv\nwhere\na.sessionid =
glv.sessionid\nand a.max_serverTime = glv.serverTime\nand
(STRPOS(glv.url,'utms=') > 0 OR STRPOS(glv.hcookie,'utms=') > 0) \nand glv.url
is not null ","plan":"00-00 Screen : rowType = RecordType(VARCHAR(255) Fragment,
BIGINT Number of records written): rowcount = 402043.18275, cumulative cost =
{5.354375750937748E8 rows, 7.821369551257525E9 cpu, 0.0 io, 1.88842215292928E11
network, 4.283794390722E9 memory}, id = 7873881\n00-01 Project(Fragment=[$0],
Number of records written=[$1]) : rowType = RecordType(VARCHAR(255) Fragment,
BIGINT Number of records written): rowcount = 402043.18275, cumulative cost =
{5.353973707754998E8 rows, 7.82132934693925E9 cpu, 0.0 io, 1.88842215292928E11
network, 4.283794390722E9 memory}, id = 7873880\n00-02 UnionExchange : rowType =
RecordType(VARCHAR(255) Fragment, BIGINT Number of records written): rowcount =
402043.18275, cumulative cost = {5.353973707754998E8 rows, 7.82132934693925E9
cpu, 0.0 io, 1.88842215292928E11 network, 4.283794390722E9 memory}, id =
7873879\n01-01 Writer : rowType = RecordType(VARCHAR(255) Fragment, BIGINT
Number of records written): rowcount = 402043.18275, cumulative cost =
{5.3499532759274983E8 rows, 7.81811300147725E9 cpu, 0.0 io, 1.8554867753984E11
network, 4.283794390722E9 memory}, id = 7873878\n01-02
ProjectAllowDup(sessionid=[$0], utms=[$1], utmm=[$2], utmc=[$3], utmt=[$4],
utmcmpn=[$5], utmplnt=[$6], uid=[$7], cid=[$8], gclid=[$9]) : rowType =
RecordType(ANY sessionid, ANY utms, ANY utmm, ANY utmc, ANY utmt, ANY utmcmpn,
ANY utmplnt, ANY uid, ANY cid, ANY gclid): rowcount = 402043.18275, cumulative
cost = {5.345932844085E8 rows, 7.8177109582945E9 cpu, 0.0 io,
1.8554867753984E11 network, 4.283794390722E9 memory}, id = 7873877\n01-03
Project(sessionid=[$0], utms=[$1], utmm=[$2], utmc=[$3], utmt=[$4],
utmcmpn=[$5], utmplnt=[$6], uid=[$7], cid=[$8], gclid=[$9]) : rowType =
RecordType(ANY sessionid, ANY utms, ANY utmm, ANY utmc, ANY utmt, ANY utmcmpn,
ANY utmplnt, ANY uid, ANY cid, ANY gclid): rowcount = 402043.18275, cumulative
cost = {5.345932844085E8 rows, 7.8177109582945E9 cpu, 0.0 io,
1.8554867753984E11 network, 4.283794390722E9 memory}, id = 7873876\n01-04
HashAgg(group=[{0, 1, 2, 3, 4, 5, 6, 7, 8, 9}]) : rowType = RecordType(ANY
sessionid, ANY utms, ANY utmm, ANY utmc, ANY utmt, ANY utmcmpn, ANY utmplnt, ANY
uid, ANY cid, ANY gclid): rowcount = 402043.18275, cumulative cost =
{5.345932844085E8 rows, 7.8177109582945E9 cpu, 0.0 io, 1.8554867753984E11
network, 4.283794390722E9 memory}, id = 7873875\n01-05 Project(sessionid=[$0],
utms=[$1], utmm=[$2], utmc=[$3], utmt=[$4], utmcmpn=[$5], utmplnt=[$6],
uid=[$7], cid=[$8], gclid=[$9]) : rowType = RecordType(ANY sessionid, ANY utms,
ANY utmm, ANY utmc, ANY utmt, ANY utmcmpn, ANY utmplnt, ANY uid, ANY cid, ANY
gclid): rowcount = 4020431.827497, cumulative cost = {5.3057285258249986E8
rows, 7.496076412094501E9 cpu, 0.0 io, 1.8554867753984E11 network,
3.89461658982E9 memory}, id = 7873874\n01-06 HashToRandomExchange(dist0=[[$0]],
dist1=[[$1]], dist2=[[$2]], dist3=[[$3]], dist4=[[$4]], dist5=[[$5]],
dist6=[[$6]], dist7=[[$7]], dist8=[[$8]], dist9=[[$9]]) : rowType =
RecordType(ANY sessionid, ANY utms, ANY utmm, ANY utmc, ANY utmt, ANY utmcmpn,
ANY utmplnt, ANY uid, ANY cid, ANY gclid, ANY E_X_P_R_H_A_S_H_F_I_E_L_D):
rowcount = 4020431.827497, cumulative cost = {5.3057285258249986E8 rows,
7.496076412094501E9 cpu, 0.0 io, 1.8554867753984E11 network, 3.89461658982E9
memory}, id = 7873873\n02-01 

Re: [Drill 1.12.0] : RESOURCE ERROR: Not enough memory for internal partitioning and fallback mechanism for HashAgg to use unbounded memory is disabled

[Anup Tiwari]

Hi Kunal,
Please find below cluster/platform details :-
Number of Nodes : 5
RAM/Node : 32GBCore/Node : 8DRILL_MAX_DIRECT_MEMORY="20G"DRILL_HEAP="8G"DRILL
VERSION = 1.12.0HADOOP VERSION = 2.7.3ZOOKEEPER VERSION = 3.4.8(Installed in
Distributed Mode on 3 nodes)planner.memory.max_query_memory_per_node  = 4G
Please find below query which was giving error before setting "alter session set
`drill.exec.hashagg.fallback.enabled`=TRUE;" and its json profile for
success(variable is set) and failure(variable is not set). Please note this
query was working fine in Drill 1.10.0 in same environment.
Query :-
create table a_games_log_visit_utm asselectdistinctglv.sessionid,case when
(UFG('utms=', glv.url, '&') <> 'null') then UFG('utms=', glv.url, '&') else
UFG('utms=',glv.hcookie, '&') end as utms,
case when (UFG('utms=', glv.url, '&') <> 'null') then UFG('utmm=', glv.url, '&')
else UFG('utmm=',glv.hcookie, '&') end as utmm,
case when (UFG('utms=', glv.url, '&') <> 'null') then UFG('utmc=', glv.url, '&')
else UFG('utmc=',glv.hcookie, '&') end as utmc,
case when (UFG('utms=', glv.url, '&') <> 'null') then UFG('utmt=', glv.url, '&')
else UFG('utmt=',glv.hcookie, '&') end as utmt,
case when (UFG('utms=', glv.url, '&') <> 'null') then UFG('utmcmpn=', glv.url,
'&') else UFG('utmcmpn=',glv.hcookie, '&') end as utmcmpn,
case when (UFG('utms=', glv.url, '&') <> 'null') then UFG('utmplnt=', glv.url,
'&') else UFG('utmplnt=',glv.hcookie, '&') end as utmplnt,
case when (UFG('utms=', glv.url, '&') <> 'null') then UFG('uid=', glv.url, '&')
else UFG('uid=', glv.hcookie, '&') end as uid,
case when (UFG('utms=', glv.url, '&') <> 'null') then UFG('cid=', glv.url, '&')
else UFG('cid=', glv.hcookie, '&') end as cid,
case when (UFG('utms=', glv.url, '&') <> 'null') then UFG('gclid=', glv.url,
'&') else UFG('gclid=', glv.hcookie, '&') end as gclid
fromglv_utm_1 a,a_games_log_visit_base glvwherea.sessionid = glv.sessionidand
a.max_serverTime = glv.serverTimeand (STRPOS(glv.url,'utms=') > 0 OR
STRPOS(glv.hcookie,'utms=') > 0) and glv.url is not null ;

JSON Profile when Failed :-
{"id":{"part1":2690698323716856156,"part2":-8447970446519047466},"type":1,"start":1521006624202,"end":1521006722666,"query":"create
table a_games_log_visit_utm as\nselect\ndistinct\nglv.sessionid,\ncase when
(UFG('utms=', glv.url, '&') <> 'null') then UFG('utms=', glv.url, '&') else
UFG('utms=',glv.hcookie, '&') end as utms,\n\ncase when (UFG('utms=', glv.url,
'&') <> 'null') then UFG('utmm=', glv.url, '&') else UFG('utmm=',glv.hcookie,
'&') end as utmm,\n\ncase when (UFG('utms=', glv.url, '&') <> 'null') then
UFG('utmc=', glv.url, '&') else UFG('utmc=',glv.hcookie, '&') end as
utmc,\n\ncase when (UFG('utms=', glv.url, '&') <> 'null') then UFG('utmt=',
glv.url, '&') else UFG('utmt=',glv.hcookie, '&') end as utmt,\n\ncase when
(UFG('utms=', glv.url, '&') <> 'null') then UFG('utmcmpn=', glv.url, '&') else
UFG('utmcmpn=',glv.hcookie, '&') end as utmcmpn,\n\ncase when (UFG('utms=',
glv.url, '&') <> 'null') then UFG('utmplnt=', glv.url, '&') else
UFG('utmplnt=',glv.hcookie, '&') end as utmplnt,\n\ncase when (UFG('utms=',
glv.url, '&') <> 'null') then UFG('uid=', glv.url, '&') else UFG('uid=',
glv.hcookie, '&') end as uid,\n\ncase when (UFG('utms=', glv.url, '&') <>
'null') then UFG('cid=', glv.url, '&') else UFG('cid=', glv.hcookie, '&') end as
cid,\n\ncase when (UFG('utms=', glv.url, '&') <> 'null') then UFG('gclid=',
glv.url, '&') else UFG('gclid=', glv.hcookie, '&') end as
gclid\n\nfrom\nglv_utm_1 a,\na_games_log_visit_base glv\nwhere\na.sessionid =
glv.sessionid\nand a.max_serverTime = glv.serverTime\nand
(STRPOS(glv.url,'utms=') > 0 OR STRPOS(glv.hcookie,'utms=') > 0)\nand glv.url is
not null ","plan":"00-00 Screen : rowType = RecordType(VARCHAR(255) Fragment,
BIGINT Number of records written): rowcount = 402043.18275, cumulative cost =
{5.354375750937748E8 rows, 7.821369551257525E9 cpu, 0.0 io, 1.88842215292928E11
network, 4.283794390722E9 memory}, id = 7755274\n00-01 Project(Fragment=[$0],
Number of records written=[$1]) : rowType = RecordType(VARCHAR(255) Fragment,
BIGINT Number of records written): rowcount = 402043.18275, cumulative cost =
{5.353973707754998E8 rows, 7.82132934693925E9 cpu, 0.0 io, 1.88842215292928E11
network, 4.283794390722E9 memory}, id = 7755273\n00-02 UnionExchange : rowType =
RecordType(VARCHAR(255) Fragment, BIGINT Number of records written): rowcount =
402043.18275, cumulative cost = {5.353973707754998E8 rows, 7.82132934693925E9
cpu, 0.0 io, 1.88842215292928E11 network, 4.283794390722E9 memory}, id =
7755272\n01-01 Writer : rowType = RecordType(VARCHAR(255) Fragment, BIGINT
Number of records written): rowcount = 402043.18275, cumulative cost =
{5.3499532759274983E8 rows, 7.81811300147725E9 cpu, 0.0 io, 1.8554867753984E11
network, 4.283794390722E9 memory}, id = 7755271\n01-02
ProjectAllowDup(sessionid=[$0], utms=[$1], utmm=[$2], utmc=[$3], utmt=[$4],
utmcmpn=[$5], utmplnt=[$6], uid=[$7], cid=[$8], gclid=[$9]) : rowType =
RecordType(ANY 

Re: [1.9.0] : UserException: SYSTEM ERROR: IllegalReferenceCountException: refCnt: 0 and then SYSTEM ERROR: IOException: Failed to shutdown streamer

[Anup Tiwari]

Hi All,
We are getting "IllegalReferenceCountException" issue again in for few queries
from last 2 days and currently we are on Drill 1.12.0. Can anybody help me here
to understand what is the exact reason behind this?  






On Thu, Dec 14, 2017 4:52 PM, Anup Tiwari anup.tiw...@games24x7.com  wrote:
Hi Kunal,

Please find below answers to your question :-

1. Setup description :- 
Number of Nodes : 5

RAM/Node : 32GB
Core/Node : 8
DRILL_MAX_DIRECT_MEMORY="20G"
DRILL_HEAP="16G"

2. What queries were you running and against what kind of dataset :-  Same type
of queries as mentioned in trail mail and 
dataset :- Drill Tables created from Hive Parquet Table which is created from

Json Log Files.

3. How frequently is it occurring :- 2-3 times in a month.


Please find below Drill Logs  :-

[Error Id: e4cf470d-5aa8-4b9a-b8dd-d6201996cabe on host1:31010]
org.apache.drill.common.exceptions.UserException: SYSTEM ERROR:
IllegalReferenceCountException: refCnt: 0

Fragment 3:13

[Error Id: e4cf470d-5aa8-4b9a-b8dd-d6201996cabe on host1:31010]
at
org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:544)
~[drill-common-1.10.0.jar:1.10.0]
at
org.apache.drill.exec.work.fragment.FragmentExecutor.sendFinalState(FragmentExecutor.java:293)
[drill-java-exec-1.10.0.jar:1.10.0]
at
org.apache.drill.exec.work.fragment.FragmentExecutor.cleanup(FragmentExecutor.java:160)
[drill-java-exec-1.10.0.jar:1.10.0]
at
org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:262)
[drill-java-exec-1.10.0.jar:1.10.0]
at
org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38)
[drill-common-1.10.0.jar:1.10.0]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[na:1.8.0_72]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[na:1.8.0_72]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_72]
Caused by: io.netty.util.IllegalReferenceCountException: refCnt: 0
at io.netty.buffer.AbstractByteBuf.ensureAccessible(AbstractByteBuf.java:1178)
~[netty-buffer-4.0.27.Final.jar:4.0.27.Final]
at io.netty.buffer.DrillBuf.checkIndexD(DrillBuf.java:115)
~[drill-memory-base-1.10.0.jar:4.0.27.Final]
at io.netty.buffer.DrillBuf.checkBytes(DrillBuf.java:141)
~[drill-memory-base-1.10.0.jar:4.0.27.Final]
at
org.apache.drill.exec.expr.fn.impl.ByteFunctionHelpers.compare(ByteFunctionHelpers.java:99)
~[vector-1.10.0.jar:1.10.0]
at
org.apache.drill.exec.test.generated.ProjectorGen3570.doEval(ProjectorTemplate.java:187)
~[na:na]
at
org.apache.drill.exec.test.generated.ProjectorGen3570.projectRecords(ProjectorTemplate.java:67)
~[na:na]
at
org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.doWork(ProjectRecordBatch.java:199)
~[drill-java-exec-1.10.0.jar:1.10.0]
at
org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:93)
~[drill-java-exec-1.10.0.jar:1.10.0]
at
org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext(ProjectRecordBatch.java:135)
~[drill-java-exec-1.10.0.jar:1.10.0]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:162)
~[drill-java-exec-1.10.0.jar:1.10.0]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119)
~[drill-java-exec-1.10.0.jar:1.10.0]
at
org.apache.drill.exec.test.generated.HashAggregatorGen120.doWork(HashAggTemplate.java:312)
~[na:na]
at
org.apache.drill.exec.physical.impl.aggregate.HashAggBatch.innerNext(HashAggBatch.java:143)
~[drill-java-exec-1.10.0.jar:1.10.0]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:162)
~[drill-java-exec-1.10.0.jar:1.10.0]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119)
~[drill-java-exec-1.10.0.jar:1.10.0]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:109)
~[drill-java-exec-1.10.0.jar:1.10.0]
at
org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:51)
~[drill-java-exec-1.10.0.jar:1.10.0]
at
org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext(ProjectRecordBatch.java:135)
~[drill-java-exec-1.10.0.jar:1.10.0]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:162)
~[drill-java-exec-1.10.0.jar:1.10.0]
at org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:104)
~[drill-java-exec-1.10.0.jar:1.10.0]
at
org.apache.drill.exec.physical.impl.SingleSenderCreator$SingleSenderRootExec.innerNext(SingleSenderCreator.java:92)
~[drill-java-exec-1.10.0.jar:1.10.0]
at org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:94)
~[drill-java-exec-1.10.0.jar:1.10.0]
at
org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:232)
~[drill-java-exec-1.10.0.jar:1.10.0]
at
org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:226)
~[dri

Re: [Drill 1.12.0] : RESOURCE ERROR: Not enough memory for internal partitioning and fallback mechanism for HashAgg to use unbounded memory is disabled

[Anup Tiwari]

Hi Kunal,
I have executed below command and query got executed in 38.763 sec.
alter session set `drill.exec.hashagg.fallback.enabled`=TRUE;
Can you tell me what is the problems in setting this variable? Since you have
mentioned it will risk instability.  






On Mon, Mar 12, 2018 6:27 PM, Anup Tiwari anup.tiw...@games24x7.com  wrote:
Hi Kunal,
I am still getting this error for some other query and i have increased
planner.memory.max_query_memory_per_node variable from 2 GB to 10 GB on session
level but still getting this issue.
Can you tell me how this was getting handled in Earlier Drill Versions(<1.11.0)? 







On Mon, Mar 12, 2018 1:59 PM, Anup Tiwari anup.tiw...@games24x7.com  wrote:
Hi Kunal,
Thanks for info and i went with option 1 and increased
planner.memory.max_query_memory_per_node and now queries are working fine. Will
let you in case of any issues.  






On Mon, Mar 12, 2018 2:30 AM, Kunal Khatua ku...@apache.org  wrote:
Here is the background of your issue:

https://drill.apache.org/docs/sort-based-and-hash-based-memory-constrained-operators/#spill-to-disk




HashAgg introduced a Spill-to-disk capability in 1.11.0 that allows for

Drill to run a query's HashAgg in a memory constrained environment. The

memory required for the operator is based on the cumulative memory required

by the operator's minor fragments (I believe it is 32MB per fragment).




The message you get is because this total exceeds the calculated memory.

With this you have two options.




 1. Reduce the number of minor fragments such that the total is within

 the available memory, *or* increase the memory per query per node

 (planner.memory.max_query_memory_per_node).

 2. Set the fallback as *TRUE* (default is *FALSE*) and let the operator

 run with unconstrained memory

 (i.e. `planner.memory.max_query_memory_per_node` is not honoured)




My recommendation is to go with #1. Going with #2 will risk instability

which is worse than a query failing IMHO.













On Sun, Mar 11, 2018 at 11:56 AM, Anup Tiwari <anup.tiw...@games24x7.com>

wrote:





Hi All,



I recently upgraded from 1.10.0 to 1.12.0 and in my one of query I got



below



error :-



INFO o.a.d.e.p.i.aggregate.HashAggregator - User Error Occurred: Not



enough



memory for internal partitioning and fallback mechanism for HashAgg to use



unbounded memory is disabled. Either enable fallback config



drill.exec.hashagg.fallback.enabled using Alter session/system command or



increase memory limit for Drillbit



org.apache.drill.common.exceptions.UserException: RESOURCE ERROR: Not



enough



memory for internal partitioning and fallback mechanism for HashAgg to use



unbounded memory is disabled. Either enable fallback config



drill.exec.hashagg.fallback.enabled using Alter session/system command or



increase memory limit for Drillbit







Can anybody tell me working of "drill.exec.hashagg.fallback.enabled"



variable.



Should we always set it to true as it is false by default?



Regards,



Anup Tiwari






Regards,
Anup Tiwari



Regards,
Anup Tiwari


Regards,
Anup Tiwari

Re: [Drill 1.12.0] : RESOURCE ERROR: Not enough memory for internal partitioning and fallback mechanism for HashAgg to use unbounded memory is disabled

[Anup Tiwari]

Hi Kunal,
I am still getting this error for some other query and i have increased
planner.memory.max_query_memory_per_node variable from 2 GB to 10 GB on session
level but still getting this issue.
Can you tell me how this was getting handled in Earlier Drill Versions(<1.11.0)? 







On Mon, Mar 12, 2018 1:59 PM, Anup Tiwari anup.tiw...@games24x7.com  wrote:
Hi Kunal,
Thanks for info and i went with option 1 and increased
planner.memory.max_query_memory_per_node and now queries are working fine. Will
let you in case of any issues.  






On Mon, Mar 12, 2018 2:30 AM, Kunal Khatua ku...@apache.org  wrote:
Here is the background of your issue:

https://drill.apache.org/docs/sort-based-and-hash-based-memory-constrained-operators/#spill-to-disk




HashAgg introduced a Spill-to-disk capability in 1.11.0 that allows for

Drill to run a query's HashAgg in a memory constrained environment. The

memory required for the operator is based on the cumulative memory required

by the operator's minor fragments (I believe it is 32MB per fragment).




The message you get is because this total exceeds the calculated memory.

With this you have two options.




 1. Reduce the number of minor fragments such that the total is within

 the available memory, *or* increase the memory per query per node

 (planner.memory.max_query_memory_per_node).

 2. Set the fallback as *TRUE* (default is *FALSE*) and let the operator

 run with unconstrained memory

 (i.e. `planner.memory.max_query_memory_per_node` is not honoured)




My recommendation is to go with #1. Going with #2 will risk instability

which is worse than a query failing IMHO.













On Sun, Mar 11, 2018 at 11:56 AM, Anup Tiwari <anup.tiw...@games24x7.com>

wrote:





Hi All,



I recently upgraded from 1.10.0 to 1.12.0 and in my one of query I got



below



error :-



INFO o.a.d.e.p.i.aggregate.HashAggregator - User Error Occurred: Not



enough



memory for internal partitioning and fallback mechanism for HashAgg to use



unbounded memory is disabled. Either enable fallback config



drill.exec.hashagg.fallback.enabled using Alter session/system command or



increase memory limit for Drillbit



org.apache.drill.common.exceptions.UserException: RESOURCE ERROR: Not



enough



memory for internal partitioning and fallback mechanism for HashAgg to use



unbounded memory is disabled. Either enable fallback config



drill.exec.hashagg.fallback.enabled using Alter session/system command or



increase memory limit for Drillbit







Can anybody tell me working of "drill.exec.hashagg.fallback.enabled"



variable.



Should we always set it to true as it is false by default?



Regards,



Anup Tiwari






Regards,
Anup Tiwari



Regards,
Anup Tiwari

Re: [Drill 1.12.0] : RESOURCE ERROR: Not enough memory for internal partitioning and fallback mechanism for HashAgg to use unbounded memory is disabled

[Anup Tiwari]

Hi Kunal,
Thanks for info and i went with option 1 and increased
planner.memory.max_query_memory_per_node and now queries are working fine. Will
let you in case of any issues.  






On Mon, Mar 12, 2018 2:30 AM, Kunal Khatua ku...@apache.org  wrote:
Here is the background of your issue:

https://drill.apache.org/docs/sort-based-and-hash-based-memory-constrained-operators/#spill-to-disk




HashAgg introduced a Spill-to-disk capability in 1.11.0 that allows for

Drill to run a query's HashAgg in a memory constrained environment. The

memory required for the operator is based on the cumulative memory required

by the operator's minor fragments (I believe it is 32MB per fragment).




The message you get is because this total exceeds the calculated memory.

With this you have two options.




 1. Reduce the number of minor fragments such that the total is within

 the available memory, *or* increase the memory per query per node

 (planner.memory.max_query_memory_per_node).

 2. Set the fallback as *TRUE* (default is *FALSE*) and let the operator

 run with unconstrained memory

 (i.e. `planner.memory.max_query_memory_per_node` is not honoured)




My recommendation is to go with #1. Going with #2 will risk instability

which is worse than a query failing IMHO.













On Sun, Mar 11, 2018 at 11:56 AM, Anup Tiwari <anup.tiw...@games24x7.com>

wrote:





Hi All,



I recently upgraded from 1.10.0 to 1.12.0 and in my one of query I got



below



error :-



INFO o.a.d.e.p.i.aggregate.HashAggregator - User Error Occurred: Not



enough



memory for internal partitioning and fallback mechanism for HashAgg to use



unbounded memory is disabled. Either enable fallback config



drill.exec.hashagg.fallback.enabled using Alter session/system command or



increase memory limit for Drillbit



org.apache.drill.common.exceptions.UserException: RESOURCE ERROR: Not



enough



memory for internal partitioning and fallback mechanism for HashAgg to use



unbounded memory is disabled. Either enable fallback config



drill.exec.hashagg.fallback.enabled using Alter session/system command or



increase memory limit for Drillbit







Can anybody tell me working of "drill.exec.hashagg.fallback.enabled"



variable.



Should we always set it to true as it is false by default?



Regards,



Anup Tiwari






Regards,
Anup Tiwari

[Drill 1.10.0/1.12.0] Query Started Taking Time + frequent one or more node lost connectivity error

[Anup Tiwari]

Hi All,

From last couple of days i am stuck in a problem. I have a query which left

joins 3 drill tables(parquet), everyday it is used to take around 15-20 mins but
from last couple of days it is taking more than 45 mins and when i tried to
drill down i can see in operator profile that 40% query time is going to
PARQUET_WRITER and 28% time in PARQUET_ROW_GROUP_SCAN. I am not sure if before
this issue the stats were same or not as earlier it gets executed in 15-20 min
max.Also on top of this a table, we used to create a table which is now showing
below error :-
SYSTEM ERROR: BlockMissingException: Could not obtain block:
BP-1083556055-10.51.2.101-148327179:blk_1094763477_21022752
Also in last few days i am getting frequent one or more node lost connectivity
error.
I just upgraded to Drill 1.12.0 from 1.10.0 but above issues are still there.
Any help will be appreciated.
Regards,
Anup Tiwari

[Drill 1.12.0] : RESOURCE ERROR: Not enough memory for internal partitioning and fallback mechanism for HashAgg to use unbounded memory is disabled

[Anup Tiwari]

Hi All,
I recently upgraded from 1.10.0 to 1.12.0 and in my one of query I got below
error :-
INFO o.a.d.e.p.i.aggregate.HashAggregator - User Error Occurred: Not enough
memory for internal partitioning and fallback mechanism for HashAgg to use
unbounded memory is disabled. Either enable fallback config
drill.exec.hashagg.fallback.enabled using Alter session/system command or
increase memory limit for Drillbit
org.apache.drill.common.exceptions.UserException: RESOURCE ERROR: Not enough
memory for internal partitioning and fallback mechanism for HashAgg to use
unbounded memory is disabled. Either enable fallback config
drill.exec.hashagg.fallback.enabled using Alter session/system command or
increase memory limit for Drillbit

Can anybody tell me working of "drill.exec.hashagg.fallback.enabled" variable.
Should we always set it to true as it is false by default?
Regards,
Anup Tiwari

Re: Code too large

[Anup Tiwari]

Hi Khurram/Arjun,
Anyone got time to look into it?  





On Fri, Feb 16, 2018 4:53 PM, Anup Tiwari anup.tiw...@games24x7.com  wrote:
Hi Arjun,
After posting this reply ; i have found the same answer on net and that
parameter to 30 and then query worked but it took bit more time than expected.
Also don't you think these type of things should be adjusted automatically?
@khurram, Please find below query and logs(since log is huge in drillbit.log for
this query so i have divided logs into 3 parts in an order which i got for the
query - error + some drill code(which was too large) + error). FYI : hive.cs_all
is a hive(2.1.1) parquet table.
Query :-
create table dfs.tmp.cs_all_test AS select log_date,ssid ,select log_date,ssid ,
count((case when ((id like 'cta-action-%' and event = 'click' and sit = 'pnow'
and ptype = '1' and stype = '1') OR (id like '1:100%' and event = 'pnowclick'
and STRPOS(url,'mrc/player/sit.html') > 0) OR (id like '/fmg/110%/pn/pnow.html'
or (id like '110%/fmgopt/pnow'))) then ssid end)) as pnow_prac_c , count((case
when ((id like 'btsit%' and event = 'click' and sit like '%TSit%' and ptype1 =
'1' and stype1 = '1') OR (event = 'ts.click' and id like '1:100%') OR (id like
'/mgems/over/110%/ts.html')) then ssid end)) as ts_prac_c , count((case when
((id = '/res/vinit/' and mptype = '1' and (mgtype = 'cfp' OR mftype = '100')) OR
(id like '/dl%/fmg/110%/pn/ftpracga/vinit.html' or id like
'/dl%/fmg/110%/pn/vinit.html')) then ssid end)) as vinit_prac_c , count((case
when (id = '/res/tiu/' and mptype = '1' and (mgtype = 'cfp' OR mftype = '100'))
then ssid end)) as tiu_prac_c , count((case when (id = '/onstClick/btnStHr/' and
event='click' and mptype = '1' and (mgtype = 'cfp' OR mftype = '100')) then ssid
end)) as StHr_prac_c , count((case when ((id = '/res/dcd/' and mptype = '1' and
(mgtype = 'cfp' OR mftype = '100')) OR (id like
'/dl%/fmg/110%/pn/ftpracga/dcd.html' or id like '/dl%/fmg/110%/pn/dcd.html'))
then ssid end)) as dcd_prac_c , count((case when ((id = '/confirmdr/btnY/' and
event in ('click','Click') and mptype = '1' and (mgtype = 'cfp' OR mftype =
'100')) OR (id like '/dl%/fmg/110%/pn/dr.html')) then ssid end)) as dr_prac_c ,
count((case when ((id = '/res/finish/' and mptype = '1' and (mgtype = 'cfp' OR
mftype = '100')) OR (id like '/dl%/fmg/110%/pn/ftpracga/finish.html' or id like
'/dl%/fmg/110%/pn/finish.html')) then ssid end)) as finish_prac_c , count((case
when ((id like 'cta-action-%' and event = 'click' and sit = 'pnow' and ptype =
'2' and stype = '1') OR (id like '2:100%' and event = 'pnowclick' and
STRPOS(url,'mrc/player/sit.html') > 0) OR (id like '/fmg/210%/pn/pnow.html' or
(id like '210%/fmgopt/pnow'))) then ssid end)) as pnow_cash_c , count((case when
(id like '2:100%' and event = 'pnowclick' and STRPOS(url,'mrc/player/sit.html')
= 0) then ssid end)) as pnow_cash_c_pac , count((case when ((id like 'btsit%'
and event = 'click' and sit like '%TSit%' and ptype1 = '2' and stype1 = '1') OR
(event = 'ts.click' and id like '2:100%') OR (id like
'/mgems/over/210%/ts.html')) then ssid end)) as ts_cash_c , count((case when
((id = '/res/vinit/' and mptype = '2' and (mgtype = 'cfp' OR mftype = '100')) OR
(id like '/dl%/fmg/210%/pn/ftpracga/vinit.html' or id like
'/dl%/fmg/210%/pn/vinit.html')) then ssid end)) as vinit_cash_c , count((case
when (id = '/res/tiu/' and mptype = '2' and (mgtype = 'cfp' OR mftype = '100'))
then ssid end)) as tiu_cash_c , count((case when (id = '/onstClick/btnStHr/' and
event='click' and mptype = '2' and (mgtype = 'cfp' OR mftype = '100')) then ssid
end)) as StHr_cash_c , count((case when ((id = '/res/dcd/' and mptype = '2' and
(mgtype = 'cfp' OR mftype = '100')) OR (id like
'/dl%/fmg/210%/pn/ftpracga/dcd.html' or id like '/dl%/fmg/210%/pn/dcd.html'))
then ssid end)) as dcd_cash_c , count((case when ((id = '/confirmdr/btnY/' and
event in ('click','Click') and mptype = '2' and (mgtype = 'cfp' OR mftype =
'100')) OR (id like '/dl%/fmg/210%/pn/dr.html')) then ssid end)) as dr_cash_c ,
count((case when ((id = '/res/finish/' and mptype = '2' and (mgtype = 'cfp' OR
mftype = '100')) OR (id like '/dl%/fmg/210%/pn/ftpracga/finish.html' or id like
'/dl%/fmg/210%/pn/finish.html')) then ssid end)) as finish_cash_c , count((case
when event = 'event.ajax' and ajaxUrl = '/pyr/ac/change-uname' and ajaxResponse
like '%validationResponse%true%updateResponse%true%' then ssid end)) as
changeunamesuccess , count((case when unameautoassign = 'true' then ssid end))
as unameautoassign , count((case when (id = '/res/cdsdlt/' or id like
'/dl%/fmg/210%/pn/cdsdlt.html%') and mptype = '2' and (mgtype = 'cfp' OR mftype
= '100') then ssid end)) as cds_dlt_cfp_cash ,
count((case when ((id like 'cta-action-%' and event = 'click' and sit = 'pnow'
and ptype = 1 and stype = 2) OR (id like '1:20%' and event = 'pnowclick' and
STRPOS(url,'mrc/player/sit.html') > 0) OR (id like '/fmg/120%/pn/pnow.html' or
(id like '120%/fmgopt/pnow'))) then ssid end)) as pnow_prac_pl , count((case
when 

Re: Code too large

[Anup Tiwari]

 too largepublic
void doEval(int inIndex, int outIndex)^
(compiler.err.limit.code) at
org.apache.drill.exec.compile.DrillDiagnosticListener.report(DrillDiagnosticListener.java:43)
~[drill-java-exec-1.11.0.jar:1.11.0] at
com.sun.tools.javac.api.ClientCodeWrapper$WrappedDiagnosticListener.report(ClientCodeWrapper.java:559)
~[jdk.tools-1.7.jar:na] at
com.sun.tools.javac.util.Log.writeDiagnostic(Log.java:428)
~[jdk.tools-1.7.jar:na] at com.sun.tools.javac.util.Log.report(Log.java:416)
~[jdk.tools-1.7.jar:na] at
com.sun.tools.javac.util.AbstractLog.error(AbstractLog.java:94)
~[jdk.tools-1.7.jar:na] at
com.sun.tools.javac.jvm.Code.checkLimits(Code.java:95) ~[jdk.tools-1.7.jar:na]
 at com.sun.tools.javac.jvm.Gen.genMethod(Gen.java:948) ~[jdk.tools-1.7.jar:na]
 at com.sun.tools.javac.jvm.Gen.visitMethodDef(Gen.java:885)
~[jdk.tools-1.7.jar:na] at
com.sun.tools.javac.tree.JCTree$JCMethodDecl.accept(JCTree.java:669)
~[jdk.tools-1.7.jar:na] at com.sun.tools.javac.jvm.Gen.genDef(Gen.java:684)
~[jdk.tools-1.7.jar:na] at com.sun.tools.javac.jvm.Gen.genClass(Gen.java:2235)
~[jdk.tools-1.7.jar:na] at
com.sun.tools.javac.main.JavaCompiler.genCode(JavaCompiler.java:712)
~[jdk.tools-1.7.jar:na] at
com.sun.tools.javac.main.JavaCompiler.generate(JavaCompiler.java:1451)
~[jdk.tools-1.7.jar:na] at
com.sun.tools.javac.main.JavaCompiler.generate(JavaCompiler.java:1419)
~[jdk.tools-1.7.jar:na] at
com.sun.tools.javac.main.JavaCompiler.compile2(JavaCompiler.java:870)
~[jdk.tools-1.7.jar:na] at
com.sun.tools.javac.main.JavaCompiler.compile(JavaCompiler.java:829)
~[jdk.tools-1.7.jar:na] at com.sun.tools.javac.main.Main.compile(Main.java:439)
~[jdk.tools-1.7.jar:na] at
com.sun.tools.javac.api.JavacTaskImpl.call(JavacTaskImpl.java:132)
~[jdk.tools-1.7.jar:na] at
org.apache.drill.exec.compile.JDKClassCompiler.doCompile(JDKClassCompiler.java:82)
~[drill-java-exec-1.11.0.jar:1.11.0] at
org.apache.drill.exec.compile.JDKClassCompiler.getByteCode(JDKClassCompiler.java:64)
~[drill-java-exec-1.11.0.jar:1.11.0] at
org.apache.drill.exec.compile.AbstractClassCompiler.getClassByteCode(AbstractClassCompiler.java:55)
~[drill-java-exec-1.11.0.jar:1.11.0] at
org.apache.drill.exec.compile.ClassCompilerSelector.getClassByteCode(ClassCompilerSelector.java:123)
~[drill-java-exec-1.11.0.jar:1.11.0] at
org.apache.drill.exec.compile.QueryClassLoader.getClassByteCode(QueryClassLoader.java:90)
~[drill-java-exec-1.11.0.jar:1.11.0] at
org.apache.drill.exec.compile.ClassTransformer.getImplementationClass(ClassTransformer.java:243)
~[drill-java-exec-1.11.0.jar:1.11.0] ... 42 common frames omitted






On Thu, Feb 15, 2018 2:21 AM, Arjun kr arjun...@outlook.com  wrote:
Hi Anup,







You may try setting configuration option 'exec.java.compiler.exp_in_method_size'
to lower value from default of 50 and run the query to see if it helps. Even
lowering to a value of 1 doesn't help, the query details and stack trace may
helpful for analysis as Khurram mentioned.







alter session set `exec.java.compiler.exp_in_method_size` = ;










Refer doc for more details on this configuration property:
https://drill.apache.org/docs/configuration-options-introduction/







Thanks,







Arjun









From: Khurram Faraaz <kfar...@mapr.com>

Sent: Thursday, February 15, 2018 12:55 AM

To: user@drill.apache.org

Subject: Re: Code too large




Can you please share the exact query as is (without formatting) here. And the
error that you see on Drill 1.11.0, so we can try and repro it.







Thanks,




Khurram






From: Anup Tiwari <anup.tiw...@games24x7.com>

Sent: Wednesday, February 14, 2018 3:14:01 AM

To: user@drill.apache.org

Subject: Re: Code too large




Hi Team,

Do we have any updates on this? It is coming even in Drill 1.11.0 and i tried

same query on hive, it is working perfectly.
















On Fri, Mar 25, 2016 2:30 AM, Jacques Nadeau jacq...@dremio.com wrote:

Looking at the trace it looks like Drill is correctly using the JDK




compiler which is good. However, it looks like we're not breaking apart the




expressions into submethods. @Aditya, since you worked on DRILL-1155, do




you have any sense to why we're not subdividing the code in this situation?













If I recall, we said we would only include 100 expressions per project




method. Or are we hitting a limit of fields or some other limit?













We test large projects here:













https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_apache_drill_blob_master_exec_java-2Dexec_src_test_java_org_apache_drill_exec_compile_TestLargeFileCompilation.java=DwIFAg=cskdkSMqhcnjZxdQVpwTXg=H5JEl9vb-mBIjic10QAbDD2vkUUKAxjO6wZO322RtdI=7Q3xDEBNDJpTxOv5CZjQjvw57jo8TxbjdDjkojonXp8=uiwsx3NwkuLNc6GpAJfcs0KPM0FQcNoAt4OD7TcMr8w=

[https://avatars3.githubusercontent.com/u/47359?s=400=4]<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_apache_drill_blob_master_exec_java-2Dexec_src_test_java_org_apache_drill

Re: Code too large

[Anup Tiwari]

THEN NULL ELSE



> > TO_DATE(columns[192], 'MMdd') END as `PRCDR_DT6`,



> > NULLIF(columns[193],'') as `ICD_PRCDR_CD7`, NULLIF(columns[194],'')



as



> > `ICD_PRCDR_VRSN_CD7`, CASE WHEN columns[195] =3D '' THEN NULL ELSE



> > TO_DATE(columns[195], 'MMdd') END as `PRCDR_DT7`,



> > NULLIF(columns[196],'') as `ICD_PRCDR_CD8`, NULLIF(columns[197],'')



as



> > `ICD_PRCDR_VRSN_CD8`, CASE WHEN columns[198] =3D '' THEN NULL ELSE



> > TO_DATE(columns[198], 'MMdd') END as `PRCDR_DT8`,



> > NULLIF(columns[199],'') as `ICD_PRCDR_CD9`, NULLIF(columns[200],'')



as



> > `ICD_PRCDR_VRSN_CD9`, CASE WHEN columns[201] =3D '' THEN NULL ELSE



> > TO_DATE(columns[201], 'MMdd') END as `PRCDR_DT9`,



> > NULLIF(columns[202],'') as `ICD_PRCDR_CD10`, NULLIF(columns[203],'')



as



> > `ICD_PRCDR_VRSN_CD10`, CASE WHEN columns[204] =3D '' THEN NULL ELSE



> > TO_DATE(columns[204], 'MMdd') END as `PRCDR_DT10`,



> > NULLIF(columns[205],'') as `ICD_PRCDR_CD11`, NULLIF(columns[206],'')



as



> > `ICD_PRCDR_VRSN_CD11`, CASE WHEN columns[207] =3D '' THEN NULL ELSE



> > TO_DATE(columns[207], 'MMdd') END as `PRCDR_DT11`,



> > NULLIF(columns[208],'') as `ICD_PRCDR_CD12`,



> > NULLIF(columns[209],'') as `ICD_PRCDR_VRSN_CD12`, CASE WHEN



columns[210]



> =



> > =3D



> > '' THEN NULL ELSE TO_DATE(columns[210], 'MMdd') END as



`PRCDR_DT12`,



> > NULLIF(columns[211],'') as `ICD_PRCDR_CD13`, NULLIF(columns[212],'')



as



> > `ICD_PRCDR_VRSN_CD13`, CASE WHEN columns[213] =3D '' THEN NULL ELSE



> > TO_DATE(columns[213], 'MMdd') END as `PRCDR_DT13`,



> > NULLIF(columns[214],'') as `ICD_PRCDR_CD14`, NULLIF(columns[215],'')



as



> > `ICD_PRCDR_VRSN_CD14`, CASE WHEN columns[216] =3D '' THEN NULL ELSE



> > TO_DATE(columns[216], 'MMdd') END as `PRCDR_DT14`,



> > NULLIF(columns[217],'') as `ICD_PRCDR_CD15`, NULLIF(columns[218],'')



as



> > `ICD_PRCDR_VRSN_CD15`, CASE WHEN columns[219] =3D '' THEN NULL ELSE



> > TO_DATE(columns[219], 'MMdd') END as `PRCDR_DT15`,



> > NULLIF(columns[220],'') as `ICD_PRCDR_CD16`, NULLIF(columns[221],'')



as



> > `ICD_PRCDR_VRSN_CD16`, CASE WHEN columns[222] =3D '' THEN NULL ELSE



> > TO_DATE(columns[222], 'MMdd') END as `PRCDR_DT16`,



> > NULLIF(columns[223],'') as `ICD_PRCDR_CD17`, NULLIF(columns[224],'')



as



> > `ICD_PRCDR_VRSN_CD17`, CASE WHEN columns[225] =3D '' THEN NULL ELSE



> > TO_DATE(columns[225], 'MMdd') END as `PRCDR_DT17`,



> > NULLIF(columns[226],'') as `ICD_PRCDR_CD18`, NULLIF(columns[227],'')



as



> > `ICD_PRCDR_VRSN_CD18`, CASE WHEN columns[228] =3D '' THEN NULL ELSE



> > TO_DATE(columns[228], 'MMdd') END as `PRCDR_DT18`,



> > NULLIF(columns[229],'') as `ICD_PRCDR_CD19`, NULLIF(columns[230],'')



as



> > `ICD_PRCDR_VRSN_CD19`, CASE WHEN columns[231] =3D '' THEN NULL ELSE



> > TO_DATE(columns[231], 'MMdd') END as `PRCDR_DT19`,



> > NULLIF(columns[232],'') as `ICD_PRCDR_CD20`,



> > NULLIF(columns[233],'') as `ICD_PRCDR_VRSN_CD20`, CASE WHEN



columns[234]



> =



> > =3D



> > '' THEN NULL ELSE TO_DATE(columns[234], 'MMdd') END as



`PRCDR_DT20`,



> > NULLIF(columns[235],'') as `ICD_PRCDR_CD21`, NULLIF(columns[236],'')



as



> > `ICD_PRCDR_VRSN_CD21`, CASE WHEN columns[237] =3D '' THEN NULL ELSE



> > TO_DATE(columns[237], 'MMdd') END as `PRCDR_DT21`,



> > NULLIF(columns[238],'') as `ICD_PRCDR_CD22`, NULLIF(columns[239],'')



as



> > `ICD_PRCDR_VRSN_CD22`, CASE WHEN columns[240] =3D '' THEN NULL ELSE



> > TO_DATE(columns[240], 'MMdd') END as `PRCDR_DT22`,



> > NULLIF(columns[241],'') as `ICD_PRCDR_CD23`, NULLIF(columns[242],'')



as



> > `ICD_PRCDR_VRSN_CD23`,



> > CASE WHEN columns[243] =3D '' THEN NULL ELSE TO_DATE(columns[243],



> > 'MMdd') END as `PRCDR_DT23`, NULLIF(columns[244],'') as



> > `ICD_PRCDR_CD24`, NULLIF(columns[245],'') as `ICD_PRCDR_VRSN_CD24`,



CASE



> > WHEN columns[246] =3D '' THEN NULL ELSE TO_DATE(columns[246],



'MMdd')



> > E=



> > ND



> > as `PRCDR_DT24`, NULLIF(columns[247],'') as `ICD_PRCDR_CD25`,



> > NULLIF(columns[248],'') as `ICD_PRCDR_VRSN_CD25`, CASE WHEN



columns[249]



> =



> > =3D



> > '' THEN NULL ELSE TO_DATE(columns[249], 'MMdd') END as



`PRCDR_DT25`,



> > CASE WHEN columns[250] =3D '' THEN NULL ELSE CAST(columns[250] as



DOUBLE)



> > E=



> > ND



> > as `DOB_DT`, NULLIF(columns[251],'') as `GNDR_CD`,



> NULLIF(columns[252],'')



> > as `RACE_CD`, NULLIF(columns[253],'') as `CNTY_CD`,



> NULLIF(columns[254],'')



> > as `STATE_CD`,



> > NULLIF(columns[255],'') as `CWF_BENE_MDCR_STUS_CD`



> > FROM cms.`blair`.`ALL_IP_OS.csv`



> > WHERE columns[58] =3D '70583' OR columns[62] =3D '70583' OR



columns[66]



> =3D



> > '70583' ;



> >



> >



> > On Thu, Mar 24, 2016 at 9:22 AM, Jacques Nadeau <jacq...@dremio.com>



> > wrote:



> >



> > > It would also good to get the full stack trace. Do you have jdk or



only



> > Joe



> > > on these machines?



> > > On Mar 24, 2016 5:27 AM, "Edmon Begoli" <ebeg...@gmail.com> wrote:



> > >



> > > > Does anyone know what might be causing this exception:



> > > >



> > > > *Error: SYSTEM ERROR: CompileException: File



> > > >



> >



'org.apache.drill.exec.compile.DrillJavaFileObject[ProjectorGen10.java]',



> > > > Line 7275, Column 17: ProjectorGen10.java:7275: error: code too



> large*



> > > >



> > > > * public void doEval(int inIndex, int outIndex)*



> > > >



> > > > * ^ (compiler.err.limit.code)*



> > > >



> > > >



> > > > *Fragment 0:0*



> > > >



> > > >



> > > > *[Error Id: 687009ec-4d55-443a-9066-218fb3ac8adb on



localhost:31010]



> > > > (state=,code=0)*



> > > >



> > >



> >



>























Regards,
Anup Tiwari

Re: Reading drill(1.10.0) created parquet table in hive(2.1.1) using external table

[Anup Tiwari]

at
org.apache.http.impl.conn.PoolingClientConnectionManager.leaseConnection(PoolingClientConnectionManager.java:232)
~[httpclient-4.2.5.jar:4.2.5]at
org.apache.http.impl.conn.PoolingClientConnectionManager$1.getConnection(PoolingClientConnectionManager.java:199)
~[httpclient-4.2.5.jar:4.2.5]at
sun.reflect.GeneratedMethodAccessor57.invoke(Unknown Source) ~[na:na]at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
~[na:1.8.0_72]at java.lang.reflect.Method.invoke(Method.java:498)
~[na:1.8.0_72]at
com.amazonaws.http.conn.ClientConnectionRequestFactory$Handler.invoke(ClientConnectionRequestFactory.java:70)
~[aws-java-sdk-1.7.4.jar:na]at
com.amazonaws.http.conn.$Proxy79.getConnection(Unknown Source) ~[na:na]
at
org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:456)
~[httpclient-4.2.5.jar:4.2.5]at
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
~[httpclient-4.2.5.jar:4.2.5]at
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
~[httpclient-4.2.5.jar:4.2.5]at
com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:384)
~[aws-java-sdk-1.7.4.jar:na]... 25 common frames omitted

Note :-  The parquet file which i want to access contains 43 column with all
column are of the below type :-"optional binary col1 (UTF8);" except one which
is "optional int32 col2";  





On Tue, Feb 13, 2018 10:59 PM, Kunal Khatua kkha...@mapr.com  wrote:
Can you share what the error is? Without that, it is anybody's guess on what the
issue is.




-Original Message-

From: Anup Tiwari [mailto:anup.tiw...@games24x7.com] 

Sent: Tuesday, February 13, 2018 6:19 AM

To: user@drill.apache.org

Subject: Reading drill(1.10.0) created parquet table in hive(2.1.1) using
external table




Hi Team,

I am trying to read drill(1.10.0) created parquet table in hive(2.1.1) using
external table and getting some error which seems not related to drill. Just
asking anyone have tried this ? If yes then do we have any best practices/link
for this?

Regards,

Anup Tiwari






Regards,
Anup Tiwari

Re: S3 Connection Issues

[Anup Tiwari]

Hi Arjun,
Thanks!! This worked and now i am able to query s3. But i didn't understand your
last line and also how this worked with 2.7 jar. Can you please explain this a
bit or provide any reference link?
@padma, i was trying to build from source and executed below steps but got error
:-
JAVA Version : 1.8.0_151Maven Version : 3.5.2
1. git clone https://git-wip-us.apache.org/repos/asf/drill.git2. cd drill && vi
pom.xml --> changed hadoop version to 2.9.03. mvn clean install -DskipTests
Error :-
[WARNING] Rule 0: org.apache.maven.plugins.enforcer.RequireFilesSize failed with
message:The file drill-jdbc-all-1.13.0-SNAPSHOT.jar is outside the expected size
range.
This is likely due to you adding new dependencies to a java-exec and not
updating the excludes in this module. This is important as it minimizes the size
of the dependency of Drill application users.
/opt/apache-s/apache-drill-s/drill/exec/jdbc-all/target/drill-jdbc-all-1.13.0-SNAPSHOT.jar
size (35620228) too large. Max. is 3500/opt/apache
-s/apache-drill-s/drill/exec/jdbc-all/target/drill-jdbc-all-1.13.0-SNAPSHOT.jar
[INFO] 
[INFO] Reactor Summary:[INFO][INFO] Apache Drill Root POM
.. SUCCESS [03:02 min][INFO] tools/Parent Pom
... SUCCESS [ 0.510 s][INFO] tools/freemarker
codegen tooling ... SUCCESS [02:42 min][INFO] Drill Protocol
. SUCCESS [ 16.736 s][INFO] Common (Logical
Plan, Base expressions)  SUCCESS [01:31 min][INFO] Logical Plan,
Base expressions . SUCCESS [ 17.550 s][INFO] exec/Parent Pom
 SUCCESS [ 0.572 s][INFO] exec/memory/Parent
Pom . SUCCESS [ 0.525 s][INFO] exec/memory/base
... SUCCESS [ 8.056 s][INFO] exec/rpc
... SUCCESS [ 6.208 s][INFO]
exec/Vectors ... SUCCESS [01:28 min][INFO]
contrib/Parent Pom . SUCCESS [ 0.461 s][INFO]
contrib/data/Parent Pom  SUCCESS [ 0.443 s][INFO]
contrib/data/tpch-sample-data .. SUCCESS [ 21.941 s][INFO]
exec/Java Execution Engine . SUCCESS [05:54 min][INFO]
exec/JDBC Driver using dependencies  SUCCESS [ 14.827 s][INFO]
JDBC JAR with all dependencies . FAILURE [ 44.417 s][INFO]
contrib/kudu-storage-plugin  SKIPPED[INFO]
contrib/opentsdb-storage-plugin  SKIPPED[INFO]
contrib/mongo-storage-plugin ... SKIPPED[INFO]
contrib/hbase-storage-plugin ... SKIPPED[INFO]
contrib/jdbc-storage-plugin  SKIPPED[INFO]
contrib/hive-storage-plugin/Parent Pom . SKIPPED[INFO]
contrib/hive-storage-plugin/hive-exec-shaded ... SKIPPED[INFO]
contrib/hive-storage-plugin/core ... SKIPPED[INFO]
contrib/drill-gis-plugin ... SKIPPED[INFO]
contrib/kafka-storage-plugin ... SKIPPED[INFO] Packaging and
Distribution Assembly  SKIPPED[INFO] contrib/mapr-format-plugin
. SKIPPED[INFO] contrib/sqlline
 SKIPPED[INFO]
[INFO]
BUILD FAILURE[INFO]
[INFO]
Total time: 16:52 min[INFO] Finished at: 2018-02-14T12:27:17+05:30[INFO] Final
Memory: 156M/1571M[INFO]
[ERROR]
Failed to execute goal
org.apache.maven.plugins:maven-enforcer-plugin:1.3.1:enforce
(enforce-jdbc-jar-compactness) on project drill-jdbc-all: Some Enforcer rules
have failed. Look above for specific messages explaining why the rule failed. ->
[Help 1][ERROR]  





On Wed, Feb 14, 2018 12:32 PM, Arjun kr arjun...@outlook.com  wrote:
If you have 'hadoop-aws-2.9.0.jar' jar in drill classpath, replace it with
original aws jar that comes with tarball.







The class 'org/apache/hadoop/fs/GlobalStorageStatistics' is not available in
hadoop common jar - hadoop-common-2.7.1.jar ( this was added in 2.8.0). You can
try with original tarball installation jars.







Thanks,







Arjun




____

From: Anup Tiwari <anup.tiw...@games24x7.com>

Sent: Wednesday, February 14, 2018 11:49 AM

To: user@drill.apache.org

Subject: Re: S3 Connection Issues




Hi Arjun,

I tried what you said but its not working and queries are going inENQUEUED

state. Please find below log :-

Error

[drill-executor-1] ERROR o.a.d.exec.server.BootStrapContext -

org.apache.drill.exec.work.foreman.Foreman.run() leaked an exception.

java.lang.NoClassDefFoundError:

Re: S3 Connection Issues

[Anup Tiwari]

:1.8.0_72]
at java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[na:1.8.0_72]
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
~[na:1.8.0_72]at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
~[na:1.8.0_72]... 38 common frames omitted

@padma, thanks for help but i will try to build it out using below link and if
things didn't worked out then will surely need your help :-
https://drill.apache.org/docs/compiling-drill-from-source/
Also as you have mentioned, will change hadoop version to 2.9.0 in pom file and
then build it.Let me know if anything needs to be taken care of.  





On Wed, Feb 14, 2018 9:17 AM, Padma Penumarthy ppenumar...@mapr.com  wrote:
Yes, I built it by changing the version in pom file. 

Try and see if what Arjun suggested works. 

If not, you can download the source, change the version and build or 

if you prefer, I can provide you with a private build that you can try with. 

  

Thanks 

Padma 

  

  

On Feb 13, 2018, at 1:46 AM, Anup Tiwari
<anup.tiw...@games24x7.com<mailto:anup.tiw...@games24x7.com>> wrote: 

  

Hi Padma, 

As you have mentioned "Last time I tried, using Hadoop 2.8.1 worked for me." so 

have you build drill with hadoop 2.8.1 ? If yes then can you provide steps ? 

Since i have downloaded tar ball of 1.11.0 and replaced hadoop-aws-2.7.1.jar 

with hadoop-aws-2.9.0.jar but still not able to query successfully to s3 bucket;
 

queries are going in starting state. 

We are trying to query : "ap-south-1" region which supports only v4 signature. 

  

  

  

  

  

On Thu, Oct 19, 2017 9:44 AM, Padma Penumarthy
ppenumar...@mapr.com<mailto:ppenumar...@mapr.com> wrote: 

Which AWS region are you trying to connect to ? 

  

We have a problem connecting to regions which support only v4 signature 

  

since the version of hadoop we include in Drill is old. 

  

Last time I tried, using Hadoop 2.8.1 worked for me. 

  

  

  

Thanks 

  

Padma 

  

  

  

  

  

On Oct 18, 2017, at 8:14 PM, Charles Givre
<cgi...@gmail.com<mailto:cgi...@gmail.com>> wrote: 

  

  

  

Hello all, 

  

I’m trying to use Drill to query data in an S3 bucket and running into some 

issues which I can’t seem to fix. I followed the various instructions online to 

set up Drill with S3, and put my keys in both the conf-site.xml and in the 

plugin config, but every time I attempt to do anything I get the following 

errors: 

  

  

  

  

  

jdbc:drill:zk=local> show databases; 

  

Error: SYSTEM ERROR: AmazonS3Exception: Status Code: 403, AWS Service: Amazon 

S3, AWS Request ID: 56D1999BD1E62DEB, AWS Error Code: null, AWS Error Message: 

Forbidden 

  

  

  

  

  

[Error Id: 65d0bb52-a923-4e98-8ab1-65678169140e on 

charless-mbp-2.fios-router.home:31010] (state=,code=0) 

  

0: jdbc:drill:zk=local> show databases; 

  

Error: SYSTEM ERROR: AmazonS3Exception: Status Code: 403, AWS Service: Amazon 

S3, AWS Request ID: 4D2CBA8D42A9ECA0, AWS Error Code: null, AWS Error Message: 

Forbidden 

  

  

  

  

  

[Error Id: 25a2d008-2f4d-4433-a809-b91ae063e61a on 

charless-mbp-2.fios-router.home:31010] (state=,code=0) 

  

0: jdbc:drill:zk=local> show files in s3.root; 

  

Error: SYSTEM ERROR: AmazonS3Exception: Status Code: 403, AWS Service: Amazon 

S3, AWS Request ID: 2C635944EDE591F0, AWS Error Code: null, AWS Error Message: 

Forbidden 

  

  

  

  

  

[Error Id: 02e136f5-68c0-4b47-9175-a9935bda5e1c on 

charless-mbp-2.fios-router.home:31010] (state=,code=0) 

  

0: jdbc:drill:zk=local> show schemas; 

  

Error: SYSTEM ERROR: AmazonS3Exception: Status Code: 403, AWS Service: Amazon 

S3, AWS Request ID: 646EB5B2EBCF7CD2, AWS Error Code: null, AWS Error Message: 

Forbidden 

  

  

  

  

  

[Error Id: 954aaffe-616a-4f40-9ba5-d4b7c04fe238 on 

charless-mbp-2.fios-router.home:31010] (state=,code=0) 

  

  

  

I have verified that the keys are correct but using the AWS CLI and downloaded 

some of the files, but I’m kind of at a loss as to how to debug. Any 

suggestions? 

  

Thanks in advance, 

  

— C 

  

  

  

  

  

  

  

Regards, 

Anup Tiwari 

  

Sent with Mixmax 

  





Regards,
Anup Tiwari

Sent with Mixmax

Reading drill(1.10.0) created parquet table in hive(2.1.1) using external table

[Anup Tiwari]

Hi Team,
I am trying to read drill(1.10.0) created parquet table in hive(2.1.1) using
external table and getting some error which seems not related to drill. Just
asking anyone have tried this ? If yes then do we have any best practices/link
for this?
Regards,
Anup Tiwari

Re: Unable to setup hive plugin in Drill 1.11.0

[Anup Tiwari]

Hi Arjun,
You have pointed correctly; i have removed all hive-2.1 jar from 3rd party and
kept only 3 jars which come with tar ball and it worked.
Thanks!!  






On Tue, Feb 13, 2018 1:33 PM, Arjun kr arjun...@outlook.com  wrote:
Hi Anup,







As Sorabh mentioned, you seem to be using hive 2.1.1 jars in Drill classpath
based on the stack trace. Did you build drill package by customizing hive
version to 2.1.1 or added hive 2.1.1 jars in Drill classpath manually? I could
see that Drill 1.12 (latest released), 1.11 and 1.10 are bundled with hive 1.2.1
client.







You could try with drill 1.11 tarball that includes hive 1.2.1 client. Also, you
may check the version of hive client jars in $DRILL_HOME/jars/3rdparty/
directory from the Drill 1.10 cluster where it is working fine.







http://archive.apache.org/dist/drill/drill-1.11.0/







# List of hive client jars from Drill 1.11 tarball extracted.







$ ls -1 apache-drill-1.11.0/jars/3rdparty/hive-*

apache-drill-1.11.0/jars/3rdparty/hive-contrib-1.2.1.jar

apache-drill-1.11.0/jars/3rdparty/hive-hbase-handler-1.2.1.jar

apache-drill-1.11.0/jars/3rdparty/hive-metastore-1.2.1.jar

$







Thanks,







Arjun









From: Anup Tiwari <anup.tiw...@games24x7.com>

Sent: Tuesday, February 13, 2018 12:01 PM

To: user@drill.apache.org

Subject: Re: Unable to setup hive plugin in Drill 1.11.0




Also forgot to mention that we are using Drill 1.10 with 2.1 on our one of

clusters and it is working properly. Just the change is in Hadoop Version and we

are using MySQL community version 5.7.16. Please find below details :-

Apache Drill Version : 1.10.0MySQL Community :5.7.16Apache Hive : 2.1.1Apache

Hadoop : 2.7.3

So does this mean the issue is with hadoop version ? As i can seehadoop 2.7.1

related jars in 3rdparty jar of drill.
















On Tue, Feb 13, 2018 11:33 AM, Anup Tiwari anup.tiw...@games24x7.com wrote:

Hi Sorabh,

Thanks for reply. We are using below combination :-

Apache Drill Version : 1.11.0MySQL Percona : 5.6.35-81.0Apache Hive : 2.1.1

Apache Hadoop : 2.8.0

As you have mentioned that Drill 1.11 with hive 2.1 combination is not supported

so do we have any link from where i can get an idea of which combination is

supported? Also does not supported means we can't use this combination(i mean

don't we have any workaround)?
















On Tue, Feb 13, 2018 12:57 AM, Sorabh Hamirwasia shamirwa...@mapr.com wrote:

Hi Anup,














From the stack trace which you have shared it looks like you are trying to use


Hive2.1 with drill-1.11.0. This s not a supported combination[1]. There is a

PR[2] open for it in 1.13 though.






















Also from the stack trace it's pointing towards an invalid field in HiveConf

METASTORE_BATCH_RETRIEVE_OBJECTS_MAX which got introduced in hive-2.1 only.






















java.lang.Thread.run(Thread.java:745) [na:1.8.0_72]Caused by:













java.lang.NoSuchFieldError: METASTORE_BATCH_RETRIEVE_OBJECTS_MAX at













org.apache.hadoop.hive.metastore.HiveMetaStoreClient.(HiveMetaStoreClient.java:222)













~[hive-metastore-2.1.1.jar:2.1.1] at













org.apache.hadoop.hive.metastore.HiveMetaStoreClient.(HiveMetaStoreClient.java:210)













~[hive-metastore-2.1.1.jar:2.1.1] at













org.apache.drill.exec.store.hive.DrillHiveMetaStoreClient.(DrillHiveMetaStoreClient.java:140)






















Usually any error/exxception should get logged into drillbit.out as well. May be

you can try seeing there.






















[1]: https://github.com/apache/drill/blob/1.11.0/pom.xml#L51

[https://avatars3.githubusercontent.com/u/47359?s=400=4]<https://github.com/apache/drill/blob/1.11.0/pom.xml#L51>




apache/drill<https://github.com/apache/drill/blob/1.11.0/pom.xml#L51>

github.com

drill - Mirror of Apache Drill






















[2]: https://github.com/apache/drill/pull/






















Thanks,




Sorabh






















____




From: Anup Tiwari <anup.tiw...@games24x7.com>




Sent: Monday, February 12, 2018 9:21 AM




To: user@drill.apache.org




Subject: Re: Unable to setup hive plugin in Drill 1.11.0













I don't know where to look for logs as i have tried multiple time creating




hive plugin. But one time i can see below stack is printed in drillbit.log(i




don't know if it came due to plugin creation or not as i have tried multiple




time but nothing got printed in drillbit.log except below which is present in




logs only once) :-




Error




org.apache.drill.common.exceptions.ExecutionSetupException: Failure setting up




new storage plugin configuration for config org.apache.drill.exec.




store.hive.HiveStoragePluginConfig@f149f108 at




org.apache.drill.exec.store.StoragePluginRegistryImpl.create(StoragePluginRegistryImpl.java:354)




~[drill-jav

Re: S3 Connection Issues

[Anup Tiwari]

Hi Padma,
As you have mentioned "Last time I tried, using Hadoop 2.8.1 worked for me." so
have you build drill with hadoop 2.8.1 ? If yes then can you provide steps ?
Since i have downloaded tar ball of 1.11.0 and replaced hadoop-aws-2.7.1.jar
with hadoop-aws-2.9.0.jar but still not able to query successfully to s3 bucket;
queries are going in starting state.
We are trying to query : "ap-south-1" region which supports only v4 signature.  





On Thu, Oct 19, 2017 9:44 AM, Padma Penumarthy ppenumar...@mapr.com  wrote:
Which AWS region are you trying to connect to ? 

We have a problem connecting to regions which support only v4 signature 

since the version of hadoop we include in Drill is old. 

Last time I tried, using Hadoop 2.8.1 worked for me. 

  

Thanks 

Padma 

  

  

> On Oct 18, 2017, at 8:14 PM, Charles Givre <cgi...@gmail.com> wrote: 

> 

> Hello all, 

> I’m trying to use Drill to query data in an S3 bucket and running into some
issues which I can’t seem to fix. I followed the various instructions online to
set up Drill with S3, and put my keys in both the conf-site.xml and in the
plugin config, but every time I attempt to do anything I get the following
errors: 

> 

> 

> jdbc:drill:zk=local> show databases; 

> Error: SYSTEM ERROR: AmazonS3Exception: Status Code: 403, AWS Service: Amazon
S3, AWS Request ID: 56D1999BD1E62DEB, AWS Error Code: null, AWS Error Message:
Forbidden 

> 

> 

> [Error Id: 65d0bb52-a923-4e98-8ab1-65678169140e on
charless-mbp-2.fios-router.home:31010] (state=,code=0) 

> 0: jdbc:drill:zk=local> show databases; 

> Error: SYSTEM ERROR: AmazonS3Exception: Status Code: 403, AWS Service: Amazon
S3, AWS Request ID: 4D2CBA8D42A9ECA0, AWS Error Code: null, AWS Error Message:
Forbidden 

> 

> 

> [Error Id: 25a2d008-2f4d-4433-a809-b91ae063e61a on
charless-mbp-2.fios-router.home:31010] (state=,code=0) 

> 0: jdbc:drill:zk=local> show files in s3.root; 

> Error: SYSTEM ERROR: AmazonS3Exception: Status Code: 403, AWS Service: Amazon
S3, AWS Request ID: 2C635944EDE591F0, AWS Error Code: null, AWS Error Message:
Forbidden 

> 

> 

> [Error Id: 02e136f5-68c0-4b47-9175-a9935bda5e1c on
charless-mbp-2.fios-router.home:31010] (state=,code=0) 

> 0: jdbc:drill:zk=local> show schemas; 

> Error: SYSTEM ERROR: AmazonS3Exception: Status Code: 403, AWS Service: Amazon
S3, AWS Request ID: 646EB5B2EBCF7CD2, AWS Error Code: null, AWS Error Message:
Forbidden 

> 

> 

> [Error Id: 954aaffe-616a-4f40-9ba5-d4b7c04fe238 on
charless-mbp-2.fios-router.home:31010] (state=,code=0) 

> 

> I have verified that the keys are correct but using the AWS CLI and downloaded
some of the files, but I’m kind of at a loss as to how to debug. Any
suggestions? 

> Thanks in advance, 

> — C 

  





Regards,
Anup Tiwari

Sent with Mixmax

Re: Unable to setup hive plugin in Drill 1.11.0

[Anup Tiwari]

Also forgot to mention that we are using Drill 1.10 with 2.1 on our one of
clusters and it is working properly. Just the change is in Hadoop Version and we
are using MySQL community version 5.7.16. Please find below details :-
Apache Drill Version : 1.10.0MySQL Community :5.7.16Apache Hive : 2.1.1Apache
Hadoop : 2.7.3
So does this mean the issue is with hadoop version ? As i can seehadoop 2.7.1
related jars in 3rdparty jar of drill.  






On Tue, Feb 13, 2018 11:33 AM, Anup Tiwari anup.tiw...@games24x7.com  wrote:
Hi Sorabh,
Thanks for reply. We are using below combination :-
Apache Drill Version : 1.11.0MySQL Percona : 5.6.35-81.0Apache Hive : 2.1.1
Apache Hadoop : 2.8.0
As you have mentioned that Drill 1.11 with hive 2.1 combination is not supported
so do we have any link from where i can get an idea of which combination is
supported? Also does not supported means we can't use this combination(i mean
don't we have any workaround)?  






On Tue, Feb 13, 2018 12:57 AM, Sorabh Hamirwasia shamirwa...@mapr.com  wrote:
Hi Anup,





From the stack trace which you have shared it looks like you are trying to use

Hive2.1 with drill-1.11.0. This s not a supported combination[1]. There is a
PR[2] open for it in 1.13 though.







Also from the stack trace it's pointing towards an invalid field in HiveConf
METASTORE_BATCH_RETRIEVE_OBJECTS_MAX which got introduced in hive-2.1 only.







java.lang.Thread.run(Thread.java:745) [na:1.8.0_72]Caused by:




java.lang.NoSuchFieldError: METASTORE_BATCH_RETRIEVE_OBJECTS_MAX at




org.apache.hadoop.hive.metastore.HiveMetaStoreClient.(HiveMetaStoreClient.java:222)




~[hive-metastore-2.1.1.jar:2.1.1] at




org.apache.hadoop.hive.metastore.HiveMetaStoreClient.(HiveMetaStoreClient.java:210)




~[hive-metastore-2.1.1.jar:2.1.1] at




org.apache.drill.exec.store.hive.DrillHiveMetaStoreClient.(DrillHiveMetaStoreClient.java:140)







Usually any error/exxception should get logged into drillbit.out as well. May be
you can try seeing there.







[1]: https://github.com/apache/drill/blob/1.11.0/pom.xml#L51




[2]: https://github.com/apache/drill/pull/







Thanks,

Sorabh









From: Anup Tiwari <anup.tiw...@games24x7.com>

Sent: Monday, February 12, 2018 9:21 AM

To: user@drill.apache.org

Subject: Re: Unable to setup hive plugin in Drill 1.11.0




I don't know where to look for logs as i have tried multiple time creating

hive plugin. But one time i can see below stack is printed in drillbit.log(i

don't know if it came due to plugin creation or not as i have tried multiple

time but nothing got printed in drillbit.log except below which is present in

logs only once) :-

Error

org.apache.drill.common.exceptions.ExecutionSetupException: Failure setting up

new storage plugin configuration for config org.apache.drill.exec.

store.hive.HiveStoragePluginConfig@f149f108 at

org.apache.drill.exec.store.StoragePluginRegistryImpl.create(StoragePluginRegistryImpl.java:354)

~[drill-java-exec-1.11.0.jar:1.11.0] at

org.apache.drill.exec.store.StoragePluginRegistryImpl.createOrUpdate(StoragePluginRegistryImpl.java:238)

~[drill-java-exec-1.11.0.jar:1.11.0] at

org.apache.drill.exec.store.StoragePluginRegistryImpl.getPlugin(StoragePluginRegistryImpl.java:289)

~[drill-java-exec-1.11.0.jar:1.11.0] at

org.apache.drill.exec.server.rest.StorageResources.getStoragePluginJSON(StorageResources.java:101)

[drill-java-exec-1.11.0.jar:1.11.0] at

org.apache.drill.exec.server.rest.StorageResources.getStoragePlugin(StorageResources.java:115)

[drill-java-exec-1.11.0.jar:1.11.0] at

sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_72]

 at

sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

~[na:1.8.0_72] at

sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

~[na:1.8.0_72] at java.lang.reflect.Method.invoke(Method.java:498)

~[na:1.8.0_72] at

org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java

:81) [jersey-server-2.8.jar:na] at

org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:151)

[jersey-server-2.8.jar:na] at

org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:171)

 [jersey-server-2.8.jar:na] at

org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcher

Provider.java:195) [jersey-server-2.8.jar:na] at

org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:104

) [jersey-server-2.8.jar:na] at

org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:387)

[jersey-server-2.8.

Re: Unable to setup hive plugin in Drill 1.11.0

[Anup Tiwari]

Hi Sorabh,
Thanks for reply. We are using below combination :-
Apache Drill Version : 1.11.0MySQL Percona : 5.6.35-81.0Apache Hive : 2.1.1
Apache Hadoop : 2.8.0
As you have mentioned that Drill 1.11 with hive 2.1 combination is not supported
so do we have any link from where i can get an idea of which combination is
supported? Also does not supported means we can't use this combination(i mean
don't we have any workaround)?  






On Tue, Feb 13, 2018 12:57 AM, Sorabh Hamirwasia shamirwa...@mapr.com  wrote:
Hi Anup,





From the stack trace which you have shared it looks like you are trying to use

Hive2.1 with drill-1.11.0. This s not a supported combination[1]. There is a
PR[2] open for it in 1.13 though.







Also from the stack trace it's pointing towards an invalid field in HiveConf
METASTORE_BATCH_RETRIEVE_OBJECTS_MAX which got introduced in hive-2.1 only.







java.lang.Thread.run(Thread.java:745) [na:1.8.0_72]Caused by:




java.lang.NoSuchFieldError: METASTORE_BATCH_RETRIEVE_OBJECTS_MAX at




org.apache.hadoop.hive.metastore.HiveMetaStoreClient.(HiveMetaStoreClient.java:222)




~[hive-metastore-2.1.1.jar:2.1.1] at




org.apache.hadoop.hive.metastore.HiveMetaStoreClient.(HiveMetaStoreClient.java:210)




~[hive-metastore-2.1.1.jar:2.1.1] at




org.apache.drill.exec.store.hive.DrillHiveMetaStoreClient.(DrillHiveMetaStoreClient.java:140)







Usually any error/exxception should get logged into drillbit.out as well. May be
you can try seeing there.







[1]: https://github.com/apache/drill/blob/1.11.0/pom.xml#L51




[2]: https://github.com/apache/drill/pull/







Thanks,

Sorabh









From: Anup Tiwari <anup.tiw...@games24x7.com>

Sent: Monday, February 12, 2018 9:21 AM

To: user@drill.apache.org

Subject: Re: Unable to setup hive plugin in Drill 1.11.0




I don't know where to look for logs as i have tried multiple time creating

hive plugin. But one time i can see below stack is printed in drillbit.log(i

don't know if it came due to plugin creation or not as i have tried multiple

time but nothing got printed in drillbit.log except below which is present in

logs only once) :-

Error

org.apache.drill.common.exceptions.ExecutionSetupException: Failure setting up

new storage plugin configuration for config org.apache.drill.exec.

store.hive.HiveStoragePluginConfig@f149f108 at

org.apache.drill.exec.store.StoragePluginRegistryImpl.create(StoragePluginRegistryImpl.java:354)

~[drill-java-exec-1.11.0.jar:1.11.0] at

org.apache.drill.exec.store.StoragePluginRegistryImpl.createOrUpdate(StoragePluginRegistryImpl.java:238)

~[drill-java-exec-1.11.0.jar:1.11.0] at

org.apache.drill.exec.store.StoragePluginRegistryImpl.getPlugin(StoragePluginRegistryImpl.java:289)

~[drill-java-exec-1.11.0.jar:1.11.0] at

org.apache.drill.exec.server.rest.StorageResources.getStoragePluginJSON(StorageResources.java:101)

[drill-java-exec-1.11.0.jar:1.11.0] at

org.apache.drill.exec.server.rest.StorageResources.getStoragePlugin(StorageResources.java:115)

[drill-java-exec-1.11.0.jar:1.11.0] at

sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_72]

 at

sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

~[na:1.8.0_72] at

sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

~[na:1.8.0_72] at java.lang.reflect.Method.invoke(Method.java:498)

~[na:1.8.0_72] at

org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java

:81) [jersey-server-2.8.jar:na] at

org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:151)

[jersey-server-2.8.jar:na] at

org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:171)

 [jersey-server-2.8.jar:na] at

org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcher

Provider.java:195) [jersey-server-2.8.jar:na] at

org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:104

) [jersey-server-2.8.jar:na] at

org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:387)

[jersey-server-2.8.jar:na] at

org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:331)

[jersey-server-2.8.jar:na] at

org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:103)

[jersey-server-2.8.jar:na] at

org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:269)

[jersey-server-2.8.jar:na] at

org.glassfish.jersey.internal.Errors$1.call(Errors.java:271)

[jersey-common-2.8.jar:na] at

org.glassfish.jersey.internal.Errors$1.call(Errors.java:267)

[jersey-common-2.8.

Re: Unable to setup hive plugin in Drill 1.11.0

[Anup Tiwari]

]at
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:478)
[jetty-servlet-9.1.5.v20140505.jar:9.1.5.v20140505]at
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:183)
[jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505]at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1045)
[jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505]at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
[jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505]at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
[jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505]at
org.eclipse.jetty.server.Server.handle(Server.java:462)
[jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505]at
org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:279)
[jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505]at
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:232)
[jetty-server-9.1.5.v20140505.jar:9.1.5.v20140505]at
org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:534)
[jetty-io-9.1.5.v20140505.jar:9.1.5.v20140505]at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:607)
[jetty-util-9.1.5.v20140505.jar:9.1.5.v20140505]at
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:536)
[jetty-util-9.1.5.v20140505.jar:9.1.5.v20140505]at
java.lang.Thread.run(Thread.java:745) [na:1.8.0_72]Caused by:
java.lang.NoSuchFieldError: METASTORE_BATCH_RETRIEVE_OBJECTS_MAXat
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.(HiveMetaStoreClient.java:222)
~[hive-metastore-2.1.1.jar:2.1.1]at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.(HiveMetaStoreClient.java:210)
~[hive-metastore-2.1.1.jar:2.1.1]at
org.apache.drill.exec.store.hive.DrillHiveMetaStoreClient.(DrillHiveMetaStoreClient.java:140)
~[drill-storage-hive-core-1.11.0.jar:1.11.0]at
org.apache.drill.exec.store.hive.DrillHiveMetaStoreClient.(DrillHiveMetaStoreClient.java:57)
~[drill-storage-hive-core-1.11.0.jar:1.11.0]at
org.apache.drill.exec.store.hive.DrillHiveMetaStoreClient$HiveClientWithCaching.(DrillHiveMetaStoreClient.java:471)
~[drill-storage-hive-core-1.11.0.jar:1.11.0]at
org.apache.drill.exec.store.hive.DrillHiveMetaStoreClient$HiveClientWithCaching.(DrillHiveMetaStoreClient.java:469)
~[drill-storage-hive-core-1.11.0.jar:1.11.0]at
org.apache.drill.exec.store.hive.DrillHiveMetaStoreClient.createCloseableClientWithCaching(DrillHiveMetaStoreClient.java:136)
~[drill-storage-hive-core-1.11.0.jar:1.11.0]at
org.apache.drill.exec.store.hive.schema.HiveSchemaFactory.(HiveSchemaFactory.java:75)
~[drill-storage-hive-core-1.11.0.jar:1.11.0]at
org.apache.drill.exec.store.hive.HiveStoragePlugin.(HiveStoragePlugin.java:68)
~[drill-storage-hive-core-1.11.0.jar:1.11.0]at
sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
~[na:1.8.0_72]at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
~[na:1.8.0_72]at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
~[na:1.8.0_72]at
java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[na:1.8.0_72]
at
org.apache.drill.exec.store.StoragePluginRegistryImpl.create(StoragePluginRegistryImpl.java:345)
~[drill-java-exec-1.11.0.jar:1.11.0]... 45 common frames omitted  





On Mon, Feb 12, 2018 9:23 PM, Anup Tiwari anup.tiw...@games24x7.com  wrote:
Hi All,
Please find below information :-
Apache Drill Version : 1.11.0MySQL Percona : 5.6.35-81.0Apache Hive : 2.1.1
mysql-connector-java-5.1.44-bin.jar is present in 3rdparty jar
Hive Plugin info :-
{"type" : "hive","enabled" : true,"configProps" : {
  "hive.metastore.uris" : "thrift://prod-hadoop-xxx:9083",
  "hive.metastore.sasl.enabled" : "false",  "fs.default.name" :
"hdfs://prod-hadoop-xxx:9000"}  }

Error :
"result" : "error (unable to create/ update storage)"
Regards,
Anup Tiwari

Sent with Mixmax  

Regards,
Anup Tiwari

Unable to setup hive plugin in Drill 1.11.0

[Anup Tiwari]

Hi All,
Please find below information :-
Apache Drill Version : 1.11.0MySQL Percona : 5.6.35-81.0Apache Hive : 2.1.1
mysql-connector-java-5.1.44-bin.jar is present in 3rdparty jar
Hive Plugin info :-
{"type" : "hive","enabled" : true,"configProps" : {
  "hive.metastore.uris" : "thrift://prod-hadoop-xxx:9083",
  "hive.metastore.sasl.enabled" : "false",  "fs.default.name" :
"hdfs://prod-hadoop-xxx:9000"}  }

Error :
"result" : "error (unable to create/ update storage)"
Regards,
Anup Tiwari

Sent with Mixmax

Re: [1.9.0] : UserException: SYSTEM ERROR: IllegalReferenceCountException: refCnt: 0 and then SYSTEM ERROR: IOException: Failed to shutdown streamer

[Anup Tiwari]

(Native Method)
~[na:1.8.0_72]
at javax.security.auth.Subject.doAs(Subject.java:422) ~[na:1.8.0_72]
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
~[hadoop-common-2.7.1.jar:na]
at
org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:226)
[drill-java-exec-1.10.0.jar:1.10.0]
... 4 common frames omitted

Regards,
*Anup Tiwari*

On Tue, Dec 12, 2017 at 11:46 AM, Kunal Khatua <kkha...@mapr.com> wrote:

> Sorry, I meant that Drill shutdown a *query* prematurely. When a query
> completes, all the related threads (fragments) need to perform a clean up
> and give resources back to the pool.
>
> This ideally should not have the need to be handled by the application.
> So, what would be good to know is
>
> 1. Setup description
> 2. What queries were you running and against what kind of dataset
> 3. How frequently is it occurring.
>
> The Drill logs also tend to have a stack trace for such errors, so it
> helps if you can share that too.
>
> ~Kunal
>
> -Original Message-
> From: Anup Tiwari [mailto:anup.tiw...@games24x7.com]
> Sent: Friday, December 08, 2017 12:35 AM
> To: user@drill.apache.org
> Subject: Re: [1.9.0] : UserException: SYSTEM ERROR:
> IllegalReferenceCountException: refCnt: 0 and then SYSTEM ERROR:
> IOException: Failed to shutdown streamer
>
> Hi Kunal,
>
> I was executing a similar query shared in trail mail also as you have
> mentioned :- *This is a system error and the message appears to hint that
> Drill shutdown a prematurely , *I have checked on all nodes and drill-bit
> is running properly.
>
> Note :- We are using Drill 1.10.0.
>
> Regards,
> *Anup Tiwari*
>
> On Thu, Dec 7, 2017 at 10:33 PM, Kunal Khatua <kkha...@mapr.com> wrote:
>
> > What is it that you were trying to do when you encountered this?
> >
> > This is a system error and the message appears to hint that Drill
> > shutdown a prematurely and is unable to account for that
> >
> > Kunal
> >
> >
> > From: Anup Tiwari
> > Sent: Wednesday, December 6, 7:46 PM
> > Subject: Re: [1.9.0] : UserException: SYSTEM ERROR:
> > IllegalReferenceCountException: refCnt: 0 and then SYSTEM ERROR:
> > IOException: Failed to shutdown streamer
> > To: user@drill.apache.org
> >
> >
> > Hi All, As asked in trail mail can someone explain how to handle :-
> > *UserException: SYSTEM ERROR: IllegalReferenceCountException* *: refCnt:
> > 0?* As above error doesn't explain what and where the real problem is?
> > But if we execute same query in hive for which we get above error then
> > it works. Regards, *Anup Tiwari* On Mon, Dec 12, 2016 at 5:07 PM, Anup
> > Tiwari
> > wrote: > Hi Aman, > > Sorry for delayed response, since we are
> > executing this query on our > ~150GB logs and as i have mentioned in
> > trail mail, by executing "removed > conditions alone" CTAS got
> > executed successfully, so i don't know which > sample data i should
> share(since i don't know pattern)?
> > > > Can you tell me in which scenarios we throw " >
> > IllegalReferenceCountException" and how to handle this in different >
> > scenarios? > > Regards, > *Anup Tiwari* > > On Thu, Dec 8, 2016 at
> > 10:55 PM, Aman Sinha wrote: > >> Hi Anup, >> since your original query
> > was working on 1.6 and failed in 1.9, could you >> pls file a JIRA for
> this ?
> > It sounds like a regression related to >> evaluation of a Project
> > expression (based on the stack trace). Since >> there >> are several
> > CASE exprs, quite likely something related to its evaluation. >> It
> > would be great if you can provide some sample data for someone to >>
> > debug. >> Thanks. >> >> On Thu, Dec 8, 2016 at 12:50 AM, Anup Tiwari
> > >> wrote: >> >>
> > > Hi, >> > >> > I have removed few conditions from my query then it
> > > just
> > worked fine. >> > >> > Also can someone tell me in which scenarios we
> > throw " >> > *IllegalReferenceCountException*" and how to handle it in
> > different >> > scenarios ? >> > >> > As i got this in another query
> > and by removing some conditions it worked >> > for me but when i
> > execute that removed conditions alone in CTAS , it got >> > executed
> > successfully. >> >
> > >> > Regards, >> > *Anup Tiwari* >> > >> > On Wed, Dec 7, 2016 at
> > >> > 12:22 

Re: [1.9.0] : UserException: SYSTEM ERROR: IllegalReferenceCountException: refCnt: 0 and then SYSTEM ERROR: IOException: Failed to shutdown streamer

[Anup Tiwari]

Hi Kunal,

I was executing a similar query shared in trail mail also as you have
mentioned :- *This is a system error and the message appears to hint that
Drill shutdown a prematurely , *I have checked on all nodes and drill-bit
is running properly.

Note :- We are using Drill 1.10.0.

Regards,
*Anup Tiwari*

On Thu, Dec 7, 2017 at 10:33 PM, Kunal Khatua <kkha...@mapr.com> wrote:

> What is it that you were trying to do when you encountered this?
>
> This is a system error and the message appears to hint that Drill shutdown
> a prematurely and is unable to account for that
>
> Kunal
>
>
> From: Anup Tiwari
> Sent: Wednesday, December 6, 7:46 PM
> Subject: Re: [1.9.0] : UserException: SYSTEM ERROR:
> IllegalReferenceCountException: refCnt: 0 and then SYSTEM ERROR:
> IOException: Failed to shutdown streamer
> To: user@drill.apache.org
>
>
> Hi All, As asked in trail mail can someone explain how to handle :-
> *UserException: SYSTEM ERROR: IllegalReferenceCountException* *: refCnt:
> 0?* As above error doesn't explain what and where the real problem is? But
> if we execute same query in hive for which we get above error then it
> works. Regards, *Anup Tiwari* On Mon, Dec 12, 2016 at 5:07 PM, Anup Tiwari
> wrote: > Hi Aman, > > Sorry for delayed response, since we are executing
> this query on our > ~150GB logs and as i have mentioned in trail mail, by
> executing "removed > conditions alone" CTAS got executed successfully, so i
> don't know which > sample data i should share(since i don't know pattern)?
> > > Can you tell me in which scenarios we throw " >
> IllegalReferenceCountException" and how to handle this in different >
> scenarios? > > Regards, > *Anup Tiwari* > > On Thu, Dec 8, 2016 at 10:55
> PM, Aman Sinha wrote: > >> Hi Anup, >> since your original query was
> working on 1.6 and failed in 1.9, could you >> pls file a JIRA for this ?
> It sounds like a regression related to >> evaluation of a Project
> expression (based on the stack trace). Since >> there >> are several CASE
> exprs, quite likely something related to its evaluation. >> It would be
> great if you can provide some sample data for someone to >> debug. >>
> Thanks. >> >> On Thu, Dec 8, 2016 at 12:50 AM, Anup Tiwari >> wrote: >> >>
> > Hi, >> > >> > I have removed few conditions from my query then it just
> worked fine. >> > >> > Also can someone tell me in which scenarios we throw
> " >> > *IllegalReferenceCountException*" and how to handle it in
> different >> > scenarios ? >> > >> > As i got this in another query and by
> removing some conditions it worked >> > for me but when i execute that
> removed conditions alone in CTAS , it got >> > executed successfully. >> >
> >> > Regards, >> > *Anup Tiwari* >> > >> > On Wed, Dec 7, 2016 at 12:22 PM,
> Anup Tiwari > > >> > wrote: >> > >> > > Hi Team, >> > > >> > > I am getting
> below 2 error in my one of the query which was working >> fine >> > > on
> 1.6, Please help me out in this:- >> > > >> > > 1. UserException: SYSTEM
> ERROR: IllegalReferenceCountException: >> refCnt: >> > 0 >> > > 2. SYSTEM
> ERROR: IOException: Failed to shutdown streamer >> > > >> > > Please find
> below query and its stack trace :- >> > > >> > > *Query :-* >> > > >> > >
> create table a_tt3_reg_login as >> > > select sessionid, >> > > >> > >
> count(distinct (case when ((( event = 'e.a' and ajaxUrl like >> > >
> '%/ab/pL%t=r%' ) or ( (Base64Conv(Response) like '%st%tr%' and >> > >
> Base64Conv(Response) not like '%error%') and ajaxUrl like >> '%/sign/ter%'
> >> > )) >> > > OR ( event = 'e.a' and ajaxUrl like '%/player/ter/ter.htm%'
> and >> > > Base64Conv(Response) like '%st%tr%ter%tr%') OR (id =
> '/ter/thyou.htm' >> > and >> > > url = '/pla/natlob.htm')) then sessionid
> end) ) as regs, >> > > >> > > count(distinct (case when ( ajaxUrl like
> '%/signup/poLo%t=log%' and >> event >> > > = 'e.a' ) or ( event = 'e.a' and
> ajaxUrl like >> '%j_spring_security_check%' >> > > and Base64Conv(Response)
> like '%st%tr%') then sessionid end) ) as >> login >> > , >> > > >> > >
> count(distinct (case when ((ajaxUrl like >>

Re: [1.9.0] : UserException: SYSTEM ERROR: IllegalReferenceCountException: refCnt: 0 and then SYSTEM ERROR: IOException: Failed to shutdown streamer

[Anup Tiwari]

Hi All,

As asked in trail mail can someone explain how to handle :- *UserException:
SYSTEM ERROR: IllegalReferenceCountException*

*: refCnt: 0?*
As above error doesn't explain what and where the real problem is? But if
we execute same query in hive for which we get above error then it works.

Regards,
*Anup Tiwari*

On Mon, Dec 12, 2016 at 5:07 PM, Anup Tiwari <anup.tiw...@games24x7.com>
wrote:

> Hi Aman,
>
> Sorry for delayed response, since we are executing this query on our
> ~150GB logs and as i have mentioned in trail mail, by executing "removed
> conditions alone" CTAS got executed successfully, so i don't know which
> sample data i should share(since i don't know pattern)?
>
> Can you tell me in which scenarios we throw "
> IllegalReferenceCountException" and how to handle this in different
> scenarios?
>
> Regards,
> *Anup Tiwari*
>
> On Thu, Dec 8, 2016 at 10:55 PM, Aman Sinha <amansi...@apache.org> wrote:
>
>> Hi Anup,
>> since your original query was working on 1.6 and failed in 1.9,  could you
>> pls file a JIRA for this ?  It sounds like a regression related to
>> evaluation of a Project expression (based on the stack trace).  Since
>> there
>> are several CASE exprs, quite likely something related to its evaluation.
>> It would be great if you can provide some sample data for someone to
>> debug.
>> Thanks.
>>
>> On Thu, Dec 8, 2016 at 12:50 AM, Anup Tiwari <anup.tiw...@games24x7.com>
>> wrote:
>>
>> > Hi,
>> >
>> > I have removed few conditions from my query then it just worked fine.
>> >
>> > Also can someone tell me in which scenarios we throw "
>> > *IllegalReferenceCountException*" and how to handle it in different
>> > scenarios ?
>> >
>> > As i got this in another query and by removing some conditions it worked
>> > for me but when i execute that removed conditions alone in CTAS , it got
>> > executed successfully.
>> >
>> > Regards,
>> > *Anup Tiwari*
>> >
>> > On Wed, Dec 7, 2016 at 12:22 PM, Anup Tiwari <anup.tiw...@games24x7.com
>> >
>> > wrote:
>> >
>> > > Hi Team,
>> > >
>> > > I am getting below 2 error in my one of the query which was working
>> fine
>> > > on 1.6, Please help me out in this:-
>> > >
>> > > 1. UserException: SYSTEM ERROR: IllegalReferenceCountException:
>> refCnt:
>> > 0
>> > > 2. SYSTEM ERROR: IOException: Failed to shutdown streamer
>> > >
>> > > Please find below query and its stack trace :-
>> > >
>> > > *Query :-*
>> > >
>> > > create table a_tt3_reg_login as
>> > > select sessionid,
>> > >
>> > > count(distinct (case when ((( event = 'e.a' and ajaxUrl like
>> > > '%/ab/pL%t=r%' ) or ( (Base64Conv(Response) like '%st%tr%' and
>> > > Base64Conv(Response) not like '%error%') and ajaxUrl like
>> '%/sign/ter%'
>> > ))
>> > > OR ( event = 'e.a' and ajaxUrl like '%/player/ter/ter.htm%' and
>> > > Base64Conv(Response) like '%st%tr%ter%tr%')  OR (id = '/ter/thyou.htm'
>> > and
>> > > url = '/pla/natlob.htm')) then sessionid end) )  as  regs,
>> > >
>> > > count(distinct (case when ( ajaxUrl like '%/signup/poLo%t=log%' and
>> event
>> > > = 'e.a' ) or ( event = 'e.a' and ajaxUrl like
>> '%j_spring_security_check%'
>> > > and Base64Conv(Response)  like '%st%tr%') then sessionid end) ) as
>> login
>> > ,
>> > >
>> > > count(distinct (case when ((ajaxUrl like
>> '/pl%/loadResponsePage.htm%fD=
>> > true=yes%'
>> > > or ajaxUrl like '/pl%/loadResponsePage.htm%fD=true=YES%') OR
>> > (ajaxUrl
>> > > like 'loadSuccessPage.do%fD=true=yes%' or ajaxUrl like
>> > > 'loadSuccessPage.do%fD=true=YES%'))  then sessionid end) ) as fd
>> ,
>> > >
>> > > count(distinct (case when ((ajaxUrl like
>> '/pl%/loadResponsePage.htm%fD=
>> > false=yes%'
>> > > or ajaxUrl like '/pl%/loadResponsePage.htm%fD=false=YES%') OR
>> > > (ajaxUrl like 'loadSuccessPage.do%fD=false=yes%' or ajaxUrl like
>> > > 'loadSuccessPage.do%fD=false=YES%')) then sessionid end) ) as rd
>> > >
>> > > from
>> > > tt2
>> > > group by sessionid;
>> > > Error: SYSTEM ERROR: IllegalReferenceCountException: refCnt: 0
>> &g

Re: Does s3 plugin support AWS S3 signature version 4 ?

[Anup Tiwari]

Any updates on this?
Since we have migrated to Aws Mumbai, we are not able to connect s3 and
Drill.

On 04-Apr-2017 11:02 PM, "Shankar Mane"  wrote:

> Quick question here:
>
> Does s3 plugin support S3 signature version 4  ?
>
> FYI: s3 plugin works in case when region has support for both v2 and v4
> signature. Whereas it seems problematic, for regions (eg. ap-south-1) which
> only has v4 signature version support.
>
> regards,
> shankar
>

Re: Running cartesian joins on Drill

[Anup Tiwari]

Hi,

I have one question here.. so if we have to use Cartesian join in Drill
then do we have to follow some workaround like Shadi mention : adding a
dummy column on the fly that has the value 1 in both tables and then join
on that column leading to having a match of every row of the first table
with every row of the second table, hence do a Cartesian product?
OR
If we just don't specify join condition like :
select a.*, b.* from tt1 as a, tt2 b; then will it internally treat this
query as Cartesian join.

Regards,
*Anup Tiwari*

On Mon, May 8, 2017 at 10:00 PM, Zelaine Fong <zf...@mapr.com> wrote:

> Cartesian joins in Drill are implemented as nested loop joins, and I think
> you should see that reflected in the resultant query plan when you run
> explain plan on the query.
>
> Yes, Cartesian joins/nested loop joins are expensive because you’re
> effectively doing an MxN read of your tables.  There are more efficient
> ways of processing a nested loop join, e.g., by creating an index on the
> larger table in the join and then using that index to do lookups into that
> table.  That way, the nested loop join cost is the cost of creating the
> index + M, where M is the number of rows in the smaller table and assuming
> the lookup cost into the index does minimize the amount of data read of the
> second table.  Drill currently doesn’t do this.
>
> -- Zelaine
>
> On 5/8/17, 9:09 AM, "Muhammad Gelbana" <m.gelb...@gmail.com> wrote:
>
> ​I believe ​clhubert is referring to this discussion
> <http://drill-user.incubator.apache.narkive.com/TIXWiTY4/
> cartesian-product-in-apache-drill#post1>
> .
>
> So why Drill doesn't transform this query into a nested join query ?
> Simply
> because there is no Calcite rule to transform it into a nested loop
> join ?
> Is it not technically possible to write such Rule or is it feasible so
> I
> may take on this challenge ?
>
> Also pardon me for repeating my question but I fail to find an answer
> in
> your replies, why doesn't Drill just run a cartesian join ? Because
> it's
> expensive regarding resources (i.e. CPU\Network\RAM) ?
>
> Thanks a lot Shadi for the query, it works for me.
>
> *-*
> *Muhammad Gelbana*
> http://www.linkedin.com/in/mgelbana
>
> On Mon, May 8, 2017 at 6:10 AM, Shadi Khalifa <khal...@cs.queensu.ca>
> wrote:
>
> > Hi Muhammad,
> >
> > I did the following as a workaround to have Cartesian product. The
> basic
> > idea is to create a dummy column on the fly that has the value 1 in
> both
> > tables and then join on that column leading to having a match of
> every row
> > of the first table with every row of the second table, hence do a
> Cartesian
> > product. This might not be the most efficient way but it will do the
> job.
> >
> > *Original Query:*
> > SELECT * FROM
> > ( SELECT 'ABC' `UserID` FROM `dfs`.`path_to_parquet_file` tc LIMIT
> > 2147483647) `t0`
> > INNER JOIN
> > ( SELECT 'ABC' `UserID` FROM `dfs`.`path_to_parquet_file` tc LIMIT
> > 2147483647) `t1`
> > ON (`t0`.`UserID` IS NOT DISTINCT FROM `t1`.`UserID`)
> > LIMIT 2147483647
> >
> > *Workaround (add columns **d1a381f3g73 and **d1a381f3g74 to tables
> one
> > and two, respectively. Names don't really matter, just need to be
> unique):*
> > SELECT * FROM
> > ( SELECT *1 as d1a381f3g73*, 'ABC' `UserID` FROM
> > `dfs`.`path_to_parquet_file` tc LIMIT 2147483647) `t0`
> > INNER JOIN
> > ( SELECT *1 as d1a381f3g74*, 'ABC' `UserID` FROM
> > `dfs`.`path_to_parquet_file` tc LIMIT 2147483647) `t1`
> > ON (`t0`.*d1a381f3g73 = *`t1`.*d1a381f3g74*)
> > WHERE `t0`.`UserID` IS NOT DISTINCT FROM `t1`.`UserID`
> > LIMIT 2147483647
> >
> > Regards
> >
> >
> > *Shadi Khalifa, PhD*
> > Postdoctoral Fellow
> > Cognitive Analytics Development Hub
> > Centre for Advanced Computing
> > Queen’s University
> > (613) 533-6000 x78347
> > http://cac.queensu.ca
> >
> > I'm just a neuron in the society collective brain
> >
> > *Join us for HPCS in June 2017! Register at:*  *http://2017.hpcs.ca/
> > <http://2017.hpcs.ca/>*
> >
> > P Please consider your environmental responsibility before printing
> this
> > e-mail
> >
> > *01001001 0010 01101100 0110 01110110 01100101 0010
> 01000101
> > 01100111 0001 0111 01110100 *
>   

Re: [Drill 1.10.0] : Memory was leaked by query

[Anup Tiwari]

Thanks Padma, it worked.

Regards,
*Anup Tiwari*

On Wed, Apr 19, 2017 at 1:13 AM, Kunal Khatua <kkha...@mapr.com> wrote:

> Could you also share the profiles for the failed queries as well?
>
>
> Thanks
>
> Kunal
>
> 
> From: Padma Penumarthy <ppenumar...@mapr.com>
> Sent: Tuesday, April 18, 2017 7:18:08 AM
> To: user@drill.apache.org
> Cc: d...@drill.apache.org
> Subject: Re: [Drill 1.10.0] : Memory was leaked by query
>
> Seems like you are running into  DRILL-5435<https://issues.apac
> he.org/jira/browse/DRILL-5435>.
> Try  turning off async parquet reader and see if that helps.
> alter session set `store.parquet.reader.pagereader.async`=false;
>
> Thanks,
> Padma
>
>
> On Apr 18, 2017, at 6:14 AM, Anup Tiwari <anup.tiw...@games24x7.com lto:anup.tiw...@games24x7.com>> wrote:
>
> Hi Team,
>
> Please find following information :
>
> *Cluster configuration :*
> Number of Nodes : 5
> Cores/Node : 8
> RAM : 32
>
> *Variable values :*
> planner.width.max_per_node = 5
> planner.width.max_per_query = 30
> planner.memory.max_query_memory_per_node = 4294967296
>
> I am getting following error on simple select statement which is coming 6
> times out of 10 times, let me know if i am missing anything:
>
> *Query :*
> select udf_channel,uid from dfs.tmp.tt1 where (event = 'ajax' and ajaxurl
> like '%/j_check%' and ajaxResponse like '%success%true%') limit 5;
>
> *Error :*
>
> ERROR o.a.d.e.w.fragment.FragmentExecutor - SYSTEM ERROR:
> IllegalStateException: Memory was leaked by query. Memory leaked: (1048576)
> Allocator(op:1:24:6:ParquetRowGroupScan)
> 100/1048576/27140096/100 (res/actual/peak/limit)
>
>
> Fragment 1:24
>
> [Error Id: a54cc1bf-794a-4143-bd82-0dd5fa3c8f52 on
> prod-hadoop-101.bom-prod.aws.games24x7.com<http://prod-hadoo
> p-101.bom-prod.aws.games24x7.com>:31010]
> org.apache.drill.common.exceptions.UserException: SYSTEM ERROR:
> IllegalStateException: Memory was leaked by query. Memory leaked: (1048576)
> Allocator(op:1:24:6:ParquetRowGroupScan)
> 100/1048576/27140096/100 (res/actual/peak/limit)
>
>
> Fragment 1:24
>
> [Error Id: a54cc1bf-794a-4143-bd82-0dd5fa3c8f52 on
> prod-hadoop-101.bom-prod.aws.games24x7.com<http://prod-hadoo
> p-101.bom-prod.aws.games24x7.com>:31010]
>at
> org.apache.drill.common.exceptions.UserException$Builder.
> build(UserException.java:544)
> ~[drill-common-1.10.0.jar:1.10.0]
>at
> org.apache.drill.exec.work.fragment.FragmentExecutor.sendFin
> alState(FragmentExecutor.java:293)
> [drill-java-exec-1.10.0.jar:1.10.0]
>at
> org.apache.drill.exec.work.fragment.FragmentExecutor.cleanup
> (FragmentExecutor.java:160)
> [drill-java-exec-1.10.0.jar:1.10.0]
>at
> org.apache.drill.exec.work.fragment.FragmentExecutor.run(Fra
> gmentExecutor.java:262)
> [drill-java-exec-1.10.0.jar:1.10.0]
>at
> org.apache.drill.common.SelfCleaningRunnable.run(SelfCleanin
> gRunnable.java:38)
> [drill-common-1.10.0.jar:1.10.0]
>at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool
> Executor.java:1142)
> [na:1.8.0_72]
>at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo
> lExecutor.java:617)
> [na:1.8.0_72]
>at java.lang.Thread.run(Thread.java:745) [na:1.8.0_72]
> Caused by: java.lang.IllegalStateException: Memory was leaked by query.
> Memory leaked: (1048576)
> Allocator(op:1:24:6:ParquetRowGroupScan)
> 100/1048576/27140096/100 (res/actual/peak/limit)
>
>at
> org.apache.drill.exec.memory.BaseAllocator.close(BaseAllocator.java:502)
> ~[drill-memory-base-1.10.0.jar:1.10.0]
>at
> org.apache.drill.exec.ops.OperatorContextImpl.close(Operator
> ContextImpl.java:149)
> ~[drill-java-exec-1.10.0.jar:1.10.0]
>at
> org.apache.drill.exec.ops.FragmentContext.suppressingClose(F
> ragmentContext.java:422)
> ~[drill-java-exec-1.10.0.jar:1.10.0]
>at
> org.apache.drill.exec.ops.FragmentContext.close(FragmentContext.java:411)
> ~[drill-java-exec-1.10.0.jar:1.10.0]
>at
> org.apache.drill.exec.work.fragment.FragmentExecutor.closeOu
> tResources(FragmentExecutor.java:318)
> [drill-java-exec-1.10.0.jar:1.10.0]
>at
> org.apache.drill.exec.work.fragment.FragmentExecutor.cleanup
> (FragmentExecutor.java:155)
> [drill-java-exec-1.10.0.jar:1.10.0]
>... 5 common frames omitted
> 2017-04-18 18:21:54,172 [BitServer-4] INFO
> o.a.d.e.w.fragment.FragmentExecutor -
> 2709f415-c08a-13b9-9f05-fcf9008c484f:1:21: State change requested RUNNING
> --> CANCELLATION_REQUESTED
> 2017-04-18 18:21:54,172 [BitServer-4] INFO
> o.a.d.e.w.f.FragmentStatusReporter

[Drill 1.10.0] : Memory was leaked by query

[Anup Tiwari]

ragment 1:21

[Error Id: 8b3bb6e8-77a0-4747-8602-43b40b349354 on
prod-hadoop-101.bom-prod.aws.games24x7.com:31010]
at
org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:544)
~[drill-common-1.10.0.jar:1.10.0]
at
org.apache.drill.exec.work.fragment.FragmentExecutor.sendFinalState(FragmentExecutor.java:293)
[drill-java-exec-1.10.0.jar:1.10.0]
at
org.apache.drill.exec.work.fragment.FragmentExecutor.cleanup(FragmentExecutor.java:160)
[drill-java-exec-1.10.0.jar:1.10.0]
at
org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:262)
[drill-java-exec-1.10.0.jar:1.10.0]
at
org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38)
[drill-common-1.10.0.jar:1.10.0]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[na:1.8.0_72]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[na:1.8.0_72]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_72]
Caused by: java.lang.IllegalStateException: Memory was leaked by query.
Memory leaked: (1048576)
Allocator(op:1:21:6:ParquetRowGroupScan)
100/1048576/27140096/100 (res/actual/peak/limit)

at
org.apache.drill.exec.memory.BaseAllocator.close(BaseAllocator.java:502)
~[drill-memory-base-1.10.0.jar:1.10.0]
at
org.apache.drill.exec.ops.OperatorContextImpl.close(OperatorContextImpl.java:149)
~[drill-java-exec-1.10.0.jar:1.10.0]
at
org.apache.drill.exec.ops.FragmentContext.suppressingClose(FragmentContext.java:422)
~[drill-java-exec-1.10.0.jar:1.10.0]
at
org.apache.drill.exec.ops.FragmentContext.close(FragmentContext.java:411)
~[drill-java-exec-1.10.0.jar:1.10.0]
at
org.apache.drill.exec.work.fragment.FragmentExecutor.closeOutResources(FragmentExecutor.java:318)
[drill-java-exec-1.10.0.jar:1.10.0]
at
org.apache.drill.exec.work.fragment.FragmentExecutor.cleanup(FragmentExecutor.java:155)
[drill-java-exec-1.10.0.jar:1.10.0]
... 5 common frames omitted


Regards,
*Anup Tiwari*

Re: [Drill 1.9.0] : [CONNECTION ERROR] :- (user client) closed unexpectedly. Drillbit down?

[Anup Tiwari]

Hi John,

First of all sorry for delayed response and thanks for your suggestion,
reducing value of "planner.width.max_per_node" helped me a lot, above issue
which was coming 8 out of 10 times earlier now it is coming only 2 out of
10 times.

As mentioned above occurrences of connection error came down considerably,
but now sometimes i get "Heap Space Error" for few queries and due to this
sometimes drill-bits on some/all nodes gets killed. Let me know if any
other variable i can check for this(As of now, i have 8GB of Heap and 20GB
of Direct memory) :

*Error Log :*

ERROR o.a.drill.common.CatastrophicFailure - Catastrophic Failure Occurred,
exiting. Information message: Unable to handle out of memory condition in
FragmentExecutor.
java.lang.OutOfMemoryError: Java heap space
at org.apache.xerces.dom.DeferredDocumentImpl.getNodeObject(Unknown
Source) ~[xercesImpl-2.11.0.jar:na]
at
org.apache.xerces.dom.DeferredDocumentImpl.synchronizeChildren(Unknown
Source) ~[xercesImpl-2.11.0.jar:na]
at
org.apache.xerces.dom.DeferredElementImpl.synchronizeChildren(Unknown
Source) ~[xercesImpl-2.11.0.jar:na]
at org.apache.xerces.dom.ElementImpl.normalize(Unknown Source)
~[xercesImpl-2.11.0.jar:na]
at org.apache.xerces.dom.ElementImpl.normalize(Unknown Source)
~[xercesImpl-2.11.0.jar:na]
at org.apache.xerces.dom.ElementImpl.normalize(Unknown Source)
~[xercesImpl-2.11.0.jar:na]
at com.games24x7.device.NewDeviceData.setup(NewDeviceData.java:94)
~[DeviceDataClient-0.0.1-SNAPSHOT.jar:na]
at
org.apache.drill.exec.test.generated.FiltererGen5369.doSetup(FilterTemplate2.java:97)
~[na:na]
at
org.apache.drill.exec.test.generated.FiltererGen5369.setup(FilterTemplate2.java:54)
~[na:na]
at
org.apache.drill.exec.physical.impl.filter.FilterRecordBatch.generateSV2Filterer(FilterRecordBatch.java:195)
~[drill-java-exec-1.9.0.jar:1.9.0]
at
org.apache.drill.exec.physical.impl.filter.FilterRecordBatch.setupNewSchema(FilterRecordBatch.java:107)
~[drill-java-exec-1.9.0.jar:1.9.0]
at
org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:78)
~[drill-java-exec-1.9.0.jar:1.9.0]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:162)
~[drill-java-exec-1.9.0.jar:1.9.0]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119)
~[drill-java-exec-1.9.0.jar:1.9.0]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:109)
~[drill-java-exec-1.9.0.jar:1.9.0]
at
org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:51)
~[drill-java-exec-1.9.0.jar:1.9.0]
at
org.apache.drill.exec.physical.impl.svremover.RemovingRecordBatch.innerNext(RemovingRecordBatch.java:94)
~[drill-java-exec-1.9.0.jar:1.9.0]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:162)
~[drill-java-exec-1.9.0.jar:1.9.0]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119)
~[drill-java-exec-1.9.0.jar:1.9.0]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:109)
~[drill-java-exec-1.9.0.jar:1.9.0]
at
org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:51)
~[drill-java-exec-1.9.0.jar:1.9.0]
at
org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext(ProjectRecordBatch.java:135)
~[drill-java-exec-1.9.0.jar:1.9.0]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:162)
~[drill-java-exec-1.9.0.jar:1.9.0]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119)
~[drill-java-exec-1.9.0.jar:1.9.0]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:109)
~[drill-java-exec-1.9.0.jar:1.9.0]
at
org.apache.drill.exec.physical.impl.aggregate.HashAggBatch.buildSchema(HashAggBatch.java:108)
~[drill-java-exec-1.9.0.jar:1.9.0]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:142)
~[drill-java-exec-1.9.0.jar:1.9.0]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119)
~[drill-java-exec-1.9.0.jar:1.9.0]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:109)
~[drill-java-exec-1.9.0.jar:1.9.0]
at
org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:51)
~[drill-java-exec-1.9.0.jar:1.9.0]
at
org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext(ProjectRecordBatch.java:135)
~[drill-java-exec-1.9.0.jar:1.9.0]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:162)
~[drill-java-exec-1.9.0.jar:1.9.0]


Regards,
*Anup Tiwari*

On Mon, Mar 6, 2017 at 7:30

Re: [Drill 1.9.0] : [CONNECTION ERROR] :- (user client) closed unexpectedly. Drillbit down?

[Anup Tiwari]

Hi John,

I have tried above config as well but still getting this issue.
And please note that we were using similar configuration params for Drill
1.6 where this issue was not coming.
Anything else which i can try?

Regards,
*Anup Tiwari*

On Fri, Mar 3, 2017 at 11:01 PM, Abhishek Girish <agir...@apache.org> wrote:

> +1 on John's suggestion.
>
> On Fri, Mar 3, 2017 at 6:24 AM, John Omernik <j...@omernik.com> wrote:
>
> > So your node has 32G of ram yet you are allowing Drill to use 36G.  I
> would
> > change your settings to be 8GB of Heap, and 22GB of Direct Memory. See if
> > this helps with your issues.  Also, are you using a distributed
> filesystem?
> > If so you may want to allow even more free ram...i.e. 8GB of Heap and
> 20GB
> > of Direct.
> >
> > On Fri, Mar 3, 2017 at 8:20 AM, Anup Tiwari <anup.tiw...@games24x7.com>
> > wrote:
> >
> > > Hi,
> > >
> > > Please find our configuration details :-
> > >
> > > Number of Nodes : 4
> > > RAM/Node : 32GB
> > > Core/Node : 8
> > > DRILL_MAX_DIRECT_MEMORY="20G"
> > > DRILL_HEAP="16G"
> > >
> > > And all other variables are set to default.
> > >
> > > Since we have tried some of the settings suggested above but still
> facing
> > > this issue more frequently, kindly suggest us what is best
> configuration
> > > for our environment.
> > >
> > > Regards,
> > > *Anup Tiwari*
> > >
> > > On Thu, Mar 2, 2017 at 1:26 AM, John Omernik <j...@omernik.com> wrote:
> > >
> > > > Another thing to consider is ensure you have a Spill Location setup,
> > and
> > > > then disable hashagg/hashjoin for the query...
> > > >
> > > > On Wed, Mar 1, 2017 at 1:25 PM, Abhishek Girish <agir...@apache.org>
> > > > wrote:
> > > >
> > > > > Hey Anup,
> > > > >
> > > > > This is indeed an issue, and I can understand that having an
> unstable
> > > > > environment is not something anyone wants. DRILL-4708 is still
> > > > unresolved -
> > > > > hopefully someone will get to it soon. I've bumped up the priority.
> > > > >
> > > > > Unfortunately we do not publish any sizing guidelines, so you'd
> have
> > to
> > > > > experiment to settle on the right load for your cluster. Please
> > > decrease
> > > > > the concurrency (number of queries running in parallel). And try
> > > bumping
> > > > up
> > > > > Drill DIRECT memory. Also, please set the system options
> recommended
> > by
> > > > > Sudheesh. While this may not solve the issue, it may help reduce
> it's
> > > > > occurrence.
> > > > >
> > > > > Can you also update the JIRA with your configurations, type of
> > queries
> > > > and
> > > > > the relevant logs?
> > > > >
> > > > > -Abhishek
> > > > >
> > > > > On Wed, Mar 1, 2017 at 10:17 AM, Anup Tiwari <
> > > anup.tiw...@games24x7.com>
> > > > > wrote:
> > > > >
> > > > > > Hi,
> > > > > >
> > > > > > Can someone look into it? As we are now getting this more
> > frequently
> > > in
> > > > > > Adhoc queries as well.
> > > > > > And for automation jobs, we are moving to Hive as in drill we are
> > > > getting
> > > > > > this more frequently.
> > > > > >
> > > > > > Regards,
> > > > > > *Anup Tiwari*
> > > > > >
> > > > > > On Sat, Dec 31, 2016 at 12:11 PM, Anup Tiwari <
> > > > anup.tiw...@games24x7.com
> > > > > >
> > > > > > wrote:
> > > > > >
> > > > > > > Hi,
> > > > > > >
> > > > > > > We are getting this issue bit more frequently. can someone
> please
> > > > look
> > > > > > > into it and tell us that why it is happening since as mention
> in
> > > > > earlier
> > > > > > > mail when this query gets executed no other query is running at
> > > that
> > > > > > time.
> > > > > > >
> > > > > > > Thanks in advance.
> > > > > > >
> > > > > &g

Re: [Drill 1.9.0] : [CONNECTION ERROR] :- (user client) closed unexpectedly. Drillbit down?

[Anup Tiwari]

Hi,

Please find our configuration details :-

Number of Nodes : 4
RAM/Node : 32GB
Core/Node : 8
DRILL_MAX_DIRECT_MEMORY="20G"
DRILL_HEAP="16G"

And all other variables are set to default.

Since we have tried some of the settings suggested above but still facing
this issue more frequently, kindly suggest us what is best configuration
for our environment.

Regards,
*Anup Tiwari*

On Thu, Mar 2, 2017 at 1:26 AM, John Omernik <j...@omernik.com> wrote:

> Another thing to consider is ensure you have a Spill Location setup, and
> then disable hashagg/hashjoin for the query...
>
> On Wed, Mar 1, 2017 at 1:25 PM, Abhishek Girish <agir...@apache.org>
> wrote:
>
> > Hey Anup,
> >
> > This is indeed an issue, and I can understand that having an unstable
> > environment is not something anyone wants. DRILL-4708 is still
> unresolved -
> > hopefully someone will get to it soon. I've bumped up the priority.
> >
> > Unfortunately we do not publish any sizing guidelines, so you'd have to
> > experiment to settle on the right load for your cluster. Please decrease
> > the concurrency (number of queries running in parallel). And try bumping
> up
> > Drill DIRECT memory. Also, please set the system options recommended by
> > Sudheesh. While this may not solve the issue, it may help reduce it's
> > occurrence.
> >
> > Can you also update the JIRA with your configurations, type of queries
> and
> > the relevant logs?
> >
> > -Abhishek
> >
> > On Wed, Mar 1, 2017 at 10:17 AM, Anup Tiwari <anup.tiw...@games24x7.com>
> > wrote:
> >
> > > Hi,
> > >
> > > Can someone look into it? As we are now getting this more frequently in
> > > Adhoc queries as well.
> > > And for automation jobs, we are moving to Hive as in drill we are
> getting
> > > this more frequently.
> > >
> > > Regards,
> > > *Anup Tiwari*
> > >
> > > On Sat, Dec 31, 2016 at 12:11 PM, Anup Tiwari <
> anup.tiw...@games24x7.com
> > >
> > > wrote:
> > >
> > > > Hi,
> > > >
> > > > We are getting this issue bit more frequently. can someone please
> look
> > > > into it and tell us that why it is happening since as mention in
> > earlier
> > > > mail when this query gets executed no other query is running at that
> > > time.
> > > >
> > > > Thanks in advance.
> > > >
> > > > Regards,
> > > > *Anup Tiwari*
> > > >
> > > > On Sat, Dec 24, 2016 at 10:20 AM, Anup Tiwari <
> > anup.tiw...@games24x7.com
> > > >
> > > > wrote:
> > > >
> > > >> Hi Sudheesh,
> > > >>
> > > >> Please find below ans :-
> > > >>
> > > >> 1. Total 4,(3 Datanodes, 1 namenode)
> > > >> 2. Only one query, as this query is part of daily dump and runs in
> > early
> > > >> morning.
> > > >>
> > > >> And as @chun mentioned , it seems similar to DRILL-4708 , so any
> > update
> > > >> on progress of this ticket?
> > > >>
> > > >>
> > > >> On 22-Dec-2016 12:13 AM, "Sudheesh Katkam" <skat...@maprtech.com>
> > > wrote:
> > > >>
> > > >> Two more questions..
> > > >>
> > > >> (1) How many nodes in your cluster?
> > > >> (2) How many queries are running when the failure is seen?
> > > >>
> > > >> If you have multiple large queries running at the same time, the
> load
> > on
> > > >> the system could cause those failures (which are heartbeat related).
> > > >>
> > > >> The two options I suggested decrease the parallelism of stages in a
> > > >> query, this implies lesser load but slower execution.
> > > >>
> > > >> System level option affect all queries, and session level affect
> > queries
> > > >> on a specific connection. Not sure what is preferred in your
> > > environment.
> > > >>
> > > >> Also, you may be interested in metrics. More info here:
> > > >>
> > > >> http://drill.apache.org/docs/monitoring-metrics/ <
> > > >> http://drill.apache.org/docs/monitoring-metrics/>
> > > >>
> > > >> Thank you,
> > > >> Sudheesh
> > > >>
> > > >> > On Dec 21, 2016, at 4:31 A

Re: [Drill 1.9.0] : [CONNECTION ERROR] :- (user client) closed unexpectedly. Drillbit down?

[Anup Tiwari]

Hi,

Can someone look into it? As we are now getting this more frequently in
Adhoc queries as well.
And for automation jobs, we are moving to Hive as in drill we are getting
this more frequently.

Regards,
*Anup Tiwari*

On Sat, Dec 31, 2016 at 12:11 PM, Anup Tiwari <anup.tiw...@games24x7.com>
wrote:

> Hi,
>
> We are getting this issue bit more frequently. can someone please look
> into it and tell us that why it is happening since as mention in earlier
> mail when this query gets executed no other query is running at that time.
>
> Thanks in advance.
>
> Regards,
> *Anup Tiwari*
>
> On Sat, Dec 24, 2016 at 10:20 AM, Anup Tiwari <anup.tiw...@games24x7.com>
> wrote:
>
>> Hi Sudheesh,
>>
>> Please find below ans :-
>>
>> 1. Total 4,(3 Datanodes, 1 namenode)
>> 2. Only one query, as this query is part of daily dump and runs in early
>> morning.
>>
>> And as @chun mentioned , it seems similar to DRILL-4708 , so any update
>> on progress of this ticket?
>>
>>
>> On 22-Dec-2016 12:13 AM, "Sudheesh Katkam" <skat...@maprtech.com> wrote:
>>
>> Two more questions..
>>
>> (1) How many nodes in your cluster?
>> (2) How many queries are running when the failure is seen?
>>
>> If you have multiple large queries running at the same time, the load on
>> the system could cause those failures (which are heartbeat related).
>>
>> The two options I suggested decrease the parallelism of stages in a
>> query, this implies lesser load but slower execution.
>>
>> System level option affect all queries, and session level affect queries
>> on a specific connection. Not sure what is preferred in your environment.
>>
>> Also, you may be interested in metrics. More info here:
>>
>> http://drill.apache.org/docs/monitoring-metrics/ <
>> http://drill.apache.org/docs/monitoring-metrics/>
>>
>> Thank you,
>> Sudheesh
>>
>> > On Dec 21, 2016, at 4:31 AM, Anup Tiwari <anup.tiw...@games24x7.com>
>> wrote:
>> >
>> > @sudheesh, yes drill bit is running on datanodeN/10.*.*.5:31010).
>> >
>> > Can you tell me how this will impact to query and do i have to set this
>> at
>> > session level OR system level?
>> >
>> >
>> >
>> > Regards,
>> > *Anup Tiwari*
>> >
>> > On Tue, Dec 20, 2016 at 11:59 PM, Chun Chang <cch...@maprtech.com>
>> wrote:
>> >
>> >> I am pretty sure this is the same as DRILL-4708.
>> >>
>> >> On Tue, Dec 20, 2016 at 10:27 AM, Sudheesh Katkam <
>> skat...@maprtech.com>
>> >> wrote:
>> >>
>> >>> Is the drillbit service (running on datanodeN/10.*.*.5:31010) actually
>> >>> down when the error is seen?
>> >>>
>> >>> If not, try lowering parallelism using these two session options,
>> before
>> >>> running the queries:
>> >>>
>> >>> planner.width.max_per_node (decrease this)
>> >>> planner.slice_target (increase this)
>> >>>
>> >>> Thank you,
>> >>> Sudheesh
>> >>>
>> >>>> On Dec 20, 2016, at 12:28 AM, Anup Tiwari <anup.tiw...@games24x7.com
>> >
>> >>> wrote:
>> >>>>
>> >>>> Hi Team,
>> >>>>
>> >>>> We are running some drill automation script on a daily basis and we
>> >> often
>> >>>> see that some query gets failed frequently by giving below error ,
>> >> Also i
>> >>>> came across DRILL-4708 <https://issues.apache.org/
>> >> jira/browse/DRILL-4708
>> >>>>
>> >>>> which seems similar, Can anyone give me update on that OR workaround
>> to
>> >>>> avoid such issue ?
>> >>>>
>> >>>> *Stack Trace :-*
>> >>>>
>> >>>> Error: CONNECTION ERROR: Connection /10.*.*.1:41613 <-->
>> >>>> datanodeN/10.*.*.5:31010 (user client) closed unexpectedly. Drillbit
>> >>> down?
>> >>>>
>> >>>>
>> >>>> [Error Id: 5089f2f1-0dfd-40f8-9fa0-8276c08be53f ] (state=,code=0)
>> >>>> java.sql.SQLException: CONNECTION ERROR: Connection /10.*.*.1:41613
>> >> <-->
>> >>>> datanodeN/10.*.*.5:31010 (user client) closed unexpectedly. Drillb
>> >>>> it down?
>> >>>>
>> >

Re: Storage Plugin for accessing Hive ORC Table from Drill

[Anup Tiwari]

First of all, sorry for late reply.

@Chunhui, you are right. we are using Hive 2.0

And are we planning to update hive libraries in next release of drill?

@rahul, as you said i have created another table with just "use stored as
orc" but all column and now drill is able to read it. Do you have any idea
why it worked now? Below is create table statement of new table, the
difference which i can observe is in TBLPROPERTIES ,partitioning and
bucketing:-

CREATE TABLE `logindetails_all_tmp`(
  `sid` char(40),
  `channel_id` tinyint,
  `c_t` bigint,
  `l_t` bigint)
ROW FORMAT SERDE
  'org.apache.hadoop.hive.ql.io.orc.OrcSerde'
STORED AS INPUTFORMAT
  'org.apache.hadoop.hive.ql.io.orc.OrcInputFormat'
OUTPUTFORMAT
  'org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat'
LOCATION
  'hdfs://hostname1:9000/usr/hive/warehouse/logindetails_all_tmp'
TBLPROPERTIES (
  'COLUMN_STATS_ACCURATE'='{\"BASIC_STATS\":\"true\"}',
  'numFiles'='3',
  'numRows'='1993254',
  'rawDataSize'='1143386232',
  'totalSize'='69876827',
  'transient_lastDdlTime'='1486640969');


Regards,
*Anup Tiwari*

On Sat, Jan 21, 2017 at 1:04 PM, Chunhui Shi <c...@mapr.com> wrote:

> I guess you are using Hive 2.0 as meta server while Drill has only 1.2
> libraries.
>
>
> In Hive 2.0 above, This delta format could have more than one '_' as
> separator while 1.2 has only one '_'.
>
>
> I think Drill should eventually update to use Hive's 2.0/2.1 libraries.
>
> 
> From: Anup Tiwari <anup.tiw...@games24x7.com>
> Sent: Friday, January 20, 2017 10:07:50 PM
> To: user@drill.apache.org; d...@drill.apache.org
> Subject: Re: Storage Plugin for accessing Hive ORC Table from Drill
>
> @Andries, We are using Hive 2.1.1 with Drill 1.9.0.
>
> @Zelaine, Could this be a problem in your Hive metastore?--> As i mentioned
> earlier, i am able to read hive parquet tables in Drill through hive
> storage plugin. So can you tell me a bit more like which type of
> configuration i am missing in metastore?
>
> Regards,
> *Anup Tiwari*
>
> On Sat, Jan 21, 2017 at 4:56 AM, Zelaine Fong <zf...@mapr.com> wrote:
>
> > The stack trace shows the following:
> >
> > Caused by: org.apache.drill.common.exceptions.DrillRuntimeException:
> > java.io.IOException: Failed to get numRows from HiveTable
> >
> > The Drill optimizer is trying to read rowcount information from Hive.
> > Could this be a problem in your Hive metastore?
> >
> > Has anyone else seen this before?
> >
> > -- Zelaine
> >
> > On 1/20/17, 7:35 AM, "Andries Engelbrecht" <aengelbre...@mapr.com>
> wrote:
> >
> > What version of Hive are you using?
> >
> >
> > --Andries
> >
> > 
> > From: Anup Tiwari <anup.tiw...@games24x7.com>
> > Sent: Friday, January 20, 2017 3:00:43 AM
> > To: user@drill.apache.org; d...@drill.apache.org
> > Subject: Re: Storage Plugin for accessing Hive ORC Table from Drill
> >
> > Hi,
> >
> > Please find below Create Table Statement and subsequent Drill Error
> :-
> >
> > *Table Structure :*
> >
> > CREATE TABLE `logindetails_all`(
> >   `sid` char(40),
> >   `channel_id` tinyint,
> >   `c_t` bigint,
> >   `l_t` bigint)
> > PARTITIONED BY (
> >   `login_date` char(10))
> > CLUSTERED BY (
> >   channel_id)
> > INTO 9 BUCKETS
> > ROW FORMAT SERDE
> >   'org.apache.hadoop.hive.ql.io.orc.OrcSerde'
> > STORED AS INPUTFORMAT
> >   'org.apache.hadoop.hive.ql.io.orc.OrcInputFormat'
> > OUTPUTFORMAT
> >   'org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat'
> > LOCATION
> >   'hdfs://hostname1:9000/usr/hive/warehouse/logindetails_all'
> > TBLPROPERTIES (
> >   'compactorthreshold.hive.compactor.delta.num.threshold'='6',
> >   'compactorthreshold.hive.compactor.delta.pct.threshold'='0.5',
> >   'transactional'='true',
> >   'transient_lastDdlTime'='1484313383');
> > ;
> >
> > *Drill Error :*
> >
> > *Query* : select * from hive.logindetails_all limit 1;
> >
> > *Error :*
> > 2017-01-20 16:21:12,625 [277e145e-c6bc-3372-01d0-
> 6c5b75b92d73:foreman]
> > INFO  o.a.drill.exec.work.foreman.Foreman - Query text for query id
> > 277e145e-c6bc-3372-01d0-6c5b75b92d73: select * from
> > hive.logindetails_all
> > limit 1
> > 2017-01-20 16:21:12,831 [277e145e-c6bc-3372-01d0-
> 6c5b75b92d73:foreman]
> > ERROR o.a.drill.exec.

Re: Storage Plugin for accessing Hive ORC Table from Drill

[Anup Tiwari]

can you point me to any specific line or sentence on that link?

Also please correct me if i am misinterpreting, but as written in 1st
line "*Drill
1.1 and later supports Hive 1.0*", does that mean Drill 1.1 and later
doesn't support OR partially support Hive 2.x?

Regards,
*Anup Tiwari*

On Sat, Jan 21, 2017 at 8:48 PM, Zelaine Fong <zf...@mapr.com> wrote:

> Have you taken a look at http://drill.apache.org/docs/hive-storage-plugin/
> ?
>
> -- Zelaine
>
> On 1/20/17, 10:07 PM, "Anup Tiwari" <anup.tiw...@games24x7.com> wrote:
>
> @Andries, We are using Hive 2.1.1 with Drill 1.9.0.
>
> @Zelaine, Could this be a problem in your Hive metastore?--> As i
> mentioned
> earlier, i am able to read hive parquet tables in Drill through hive
> storage plugin. So can you tell me a bit more like which type of
> configuration i am missing in metastore?
>
> Regards,
> *Anup Tiwari*
>
> On Sat, Jan 21, 2017 at 4:56 AM, Zelaine Fong <zf...@mapr.com> wrote:
>
> > The stack trace shows the following:
> >
> > Caused by: org.apache.drill.common.exceptions.DrillRuntimeException:
> > java.io.IOException: Failed to get numRows from HiveTable
> >
> > The Drill optimizer is trying to read rowcount information from Hive.
> > Could this be a problem in your Hive metastore?
> >
> > Has anyone else seen this before?
> >
> > -- Zelaine
> >
> > On 1/20/17, 7:35 AM, "Andries Engelbrecht" <aengelbre...@mapr.com>
> wrote:
> >
> > What version of Hive are you using?
> >
> >
> > --Andries
> >
> > 
> > From: Anup Tiwari <anup.tiw...@games24x7.com>
> > Sent: Friday, January 20, 2017 3:00:43 AM
> > To: user@drill.apache.org; d...@drill.apache.org
> > Subject: Re: Storage Plugin for accessing Hive ORC Table from
> Drill
> >
> > Hi,
> >
> > Please find below Create Table Statement and subsequent Drill
> Error :-
> >
> > *Table Structure :*
> >
> > CREATE TABLE `logindetails_all`(
> >   `sid` char(40),
> >   `channel_id` tinyint,
> >   `c_t` bigint,
> >   `l_t` bigint)
> > PARTITIONED BY (
> >   `login_date` char(10))
> > CLUSTERED BY (
> >   channel_id)
> > INTO 9 BUCKETS
> > ROW FORMAT SERDE
> >   'org.apache.hadoop.hive.ql.io.orc.OrcSerde'
> > STORED AS INPUTFORMAT
> >   'org.apache.hadoop.hive.ql.io.orc.OrcInputFormat'
> > OUTPUTFORMAT
> >   'org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat'
> > LOCATION
> >   'hdfs://hostname1:9000/usr/hive/warehouse/logindetails_all'
> > TBLPROPERTIES (
> >   'compactorthreshold.hive.compactor.delta.num.threshold'='6',
> >   'compactorthreshold.hive.compactor.delta.pct.threshold'='0.5',
> >   'transactional'='true',
> >   'transient_lastDdlTime'='1484313383');
> > ;
> >
> > *Drill Error :*
> >
> > *Query* : select * from hive.logindetails_all limit 1;
> >
> > *Error :*
> > 2017-01-20 16:21:12,625 [277e145e-c6bc-3372-01d0-
> 6c5b75b92d73:foreman]
> > INFO  o.a.drill.exec.work.foreman.Foreman - Query text for
> query id
> > 277e145e-c6bc-3372-01d0-6c5b75b92d73: select * from
> > hive.logindetails_all
> > limit 1
> > 2017-01-20 16:21:12,831 [277e145e-c6bc-3372-01d0-
> 6c5b75b92d73:foreman]
> > ERROR o.a.drill.exec.work.foreman.Foreman - SYSTEM ERROR:
> > NumberFormatException: For input string: "004_"
> >
> >
> > [Error Id: 53fa92e1-477e-45d2-b6f7-6eab9ef1da35 on
> > prod-hadoop-101.bom-prod.aws.games24x7.com:31010]
> > org.apache.drill.common.exceptions.UserException: SYSTEM ERROR:
> > NumberFormatException: For input string: "004_"
> >
> >
> > [Error Id: 53fa92e1-477e-45d2-b6f7-6eab9ef1da35 on
> > prod-hadoop-101.bom-prod.aws.games24x7.com:31010]
> > at
> > org.apache.drill.common.exceptions.UserException$
> > Builder.build(UserException.java:543)
> > ~[drill-common-1.9.0.jar:1.9.0]
> > at
>

Re: Storage Plugin for accessing Hive ORC Table from Drill

[Anup Tiwari]

@Andries, We are using Hive 2.1.1 with Drill 1.9.0.

@Zelaine, Could this be a problem in your Hive metastore?--> As i mentioned
earlier, i am able to read hive parquet tables in Drill through hive
storage plugin. So can you tell me a bit more like which type of
configuration i am missing in metastore?

Regards,
*Anup Tiwari*

On Sat, Jan 21, 2017 at 4:56 AM, Zelaine Fong <zf...@mapr.com> wrote:

> The stack trace shows the following:
>
> Caused by: org.apache.drill.common.exceptions.DrillRuntimeException:
> java.io.IOException: Failed to get numRows from HiveTable
>
> The Drill optimizer is trying to read rowcount information from Hive.
> Could this be a problem in your Hive metastore?
>
> Has anyone else seen this before?
>
> -- Zelaine
>
> On 1/20/17, 7:35 AM, "Andries Engelbrecht" <aengelbre...@mapr.com> wrote:
>
> What version of Hive are you using?
>
>
> --Andries
>
> 
> From: Anup Tiwari <anup.tiw...@games24x7.com>
> Sent: Friday, January 20, 2017 3:00:43 AM
> To: user@drill.apache.org; d...@drill.apache.org
> Subject: Re: Storage Plugin for accessing Hive ORC Table from Drill
>
> Hi,
>
> Please find below Create Table Statement and subsequent Drill Error :-
>
> *Table Structure :*
>
> CREATE TABLE `logindetails_all`(
>   `sid` char(40),
>   `channel_id` tinyint,
>   `c_t` bigint,
>   `l_t` bigint)
> PARTITIONED BY (
>   `login_date` char(10))
> CLUSTERED BY (
>   channel_id)
> INTO 9 BUCKETS
> ROW FORMAT SERDE
>   'org.apache.hadoop.hive.ql.io.orc.OrcSerde'
> STORED AS INPUTFORMAT
>   'org.apache.hadoop.hive.ql.io.orc.OrcInputFormat'
> OUTPUTFORMAT
>   'org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat'
> LOCATION
>   'hdfs://hostname1:9000/usr/hive/warehouse/logindetails_all'
> TBLPROPERTIES (
>   'compactorthreshold.hive.compactor.delta.num.threshold'='6',
>   'compactorthreshold.hive.compactor.delta.pct.threshold'='0.5',
>   'transactional'='true',
>   'transient_lastDdlTime'='1484313383');
> ;
>
> *Drill Error :*
>
> *Query* : select * from hive.logindetails_all limit 1;
>
> *Error :*
> 2017-01-20 16:21:12,625 [277e145e-c6bc-3372-01d0-6c5b75b92d73:foreman]
> INFO  o.a.drill.exec.work.foreman.Foreman - Query text for query id
> 277e145e-c6bc-3372-01d0-6c5b75b92d73: select * from
> hive.logindetails_all
> limit 1
> 2017-01-20 16:21:12,831 [277e145e-c6bc-3372-01d0-6c5b75b92d73:foreman]
> ERROR o.a.drill.exec.work.foreman.Foreman - SYSTEM ERROR:
> NumberFormatException: For input string: "004_"
>
>
> [Error Id: 53fa92e1-477e-45d2-b6f7-6eab9ef1da35 on
> prod-hadoop-101.bom-prod.aws.games24x7.com:31010]
> org.apache.drill.common.exceptions.UserException: SYSTEM ERROR:
> NumberFormatException: For input string: "004_"
>
>
> [Error Id: 53fa92e1-477e-45d2-b6f7-6eab9ef1da35 on
> prod-hadoop-101.bom-prod.aws.games24x7.com:31010]
> at
> org.apache.drill.common.exceptions.UserException$
> Builder.build(UserException.java:543)
> ~[drill-common-1.9.0.jar:1.9.0]
> at
> org.apache.drill.exec.work.foreman.Foreman$ForemanResult.
> close(Foreman.java:825)
> [drill-java-exec-1.9.0.jar:1.9.0]
> at
> org.apache.drill.exec.work.foreman.Foreman.moveToState(
> Foreman.java:935)
> [drill-java-exec-1.9.0.jar:1.9.0]
> at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.
> java:281)
> [drill-java-exec-1.9.0.jar:1.9.0]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(
> ThreadPoolExecutor.java:1142)
> [na:1.8.0_72]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(
> ThreadPoolExecutor.java:617)
> [na:1.8.0_72]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_72]
> Caused by: org.apache.drill.exec.work.foreman.ForemanException:
> Unexpected
> exception during fragment initialization: Internal error: Error while
> applying rule DrillPushProjIntoScan, args
> [rel#4220197:LogicalProject.NONE.ANY([]).[](input=rel#
> 4220196:Subset#0.ENUMERABLE.ANY([]).[],sid=$0,channel_id=$
> 1,c_t=$2,l_t=$3,login_date=$4),
> rel#4220181:EnumerableTableScan.ENUMERABLE.ANY([]).[](table=[hive,
> logindetails_all])]
> ... 4 common frames omitted
> Caused by: java.lang.AssertionError: Internal error: Error while
> applying
> rule DrillPushProjIntoScan, args
> [rel#4220197:LogicalProject.NONE.ANY([]).[](input=rel#
> 4220196:Subset#0

Re: Storage Plugin for accessing Hive ORC Table from Drill

[Anup Tiwari]

rames omitted
Caused by: java.io.IOException: Failed to get numRows from HiveTable
at
org.apache.drill.exec.store.hive.HiveMetadataProvider.getStats(HiveMetadataProvider.java:113)
~[drill-storage-hive-core-1.9.0.jar:1.9.0]
at
org.apache.drill.exec.store.hive.HiveScan.getScanStats(HiveScan.java:224)
~[drill-storage-hive-core-1.9.0.jar:1.9.0]
... 45 common frames omitted
Caused by: java.lang.RuntimeException: serious problem
at
org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.generateSplitsInfo(OrcInputFormat.java:1021)
~[drill-hive-exec-shaded-1.9.0.jar:1.9.0]
at
org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.getSplits(OrcInputFormat.java:1048)
~[drill-hive-exec-shaded-1.9.0.jar:1.9.0]
at
org.apache.drill.exec.store.hive.HiveMetadataProvider$1.run(HiveMetadataProvider.java:253)
~[drill-storage-hive-core-1.9.0.jar:1.9.0]
at
org.apache.drill.exec.store.hive.HiveMetadataProvider$1.run(HiveMetadataProvider.java:241)
~[drill-storage-hive-core-1.9.0.jar:1.9.0]
at java.security.AccessController.doPrivileged(Native Method)
~[na:1.8.0_72]
at javax.security.auth.Subject.doAs(Subject.java:422) ~[na:1.8.0_72]
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
~[hadoop-common-2.7.1.jar:na]
at
org.apache.drill.exec.store.hive.HiveMetadataProvider.splitInputWithUGI(HiveMetadataProvider.java:241)
~[drill-storage-hive-core-1.9.0.jar:1.9.0]
at
org.apache.drill.exec.store.hive.HiveMetadataProvider.getPartitionInputSplits(HiveMetadataProvider.java:142)
~[drill-storage-hive-core-1.9.0.jar:1.9.0]
at
org.apache.drill.exec.store.hive.HiveMetadataProvider.getStats(HiveMetadataProvider.java:105)
~[drill-storage-hive-core-1.9.0.jar:1.9.0]
... 46 common frames omitted
Caused by: java.util.concurrent.ExecutionException:
java.lang.NumberFormatException: For input string: "004_"
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
~[na:1.8.0_72]
at java.util.concurrent.FutureTask.get(FutureTask.java:192)
~[na:1.8.0_72]
at
org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.generateSplitsInfo(OrcInputFormat.java:998)
~[drill-hive-exec-shaded-1.9.0.jar:1.9.0]
... 55 common frames omitted
Caused by: java.lang.NumberFormatException: For input string: "004_"
at
java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
~[na:1.8.0_72]
at java.lang.Long.parseLong(Long.java:589) ~[na:1.8.0_72]
at java.lang.Long.parseLong(Long.java:631) ~[na:1.8.0_72]
at
org.apache.hadoop.hive.ql.io.AcidUtils.parseDelta(AcidUtils.java:310)
~[drill-hive-exec-shaded-1.9.0.jar:1.9.0]
at
org.apache.hadoop.hive.ql.io.AcidUtils.getAcidState(AcidUtils.java:379)
~[drill-hive-exec-shaded-1.9.0.jar:1.9.0]
at
org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$FileGenerator.call(OrcInputFormat.java:634)
~[drill-hive-exec-shaded-1.9.0.jar:1.9.0]
at
org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$FileGenerator.call(OrcInputFormat.java:620)
~[drill-hive-exec-shaded-1.9.0.jar:1.9.0]
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
~[na:1.8.0_72]
... 3 common frames omitted




Regards,
*Anup Tiwari*

On Thu, Jan 19, 2017 at 9:18 PM, Andries Engelbrecht <aengelbre...@mapr.com>
wrote:

> I have not seen issues reading Hive ORC data with Drill.
>
>
> What is the DDL for the table in Hive?
>
>
> --Andries
>
> 
> From: Anup Tiwari <anup.tiw...@games24x7.com>
> Sent: Thursday, January 19, 2017 12:49:20 AM
> To: user@drill.apache.org
> Cc: d...@drill.apache.org
> Subject: Re: Storage Plugin for accessing Hive ORC Table from Drill
>
> We have created a ORC format table in hive and we were trying to read it in
> drill through hive plugin, but it is giving us error. But with same hive
> plugin, we are able to read parquet table created in hive.
>
> So after searching a bit, i found a drill documentation link
> <https://drill.apache.org/docs/apache-drill-contribution-ideas/> which
> says
> that we have to create custom storage plugin to read ORC format tables. So
> can you tell me how to create custom storage plugin in this case?
>
>
>
> Regards,
> *Anup Tiwari*
>
> On Thu, Jan 19, 2017 at 1:55 PM, Nitin Pawar <nitinpawar...@gmail.com>
> wrote:
>
> > you want to use the ORC files created by hive directly in drill or you
> want
> > to use them through hive?
> >
> > On Thu, Jan 19, 2017 at 1:40 PM, Anup Tiwari <anup.tiw...@games24x7.com>
> > wrote:
> >
> > > +Dev
> > >
> > > Can someone help me in this?
> > >
> > > Regards,
> > > *Anup Tiwari*
> > >
> > > On Sun, Jan 15, 2017 at 2:21 PM, Anup Tiwari <
> anup.tiw...@games24x7.com>
> > > wrote:
> > >
> > > > Hi Team,
> > > >
> > > > Can someone tell me how to configure custom storage plugin in Drill
> for
> > > > accessing hive ORC tables?
> > > >
> > > > Thanks in advance!!
> > > >
> > > > Regards,
> > > > *Anup Tiwari*
> > > >
> > >
> >
> >
> >
> > --
> > Nitin Pawar
> >
>

Re: Storage Plugin for accessing Hive ORC Table from Drill

[Anup Tiwari]

We have created a ORC format table in hive and we were trying to read it in
drill through hive plugin, but it is giving us error. But with same hive
plugin, we are able to read parquet table created in hive.

So after searching a bit, i found a drill documentation link
<https://drill.apache.org/docs/apache-drill-contribution-ideas/> which says
that we have to create custom storage plugin to read ORC format tables. So
can you tell me how to create custom storage plugin in this case?



Regards,
*Anup Tiwari*

On Thu, Jan 19, 2017 at 1:55 PM, Nitin Pawar <nitinpawar...@gmail.com>
wrote:

> you want to use the ORC files created by hive directly in drill or you want
> to use them through hive?
>
> On Thu, Jan 19, 2017 at 1:40 PM, Anup Tiwari <anup.tiw...@games24x7.com>
> wrote:
>
> > +Dev
> >
> > Can someone help me in this?
> >
> > Regards,
> > *Anup Tiwari*
> >
> > On Sun, Jan 15, 2017 at 2:21 PM, Anup Tiwari <anup.tiw...@games24x7.com>
> > wrote:
> >
> > > Hi Team,
> > >
> > > Can someone tell me how to configure custom storage plugin in Drill for
> > > accessing hive ORC tables?
> > >
> > > Thanks in advance!!
> > >
> > > Regards,
> > > *Anup Tiwari*
> > >
> >
>
>
>
> --
> Nitin Pawar
>

Re: Storage Plugin for accessing Hive ORC Table from Drill

[Anup Tiwari]

+Dev

Can someone help me in this?

Regards,
*Anup Tiwari*

On Sun, Jan 15, 2017 at 2:21 PM, Anup Tiwari <anup.tiw...@games24x7.com>
wrote:

> Hi Team,
>
> Can someone tell me how to configure custom storage plugin in Drill for
> accessing hive ORC tables?
>
> Thanks in advance!!
>
> Regards,
> *Anup Tiwari*
>

Storage Plugin for accessing Hive ORC Table from Drill

[Anup Tiwari]

Hi Team,

Can someone tell me how to configure custom storage plugin in Drill for
accessing hive ORC tables?

Thanks in advance!!

Regards,
*Anup Tiwari*

Re: [Drill 1.9.0] : [CONNECTION ERROR] :- (user client) closed unexpectedly. Drillbit down?

[Anup Tiwari]

Hi,

We are getting this issue bit more frequently. can someone please look into
it and tell us that why it is happening since as mention in earlier mail
when this query gets executed no other query is running at that time.

Thanks in advance.

Regards,
*Anup Tiwari*

On Sat, Dec 24, 2016 at 10:20 AM, Anup Tiwari <anup.tiw...@games24x7.com>
wrote:

> Hi Sudheesh,
>
> Please find below ans :-
>
> 1. Total 4,(3 Datanodes, 1 namenode)
> 2. Only one query, as this query is part of daily dump and runs in early
> morning.
>
> And as @chun mentioned , it seems similar to DRILL-4708 , so any update
> on progress of this ticket?
>
>
> On 22-Dec-2016 12:13 AM, "Sudheesh Katkam" <skat...@maprtech.com> wrote:
>
> Two more questions..
>
> (1) How many nodes in your cluster?
> (2) How many queries are running when the failure is seen?
>
> If you have multiple large queries running at the same time, the load on
> the system could cause those failures (which are heartbeat related).
>
> The two options I suggested decrease the parallelism of stages in a query,
> this implies lesser load but slower execution.
>
> System level option affect all queries, and session level affect queries
> on a specific connection. Not sure what is preferred in your environment.
>
> Also, you may be interested in metrics. More info here:
>
> http://drill.apache.org/docs/monitoring-metrics/ <
> http://drill.apache.org/docs/monitoring-metrics/>
>
> Thank you,
> Sudheesh
>
> > On Dec 21, 2016, at 4:31 AM, Anup Tiwari <anup.tiw...@games24x7.com>
> wrote:
> >
> > @sudheesh, yes drill bit is running on datanodeN/10.*.*.5:31010).
> >
> > Can you tell me how this will impact to query and do i have to set this
> at
> > session level OR system level?
> >
> >
> >
> > Regards,
> > *Anup Tiwari*
> >
> > On Tue, Dec 20, 2016 at 11:59 PM, Chun Chang <cch...@maprtech.com>
> wrote:
> >
> >> I am pretty sure this is the same as DRILL-4708.
> >>
> >> On Tue, Dec 20, 2016 at 10:27 AM, Sudheesh Katkam <skat...@maprtech.com
> >
> >> wrote:
> >>
> >>> Is the drillbit service (running on datanodeN/10.*.*.5:31010) actually
> >>> down when the error is seen?
> >>>
> >>> If not, try lowering parallelism using these two session options,
> before
> >>> running the queries:
> >>>
> >>> planner.width.max_per_node (decrease this)
> >>> planner.slice_target (increase this)
> >>>
> >>> Thank you,
> >>> Sudheesh
> >>>
> >>>> On Dec 20, 2016, at 12:28 AM, Anup Tiwari <anup.tiw...@games24x7.com>
> >>> wrote:
> >>>>
> >>>> Hi Team,
> >>>>
> >>>> We are running some drill automation script on a daily basis and we
> >> often
> >>>> see that some query gets failed frequently by giving below error ,
> >> Also i
> >>>> came across DRILL-4708 <https://issues.apache.org/
> >> jira/browse/DRILL-4708
> >>>>
> >>>> which seems similar, Can anyone give me update on that OR workaround
> to
> >>>> avoid such issue ?
> >>>>
> >>>> *Stack Trace :-*
> >>>>
> >>>> Error: CONNECTION ERROR: Connection /10.*.*.1:41613 <-->
> >>>> datanodeN/10.*.*.5:31010 (user client) closed unexpectedly. Drillbit
> >>> down?
> >>>>
> >>>>
> >>>> [Error Id: 5089f2f1-0dfd-40f8-9fa0-8276c08be53f ] (state=,code=0)
> >>>> java.sql.SQLException: CONNECTION ERROR: Connection /10.*.*.1:41613
> >> <-->
> >>>> datanodeN/10.*.*.5:31010 (user client) closed unexpectedly. Drillb
> >>>> it down?
> >>>>
> >>>>
> >>>> [Error Id: 5089f2f1-0dfd-40f8-9fa0-8276c08be53f ]
> >>>>   at
> >>>> org.apache.drill.jdbc.impl.DrillCursor.nextRowInternally(
> >>> DrillCursor.java:232)
> >>>>   at
> >>>> org.apache.drill.jdbc.impl.DrillCursor.loadInitialSchema(
> >>> DrillCursor.java:275)
> >>>>   at
> >>>> org.apache.drill.jdbc.impl.DrillResultSetImpl.execute(
> >>> DrillResultSetImpl.java:1943)
> >>>>   at
> >>>> org.apache.drill.jdbc.impl.DrillResultSetImpl.execute(
> >>> DrillResultSetImpl.java:76)
> >>>>   at
> >>>> org.apache.calcite.

Re: [Drill 1.9.0] : [CONNECTION ERROR] :- (user client) closed unexpectedly. Drillbit down?

[Anup Tiwari]

Hi Sudheesh,

Please find below ans :-

1. Total 4,(3 Datanodes, 1 namenode)
2. Only one query, as this query is part of daily dump and runs in early
morning.

And as @chun mentioned , it seems similar to DRILL-4708 , so any update on
progress of this ticket?

On 22-Dec-2016 12:13 AM, "Sudheesh Katkam" <skat...@maprtech.com> wrote:

Two more questions..

(1) How many nodes in your cluster?
(2) How many queries are running when the failure is seen?

If you have multiple large queries running at the same time, the load on
the system could cause those failures (which are heartbeat related).

The two options I suggested decrease the parallelism of stages in a query,
this implies lesser load but slower execution.

System level option affect all queries, and session level affect queries on
a specific connection. Not sure what is preferred in your environment.

Also, you may be interested in metrics. More info here:

http://drill.apache.org/docs/monitoring-metrics/ <
http://drill.apache.org/docs/monitoring-metrics/>

Thank you,
Sudheesh

> On Dec 21, 2016, at 4:31 AM, Anup Tiwari <anup.tiw...@games24x7.com>
wrote:
>
> @sudheesh, yes drill bit is running on datanodeN/10.*.*.5:31010).
>
> Can you tell me how this will impact to query and do i have to set this at
> session level OR system level?
>
>
>
> Regards,
> *Anup Tiwari*
>
> On Tue, Dec 20, 2016 at 11:59 PM, Chun Chang <cch...@maprtech.com> wrote:
>
>> I am pretty sure this is the same as DRILL-4708.
>>
>> On Tue, Dec 20, 2016 at 10:27 AM, Sudheesh Katkam <skat...@maprtech.com>
>> wrote:
>>
>>> Is the drillbit service (running on datanodeN/10.*.*.5:31010) actually
>>> down when the error is seen?
>>>
>>> If not, try lowering parallelism using these two session options, before
>>> running the queries:
>>>
>>> planner.width.max_per_node (decrease this)
>>> planner.slice_target (increase this)
>>>
>>> Thank you,
>>> Sudheesh
>>>
>>>> On Dec 20, 2016, at 12:28 AM, Anup Tiwari <anup.tiw...@games24x7.com>
>>> wrote:
>>>>
>>>> Hi Team,
>>>>
>>>> We are running some drill automation script on a daily basis and we
>> often
>>>> see that some query gets failed frequently by giving below error ,
>> Also i
>>>> came across DRILL-4708 <https://issues.apache.org/
>> jira/browse/DRILL-4708
>>>>
>>>> which seems similar, Can anyone give me update on that OR workaround to
>>>> avoid such issue ?
>>>>
>>>> *Stack Trace :-*
>>>>
>>>> Error: CONNECTION ERROR: Connection /10.*.*.1:41613 <-->
>>>> datanodeN/10.*.*.5:31010 (user client) closed unexpectedly. Drillbit
>>> down?
>>>>
>>>>
>>>> [Error Id: 5089f2f1-0dfd-40f8-9fa0-8276c08be53f ] (state=,code=0)
>>>> java.sql.SQLException: CONNECTION ERROR: Connection /10.*.*.1:41613
>> <-->
>>>> datanodeN/10.*.*.5:31010 (user client) closed unexpectedly. Drillb
>>>> it down?
>>>>
>>>>
>>>> [Error Id: 5089f2f1-0dfd-40f8-9fa0-8276c08be53f ]
>>>>   at
>>>> org.apache.drill.jdbc.impl.DrillCursor.nextRowInternally(
>>> DrillCursor.java:232)
>>>>   at
>>>> org.apache.drill.jdbc.impl.DrillCursor.loadInitialSchema(
>>> DrillCursor.java:275)
>>>>   at
>>>> org.apache.drill.jdbc.impl.DrillResultSetImpl.execute(
>>> DrillResultSetImpl.java:1943)
>>>>   at
>>>> org.apache.drill.jdbc.impl.DrillResultSetImpl.execute(
>>> DrillResultSetImpl.java:76)
>>>>   at
>>>> org.apache.calcite.avatica.AvaticaConnection$1.execute(
>>> AvaticaConnection.java:473)
>>>>   at
>>>> org.apache.drill.jdbc.impl.DrillMetaImpl.prepareAndExecute(
>>> DrillMetaImpl.java:465)
>>>>   at
>>>> org.apache.calcite.avatica.AvaticaConnection.
>> prepareAndExecuteInternal(
>>> AvaticaConnection.java:477)
>>>>   at
>>>> org.apache.drill.jdbc.impl.DrillConnectionImpl.
>>> prepareAndExecuteInternal(DrillConnectionImpl.java:169)
>>>>   at
>>>> org.apache.calcite.avatica.AvaticaStatement.executeInternal(
>>> AvaticaStatement.java:109)
>>>>   at
>>>> org.apache.calcite.avatica.AvaticaStatement.execute(
>>> AvaticaStatement.java:121)
>>>>   at
>>>> org.apache.drill.jdbc.impl.DrillStatementImpl.execute(
>&

Re: [Drill 1.9.0] : [CONNECTION ERROR] :- (user client) closed unexpectedly. Drillbit down?

[Anup Tiwari]

@sudheesh, yes drill bit is running on datanodeN/10.*.*.5:31010).

Can you tell me how this will impact to query and do i have to set this at
session level OR system level?



Regards,
*Anup Tiwari*

On Tue, Dec 20, 2016 at 11:59 PM, Chun Chang <cch...@maprtech.com> wrote:

> I am pretty sure this is the same as DRILL-4708.
>
> On Tue, Dec 20, 2016 at 10:27 AM, Sudheesh Katkam <skat...@maprtech.com>
> wrote:
>
> > Is the drillbit service (running on datanodeN/10.*.*.5:31010) actually
> > down when the error is seen?
> >
> > If not, try lowering parallelism using these two session options, before
> > running the queries:
> >
> > planner.width.max_per_node (decrease this)
> > planner.slice_target (increase this)
> >
> > Thank you,
> > Sudheesh
> >
> > > On Dec 20, 2016, at 12:28 AM, Anup Tiwari <anup.tiw...@games24x7.com>
> > wrote:
> > >
> > > Hi Team,
> > >
> > > We are running some drill automation script on a daily basis and we
> often
> > > see that some query gets failed frequently by giving below error ,
> Also i
> > > came across DRILL-4708 <https://issues.apache.org/
> jira/browse/DRILL-4708
> > >
> > > which seems similar, Can anyone give me update on that OR workaround to
> > > avoid such issue ?
> > >
> > > *Stack Trace :-*
> > >
> > > Error: CONNECTION ERROR: Connection /10.*.*.1:41613 <-->
> > > datanodeN/10.*.*.5:31010 (user client) closed unexpectedly. Drillbit
> > down?
> > >
> > >
> > > [Error Id: 5089f2f1-0dfd-40f8-9fa0-8276c08be53f ] (state=,code=0)
> > > java.sql.SQLException: CONNECTION ERROR: Connection /10.*.*.1:41613
> <-->
> > > datanodeN/10.*.*.5:31010 (user client) closed unexpectedly. Drillb
> > > it down?
> > >
> > >
> > > [Error Id: 5089f2f1-0dfd-40f8-9fa0-8276c08be53f ]
> > >at
> > > org.apache.drill.jdbc.impl.DrillCursor.nextRowInternally(
> > DrillCursor.java:232)
> > >at
> > > org.apache.drill.jdbc.impl.DrillCursor.loadInitialSchema(
> > DrillCursor.java:275)
> > >at
> > > org.apache.drill.jdbc.impl.DrillResultSetImpl.execute(
> > DrillResultSetImpl.java:1943)
> > >at
> > > org.apache.drill.jdbc.impl.DrillResultSetImpl.execute(
> > DrillResultSetImpl.java:76)
> > >at
> > > org.apache.calcite.avatica.AvaticaConnection$1.execute(
> > AvaticaConnection.java:473)
> > >at
> > > org.apache.drill.jdbc.impl.DrillMetaImpl.prepareAndExecute(
> > DrillMetaImpl.java:465)
> > >at
> > > org.apache.calcite.avatica.AvaticaConnection.
> prepareAndExecuteInternal(
> > AvaticaConnection.java:477)
> > >at
> > > org.apache.drill.jdbc.impl.DrillConnectionImpl.
> > prepareAndExecuteInternal(DrillConnectionImpl.java:169)
> > >at
> > > org.apache.calcite.avatica.AvaticaStatement.executeInternal(
> > AvaticaStatement.java:109)
> > >at
> > > org.apache.calcite.avatica.AvaticaStatement.execute(
> > AvaticaStatement.java:121)
> > >at
> > > org.apache.drill.jdbc.impl.DrillStatementImpl.execute(
> > DrillStatementImpl.java:101)
> > >at sqlline.Commands.execute(Commands.java:841)
> > >at sqlline.Commands.sql(Commands.java:751)
> > >at sqlline.SqlLine.dispatch(SqlLine.java:746)
> > >at sqlline.SqlLine.runCommands(SqlLine.java:1651)
> > >at sqlline.Commands.run(Commands.java:1304)
> > >at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > >at
> > > sun.reflect.NativeMethodAccessorImpl.invoke(
> > NativeMethodAccessorImpl.java:62)
> > >at
> > > sun.reflect.DelegatingMethodAccessorImpl.invoke(
> > DelegatingMethodAccessorImpl.java:43)
> > >at java.lang.reflect.Method.invoke(Method.java:498)
> > >at
> > > sqlline.ReflectiveCommandHandler.execute(
> ReflectiveCommandHandler.java:
> > 36)
> > >at sqlline.SqlLine.dispatch(SqlLine.java:742)
> > >at sqlline.SqlLine.initArgs(SqlLine.java:553)
> > >at sqlline.SqlLine.begin(SqlLine.java:596)
> > >at sqlline.SqlLine.start(SqlLine.java:375)
> > >at sqlline.SqlLine.main(SqlLine.java:268)
> > > Caused by: org.apache.drill.common.exceptions.UserException:
> CONNECTION
> > > ERROR: Connection /10.*.*.1:41613 <-->

[Drill 1.9.0] : [CONNECTION ERROR] :- (user client) closed unexpectedly. Drillbit down?

[Anup Tiwari]

Hi Team,

We are running some drill automation script on a daily basis and we often
see that some query gets failed frequently by giving below error , Also i
came across DRILL-4708 <https://issues.apache.org/jira/browse/DRILL-4708>
which seems similar, Can anyone give me update on that OR workaround to
avoid such issue ?

*Stack Trace :-*

Error: CONNECTION ERROR: Connection /10.*.*.1:41613 <-->
datanodeN/10.*.*.5:31010 (user client) closed unexpectedly. Drillbit down?


[Error Id: 5089f2f1-0dfd-40f8-9fa0-8276c08be53f ] (state=,code=0)
java.sql.SQLException: CONNECTION ERROR: Connection /10.*.*.1:41613 <-->
datanodeN/10.*.*.5:31010 (user client) closed unexpectedly. Drillb
it down?


[Error Id: 5089f2f1-0dfd-40f8-9fa0-8276c08be53f ]
at
org.apache.drill.jdbc.impl.DrillCursor.nextRowInternally(DrillCursor.java:232)
at
org.apache.drill.jdbc.impl.DrillCursor.loadInitialSchema(DrillCursor.java:275)
at
org.apache.drill.jdbc.impl.DrillResultSetImpl.execute(DrillResultSetImpl.java:1943)
at
org.apache.drill.jdbc.impl.DrillResultSetImpl.execute(DrillResultSetImpl.java:76)
at
org.apache.calcite.avatica.AvaticaConnection$1.execute(AvaticaConnection.java:473)
at
org.apache.drill.jdbc.impl.DrillMetaImpl.prepareAndExecute(DrillMetaImpl.java:465)
at
org.apache.calcite.avatica.AvaticaConnection.prepareAndExecuteInternal(AvaticaConnection.java:477)
at
org.apache.drill.jdbc.impl.DrillConnectionImpl.prepareAndExecuteInternal(DrillConnectionImpl.java:169)
at
org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:109)
at
org.apache.calcite.avatica.AvaticaStatement.execute(AvaticaStatement.java:121)
at
org.apache.drill.jdbc.impl.DrillStatementImpl.execute(DrillStatementImpl.java:101)
at sqlline.Commands.execute(Commands.java:841)
at sqlline.Commands.sql(Commands.java:751)
at sqlline.SqlLine.dispatch(SqlLine.java:746)
at sqlline.SqlLine.runCommands(SqlLine.java:1651)
at sqlline.Commands.run(Commands.java:1304)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
sqlline.ReflectiveCommandHandler.execute(ReflectiveCommandHandler.java:36)
at sqlline.SqlLine.dispatch(SqlLine.java:742)
at sqlline.SqlLine.initArgs(SqlLine.java:553)
at sqlline.SqlLine.begin(SqlLine.java:596)
at sqlline.SqlLine.start(SqlLine.java:375)
at sqlline.SqlLine.main(SqlLine.java:268)
Caused by: org.apache.drill.common.exceptions.UserException: CONNECTION
ERROR: Connection /10.*.*.1:41613 <--> datanodeN/10.*.*.5:31010 (user
client) closed unexpectedly. Drillbit down?


[Error Id: 5089f2f1-0dfd-40f8-9fa0-8276c08be53f ]
at
org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:543)
at
org.apache.drill.exec.rpc.user.QueryResultHandler$ChannelClosedHandler$1.operationComplete(QueryResultHandler.java:373)
at
io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:680)
at
io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:603)
at
io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:563)
at
io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:406)
at
io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:82)
at
io.netty.channel.AbstractChannel$CloseFuture.setClosed(AbstractChannel.java:943)
at
io.netty.channel.AbstractChannel$AbstractUnsafe.doClose0(AbstractChannel.java:592)
at
io.netty.channel.AbstractChannel$AbstractUnsafe.close(AbstractChannel.java:584)
at
io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.closeOnRead(AbstractNioByteChannel.java:71)
at
io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.handleReadException(AbstractNioByteChannel.java:89)
at
io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:162)
at
io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:511)
at
io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468)
at
io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354)
at
io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111)
at java.lang.Thread.run(Thread.java:745)


Regards,
*Anup Tiwari*

Re: [1.9.0] : UserException: SYSTEM ERROR: IllegalReferenceCountException: refCnt: 0 and then SYSTEM ERROR: IOException: Failed to shutdown streamer

[Anup Tiwari]

Hi Aman,

Sorry for delayed response, since we are executing this query on our ~150GB
logs and as i have mentioned in trail mail, by executing "removed
conditions alone" CTAS got executed successfully, so i don't know which
sample data i should share(since i don't know pattern)?

Can you tell me in which scenarios we throw
"IllegalReferenceCountException" and how to handle this in different
scenarios?

Regards,
*Anup Tiwari*

On Thu, Dec 8, 2016 at 10:55 PM, Aman Sinha <amansi...@apache.org> wrote:

> Hi Anup,
> since your original query was working on 1.6 and failed in 1.9,  could you
> pls file a JIRA for this ?  It sounds like a regression related to
> evaluation of a Project expression (based on the stack trace).  Since there
> are several CASE exprs, quite likely something related to its evaluation.
> It would be great if you can provide some sample data for someone to
> debug.
> Thanks.
>
> On Thu, Dec 8, 2016 at 12:50 AM, Anup Tiwari <anup.tiw...@games24x7.com>
> wrote:
>
> > Hi,
> >
> > I have removed few conditions from my query then it just worked fine.
> >
> > Also can someone tell me in which scenarios we throw "
> > *IllegalReferenceCountException*" and how to handle it in different
> > scenarios ?
> >
> > As i got this in another query and by removing some conditions it worked
> > for me but when i execute that removed conditions alone in CTAS , it got
> > executed successfully.
> >
> > Regards,
> > *Anup Tiwari*
> >
> > On Wed, Dec 7, 2016 at 12:22 PM, Anup Tiwari <anup.tiw...@games24x7.com>
> > wrote:
> >
> > > Hi Team,
> > >
> > > I am getting below 2 error in my one of the query which was working
> fine
> > > on 1.6, Please help me out in this:-
> > >
> > > 1. UserException: SYSTEM ERROR: IllegalReferenceCountException:
> refCnt:
> > 0
> > > 2. SYSTEM ERROR: IOException: Failed to shutdown streamer
> > >
> > > Please find below query and its stack trace :-
> > >
> > > *Query :-*
> > >
> > > create table a_tt3_reg_login as
> > > select sessionid,
> > >
> > > count(distinct (case when ((( event = 'e.a' and ajaxUrl like
> > > '%/ab/pL%t=r%' ) or ( (Base64Conv(Response) like '%st%tr%' and
> > > Base64Conv(Response) not like '%error%') and ajaxUrl like '%/sign/ter%'
> > ))
> > > OR ( event = 'e.a' and ajaxUrl like '%/player/ter/ter.htm%' and
> > > Base64Conv(Response) like '%st%tr%ter%tr%')  OR (id = '/ter/thyou.htm'
> > and
> > > url = '/pla/natlob.htm')) then sessionid end) )  as  regs,
> > >
> > > count(distinct (case when ( ajaxUrl like '%/signup/poLo%t=log%' and
> event
> > > = 'e.a' ) or ( event = 'e.a' and ajaxUrl like
> '%j_spring_security_check%'
> > > and Base64Conv(Response)  like '%st%tr%') then sessionid end) ) as
> login
> > ,
> > >
> > > count(distinct (case when ((ajaxUrl like '/pl%/loadResponsePage.htm%fD=
> > true=yes%'
> > > or ajaxUrl like '/pl%/loadResponsePage.htm%fD=true=YES%') OR
> > (ajaxUrl
> > > like 'loadSuccessPage.do%fD=true=yes%' or ajaxUrl like
> > > 'loadSuccessPage.do%fD=true=YES%'))  then sessionid end) ) as fd ,
> > >
> > > count(distinct (case when ((ajaxUrl like '/pl%/loadResponsePage.htm%fD=
> > false=yes%'
> > > or ajaxUrl like '/pl%/loadResponsePage.htm%fD=false=YES%') OR
> > > (ajaxUrl like 'loadSuccessPage.do%fD=false=yes%' or ajaxUrl like
> > > 'loadSuccessPage.do%fD=false=YES%')) then sessionid end) ) as rd
> > >
> > > from
> > > tt2
> > > group by sessionid;
> > > Error: SYSTEM ERROR: IllegalReferenceCountException: refCnt: 0
> > >
> > > Fragment 14:19
> > >
> > > [Error Id: e4659753-f8d0-403c-9eec-0ff6f2e30dd9 on namenode:31010]
> > > (state=,code=0)
> > >
> > >
> > > *Stack Trace From Drillbit.log:-*
> > >
> > > [Error Id: e4659753-f8d0-403c-9eec-0ff6f2e30dd9 on namenode:31010]
> > > org.apache.drill.common.exceptions.UserException: SYSTEM ERROR:
> > > IllegalReferenceCountException: refCnt: 0
> > >
> > > Fragment 14:19
> > >
> > > [Error Id: e4659753-f8d0-403c-9eec-0ff6f2e30dd9 on namenode:31010]
> > > at org.apache.drill.common.exceptions.UserException$
> > > Builder.build(UserException.java:543) ~[drill-common-1.9.0.jar:1.9.0]
> > > at org.apache.drill.exec.work.fragment.FragmentExecutor.
> > > sendFinalState(FragmentExecutor.java:293)
> [dri

Re: [1.9.0] : UserException: SYSTEM ERROR: IllegalReferenceCountException: refCnt: 0 and then SYSTEM ERROR: IOException: Failed to shutdown streamer

[Anup Tiwari]

Hi,

I have removed few conditions from my query then it just worked fine.

Also can someone tell me in which scenarios we throw "
*IllegalReferenceCountException*" and how to handle it in different
scenarios ?

As i got this in another query and by removing some conditions it worked
for me but when i execute that removed conditions alone in CTAS , it got
executed successfully.

Regards,
*Anup Tiwari*

On Wed, Dec 7, 2016 at 12:22 PM, Anup Tiwari <anup.tiw...@games24x7.com>
wrote:

> Hi Team,
>
> I am getting below 2 error in my one of the query which was working fine
> on 1.6, Please help me out in this:-
>
> 1. UserException: SYSTEM ERROR: IllegalReferenceCountException: refCnt: 0
> 2. SYSTEM ERROR: IOException: Failed to shutdown streamer
>
> Please find below query and its stack trace :-
>
> *Query :-*
>
> create table a_tt3_reg_login as
> select sessionid,
>
> count(distinct (case when ((( event = 'e.a' and ajaxUrl like
> '%/ab/pL%t=r%' ) or ( (Base64Conv(Response) like '%st%tr%' and
> Base64Conv(Response) not like '%error%') and ajaxUrl like '%/sign/ter%' ))
> OR ( event = 'e.a' and ajaxUrl like '%/player/ter/ter.htm%' and
> Base64Conv(Response) like '%st%tr%ter%tr%')  OR (id = '/ter/thyou.htm' and
> url = '/pla/natlob.htm')) then sessionid end) )  as  regs,
>
> count(distinct (case when ( ajaxUrl like '%/signup/poLo%t=log%' and event
> = 'e.a' ) or ( event = 'e.a' and ajaxUrl like '%j_spring_security_check%'
> and Base64Conv(Response)  like '%st%tr%') then sessionid end) ) as login ,
>
> count(distinct (case when ((ajaxUrl like 
> '/pl%/loadResponsePage.htm%fD=true=yes%'
> or ajaxUrl like '/pl%/loadResponsePage.htm%fD=true=YES%') OR (ajaxUrl
> like 'loadSuccessPage.do%fD=true=yes%' or ajaxUrl like
> 'loadSuccessPage.do%fD=true=YES%'))  then sessionid end) ) as fd ,
>
> count(distinct (case when ((ajaxUrl like 
> '/pl%/loadResponsePage.htm%fD=false=yes%'
> or ajaxUrl like '/pl%/loadResponsePage.htm%fD=false=YES%') OR
> (ajaxUrl like 'loadSuccessPage.do%fD=false=yes%' or ajaxUrl like
> 'loadSuccessPage.do%fD=false=YES%')) then sessionid end) ) as rd
>
> from
> tt2
> group by sessionid;
> Error: SYSTEM ERROR: IllegalReferenceCountException: refCnt: 0
>
> Fragment 14:19
>
> [Error Id: e4659753-f8d0-403c-9eec-0ff6f2e30dd9 on namenode:31010]
> (state=,code=0)
>
>
> *Stack Trace From Drillbit.log:-*
>
> [Error Id: e4659753-f8d0-403c-9eec-0ff6f2e30dd9 on namenode:31010]
> org.apache.drill.common.exceptions.UserException: SYSTEM ERROR:
> IllegalReferenceCountException: refCnt: 0
>
> Fragment 14:19
>
> [Error Id: e4659753-f8d0-403c-9eec-0ff6f2e30dd9 on namenode:31010]
> at org.apache.drill.common.exceptions.UserException$
> Builder.build(UserException.java:543) ~[drill-common-1.9.0.jar:1.9.0]
> at org.apache.drill.exec.work.fragment.FragmentExecutor.
> sendFinalState(FragmentExecutor.java:293) [drill-java-exec-1.9.0.jar:1.
> 9.0]
> at org.apache.drill.exec.work.fragment.FragmentExecutor.
> cleanup(FragmentExecutor.java:160) [drill-java-exec-1.9.0.jar:1.9.0]
> at 
> org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:262)
> [drill-java-exec-1.9.0.jar:1.9.0]
> at 
> org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38)
> [drill-common-1.9.0.jar:1.9.0]
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [na:1.8.0_74]
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [na:1.8.0_74]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_74]
> Caused by: io.netty.util.IllegalReferenceCountException: refCnt: 0
> at 
> io.netty.buffer.AbstractByteBuf.ensureAccessible(AbstractByteBuf.java:1178)
> ~[netty-buffer-4.0.27.Final.jar:4.0.27.Final]
> at io.netty.buffer.DrillBuf.checkIndexD(DrillBuf.java:115)
> ~[drill-memory-base-1.9.0.jar:4.0.27.Final]
> at io.netty.buffer.DrillBuf.chk(DrillBuf.java:147)
> ~[drill-memory-base-1.9.0.jar:4.0.27.Final]
> at io.netty.buffer.DrillBuf.getByte(DrillBuf.java:775)
> ~[drill-memory-base-1.9.0.jar:4.0.27.Final]
> at org.apache.drill.exec.expr.fn.impl.CharSequenceWrapper.
> isAscii(CharSequenceWrapper.java:143) ~[drill-java-exec-1.9.0.jar:1.9.0]
> at org.apache.drill.exec.expr.fn.impl.CharSequenceWrapper.
> setBuffer(CharSequenceWrapper.java:106) ~[drill-java-exec-1.9.0.jar:1.9.0]
> at org.apache.drill.exec.test.generated.ProjectorGen980.
> doEval(ProjectorTemplate.java:776) ~[na:na]
> at org.apache.drill.exec.test.generated.ProjectorGen980.
> projectRecords(ProjectorTemplate.java:62) ~[na:na]

[1.9.0] : UserException: SYSTEM ERROR: IllegalReferenceCountException: refCnt: 0 and then SYSTEM ERROR: IOException: Failed to shutdown streamer

[Anup Tiwari]

e.drill.exec.work.fragment.FragmentExecutor.cleanup(FragmentExecutor.java:160)
[drill-java-exec-1.9.0.jar:1.9.0]
at
org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:262)
[drill-java-exec-1.9.0.jar:1.9.0]
at
org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38)
[drill-common-1.9.0.jar:1.9.0]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[na:1.8.0_74]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[na:1.8.0_74]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_74]
Caused by: java.io.IOException: Failed to shutdown streamer
at
org.apache.hadoop.hdfs.DFSOutputStream.closeThreads(DFSOutputStream.java:2187)
~[hadoop-hdfs-2.7.1.jar:na]
at
org.apache.hadoop.hdfs.DFSOutputStream.closeImpl(DFSOutputStream.java:2235)
~[hadoop-hdfs-2.7.1.jar:na]
at
org.apache.hadoop.hdfs.DFSOutputStream.close(DFSOutputStream.java:2204)
~[hadoop-hdfs-2.7.1.jar:na]
at
org.apache.hadoop.fs.FSDataOutputStream$PositionCache.close(FSDataOutputStream.java:72)
~[hadoop-common-2.7.1.jar:na]
at
org.apache.hadoop.fs.FSDataOutputStream.close(FSDataOutputStream.java:106)
~[hadoop-common-2.7.1.jar:na]
at
org.apache.drill.exec.store.easy.json.JsonRecordWriter.cleanup(JsonRecordWriter.java:246)
~[drill-java-exec-1.9.0.jar:1.9.0]
at
org.apache.drill.exec.physical.impl.WriterRecordBatch.closeWriter(WriterRecordBatch.java:180)
~[drill-java-exec-1.9.0.jar:1.9.0]
at
org.apache.drill.exec.physical.impl.WriterRecordBatch.innerNext(WriterRecordBatch.java:128)
~[drill-java-exec-1.9.0.jar:1.9.0]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:162)
~[drill-java-exec-1.9.0.jar:1.9.0]
at
org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:104)
~[drill-java-exec-1.9.0.jar:1.9.0]
at
org.apache.drill.exec.physical.impl.SingleSenderCreator$SingleSenderRootExec.innerNext(SingleSenderCreator.java:92)
~[drill-java-exec-1.9.0.jar:1.9.0]
at
org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:94)
~[drill-java-exec-1.9.0.jar:1.9.0]
at
org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:232)
~[drill-java-exec-1.9.0.jar:1.9.0]
at
org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:226)
~[drill-java-exec-1.9.0.jar:1.9.0]
at java.security.AccessController.doPrivileged(Native Method)
~[na:1.8.0_74]
at javax.security.auth.Subject.doAs(Subject.java:422) ~[na:1.8.0_74]
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
~[hadoop-common-2.7.1.jar:na]
at
org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:226)
[drill-java-exec-1.9.0.jar:1.9.0]
    ... 4 common frames omitted


Regards,
*Anup Tiwari*

Unable to connect Tableau 9.2 to Drill cluster using zookeeper quorum

[Anup Tiwari]

Hi Team,

I am trying to connect to my drill cluster from tableau using MapR Drill
ODBC Driver.

I followed steps given in
https://drill.apache.org/docs/using-apache-drill-with-tableau-9-server/ and
subsequent links and successfully connected to individual "direct drillbit"
reading docs. But when i am trying to connect to "zookeeper quorum" instead
of "direct drillbit", i am getting below error on MapR interface :

FAILED!
[MapR][Drill] (1010) Error occurred while trying to connect: [MapR][Drill]
(20) The hostname of '10.x.x.x' cannot be resolved. Please check your DNS
setup or connect directly to Drillbit.

Please note that since i am giving directly IP(Drill Hosts which are on
AWS) so i believe i don't have to maintain DNS entries in host file.

Also corresponding zookeeper logs are as follows :-

2016-12-01 18:08:42,541 [myid:3] - INFO  [NIOServerCxn.Factory:
0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@192] - Accepted socket connection
from /192.*.*.*:53159
2016-12-01 18:08:42,543 [myid:3] - WARN  [NIOServerCxn.Factory:
0.0.0.0/0.0.0.0:2181:ZooKeeperServer@854] - Connection request from old
client /192.*.*.*:53159; will be dropped if server is in r-o mode
2016-12-01 18:08:42,543 [myid:3] - INFO  [NIOServerCxn.Factory:
0.0.0.0/0.0.0.0:2181:ZooKeeperServer@900] - Client attempting to establish
new session at /192.*.*.*:53159
2016-12-01 18:08:42,546 [myid:3] - INFO
[CommitProcessor:3:ZooKeeperServer@645] - Established session
0x358ba2951720006 with negotiated timeout 3 for client /192.*.*.*:53159
2016-12-01 18:08:42,793 [myid:3] - WARN  [NIOServerCxn.Factory:
0.0.0.0/0.0.0.0:2181:NIOServerCnxn@357] - caught end of stream exception
EndOfStreamException: Unable to read additional data from client sessionid
0x358ba2951720006, likely client has closed socket
at
org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:230)
at
org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
at java.lang.Thread.run(Thread.java:745)
2016-12-01 18:08:42,794 [myid:3] - INFO  [NIOServerCxn.Factory:
0.0.0.0/0.0.0.0:2181:NIOServerCnxn@1008] - Closed socket connection for
client /192.*.*.*:53159 which had sessionid 0x358ba2951720006
2016-12-01 18:08:42,795 [myid:3] - ERROR
[CommitProcessor:3:NIOServerCnxn@178] - Unexpected Exception:
java.nio.channels.CancelledKeyException
at sun.nio.ch.SelectionKeyImpl.ensureValid(SelectionKeyImpl.java:73)
at sun.nio.ch.SelectionKeyImpl.interestOps(SelectionKeyImpl.java:77)
at
org.apache.zookeeper.server.NIOServerCnxn.sendBuffer(NIOServerCnxn.java:151)
at
org.apache.zookeeper.server.NIOServerCnxn.sendResponse(NIOServerCnxn.java:1082)
at
org.apache.zookeeper.server.FinalRequestProcessor.processRequest(FinalRequestProcessor.java:404)
at
org.apache.zookeeper.server.quorum.CommitProcessor.run(CommitProcessor.java:77)


I have gone through this link but this doesn't helped me :-
http://stackoverflow.com/questions/30940981/zookeeper-error-cannot-open-channel-to-x-at-election-address

Regards,
*Anup Tiwari*

Re: [Drill 1.8] : Can same key hold a JSON as well as plain value?

[Anup Tiwari]

keChannelRead(AbstractChannelHandlerContext.java:339)
at
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324)
at
io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:847)
at
io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:131)
at
io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:511)
at
io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468)
at
io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354)
at
io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111)
at java.lang.Thread.run(Thread.java:745)


Can anyone suggest any workaround or fix in these scenarios?

Regards,
*Anup Tiwari*

On Mon, Oct 17, 2016 at 10:42 PM, Abhishek Girish <abhishek.gir...@gmail.com
> wrote:

> Hey Anup,
>
> From the sample data you shared, it looks like you have JSON documents
> which differ considerably in the schema / structure. This isn't supported
> by default.
>
> You could try turning on UNION type (an experimental feature).
>
> > set `exec.enable_union_type` = true;
> +---+--+
> |  ok   | summary  |
> +---+--+
> | true  | exec.enable_union_type updated.  |
> +---+--+
> 1 row selected (0.193 seconds)
>
>
> On Mon, Oct 17, 2016 at 2:33 AM, Anup Tiwari <anup.tiw...@games24x7.com>
> wrote:
>
> > Hi Team,
> >
> > I have a data in which key name "event" normally consist of plain values
> > like 'load' OR "click" etc. Also from past few days in few JSON,
> > this("event" key) contains JSON value.
> >
> > And while reading event section, we are getting *DATA_READ ERROR: Error
> > parsing JSON. *Please provide solution asap.
> >
> >
> > Please find below eg :-
> >
> > *Case 1* :- {"body":{*"event":"click"*,"url":"https://www.rnd.com/
> > ","timestamp":1476514176276,"id":"abc","ajaxUrl":null,"
> > ajaxData":null,"metadata":null}
> >
> > *Case 2* :- {"body":{
> > *"event":{"isTrusted":true,"customTrackingData":"ACE-
> > click-MON-ABC_UL-NA"}*
> > ,"url":"https://www.rnd.com/test.html
> > ","timestamp":1476531412179,"id":"pqr","ajaxUrl":null,"
> > ajaxData":null,"metadata":null}
> >
> >
> >
> > *Error :*0: jdbc:drill:> select t.body from
> > dfs.`/user/hadoop/kafkalogs-stage/acl_15.json` t  limit 1;
> > Error: DATA_READ ERROR: Error parsing JSON - You tried to start when you
> > are using a ValueWriter of type NullableVarCharWriterImpl.
> >
> > File  /user/hadoop/kafkalogs-stage/acl_15.json
> > Record  120
> > Fragment 0:0
> >
> > [Error Id: bc33aac6-4cca-4237-9d01-01e1f1015572 on datanode1:31010]
> > (state=,code=0)
> > java.sql.SQLException: DATA_READ ERROR: Error parsing JSON - You tried to
> > start when you are using a ValueWriter of type NullableVarCharWriterImpl.
> >
> > File  /user/hadoop/kafkalogs-stage/acl_15.json
> > Record  120
> > Fragment 0:0
> >
> > [Error Id: bc33aac6-4cca-4237-9d01-01e1f1015572 on datanode1:31010]
> > at
> > org.apache.drill.jdbc.impl.DrillCursor.nextRowInternally(
> > DrillCursor.java:247)
> > at
> > org.apache.drill.jdbc.impl.DrillCursor.loadInitialSchema(
> > DrillCursor.java:290)
> > at
> > org.apache.drill.jdbc.impl.DrillResultSetImpl.execute(
> > DrillResultSetImpl.java:1933)
> > at
> > org.apache.drill.jdbc.impl.DrillResultSetImpl.execute(
> > DrillResultSetImpl.java:73)
> > at
> > net.hydromatic.avatica.AvaticaConnection.executeQueryInternal(
> > AvaticaConnection.java:404)
> > at
> > net.hydromatic.avatica.AvaticaStatement.executeQueryInternal(
> > AvaticaStatement.java:351)
> > at
> > net.hydromatic.avatica.AvaticaStatement.executeInternal(
> > AvaticaStatement.java:338)
> > at
> > net.hydromatic.avatica.AvaticaStatement.execute(
> AvaticaStatement.java:69)
> > at
> > org.apache.drill.jdbc.impl.DrillStatementImpl.execute(
> > DrillStatementImpl.java:101)
> > at sqlline.Commands.execute(Commands.java:841)
> > at sqlline.Commands.sql(Com

[Drill 1.8] : Can same key hold a JSON as well as plain value?

[Anup Tiwari]

  at
io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:242)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339)
at
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324)
at
io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339)
at
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324)
at
io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:847)
at
io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:131)
at
io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:511)
at
io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468)
at
io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354)
at
io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111)
at java.lang.Thread.run(Thread.java:745)


Regards,
*Anup Tiwari*

Re: [Drill 1.6] : Number format exception due to Empty String

[Anup Tiwari]

We will surely test this on 1.8 and let you know.. but by looking at JIRA ,
I thought it got fixed in 0.8, is it not the case?
Let me know if i missed anything and pls provide some quick fix which can
be applicable just by changing some variable like alter session mentioned
in trail mail..

On 15-Oct-2016 11:35 AM, "Nitin Pawar" <nitinpawar...@gmail.com> wrote:

is there an option where you can upgrade to 1.8 and test it?


On Sat, Oct 15, 2016 at 10:23 AM, Anup Tiwari <anup.tiw...@games24x7.com>
wrote:

> No.. on a parquet table..
>
> Regards,
> *Anup Tiwari*
>
> On Fri, Oct 14, 2016 at 6:23 PM, Nitin Pawar <nitinpawar...@gmail.com>
> wrote:
>
> > are you querying on csv files?
> >
> > On Fri, Oct 14, 2016 at 1:31 PM, Anup Tiwari <anup.tiw...@games24x7.com>
> > wrote:
> >
> > > Hi Team,
> > >
> > > I got number format exception for empty string and after googling i
> found
> > > that this issue is bit related to DRILL-1874
> > > <https://issues.apache.org/jira/browse/DRILL-1874> .
> > >
> > > Q) Is it fixed in 1.6 ? Since it is very basic problem which can occur
> > > anytime.
> > >
> > > After reading comments of drill-1874 , i used *alter session set
> > > planner.safe_cast_varchar = true;* but it is giving me below error,
> > Please
> > > let me know if i am missing anything :
> > >
> > >
> > > 0: jdbc:drill:> *alter session set planner.safe_cast_varchar = true*;
> > > Error: VALIDATION ERROR: The option 'planner.safe_cast_varchar' does
> not
> > > exist.
> > >
> > >
> > > [Error Id: b9a8bcc4-91e3-46e9-8cf8-55aada95383b on datanode3:31010]
> > > (state=,code=0)
> > > java.sql.SQLException: VALIDATION ERROR: The option
> > > 'planner.safe_cast_varchar' does not exist.
> > >
> > >
> > > [Error Id: b9a8bcc4-91e3-46e9-8cf8-55aada95383b on datanode3:31010]
> > >
> > >
> > > at
> > > org.apache.drill.jdbc.impl.DrillCursor.nextRowInternally(
> > > DrillCursor.java:247)
> > > at
> > > org.apache.drill.jdbc.impl.DrillCursor.loadInitialSchema(
> > > DrillCursor.java:290)
> > > at
> > > org.apache.drill.jdbc.impl.DrillResultSetImpl.execute(
> > > DrillResultSetImpl.java:1923)
> > > at
> > > org.apache.drill.jdbc.impl.DrillResultSetImpl.execute(
> > > DrillResultSetImpl.java:73)
> > > at
> > > net.hydromatic.avatica.AvaticaConnection.executeQueryInternal(
> > > AvaticaConnection.java:404)
> > > at
> > > net.hydromatic.avatica.AvaticaStatement.executeQueryInternal(
> > > AvaticaStatement.java:355)
> > > at
> > > net.hydromatic.avatica.AvaticaStatement.executeInternal(
> > > AvaticaStatement.java:338)
> > > at
> > > net.hydromatic.avatica.AvaticaStatement.execute(
> > AvaticaStatement.java:69)
> > > at
> > > org.apache.drill.jdbc.impl.DrillStatementImpl.execute(
> > > DrillStatementImpl.java:101)
> > > at sqlline.Commands.execute(Commands.java:841)
> > > at sqlline.Commands.sql(Commands.java:751)
> > > at sqlline.SqlLine.dispatch(SqlLine.java:746)
> > > at sqlline.SqlLine.begin(SqlLine.java:621)
> > > at sqlline.SqlLine.start(SqlLine.java:375)
> > > at sqlline.SqlLine.main(SqlLine.java:268)
> > > Caused by: org.apache.drill.common.exceptions.UserRemoteException:
> > > VALIDATION ERROR: The option 'planner.safe_cast_varchar' does not
> exist.
> > >
> > >
> > > [Error Id: b9a8bcc4-91e3-46e9-8cf8-55aada95383b on datanode3:31010]
> > >
> > >
> > > at
> > > org.apache.drill.exec.rpc.user.QueryResultHandler.resultArrived(
> > > QueryResultHandler.java:119)
> > > at
> > > org.apache.drill.exec.rpc.user.UserClient.handleReponse(
> > > UserClient.java:113)
> > > at
> > > org.apache.drill.exec.rpc.BasicClientWithConnection.handle(
> > > BasicClientWithConnection.java:46)
> > > at
> > > org.apache.drill.exec.rpc.BasicClientWithConnection.handle(
> > > BasicClientWithConnection.java:31)
> > > at org.apache.drill.exec.rpc.RpcBus.handle(RpcBus.java:67)
> > > at org.apache.drill.exec.rpc.RpcBus$RequestEvent.run(
> > RpcBus.java:374)
> > > at
> > > org.apache.drill.common.SerializedExecutor$RunnableProcessor.run(
> > > SerializedExecu

Re: [Drill 1.6] : Number format exception due to Empty String

[Anup Tiwari]

No.. on a parquet table..

Regards,
*Anup Tiwari*

On Fri, Oct 14, 2016 at 6:23 PM, Nitin Pawar <nitinpawar...@gmail.com>
wrote:

> are you querying on csv files?
>
> On Fri, Oct 14, 2016 at 1:31 PM, Anup Tiwari <anup.tiw...@games24x7.com>
> wrote:
>
> > Hi Team,
> >
> > I got number format exception for empty string and after googling i found
> > that this issue is bit related to DRILL-1874
> > <https://issues.apache.org/jira/browse/DRILL-1874> .
> >
> > Q) Is it fixed in 1.6 ? Since it is very basic problem which can occur
> > anytime.
> >
> > After reading comments of drill-1874 , i used *alter session set
> > planner.safe_cast_varchar = true;* but it is giving me below error,
> Please
> > let me know if i am missing anything :
> >
> >
> > 0: jdbc:drill:> *alter session set planner.safe_cast_varchar = true*;
> > Error: VALIDATION ERROR: The option 'planner.safe_cast_varchar' does not
> > exist.
> >
> >
> > [Error Id: b9a8bcc4-91e3-46e9-8cf8-55aada95383b on datanode3:31010]
> > (state=,code=0)
> > java.sql.SQLException: VALIDATION ERROR: The option
> > 'planner.safe_cast_varchar' does not exist.
> >
> >
> > [Error Id: b9a8bcc4-91e3-46e9-8cf8-55aada95383b on datanode3:31010]
> >
> >
> > at
> > org.apache.drill.jdbc.impl.DrillCursor.nextRowInternally(
> > DrillCursor.java:247)
> > at
> > org.apache.drill.jdbc.impl.DrillCursor.loadInitialSchema(
> > DrillCursor.java:290)
> > at
> > org.apache.drill.jdbc.impl.DrillResultSetImpl.execute(
> > DrillResultSetImpl.java:1923)
> > at
> > org.apache.drill.jdbc.impl.DrillResultSetImpl.execute(
> > DrillResultSetImpl.java:73)
> > at
> > net.hydromatic.avatica.AvaticaConnection.executeQueryInternal(
> > AvaticaConnection.java:404)
> > at
> > net.hydromatic.avatica.AvaticaStatement.executeQueryInternal(
> > AvaticaStatement.java:355)
> > at
> > net.hydromatic.avatica.AvaticaStatement.executeInternal(
> > AvaticaStatement.java:338)
> > at
> > net.hydromatic.avatica.AvaticaStatement.execute(
> AvaticaStatement.java:69)
> > at
> > org.apache.drill.jdbc.impl.DrillStatementImpl.execute(
> > DrillStatementImpl.java:101)
> > at sqlline.Commands.execute(Commands.java:841)
> > at sqlline.Commands.sql(Commands.java:751)
> > at sqlline.SqlLine.dispatch(SqlLine.java:746)
> > at sqlline.SqlLine.begin(SqlLine.java:621)
> > at sqlline.SqlLine.start(SqlLine.java:375)
> > at sqlline.SqlLine.main(SqlLine.java:268)
> > Caused by: org.apache.drill.common.exceptions.UserRemoteException:
> > VALIDATION ERROR: The option 'planner.safe_cast_varchar' does not exist.
> >
> >
> > [Error Id: b9a8bcc4-91e3-46e9-8cf8-55aada95383b on datanode3:31010]
> >
> >
> > at
> > org.apache.drill.exec.rpc.user.QueryResultHandler.resultArrived(
> > QueryResultHandler.java:119)
> > at
> > org.apache.drill.exec.rpc.user.UserClient.handleReponse(
> > UserClient.java:113)
> > at
> > org.apache.drill.exec.rpc.BasicClientWithConnection.handle(
> > BasicClientWithConnection.java:46)
> > at
> > org.apache.drill.exec.rpc.BasicClientWithConnection.handle(
> > BasicClientWithConnection.java:31)
> > at org.apache.drill.exec.rpc.RpcBus.handle(RpcBus.java:67)
> > at org.apache.drill.exec.rpc.RpcBus$RequestEvent.run(
> RpcBus.java:374)
> > at
> > org.apache.drill.common.SerializedExecutor$RunnableProcessor.run(
> > SerializedExecutor.java:89)
> > at
> > org.apache.drill.exec.rpc.RpcBus$SameExecutor.execute(RpcBus.java:252)
> > at
> > org.apache.drill.common.SerializedExecutor.execute(
> > SerializedExecutor.java:123)
> > at
> > org.apache.drill.exec.rpc.RpcBus$InboundHandler.decode(RpcBus.java:285)
> > at
> > org.apache.drill.exec.rpc.RpcBus$InboundHandler.decode(RpcBus.java:257)
> > at
> > io.netty.handler.codec.MessageToMessageDecoder.channelRead(
> > MessageToMessageDecoder.java:89)
> > at
> > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(
> > AbstractChannelHandlerContext.java:339)
> > at
> > io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(
> > AbstractChannelHandlerContext.java:324)
> > at
> > io.netty.handler.timeout.IdleStateHandler.channelRead(
> > IdleStateHandler.java:254)
> > at
> > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(
> > AbstractChannelHa

[Drill 1.6] : Number format exception due to Empty String

[Anup Tiwari]

.java:329)
at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:250)
at
io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111)
at java.lang.Thread.run(Thread.java:745)


Also when i am trying to exclude empty string i.e. *col_name <> ''* then it
is excluding null values as well.

Regards,
*Anup Tiwari*

Re: move drill log directory to HDFS

[Anup Tiwari]

Also please note that I have tried below in all node's drill-env.sh but its
not working.

export DRILL_LOG_DIR="hdfs://namenode:9000/tmp/drilllogs/"


Regards,
*Anup Tiwari*


On Fri, Aug 26, 2016 at 4:06 PM, Anup Tiwari <anup.tiw...@games24x7.com>
wrote:

> Hi All,
>
> We are trying to move drill logs directory from local file system to HDFS
> so that we can refer only one location rather than each node's log
> directory.
>
> Can anyone help me on this?
>
> Regards,
> *Anup Tiwari*
>
>

move drill log directory to HDFS

[Anup Tiwari]

Hi All,

We are trying to move drill logs directory from local file system to HDFS
so that we can refer only one location rather than each node's log
directory.

Can anyone help me on this?

Regards,
*Anup Tiwari*

Re: Fetch queries status from drill prompt

[Anup Tiwari]

thanks for link.. but till then is their any other way? OR way to read
zookeper logs in drill, as we are showing on profile UI?

Regards,
*Anup Tiwari*
Software Engineer(BI-Team),PlayGames24x7 Pvt Ltd

On Fri, Aug 19, 2016 at 6:40 PM, Khurram Faraaz <kfar...@maprtech.com>
wrote:

> There is an open JIRA DRILL-4258
> <https://issues.apache.org/jira/browse/DRILL-4258>
> Once that is fixed/implemented you can query from sys.queries or
> sys.statements to know about all SQL statements that are in different
> states.
>
> Khurram
>
> On Fri, Aug 19, 2016 at 4:45 PM, Anup Tiwari <anup.tiw...@games24x7.com>
> wrote:
>
> > Thanks for above info.. i found running queries information in Zookeeper
> > logs but i guess it is in binary or some other form.
> > Is their any way to read it in drill?
> >
> > Regards,
> > *Anup Tiwari*
> > Software Engineer(BI-Team),PlayGames24x7 Pvt Ltd
> >
> > On Thu, Aug 18, 2016 at 11:59 PM, Sudheesh Katkam <skat...@maprtech.com>
> > wrote:
> >
> > > Profiles of running queries are stored in Zookeeper (or the configured
> > > transient store).
> > >
> > > Thank you,
> > > Sudheesh
> > >
> > > > On Aug 18, 2016, at 11:23 AM, Anup Tiwari <anup.tiw...@games24x7.com
> >
> > > wrote:
> > > >
> > > > Thanks chun for info..
> > > >
> > > > But can you tell me from where, running queries status come on
> profile
> > > > user-interface(UI)? Because if it's coming on profile UI then it must
> > > have
> > > > some back end file or something like that..
> > > > On 18-Aug-2016 11:37 PM, "Chun Chang" <cch...@maprtech.com> wrote:
> > > >
> > > > Anup,
> > > >
> > > > I believe only when a query is in a "terminal", i.e.
> > > > cancelled/completed/failed state, then it is written to the
> > > > drillbit_queries.json file on the foreman node. If what you want to
> do
> > is
> > > > monitoring queries running on your cluster, your best bet is to
> > configure
> > > > your cluster to store profile information on HDFS and monitor through
> > > query
> > > > profile. Remember if you have a cluster, you will have a
> > > > drillbit_queries.json file on very cluster node where drillbit is
> > > running.
> > > > And each file only contains completed queries that were run on that
> > node
> > > as
> > > > foreman. You would have to aggregate to get the whole picture of your
> > > > cluster. Even that, you will not see running queries.
> > > >
> > > > Hope this helps.
> > > >
> > > > On Thu, Aug 18, 2016 at 12:34 AM, Anup Tiwari <
> > anup.tiw...@games24x7.com
> > > >
> > > > wrote:
> > > >
> > > >> Hi All,
> > > >>
> > > >> We want to see all types of queries which ran on drill cluster or
> > > > currently
> > > >> running from drill prompt, Can someone help us on this?
> > > >>
> > > >> To achieve above , we read the drill documentation and set up a
> > storage
> > > >> plugin to access local file system and able to query
> > > >> *"drillbit_queries.json"* log file, but in above file we are getting
> > > > status
> > > >> of all queries whose status is either "cancelled","completed" or
> > > "failed"
> > > >> but missing "running". At the same time we check drill profile
> > interface
> > > >> where we can see running queries.
> > > >>
> > > >> I am sure if we can see on User-Interface then it must be coming
> from
> > > >> somewhere.
> > > >>
> > > >> Kindly help me on this.
> > > >>
> > > >> Regards,
> > > >> *Anup Tiwari*
> > > >> Software Engineer(BI-Team),PlayGames24x7 Pvt Ltd
> > > >>
> > >
> > >
> >
>

Re: Fetch queries status from drill prompt

[Anup Tiwari]

Thanks chun for info..

But can you tell me from where, running queries status come on profile
user-interface(UI)? Because if it's coming on profile UI then it must have
some back end file or something like that..
On 18-Aug-2016 11:37 PM, "Chun Chang" <cch...@maprtech.com> wrote:

Anup,

I believe only when a query is in a "terminal", i.e.
cancelled/completed/failed state, then it is written to the
drillbit_queries.json file on the foreman node. If what you want to do is
monitoring queries running on your cluster, your best bet is to configure
your cluster to store profile information on HDFS and monitor through query
profile. Remember if you have a cluster, you will have a
drillbit_queries.json file on very cluster node where drillbit is running.
And each file only contains completed queries that were run on that node as
foreman. You would have to aggregate to get the whole picture of your
cluster. Even that, you will not see running queries.

Hope this helps.

On Thu, Aug 18, 2016 at 12:34 AM, Anup Tiwari <anup.tiw...@games24x7.com>
wrote:

> Hi All,
>
> We want to see all types of queries which ran on drill cluster or
currently
> running from drill prompt, Can someone help us on this?
>
> To achieve above , we read the drill documentation and set up a storage
> plugin to access local file system and able to query
> *"drillbit_queries.json"* log file, but in above file we are getting
status
> of all queries whose status is either "cancelled","completed" or "failed"
> but missing "running". At the same time we check drill profile interface
> where we can see running queries.
>
> I am sure if we can see on User-Interface then it must be coming from
> somewhere.
>
> Kindly help me on this.
>
> Regards,
> *Anup Tiwari*
> Software Engineer(BI-Team),PlayGames24x7 Pvt Ltd
>

Date-Time Conversion

[Anup Tiwari]

Hi All,

We have a column in table in which date time is coming in below format :-

Thu Jun 09 2016 17:00:25 GMT+0530 (IST)

We want to extract date-time in "-MM-dd hh:mm:ss" (2016-06-09
17:00:25") format.

As far as my knowledge their is no in build function to achieve this.
Kindly let me know how to do it.


Regards,
*Anup*

"java.lang.OutOfMemoryError: Java heap space" error which in-turn kills drill bit of one of the node

[Anup Tiwari]

Hi All,

Sometimes I am getting below error while creating a table in drill using a
hive table :-

*"*java.lang.OutOfMemoryError: Java heap space*"* which in-turn kills drill
bit of one of the node where i have executed respective query.

*Query Type :-*

create table glv_abc as select sessionid, max(serverTime) as max_serverTime
from hive.hive_logs_daily
where log_date = '2016-05-02'
group by sessionid;


Kindly help me in this.

Please find *output of drillbit.log* below :-

2016-05-03 15:33:15,628 [28d7890f-a7d6-b55e-3853-23f1ea828751:frag:2:12]
ERROR o.a.drill.common.CatastrophicFailure - Catastrophic Failure Occurr
ed, exiting. Information message: Unable to handle out of memory condition
in FragmentExecutor.
java.lang.OutOfMemoryError: Java heap space
at
hive.parquet.hadoop.ParquetFileReader$ConsecutiveChunkList.readAll(ParquetFileReader.java:755)
~[drill-hive-exec-shaded-1.6.0.jar:1.6.
0]
at
hive.parquet.hadoop.ParquetFileReader.readNextRowGroup(ParquetFileReader.java:494)
~[drill-hive-exec-shaded-1.6.0.jar:1.6.0]
at
hive.parquet.hadoop.InternalParquetRecordReader.checkRead(InternalParquetRecordReader.java:127)
~[drill-hive-exec-shaded-1.6.0.jar:1.6
.0]
at
hive.parquet.hadoop.InternalParquetRecordReader.nextKeyValue(InternalParquetRecordReader.java:208)
~[drill-hive-exec-shaded-1.6.0.jar:
1.6.0]
at
hive.parquet.hadoop.ParquetRecordReader.nextKeyValue(ParquetRecordReader.java:201)
~[drill-hive-exec-shaded-1.6.0.jar:1.6.0]
at
org.apache.hadoop.hive.ql.io.parquet.read.ParquetRecordReaderWrapper.next(ParquetRecordReaderWrapper.java:206)
~[drill-hive-exec-shade
d-1.6.0.jar:1.6.0]
at
org.apache.hadoop.hive.ql.io.parquet.read.ParquetRecordReaderWrapper.next(ParquetRecordReaderWrapper.java:62)
~[drill-hive-exec-shaded
-1.6.0.jar:1.6.0]
at
org.apache.drill.exec.store.hive.HiveRecordReader.next(HiveRecordReader.java:321)
~[drill-storage-hive-core-1.6.0.jar:1.6.0]
at
org.apache.drill.exec.physical.impl.ScanBatch.next(ScanBatch.java:191)
~[drill-java-exec-1.6.0.jar:1.6.0]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119)
~[drill-java-exec-1.6.0.jar:1.6.0]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:109)
~[drill-java-exec-1.6.0.jar:1.6.0]
at
org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:51)
~[drill-java-exec-1.6.0.jar:1.6.0]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:162)
~[drill-java-exec-1.6.0.jar:1.6.0]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119)
~[drill-java-exec-1.6.0.jar:1.6.0]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:109)
~[drill-java-exec-1.6.0.jar:1.6.0]
at
org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:51)
~[drill-java-exec-1.6.0.jar:1.6.0]
at
org.apache.drill.exec.physical.impl.svremover.RemovingRecordBatch.innerNext(RemovingRecordBatch.java:94)
~[drill-java-exec-1.6.0.jar:1
.6.0]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:162)
~[drill-java-exec-1.6.0.jar:1.6.0]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119)
~[drill-java-exec-1.6.0.jar:1.6.0]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:109)
~[drill-java-exec-1.6.0.jar:1.6.0]
at
org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:51)
~[drill-java-exec-1.6.0.jar:1.6.0]
at
org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext(ProjectRecordBatch.java:129)
~[drill-java-exec-1.6.0.jar:1.6.0]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:162)
~[drill-java-exec-1.6.0.jar:1.6.0]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119)
~[drill-java-exec-1.6.0.jar:1.6.0]
at
org.apache.drill.exec.test.generated.HashAggregatorGen731.doWork(HashAggTemplate.java:314)
~[na:na]
at
org.apache.drill.exec.physical.impl.aggregate.HashAggBatch.innerNext(HashAggBatch.java:133)
~[drill-java-exec-1.6.0.jar:1.6.0]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:162)
~[drill-java-exec-1.6.0.jar:1.6.0]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119)
~[drill-java-exec-1.6.0.jar:1.6.0]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:109)
~[drill-java-exec-1.6.0.jar:1.6.0]
at
org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:51)
~[drill-java-exec-1.6.0.jar:1.6.0]
at

Re: Two different output for Same condition

[Anup Tiwari]

Ok Team.. So it's a bug , please find below jira link:-

https://issues.apache.org/jira/browse/DRILL-4474
On 04-Mar-2016 11:23 PM, "Anup Tiwari" <anup.tiw...@games24x7.com> wrote:

> Hi Team,
>
> I am getting different output for same condition in drill.. In 1st query I
> have counted sessions in select clause directly with my desired conditions(
> t.id = '/confirm/btnYes/' and t.event = 'Click')and in 2nd one counting
> same along with another column..
> Please find below 2 queries with output:-
>
> 0: jdbc:drill:> select
> . . . . . . . > count(case when t.id = '/confirm/btnYes/' and t.event =
> 'Click' then sessionid end) as cnt
> . . . . . . . > from dfs.tmp.tt1 t
> . . . . . . . > ;
> +---+
> |   count   |
> +---+
> | 27645752  |
> +---+
> 1 row selected (0.281 seconds)
> 0: jdbc:drill:> select
> . . . . . . . > count(distinct sessionid),
> . . . . . . . > count(case when t.id = '/confirm/btnYes/' and t.event =
> 'Click' then sessionid end) as cnt
> . . . . . . . > from dfs.tmp.tt1 t
> . . . . . . . > ;
> +-+---+
> | EXPR$0  |cnt|
> +-+---+
> | 201941  | 37772844  |
> +-+---+
> 1 row selected (8.259 seconds)
>
> Kindly let me know if I am Missing anything.
>
>
> Regards,
>
> Anup
>

Re: Failed to Query Cassandra using Drill

[Anup Tiwari]

Can anyone help me on this?
On 21-Jan-2016 11:29 pm, "Anup Tiwari" <anup.tiw...@games24x7.com> wrote:

> @jim I have already  followed steps given in that post but its not working.
> On 21-Jan-2016 8:45 pm, "Devender Yadav" <dev@gmail.com> wrote:
>
>> This patch is not working with Drill 1.0+ version. Check details:
>>
>>
>> http://stackoverflow.com/questions/31017755/how-to-use-apache-drill-with-cassandra
>>
>>
>>
>>
>>
>> Regards,
>> Devender
>>
>> On Thu, Jan 21, 2016 at 7:57 PM, Jim Bates <jba...@maprtech.com> wrote:
>>
>> > Here is a post I saw on cassandra/drill a while back.
>> >
>> >
>> >
>> http://www.confusedcoders.com/bigdata/apache-drill/sql-on-cassandra-querying-cassandra-via-apache-drill
>> >
>> > On Thu, Jan 21, 2016 at 6:07 AM, Anup Tiwari <anupsdtiw...@gmail.com>
>> > wrote:
>> >
>> > > Hi,
>> > >
>> > > I am using Drill 1.2 and want to query Cassandra(2.1.9). Is there any
>> way
>> > > of doing that?
>> > >
>> > > Because after searching a lot I found that there is no proper storage
>> > > plugin is available to achieve this?
>> > >
>> > > Can anyone let me know how can I achieve this?
>> > >
>> > >
>> > > Regards,
>> > > Anup Tiwari
>> > >
>> >
>>
>

Re: Failed to Query Cassandra using Drill

[Anup Tiwari]

@jim I have already  followed steps given in that post but its not working.
On 21-Jan-2016 8:45 pm, "Devender Yadav" <dev@gmail.com> wrote:

> This patch is not working with Drill 1.0+ version. Check details:
>
>
> http://stackoverflow.com/questions/31017755/how-to-use-apache-drill-with-cassandra
>
>
>
>
>
> Regards,
> Devender
>
> On Thu, Jan 21, 2016 at 7:57 PM, Jim Bates <jba...@maprtech.com> wrote:
>
> > Here is a post I saw on cassandra/drill a while back.
> >
> >
> >
> http://www.confusedcoders.com/bigdata/apache-drill/sql-on-cassandra-querying-cassandra-via-apache-drill
> >
> > On Thu, Jan 21, 2016 at 6:07 AM, Anup Tiwari <anupsdtiw...@gmail.com>
> > wrote:
> >
> > > Hi,
> > >
> > > I am using Drill 1.2 and want to query Cassandra(2.1.9). Is there any
> way
> > > of doing that?
> > >
> > > Because after searching a lot I found that there is no proper storage
> > > plugin is available to achieve this?
> > >
> > > Can anyone let me know how can I achieve this?
> > >
> > >
> > > Regards,
> > > Anup Tiwari
> > >
> >
>