Re: Best way to set schema to handle different json structures

[Scott Kinney]

No i haven't. 
I'll give that a try.
Thanks.



Scott Kinney | DevOps
stem   |   m  510.282.1299
100 Rollins Road, Millbrae, California 94030

This e-mail and/or any attachments contain Stem, Inc. confidential and 
proprietary information and material for the sole use of the intended 
recipient(s). Any review, use or distribution that has not been expressly 
authorized by Stem, Inc. is strictly prohibited. If you are not the intended 
recipient, please contact the sender and delete all copies. Thank you.


From: rahul challapalli 
Sent: Monday, July 11, 2016 3:51 PM
To: user
Subject: Re: Best way to set schema to handle different json structures

Did you try creating a view with the merged schema? Then you can try
running all your queries on top of that view.

- Rahul

On Mon, Jul 11, 2016 at 3:23 PM, Scott Kinney  wrote:

> We have several different json structures we want to run queries across. I
> can take a sample of each and merge the json together as python
> dictionaries then write that out to a file and have drill read that file
> first to set the schema but I dont think this will be very practical as we
> will have out data in s3 in a name/year/month/day and I dont want to have
> to put this schema file in every directory is s3. that seems unmanageable.
>
>
> Is there a way to set the schema from a file before making a query via the
> REST api?
>
>
>
> 
> Scott Kinney | DevOps
> stem    |   m  510.282.1299
> 100 Rollins Road, Millbrae, California 94030
>
> This e-mail and/or any attachments contain Stem, Inc. confidential and
> proprietary information and material for the sole use of the intended
> recipient(s). Any review, use or distribution that has not been expressly
> authorized by Stem, Inc. is strictly prohibited. If you are not the
> intended recipient, please contact the sender and delete all copies. Thank
> you.
>

Re: Best way to set schema to handle different json structures

[rahul challapalli]

Did you try creating a view with the merged schema? Then you can try
running all your queries on top of that view.

- Rahul

On Mon, Jul 11, 2016 at 3:23 PM, Scott Kinney  wrote:

> We have several different json structures we want to run queries across. I
> can take a sample of each and merge the json together as python
> dictionaries then write that out to a file and have drill read that file
> first to set the schema but I dont think this will be very practical as we
> will have out data in s3 in a name/year/month/day and I dont want to have
> to put this schema file in every directory is s3. that seems unmanageable.
>
>
> Is there a way to set the schema from a file before making a query via the
> REST api?
>
>
>
> 
> Scott Kinney | DevOps
> stem    |   m  510.282.1299
> 100 Rollins Road, Millbrae, California 94030
>
> This e-mail and/or any attachments contain Stem, Inc. confidential and
> proprietary information and material for the sole use of the intended
> recipient(s). Any review, use or distribution that has not been expressly
> authorized by Stem, Inc. is strictly prohibited. If you are not the
> intended recipient, please contact the sender and delete all copies. Thank
> you.
>

Best way to set schema to handle different json structures

[Scott Kinney]

We have several different json structures we want to run queries across. I can 
take a sample of each and merge the json together as python dictionaries then 
write that out to a file and have drill read that file first to set the schema 
but I dont think this will be very practical as we will have out data in s3 in 
a name/year/month/day and I dont want to have to put this schema file in every 
directory is s3. that seems unmanageable.


Is there a way to set the schema from a file before making a query via the REST 
api?




Scott Kinney | DevOps
stem    |   m  510.282.1299
100 Rollins Road, Millbrae, California 94030

This e-mail and/or any attachments contain Stem, Inc. confidential and 
proprietary information and material for the sole use of the intended 
recipient(s). Any review, use or distribution that has not been expressly 
authorized by Stem, Inc. is strictly prohibited. If you are not the intended 
recipient, please contact the sender and delete all copies. Thank you.

Re: Drill JDBC Interpreter: Connection Refused

[Andries Engelbrecht]

Have you tried to connect to a drillbit directly from Zeppelin using the jdbc 
url jdbc:drill:drillbit=:31010?

What does your drill-override.conf file look like?


> On Jul 11, 2016, at 2:33 PM, Krishnaprasad A S  
> wrote:
> 
> I can see all the 4 drillbits in ui. Aso I tried the query in web ui before
> running it in zeppelin. In web ui it works currently. Then what may be the
> issue.?
> On Jul 12, 2016 3:00 AM, "Andries Engelbrecht" 
> wrote:
> 
> What happens if you try to connect to a drillbit directly?
> 
> This will help to see if it is a zk connection issue.
> 
> Also I assume the dill cluster is up and running, and if you go to the
> webUI it shows all drillbits in the cluster connected and running.
> 
> 
>> On Jul 11, 2016, at 2:27 PM, Krishnaprasad A S 
> wrote:
>> 
>> I went through the same link before configuring the interpreter, also
> there
>> is no security configured.
>> On Jul 12, 2016 2:48 AM, "Andries Engelbrecht" 
>> wrote:
>> 
>>> Do you have security configured on the Drill cluster? If so make sure to
>>> add the user and password info for the Drill connection.
>>> 
>>> Some good info for configuring Zeppelin with Drill here
>>> 
>>> https://community.mapr.com/docs/DOC-1493 <
>>> https://community.mapr.com/docs/DOC-1493>
>>> 
>>> --Andries
>>> 
>>> 
 On Jul 11, 2016, at 2:13 PM, Krishnaprasad A S <
>>> krishna.pra...@flytxt.com> wrote:
 
 My drill runs in clustered mode, with 4 drillbits running in 4 nodes. I
 started it using drillbit.sh start command.
 On Jul 12, 2016 2:18 AM, "Andries Engelbrecht" <
>>> aengelbre...@maprtech.com>
 wrote:
 
> Are you running Drill in embedded mode or clustered mode?
> 
> If in embedded mode you may want to try to connect directly to the
>>> drillbit
> jdbc:drill:drillbit=:31010
> 
> It looks like you are trying to connect to a zk with a drill cluster,
>>> and
> your setup may just be embedded mode.
> 
> --Andries
> 
> 
>> On Jul 11, 2016, at 12:55 PM, Krishnaprasad A S <
> krishna.pra...@flytxt.com> wrote:
>> 
>> hi,
>> I'm trying to create a drill interpreter in zeppelin using the
> existing
>> jdbc interpreter.
>> 
>> *drill.url = jdbc:drill:zk=:2181/drill/drillbits1drill.driver =
>> org.apache.drill.jdbc.Driver*
>> my drillbit runs on the same server as of zeppelin.
>> 
>> Added the dependency
> 'apache-drill-1.7.0/jars/drill-jdbc-all-1.7.0.jar'
> in
>> zeppelin
>> I'm the getting the following error while running a sample SQL from
>> zeppelin,
>> 
>> ERROR [2016-07-12 01:11:41,946] ({pool-1-thread-4}
>> NotebookServer.java[afterStatusChange]:1135) - Error
>> org.apache.zeppelin.interpreter.InterpreterException:
>> org.apache.zeppelin.interpreter.InterpreterException:
>> org.apache.thrift.transport.TTransportException:
> java.net.ConnectException:
>> Connection refused
>> at
>> 
> 
>>> 
> org.apache.zeppelin.interpreter.remote.RemoteInterpreter.init(RemoteInterpreter.java:165)
>> at
>> 
> 
>>> 
> org.apache.zeppelin.interpreter.remote.RemoteInterpreter.getFormType(RemoteInterpreter.java:328)
>> at
>> 
> 
>>> 
> org.apache.zeppelin.interpreter.LazyOpenInterpreter.getFormType(LazyOpenInterpreter.java:105)
>> at
> org.apache.zeppelin.notebook.Paragraph.jobRun(Paragraph.java:260)
>> at org.apache.zeppelin.scheduler.Job.run(Job.java:176)
>> at
>> 
> 
>>> 
> org.apache.zeppelin.scheduler.RemoteScheduler$JobRunner.run(RemoteScheduler.java:328)
>> at
>> 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>> at
>> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
>> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
>> at
>> 
> 
>>> 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
>> at
>> 
> 
>>> 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
>> at
>> 
> 
>>> 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
>> at
>> 
> 
>>> 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
>> at java.lang.Thread.run(Thread.java:722)
>> Caused by: org.apache.zeppelin.interpreter.InterpreterException:
>> org.apache.thrift.transport.TTransportException:
> java.net.ConnectException:
>> Connection refused
>> at
>> 
> 
>>> 
> org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:53)
>> at
>> 
> 
>>> 
> org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:37)
>> 

Re: Drill JDBC Interpreter: Connection Refused

[Krishnaprasad A S]

I can see all the 4 drillbits in ui. Aso I tried the query in web ui before
running it in zeppelin. In web ui it works currently. Then what may be the
issue.?
On Jul 12, 2016 3:00 AM, "Andries Engelbrecht" 
wrote:

What happens if you try to connect to a drillbit directly?

This will help to see if it is a zk connection issue.

Also I assume the dill cluster is up and running, and if you go to the
webUI it shows all drillbits in the cluster connected and running.


> On Jul 11, 2016, at 2:27 PM, Krishnaprasad A S 
wrote:
>
> I went through the same link before configuring the interpreter, also
there
> is no security configured.
> On Jul 12, 2016 2:48 AM, "Andries Engelbrecht" 
> wrote:
>
>> Do you have security configured on the Drill cluster? If so make sure to
>> add the user and password info for the Drill connection.
>>
>> Some good info for configuring Zeppelin with Drill here
>>
>> https://community.mapr.com/docs/DOC-1493 <
>> https://community.mapr.com/docs/DOC-1493>
>>
>> --Andries
>>
>>
>>> On Jul 11, 2016, at 2:13 PM, Krishnaprasad A S <
>> krishna.pra...@flytxt.com> wrote:
>>>
>>> My drill runs in clustered mode, with 4 drillbits running in 4 nodes. I
>>> started it using drillbit.sh start command.
>>> On Jul 12, 2016 2:18 AM, "Andries Engelbrecht" <
>> aengelbre...@maprtech.com>
>>> wrote:
>>>
 Are you running Drill in embedded mode or clustered mode?

 If in embedded mode you may want to try to connect directly to the
>> drillbit
 jdbc:drill:drillbit=:31010

 It looks like you are trying to connect to a zk with a drill cluster,
>> and
 your setup may just be embedded mode.

 --Andries


> On Jul 11, 2016, at 12:55 PM, Krishnaprasad A S <
 krishna.pra...@flytxt.com> wrote:
>
> hi,
> I'm trying to create a drill interpreter in zeppelin using the
existing
> jdbc interpreter.
>
> *drill.url = jdbc:drill:zk=:2181/drill/drillbits1drill.driver =
> org.apache.drill.jdbc.Driver*
> my drillbit runs on the same server as of zeppelin.
>
> Added the dependency
'apache-drill-1.7.0/jars/drill-jdbc-all-1.7.0.jar'
 in
> zeppelin
> I'm the getting the following error while running a sample SQL from
> zeppelin,
>
> ERROR [2016-07-12 01:11:41,946] ({pool-1-thread-4}
> NotebookServer.java[afterStatusChange]:1135) - Error
> org.apache.zeppelin.interpreter.InterpreterException:
> org.apache.zeppelin.interpreter.InterpreterException:
> org.apache.thrift.transport.TTransportException:
 java.net.ConnectException:
> Connection refused
>  at
>

>>
org.apache.zeppelin.interpreter.remote.RemoteInterpreter.init(RemoteInterpreter.java:165)
>  at
>

>>
org.apache.zeppelin.interpreter.remote.RemoteInterpreter.getFormType(RemoteInterpreter.java:328)
>  at
>

>>
org.apache.zeppelin.interpreter.LazyOpenInterpreter.getFormType(LazyOpenInterpreter.java:105)
>  at
 org.apache.zeppelin.notebook.Paragraph.jobRun(Paragraph.java:260)
>  at org.apache.zeppelin.scheduler.Job.run(Job.java:176)
>  at
>

>>
org.apache.zeppelin.scheduler.RemoteScheduler$JobRunner.run(RemoteScheduler.java:328)
>  at
>
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>  at
> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
>  at java.util.concurrent.FutureTask.run(FutureTask.java:166)
>  at
>

>>
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
>  at
>

>>
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
>  at
>

>>
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
>  at
>

>>
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
>  at java.lang.Thread.run(Thread.java:722)
> Caused by: org.apache.zeppelin.interpreter.InterpreterException:
> org.apache.thrift.transport.TTransportException:
 java.net.ConnectException:
> Connection refused
>  at
>

>>
org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:53)
>  at
>

>>
org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:37)
>  at
>

>>
org.apache.commons.pool2.BasePooledObjectFactory.makeObject(BasePooledObjectFactory.java:60)
>  at
>

>>
org.apache.commons.pool2.impl.GenericObjectPool.create(GenericObjectPool.java:861)
>  at
>

>>
org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:435)
>  at
>

>>
org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:363)
>

Re: Drill JDBC Interpreter: Connection Refused

[Andries Engelbrecht]

What happens if you try to connect to a drillbit directly?

This will help to see if it is a zk connection issue.

Also I assume the dill cluster is up and running, and if you go to the webUI it 
shows all drillbits in the cluster connected and running.


> On Jul 11, 2016, at 2:27 PM, Krishnaprasad A S  
> wrote:
> 
> I went through the same link before configuring the interpreter, also there
> is no security configured.
> On Jul 12, 2016 2:48 AM, "Andries Engelbrecht" 
> wrote:
> 
>> Do you have security configured on the Drill cluster? If so make sure to
>> add the user and password info for the Drill connection.
>> 
>> Some good info for configuring Zeppelin with Drill here
>> 
>> https://community.mapr.com/docs/DOC-1493 <
>> https://community.mapr.com/docs/DOC-1493>
>> 
>> --Andries
>> 
>> 
>>> On Jul 11, 2016, at 2:13 PM, Krishnaprasad A S <
>> krishna.pra...@flytxt.com> wrote:
>>> 
>>> My drill runs in clustered mode, with 4 drillbits running in 4 nodes. I
>>> started it using drillbit.sh start command.
>>> On Jul 12, 2016 2:18 AM, "Andries Engelbrecht" <
>> aengelbre...@maprtech.com>
>>> wrote:
>>> 
 Are you running Drill in embedded mode or clustered mode?
 
 If in embedded mode you may want to try to connect directly to the
>> drillbit
 jdbc:drill:drillbit=:31010
 
 It looks like you are trying to connect to a zk with a drill cluster,
>> and
 your setup may just be embedded mode.
 
 --Andries
 
 
> On Jul 11, 2016, at 12:55 PM, Krishnaprasad A S <
 krishna.pra...@flytxt.com> wrote:
> 
> hi,
> I'm trying to create a drill interpreter in zeppelin using the existing
> jdbc interpreter.
> 
> *drill.url = jdbc:drill:zk=:2181/drill/drillbits1drill.driver =
> org.apache.drill.jdbc.Driver*
> my drillbit runs on the same server as of zeppelin.
> 
> Added the dependency 'apache-drill-1.7.0/jars/drill-jdbc-all-1.7.0.jar'
 in
> zeppelin
> I'm the getting the following error while running a sample SQL from
> zeppelin,
> 
> ERROR [2016-07-12 01:11:41,946] ({pool-1-thread-4}
> NotebookServer.java[afterStatusChange]:1135) - Error
> org.apache.zeppelin.interpreter.InterpreterException:
> org.apache.zeppelin.interpreter.InterpreterException:
> org.apache.thrift.transport.TTransportException:
 java.net.ConnectException:
> Connection refused
>  at
> 
 
>> org.apache.zeppelin.interpreter.remote.RemoteInterpreter.init(RemoteInterpreter.java:165)
>  at
> 
 
>> org.apache.zeppelin.interpreter.remote.RemoteInterpreter.getFormType(RemoteInterpreter.java:328)
>  at
> 
 
>> org.apache.zeppelin.interpreter.LazyOpenInterpreter.getFormType(LazyOpenInterpreter.java:105)
>  at
 org.apache.zeppelin.notebook.Paragraph.jobRun(Paragraph.java:260)
>  at org.apache.zeppelin.scheduler.Job.run(Job.java:176)
>  at
> 
 
>> org.apache.zeppelin.scheduler.RemoteScheduler$JobRunner.run(RemoteScheduler.java:328)
>  at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>  at
> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
>  at java.util.concurrent.FutureTask.run(FutureTask.java:166)
>  at
> 
 
>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
>  at
> 
 
>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
>  at
> 
 
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
>  at
> 
 
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
>  at java.lang.Thread.run(Thread.java:722)
> Caused by: org.apache.zeppelin.interpreter.InterpreterException:
> org.apache.thrift.transport.TTransportException:
 java.net.ConnectException:
> Connection refused
>  at
> 
 
>> org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:53)
>  at
> 
 
>> org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:37)
>  at
> 
 
>> org.apache.commons.pool2.BasePooledObjectFactory.makeObject(BasePooledObjectFactory.java:60)
>  at
> 
 
>> org.apache.commons.pool2.impl.GenericObjectPool.create(GenericObjectPool.java:861)
>  at
> 
 
>> org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:435)
>  at
> 
 
>> org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:363)
>  at
> 
 
>> org.apache.zeppelin.interpreter.remote.RemoteInterpreterProcess.getClient(RemoteInterpreterProcess.java:184)
>  at
> 
 
>> 

Re: Drill JDBC Interpreter: Connection Refused

[Krishnaprasad A S]

I went through the same link before configuring the interpreter, also there
is no security configured.
On Jul 12, 2016 2:48 AM, "Andries Engelbrecht" 
wrote:

> Do you have security configured on the Drill cluster? If so make sure to
> add the user and password info for the Drill connection.
>
> Some good info for configuring Zeppelin with Drill here
>
> https://community.mapr.com/docs/DOC-1493 <
> https://community.mapr.com/docs/DOC-1493>
>
> --Andries
>
>
> > On Jul 11, 2016, at 2:13 PM, Krishnaprasad A S <
> krishna.pra...@flytxt.com> wrote:
> >
> > My drill runs in clustered mode, with 4 drillbits running in 4 nodes. I
> > started it using drillbit.sh start command.
> > On Jul 12, 2016 2:18 AM, "Andries Engelbrecht" <
> aengelbre...@maprtech.com>
> > wrote:
> >
> >> Are you running Drill in embedded mode or clustered mode?
> >>
> >> If in embedded mode you may want to try to connect directly to the
> drillbit
> >> jdbc:drill:drillbit=:31010
> >>
> >> It looks like you are trying to connect to a zk with a drill cluster,
> and
> >> your setup may just be embedded mode.
> >>
> >> --Andries
> >>
> >>
> >>> On Jul 11, 2016, at 12:55 PM, Krishnaprasad A S <
> >> krishna.pra...@flytxt.com> wrote:
> >>>
> >>> hi,
> >>> I'm trying to create a drill interpreter in zeppelin using the existing
> >>> jdbc interpreter.
> >>>
> >>> *drill.url = jdbc:drill:zk=:2181/drill/drillbits1drill.driver =
> >>> org.apache.drill.jdbc.Driver*
> >>> my drillbit runs on the same server as of zeppelin.
> >>>
> >>> Added the dependency 'apache-drill-1.7.0/jars/drill-jdbc-all-1.7.0.jar'
> >> in
> >>> zeppelin
> >>> I'm the getting the following error while running a sample SQL from
> >>> zeppelin,
> >>>
> >>> ERROR [2016-07-12 01:11:41,946] ({pool-1-thread-4}
> >>> NotebookServer.java[afterStatusChange]:1135) - Error
> >>> org.apache.zeppelin.interpreter.InterpreterException:
> >>> org.apache.zeppelin.interpreter.InterpreterException:
> >>> org.apache.thrift.transport.TTransportException:
> >> java.net.ConnectException:
> >>> Connection refused
> >>>   at
> >>>
> >>
> org.apache.zeppelin.interpreter.remote.RemoteInterpreter.init(RemoteInterpreter.java:165)
> >>>   at
> >>>
> >>
> org.apache.zeppelin.interpreter.remote.RemoteInterpreter.getFormType(RemoteInterpreter.java:328)
> >>>   at
> >>>
> >>
> org.apache.zeppelin.interpreter.LazyOpenInterpreter.getFormType(LazyOpenInterpreter.java:105)
> >>>   at
> >> org.apache.zeppelin.notebook.Paragraph.jobRun(Paragraph.java:260)
> >>>   at org.apache.zeppelin.scheduler.Job.run(Job.java:176)
> >>>   at
> >>>
> >>
> org.apache.zeppelin.scheduler.RemoteScheduler$JobRunner.run(RemoteScheduler.java:328)
> >>>   at
> >>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> >>>   at
> >>> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> >>>   at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> >>>   at
> >>>
> >>
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
> >>>   at
> >>>
> >>
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
> >>>   at
> >>>
> >>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> >>>   at
> >>>
> >>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> >>>   at java.lang.Thread.run(Thread.java:722)
> >>> Caused by: org.apache.zeppelin.interpreter.InterpreterException:
> >>> org.apache.thrift.transport.TTransportException:
> >> java.net.ConnectException:
> >>> Connection refused
> >>>   at
> >>>
> >>
> org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:53)
> >>>   at
> >>>
> >>
> org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:37)
> >>>   at
> >>>
> >>
> org.apache.commons.pool2.BasePooledObjectFactory.makeObject(BasePooledObjectFactory.java:60)
> >>>   at
> >>>
> >>
> org.apache.commons.pool2.impl.GenericObjectPool.create(GenericObjectPool.java:861)
> >>>   at
> >>>
> >>
> org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:435)
> >>>   at
> >>>
> >>
> org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:363)
> >>>   at
> >>>
> >>
> org.apache.zeppelin.interpreter.remote.RemoteInterpreterProcess.getClient(RemoteInterpreterProcess.java:184)
> >>>   at
> >>>
> >>
> org.apache.zeppelin.interpreter.remote.RemoteInterpreter.init(RemoteInterpreter.java:163)
> >>>   ... 13 more
> >>> Caused by: org.apache.thrift.transport.TTransportException:
> >>> java.net.ConnectException: Connection refused
> >>>   at org.apache.thrift.transport.TSocket.open(TSocket.java:187)
> >>>   at
> >>>
> >>
> org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:51)
> >>>   ... 20 more
> 

Re: Drill JDBC Interpreter: Connection Refused

[Andries Engelbrecht]

Do you have security configured on the Drill cluster? If so make sure to add 
the user and password info for the Drill connection.

Some good info for configuring Zeppelin with Drill here

https://community.mapr.com/docs/DOC-1493 


--Andries


> On Jul 11, 2016, at 2:13 PM, Krishnaprasad A S  
> wrote:
> 
> My drill runs in clustered mode, with 4 drillbits running in 4 nodes. I
> started it using drillbit.sh start command.
> On Jul 12, 2016 2:18 AM, "Andries Engelbrecht" 
> wrote:
> 
>> Are you running Drill in embedded mode or clustered mode?
>> 
>> If in embedded mode you may want to try to connect directly to the drillbit
>> jdbc:drill:drillbit=:31010
>> 
>> It looks like you are trying to connect to a zk with a drill cluster, and
>> your setup may just be embedded mode.
>> 
>> --Andries
>> 
>> 
>>> On Jul 11, 2016, at 12:55 PM, Krishnaprasad A S <
>> krishna.pra...@flytxt.com> wrote:
>>> 
>>> hi,
>>> I'm trying to create a drill interpreter in zeppelin using the existing
>>> jdbc interpreter.
>>> 
>>> *drill.url = jdbc:drill:zk=:2181/drill/drillbits1drill.driver =
>>> org.apache.drill.jdbc.Driver*
>>> my drillbit runs on the same server as of zeppelin.
>>> 
>>> Added the dependency 'apache-drill-1.7.0/jars/drill-jdbc-all-1.7.0.jar'
>> in
>>> zeppelin
>>> I'm the getting the following error while running a sample SQL from
>>> zeppelin,
>>> 
>>> ERROR [2016-07-12 01:11:41,946] ({pool-1-thread-4}
>>> NotebookServer.java[afterStatusChange]:1135) - Error
>>> org.apache.zeppelin.interpreter.InterpreterException:
>>> org.apache.zeppelin.interpreter.InterpreterException:
>>> org.apache.thrift.transport.TTransportException:
>> java.net.ConnectException:
>>> Connection refused
>>>   at
>>> 
>> org.apache.zeppelin.interpreter.remote.RemoteInterpreter.init(RemoteInterpreter.java:165)
>>>   at
>>> 
>> org.apache.zeppelin.interpreter.remote.RemoteInterpreter.getFormType(RemoteInterpreter.java:328)
>>>   at
>>> 
>> org.apache.zeppelin.interpreter.LazyOpenInterpreter.getFormType(LazyOpenInterpreter.java:105)
>>>   at
>> org.apache.zeppelin.notebook.Paragraph.jobRun(Paragraph.java:260)
>>>   at org.apache.zeppelin.scheduler.Job.run(Job.java:176)
>>>   at
>>> 
>> org.apache.zeppelin.scheduler.RemoteScheduler$JobRunner.run(RemoteScheduler.java:328)
>>>   at
>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>>>   at
>>> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
>>>   at java.util.concurrent.FutureTask.run(FutureTask.java:166)
>>>   at
>>> 
>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
>>>   at
>>> 
>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
>>>   at
>>> 
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
>>>   at
>>> 
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
>>>   at java.lang.Thread.run(Thread.java:722)
>>> Caused by: org.apache.zeppelin.interpreter.InterpreterException:
>>> org.apache.thrift.transport.TTransportException:
>> java.net.ConnectException:
>>> Connection refused
>>>   at
>>> 
>> org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:53)
>>>   at
>>> 
>> org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:37)
>>>   at
>>> 
>> org.apache.commons.pool2.BasePooledObjectFactory.makeObject(BasePooledObjectFactory.java:60)
>>>   at
>>> 
>> org.apache.commons.pool2.impl.GenericObjectPool.create(GenericObjectPool.java:861)
>>>   at
>>> 
>> org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:435)
>>>   at
>>> 
>> org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:363)
>>>   at
>>> 
>> org.apache.zeppelin.interpreter.remote.RemoteInterpreterProcess.getClient(RemoteInterpreterProcess.java:184)
>>>   at
>>> 
>> org.apache.zeppelin.interpreter.remote.RemoteInterpreter.init(RemoteInterpreter.java:163)
>>>   ... 13 more
>>> Caused by: org.apache.thrift.transport.TTransportException:
>>> java.net.ConnectException: Connection refused
>>>   at org.apache.thrift.transport.TSocket.open(TSocket.java:187)
>>>   at
>>> 
>> org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:51)
>>>   ... 20 more
>>> Caused by: java.net.ConnectException: Connection refused
>> 
>> 

Re: Drill JDBC Interpreter: Connection Refused

[Krishnaprasad A S]

My drill runs in clustered mode, with 4 drillbits running in 4 nodes. I
started it using drillbit.sh start command.
On Jul 12, 2016 2:18 AM, "Andries Engelbrecht" 
wrote:

> Are you running Drill in embedded mode or clustered mode?
>
> If in embedded mode you may want to try to connect directly to the drillbit
> jdbc:drill:drillbit=:31010
>
> It looks like you are trying to connect to a zk with a drill cluster, and
> your setup may just be embedded mode.
>
> --Andries
>
>
> > On Jul 11, 2016, at 12:55 PM, Krishnaprasad A S <
> krishna.pra...@flytxt.com> wrote:
> >
> > hi,
> > I'm trying to create a drill interpreter in zeppelin using the existing
> > jdbc interpreter.
> >
> > *drill.url = jdbc:drill:zk=:2181/drill/drillbits1drill.driver =
> > org.apache.drill.jdbc.Driver*
> > my drillbit runs on the same server as of zeppelin.
> >
> > Added the dependency 'apache-drill-1.7.0/jars/drill-jdbc-all-1.7.0.jar'
> in
> > zeppelin
> > I'm the getting the following error while running a sample SQL from
> > zeppelin,
> >
> > ERROR [2016-07-12 01:11:41,946] ({pool-1-thread-4}
> > NotebookServer.java[afterStatusChange]:1135) - Error
> > org.apache.zeppelin.interpreter.InterpreterException:
> > org.apache.zeppelin.interpreter.InterpreterException:
> > org.apache.thrift.transport.TTransportException:
> java.net.ConnectException:
> > Connection refused
> >at
> >
> org.apache.zeppelin.interpreter.remote.RemoteInterpreter.init(RemoteInterpreter.java:165)
> >at
> >
> org.apache.zeppelin.interpreter.remote.RemoteInterpreter.getFormType(RemoteInterpreter.java:328)
> >at
> >
> org.apache.zeppelin.interpreter.LazyOpenInterpreter.getFormType(LazyOpenInterpreter.java:105)
> >at
> org.apache.zeppelin.notebook.Paragraph.jobRun(Paragraph.java:260)
> >at org.apache.zeppelin.scheduler.Job.run(Job.java:176)
> >at
> >
> org.apache.zeppelin.scheduler.RemoteScheduler$JobRunner.run(RemoteScheduler.java:328)
> >at
> > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> >at
> > java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> >at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> >at
> >
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
> >at
> >
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
> >at
> >
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> >at
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> >at java.lang.Thread.run(Thread.java:722)
> > Caused by: org.apache.zeppelin.interpreter.InterpreterException:
> > org.apache.thrift.transport.TTransportException:
> java.net.ConnectException:
> > Connection refused
> >at
> >
> org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:53)
> >at
> >
> org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:37)
> >at
> >
> org.apache.commons.pool2.BasePooledObjectFactory.makeObject(BasePooledObjectFactory.java:60)
> >at
> >
> org.apache.commons.pool2.impl.GenericObjectPool.create(GenericObjectPool.java:861)
> >at
> >
> org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:435)
> >at
> >
> org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:363)
> >at
> >
> org.apache.zeppelin.interpreter.remote.RemoteInterpreterProcess.getClient(RemoteInterpreterProcess.java:184)
> >at
> >
> org.apache.zeppelin.interpreter.remote.RemoteInterpreter.init(RemoteInterpreter.java:163)
> >... 13 more
> > Caused by: org.apache.thrift.transport.TTransportException:
> > java.net.ConnectException: Connection refused
> >at org.apache.thrift.transport.TSocket.open(TSocket.java:187)
> >at
> >
> org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:51)
> >... 20 more
> > Caused by: java.net.ConnectException: Connection refused
>
>

Re: Drill JDBC Interpreter: Connection Refused

[Andries Engelbrecht]

Are you running Drill in embedded mode or clustered mode?

If in embedded mode you may want to try to connect directly to the drillbit
jdbc:drill:drillbit=:31010

It looks like you are trying to connect to a zk with a drill cluster, and your 
setup may just be embedded mode.

--Andries


> On Jul 11, 2016, at 12:55 PM, Krishnaprasad A S  
> wrote:
> 
> hi,
> I'm trying to create a drill interpreter in zeppelin using the existing
> jdbc interpreter.
> 
> *drill.url = jdbc:drill:zk=:2181/drill/drillbits1drill.driver =
> org.apache.drill.jdbc.Driver*
> my drillbit runs on the same server as of zeppelin.
> 
> Added the dependency 'apache-drill-1.7.0/jars/drill-jdbc-all-1.7.0.jar' in
> zeppelin
> I'm the getting the following error while running a sample SQL from
> zeppelin,
> 
> ERROR [2016-07-12 01:11:41,946] ({pool-1-thread-4}
> NotebookServer.java[afterStatusChange]:1135) - Error
> org.apache.zeppelin.interpreter.InterpreterException:
> org.apache.zeppelin.interpreter.InterpreterException:
> org.apache.thrift.transport.TTransportException: java.net.ConnectException:
> Connection refused
>at
> org.apache.zeppelin.interpreter.remote.RemoteInterpreter.init(RemoteInterpreter.java:165)
>at
> org.apache.zeppelin.interpreter.remote.RemoteInterpreter.getFormType(RemoteInterpreter.java:328)
>at
> org.apache.zeppelin.interpreter.LazyOpenInterpreter.getFormType(LazyOpenInterpreter.java:105)
>at org.apache.zeppelin.notebook.Paragraph.jobRun(Paragraph.java:260)
>at org.apache.zeppelin.scheduler.Job.run(Job.java:176)
>at
> org.apache.zeppelin.scheduler.RemoteScheduler$JobRunner.run(RemoteScheduler.java:328)
>at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>at
> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
>at java.util.concurrent.FutureTask.run(FutureTask.java:166)
>at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
>at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
>at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
>at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
>at java.lang.Thread.run(Thread.java:722)
> Caused by: org.apache.zeppelin.interpreter.InterpreterException:
> org.apache.thrift.transport.TTransportException: java.net.ConnectException:
> Connection refused
>at
> org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:53)
>at
> org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:37)
>at
> org.apache.commons.pool2.BasePooledObjectFactory.makeObject(BasePooledObjectFactory.java:60)
>at
> org.apache.commons.pool2.impl.GenericObjectPool.create(GenericObjectPool.java:861)
>at
> org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:435)
>at
> org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:363)
>at
> org.apache.zeppelin.interpreter.remote.RemoteInterpreterProcess.getClient(RemoteInterpreterProcess.java:184)
>at
> org.apache.zeppelin.interpreter.remote.RemoteInterpreter.init(RemoteInterpreter.java:163)
>... 13 more
> Caused by: org.apache.thrift.transport.TTransportException:
> java.net.ConnectException: Connection refused
>at org.apache.thrift.transport.TSocket.open(TSocket.java:187)
>at
> org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:51)
>... 20 more
> Caused by: java.net.ConnectException: Connection refused

Drill JDBC Interpreter: Connection Refused

[Krishnaprasad A S]

hi,
I'm trying to create a drill interpreter in zeppelin using the existing
jdbc interpreter.

*drill.url = jdbc:drill:zk=:2181/drill/drillbits1drill.driver =
org.apache.drill.jdbc.Driver*
my drillbit runs on the same server as of zeppelin.

Added the dependency 'apache-drill-1.7.0/jars/drill-jdbc-all-1.7.0.jar' in
zeppelin
I'm the getting the following error while running a sample SQL from
zeppelin,

ERROR [2016-07-12 01:11:41,946] ({pool-1-thread-4}
NotebookServer.java[afterStatusChange]:1135) - Error
org.apache.zeppelin.interpreter.InterpreterException:
org.apache.zeppelin.interpreter.InterpreterException:
org.apache.thrift.transport.TTransportException: java.net.ConnectException:
Connection refused
at
org.apache.zeppelin.interpreter.remote.RemoteInterpreter.init(RemoteInterpreter.java:165)
at
org.apache.zeppelin.interpreter.remote.RemoteInterpreter.getFormType(RemoteInterpreter.java:328)
at
org.apache.zeppelin.interpreter.LazyOpenInterpreter.getFormType(LazyOpenInterpreter.java:105)
at org.apache.zeppelin.notebook.Paragraph.jobRun(Paragraph.java:260)
at org.apache.zeppelin.scheduler.Job.run(Job.java:176)
at
org.apache.zeppelin.scheduler.RemoteScheduler$JobRunner.run(RemoteScheduler.java:328)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at
java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
Caused by: org.apache.zeppelin.interpreter.InterpreterException:
org.apache.thrift.transport.TTransportException: java.net.ConnectException:
Connection refused
at
org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:53)
at
org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:37)
at
org.apache.commons.pool2.BasePooledObjectFactory.makeObject(BasePooledObjectFactory.java:60)
at
org.apache.commons.pool2.impl.GenericObjectPool.create(GenericObjectPool.java:861)
at
org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:435)
at
org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:363)
at
org.apache.zeppelin.interpreter.remote.RemoteInterpreterProcess.getClient(RemoteInterpreterProcess.java:184)
at
org.apache.zeppelin.interpreter.remote.RemoteInterpreter.init(RemoteInterpreter.java:163)
... 13 more
Caused by: org.apache.thrift.transport.TTransportException:
java.net.ConnectException: Connection refused
at org.apache.thrift.transport.TSocket.open(TSocket.java:187)
at
org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:51)
... 20 more
Caused by: java.net.ConnectException: Connection refused

Re: CHAR data type

[rahul challapalli]

I raised https://issues.apache.org/jira/browse/DRILL-4772 to fix the doc
issue. Thanks Santosh.

- Rahul

On Mon, Jul 4, 2016 at 10:11 AM, Santosh Kulkarni <
santoshskulkarn...@gmail.com> wrote:

> Here is the link:
>
> https://drill.apache.org/docs/supported-data-types/
>
>
>
> On Mon, Jul 4, 2016 at 12:06 PM, rahul challapalli <
> challapallira...@gmail.com> wrote:
>
> > Can you point us to where you are looking? The documentation should only
> > say that "CHAR" datatype in hive is supported from Drill 1.7 onward.
> >
> > - Rahul
> >
> > On Mon, Jul 4, 2016 at 9:53 AM, Santosh Kulkarni <
> > santoshskulkarn...@gmail.com> wrote:
> >
> > > Thanks Shankar. I was looking in Drill documentation but did not
> realize
> > to
> > > check in 1.7 Release notes.
> > >
> > >
> > >
> > > On Mon, Jul 4, 2016 at 10:02 AM, Shankar Mane <
> > shankar.m...@games24x7.com>
> > > wrote:
> > >
> > > > It is being supported since 1.7.0. Please check this link
> > > > https://drill.apache.org/docs/apache-drill-1-7-0-release-notes/
> > > >
> > > > On 04-Jul-2016 8:07 PM, "Santosh Kulkarni" <
> > santoshskulkarn...@gmail.com
> > > >
> > > > wrote:
> > > >
> > > > Hello,
> > > >
> > > > While running another simple query for select count(*) from
> table_name,
> > > > Drill gave an error for Unsupported Hive data type CHAR.
> > > >
> > > > The column is of CHAR(6) data type. Drill documentation shows CHAR as
> > > > supported data type.
> > > >
> > > > This is on Drill version 1.6
> > > >
> > > > Thanks,
> > > >
> > > > Santosh
> > > >
> > >
> >
>

RE: Drill Hangout Topics for 07/12/2016

[Dave Oshinsky]

Please complete review of DRILL-4704:
https://github.com/apache/drill/pull/517

Thanks,
Dave Oshinsky

-Original Message-
From: Jinfeng Ni [mailto:j...@apache.org] 
Sent: Monday, July 11, 2016 11:49 AM
To: dev; user
Subject: Drill Hangout Topics for 07/12/2016

Hey,

Just a reminder that we will have Drill hangout at 10am-11am, 07/12/16 PST.

If you have any topics to discuss on tomorrow's hangout, please reply this 
thread and list the topics you want to discuss. I'll also collect topics at the 
beginning of the hangout.

Thank you,

Jinfeng




***Legal Disclaimer***
"This communication may contain confidential and privileged material for the
sole use of the intended recipient. Any unauthorized review, use or distribution
by others is strictly prohibited. If you have received the message by mistake,
please advise the sender by reply email and delete the message. Thank you."
**

Re: Drill Hangout Topics for 07/12/2016

[Subbu Srinivasan]

Can drill-4653 be reviewed?

On Mon, Jul 11, 2016 at 8:49 AM, Jinfeng Ni  wrote:

> Hey,
>
> Just a reminder that we will have Drill hangout at 10am-11am, 07/12/16 PST.
>
> If you have any topics to discuss on tomorrow's hangout, please reply
> this thread and list the topics you want to discuss. I'll also collect
> topics at the beginning of the hangout.
>
> Thank you,
>
> Jinfeng
>

Drill Hangout Topics for 07/12/2016

[Jinfeng Ni]

Hey,

Just a reminder that we will have Drill hangout at 10am-11am, 07/12/16 PST.

If you have any topics to discuss on tomorrow's hangout, please reply
this thread and list the topics you want to discuss. I'll also collect
topics at the beginning of the hangout.

Thank you,

Jinfeng