RE: Apache Phoenix integration

2018-02-06 Thread Kunal Khatua 
The downside of integrating first is that you might spend a lot of time 
fire-fighting 'bugs' that are already resolved with the later versions.

Ideally, you want to go the other way around from the ground up. Bringing up 
the platform (CDH in this case) to more recent versions, and then work on 
integrating on top of that. 

As for merging with Drill, if you are looking to just drop the Phoenix JDBC Jar 
within Drill's 3rdparty folder, it's best you use the Dev mailing list to 
identify a process.

All the current jars in that folder and dependencies which are integral to 
Drill and have been extensively tested. The Phoenix JDBC driver would have to 
go through a similar test-of-fire initiation to make it to that list, IMO. 

Since the effort for this would not be trivial, with engineering resources 
already working on more critical aspects, I think we can publish these drivers 
(and steps, like you've provided on your Github fork) for folks to consume the 
drivers until integration. We have been thinking of creating a repository of 
such supporting projects for users to consume, so this would be a good start 
IMO. 



-Original Message-
From: Flavio Pompermaier [mailto:pomperma...@okkam.it] 
Sent: Tuesday, February 06, 2018 2:36 AM
To: user@drill.apache.org
Cc: u...@phoenix.apache.org; Bridget Bevens <bbev...@mapr.com>; James Taylor 
<jamestay...@apache.org>; Pedro Boado <pedro.bo...@gmail.com>
Subject: Re: Apache Phoenix integration

While it should not be a big problem to provide a cdh5 profile to Apache Drill, 
my current contribution was just a quick and dirty way to integrate Drill with 
Phoenix.
I think it will be much better to avoid a fork of Apache Drill (i.e.
Drillix) and try to merge the 2 things.
Then I can work on creating a CDH5  profile for Drill without any problem.

What do you think? Is there any possibility to have Drill and Drillix merged 
soon?

Best,
Flavio

On Mon, Feb 5, 2018 at 9:10 PM, Kunal Khatua <kkha...@mapr.com> wrote:

> Hi Flavio
>
> I'm wondering whether you tried modifying the pom.xml on your private 
> Github fork to work with CDH 5.11.2 .
>
> https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_apache
> _drill_blob_master_pom.xml-23L2377=DwIBaQ=cskdkSMqhcnjZxdQVpwTXg
> =-cT6otg6lpT_XkmYy7yg3A=-VMIfEawApGEBf2--L454_tJU9ybI5yYsJ2btpBtgeo&
> s=kFEHv5Vxj0eiw4cwPR559turgyJ2dG3LiZbRxtWINJ8=
>
> We know folks have been using Drill with a variety of Hadoop (and
> non-Hadoop) platforms by modifying jars on their private forks, just 
> like you are.
>
> However, it benefits the community and Drill when we all can 
> contribute back such enhancements. Do you think you can try modifying 
> the pom.xml to successfully build ? If you did hit issues, can you 
> file JIRAs for the same, so that someone can take a look at it?
>
> Thanks
> Kunal
>
> -Original Message-
> From: Flavio Pompermaier [mailto:pomperma...@okkam.it]
> Sent: Monday, February 05, 2018 7:29 AM
> To: user@drill.apache.org; u...@phoenix.apache.org
> Cc: Bridget Bevens <bbev...@mapr.com>; James Taylor < 
> jamestay...@apache.org>; Pedro Boado <pedro.bo...@gmail.com>
> Subject: Re: Apache Phoenix integration
>
> Hi all,
> in the hope of helping many other enthusiastic users of Apache Phoneix 
> and Apache Drill, I've just finished to create a dedicated github 
> repository [1] with all the instruction about how to modify the 
> current Apache Drill
> 1.12.0 in order to make it work with Phoenix 4.13.2 on Cloudera CDH 5.11.2.
> I've used this version because it is the latest stable one available 
> on Maven Central containing some important fix about namespaces[2].
> In the README I've also reported some known issues I've encountered 
> but that probably need some further work in the Apache Drill code base.
> Since I use Drill just to sample tables this could be enough for me...
>
> Looking forward for the deeper integration provided by Drillix 
> (dreaming in a common effort to integrate its benefint also into the 
> official Drill version)!
>
> Best,
> Flavio
>
> [1] https://urldefense.proofpoint.com/v2/url?u=https-3A__github.
> com_okkam-2Dit_drill-2Dphoenix-2Dintegration=DwIBaQ=
> cskdkSMqhcnjZxdQVpwTXg=-cT6otg6lpT_XkmYy7yg3A=
> BQrc4m6Ki2hXeQbOe8XIbthhiEgygxzD16DmbvRBW-I=
> cwbSaV0Z9ZgnlzVsRduf6IJwXtjFZwJsHgvKPpKvsbo=
> [2] https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.
> apache.org_jira_browse_PHOENIX-2D4523=DwIBaQ=
> cskdkSMqhcnjZxdQVpwTXg=-cT6otg6lpT_XkmYy7yg3A=
> BQrc4m6Ki2hXeQbOe8XIbthhiEgygxzD16DmbvRBW-I=zx8vNOk1VKYBp4KV1Iya4XU_
> JyLRlHKSxQvA1WcFHW4=
>
>
> On Fri, Feb 2, 2018 at 7:21 PM, Kunal Khatua <kkha...@mapr.com> wrote:
>
> > That's great, Flavio!
> >
> > You can create a Google doc for review and sha

Re: Apache Phoenix integration

2018-02-06 Thread Flavio Pompermaier 
While it should not be a big problem to provide a cdh5 profile to Apache
Drill, my current contribution was just a quick and dirty way to integrate
Drill with Phoenix.
I think it will be much better to avoid a fork of Apache Drill (i.e.
Drillix) and try to merge the 2 things.
Then I can work on creating a CDH5  profile for Drill without any problem.

What do you think? Is there any possibility to have Drill and Drillix
merged soon?

Best,
Flavio

On Mon, Feb 5, 2018 at 9:10 PM, Kunal Khatua <kkha...@mapr.com> wrote:

> Hi Flavio
>
> I'm wondering whether you tried modifying the pom.xml on your private
> Github fork to work with CDH 5.11.2 .
>
> https://github.com/apache/drill/blob/master/pom.xml#L2377
>
> We know folks have been using Drill with a variety of Hadoop (and
> non-Hadoop) platforms by modifying jars on their private forks, just like
> you are.
>
> However, it benefits the community and Drill when we all can contribute
> back such enhancements. Do you think you can try modifying the pom.xml to
> successfully build ? If you did hit issues, can you file JIRAs for the
> same, so that someone can take a look at it?
>
> Thanks
> Kunal
>
> -Original Message-
> From: Flavio Pompermaier [mailto:pomperma...@okkam.it]
> Sent: Monday, February 05, 2018 7:29 AM
> To: user@drill.apache.org; u...@phoenix.apache.org
> Cc: Bridget Bevens <bbev...@mapr.com>; James Taylor <
> jamestay...@apache.org>; Pedro Boado <pedro.bo...@gmail.com>
> Subject: Re: Apache Phoenix integration
>
> Hi all,
> in the hope of helping many other enthusiastic users of Apache Phoneix and
> Apache Drill, I've just finished to create a dedicated github repository
> [1] with all the instruction about how to modify the current Apache Drill
> 1.12.0 in order to make it work with Phoenix 4.13.2 on Cloudera CDH 5.11.2.
> I've used this version because it is the latest stable one available on
> Maven Central containing some important fix about namespaces[2].
> In the README I've also reported some known issues I've encountered but
> that probably need some further work in the Apache Drill code base.
> Since I use Drill just to sample tables this could be enough for me...
>
> Looking forward for the deeper integration provided by Drillix (dreaming
> in a common effort to integrate its benefint also into the official Drill
> version)!
>
> Best,
> Flavio
>
> [1] https://urldefense.proofpoint.com/v2/url?u=https-3A__github.
> com_okkam-2Dit_drill-2Dphoenix-2Dintegration=DwIBaQ=
> cskdkSMqhcnjZxdQVpwTXg=-cT6otg6lpT_XkmYy7yg3A=
> BQrc4m6Ki2hXeQbOe8XIbthhiEgygxzD16DmbvRBW-I=
> cwbSaV0Z9ZgnlzVsRduf6IJwXtjFZwJsHgvKPpKvsbo=
> [2] https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.
> apache.org_jira_browse_PHOENIX-2D4523=DwIBaQ=
> cskdkSMqhcnjZxdQVpwTXg=-cT6otg6lpT_XkmYy7yg3A=
> BQrc4m6Ki2hXeQbOe8XIbthhiEgygxzD16DmbvRBW-I=zx8vNOk1VKYBp4KV1Iya4XU_
> JyLRlHKSxQvA1WcFHW4=
>
>
> On Fri, Feb 2, 2018 at 7:21 PM, Kunal Khatua <kkha...@mapr.com> wrote:
>
> > That's great, Flavio!
> >
> > You can create a Google doc for review and share it on the user list.
> >
> > @Bridget handles the documentation on the Apache website, so she can
> > do the final touches and help it find a home on the website.
> >
> > -Original Message-
> > From: Flavio Pompermaier [mailto:pomperma...@okkam.it]
> > Sent: Friday, February 02, 2018 9:04 AM
> > To: user@drill.apache.org
> > Cc: James Taylor <jamestay...@apache.org>
> > Subject: Re: Apache Phoenix integration
> >
> > Eventually I made it to integrate Phoenix with Drill! I debugged
> > remotely the drill-embedded via Eclipse and I discovered that the
> > problem was that you need some extra jars to make it work!
> > Where can I write some documentation about debugging remotely Drill
> > from Eclipse and integration with Drill?
> >
> > On Fri, Feb 2, 2018 at 5:28 PM, Flavio Pompermaier
> > <pomperma...@okkam.it>
> > wrote:
> >
> > > What is the fastest way to debug the JDBC plugin from Eclipse? I
> > > don't see anything in the logs that could help...
> > > Is it possible to connect directly to the external embedded drill
> > > running on my machine if I enable jmx?
> > > it seems that the JDBC connection is established correctly but Drill
> > > throws an Exception (that is not well unwrapped by Jersey):
> > >
> > > 2018-02-02 16:54:04,520 [qtp159619134-56] INFO
> > > o.a.p.q.ConnectionQueryServicesImpl
> > > - HConnection established. Stacktrace for informational purposes:
> > > hconnection-0x1b9fe9f8
> > >

RE: Apache Phoenix integration

2018-02-05 Thread Kunal Khatua 
Hi Flavio

I'm wondering whether you tried modifying the pom.xml on your private Github 
fork to work with CDH 5.11.2 . 

https://github.com/apache/drill/blob/master/pom.xml#L2377 

We know folks have been using Drill with a variety of Hadoop (and non-Hadoop) 
platforms by modifying jars on their private forks, just like you are. 

However, it benefits the community and Drill when we all can contribute back 
such enhancements. Do you think you can try modifying the pom.xml to 
successfully build ? If you did hit issues, can you file JIRAs for the same, so 
that someone can take a look at it?

Thanks
Kunal 

-Original Message-
From: Flavio Pompermaier [mailto:pomperma...@okkam.it] 
Sent: Monday, February 05, 2018 7:29 AM
To: user@drill.apache.org; u...@phoenix.apache.org
Cc: Bridget Bevens <bbev...@mapr.com>; James Taylor <jamestay...@apache.org>; 
Pedro Boado <pedro.bo...@gmail.com>
Subject: Re: Apache Phoenix integration

Hi all,
in the hope of helping many other enthusiastic users of Apache Phoneix and 
Apache Drill, I've just finished to create a dedicated github repository [1] 
with all the instruction about how to modify the current Apache Drill
1.12.0 in order to make it work with Phoenix 4.13.2 on Cloudera CDH 5.11.2.
I've used this version because it is the latest stable one available on Maven 
Central containing some important fix about namespaces[2].
In the README I've also reported some known issues I've encountered but that 
probably need some further work in the Apache Drill code base.
Since I use Drill just to sample tables this could be enough for me...

Looking forward for the deeper integration provided by Drillix (dreaming in a 
common effort to integrate its benefint also into the official Drill version)!

Best,
Flavio

[1] 
https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_okkam-2Dit_drill-2Dphoenix-2Dintegration=DwIBaQ=cskdkSMqhcnjZxdQVpwTXg=-cT6otg6lpT_XkmYy7yg3A=BQrc4m6Ki2hXeQbOe8XIbthhiEgygxzD16DmbvRBW-I=cwbSaV0Z9ZgnlzVsRduf6IJwXtjFZwJsHgvKPpKvsbo=
[2] 
https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.apache.org_jira_browse_PHOENIX-2D4523=DwIBaQ=cskdkSMqhcnjZxdQVpwTXg=-cT6otg6lpT_XkmYy7yg3A=BQrc4m6Ki2hXeQbOe8XIbthhiEgygxzD16DmbvRBW-I=zx8vNOk1VKYBp4KV1Iya4XU_JyLRlHKSxQvA1WcFHW4=


On Fri, Feb 2, 2018 at 7:21 PM, Kunal Khatua <kkha...@mapr.com> wrote:

> That's great, Flavio!
>
> You can create a Google doc for review and share it on the user list.
>
> @Bridget handles the documentation on the Apache website, so she can 
> do the final touches and help it find a home on the website.
>
> -Original Message-
> From: Flavio Pompermaier [mailto:pomperma...@okkam.it]
> Sent: Friday, February 02, 2018 9:04 AM
> To: user@drill.apache.org
> Cc: James Taylor <jamestay...@apache.org>
> Subject: Re: Apache Phoenix integration
>
> Eventually I made it to integrate Phoenix with Drill! I debugged 
> remotely the drill-embedded via Eclipse and I discovered that the 
> problem was that you need some extra jars to make it work!
> Where can I write some documentation about debugging remotely Drill 
> from Eclipse and integration with Drill?
>
> On Fri, Feb 2, 2018 at 5:28 PM, Flavio Pompermaier 
> <pomperma...@okkam.it>
> wrote:
>
> > What is the fastest way to debug the JDBC plugin from Eclipse? I 
> > don't see anything in the logs that could help...
> > Is it possible to connect directly to the external embedded drill 
> > running on my machine if I enable jmx?
> > it seems that the JDBC connection is established correctly but Drill 
> > throws an Exception (that is not well unwrapped by Jersey):
> >
> > 2018-02-02 16:54:04,520 [qtp159619134-56] INFO 
> > o.a.p.q.ConnectionQueryServicesImpl
> > - HConnection established. Stacktrace for informational purposes:
> > hconnection-0x1b9fe9f8
> > java.lang.Thread.getStackTrace(Thread.java:1552)
> > org.apache.phoenix.util.LogUtil.getCallerStackTrace(LogUtil.java:55)
> > org.apache.phoenix.query.ConnectionQueryServicesImpl.openConnection(
> > ConnectionQueryServicesImpl.java:410)
> > org.apache.phoenix.query.ConnectionQueryServicesImpl.access$400(
> > ConnectionQueryServicesImpl.java:256)
> > org.apache.phoenix.query.ConnectionQueryServicesImpl$12.call(
> > ConnectionQueryServicesImpl.java:2408)
> > org.apache.phoenix.query.ConnectionQueryServicesImpl$12.call(
> > ConnectionQueryServicesImpl.java:2384)
> > org.apache.phoenix.util.PhoenixContextExecutor.call(
> > PhoenixContextExecutor.java:76)
> > org.apache.phoenix.query.ConnectionQueryServicesImpl.init(
> > ConnectionQueryServicesImpl.java:2384)
> > org.apache.phoenix.jdbc.PhoenixDriver.getConnectionQueryServices(
> > PhoenixDriver.java:255)
> > org.

Re: Apache Phoenix integration

2018-02-05 Thread Flavio Pompermaier 
Hi all,
in the hope of helping many other enthusiastic users of Apache Phoneix and
Apache Drill, I've just finished to create a dedicated github repository
[1] with all the instruction about how to modify the current Apache Drill
1.12.0 in order to make it work with Phoenix 4.13.2 on Cloudera CDH 5.11.2.
I've used this version because it is the latest stable one available on
Maven Central containing some important fix about namespaces[2].
In the README I've also reported some known issues I've encountered but
that probably need some further work in the Apache Drill code base.
Since I use Drill just to sample tables this could be enough for me...

Looking forward for the deeper integration provided by Drillix (dreaming in
a common effort to integrate its benefint also into the official Drill
version)!

Best,
Flavio

[1] https://github.com/okkam-it/drill-phoenix-integration
[2] https://issues.apache.org/jira/browse/PHOENIX-4523


On Fri, Feb 2, 2018 at 7:21 PM, Kunal Khatua <kkha...@mapr.com> wrote:

> That's great, Flavio!
>
> You can create a Google doc for review and share it on the user list.
>
> @Bridget handles the documentation on the Apache website, so she can do
> the final touches and help it find a home on the website.
>
> -Original Message-
> From: Flavio Pompermaier [mailto:pomperma...@okkam.it]
> Sent: Friday, February 02, 2018 9:04 AM
> To: user@drill.apache.org
> Cc: James Taylor <jamestay...@apache.org>
> Subject: Re: Apache Phoenix integration
>
> Eventually I made it to integrate Phoenix with Drill! I debugged remotely
> the drill-embedded via Eclipse and I discovered that the problem was that
> you need some extra jars to make it work!
> Where can I write some documentation about debugging remotely Drill from
> Eclipse and integration with Drill?
>
> On Fri, Feb 2, 2018 at 5:28 PM, Flavio Pompermaier <pomperma...@okkam.it>
> wrote:
>
> > What is the fastest way to debug the JDBC plugin from Eclipse? I don't
> > see anything in the logs that could help...
> > Is it possible to connect directly to the external embedded drill
> > running on my machine if I enable jmx?
> > it seems that the JDBC connection is established correctly but Drill
> > throws an Exception (that is not well unwrapped by Jersey):
> >
> > 2018-02-02 16:54:04,520 [qtp159619134-56] INFO
> > o.a.p.q.ConnectionQueryServicesImpl
> > - HConnection established. Stacktrace for informational purposes:
> > hconnection-0x1b9fe9f8
> > java.lang.Thread.getStackTrace(Thread.java:1552)
> > org.apache.phoenix.util.LogUtil.getCallerStackTrace(LogUtil.java:55)
> > org.apache.phoenix.query.ConnectionQueryServicesImpl.openConnection(
> > ConnectionQueryServicesImpl.java:410)
> > org.apache.phoenix.query.ConnectionQueryServicesImpl.access$400(
> > ConnectionQueryServicesImpl.java:256)
> > org.apache.phoenix.query.ConnectionQueryServicesImpl$12.call(
> > ConnectionQueryServicesImpl.java:2408)
> > org.apache.phoenix.query.ConnectionQueryServicesImpl$12.call(
> > ConnectionQueryServicesImpl.java:2384)
> > org.apache.phoenix.util.PhoenixContextExecutor.call(
> > PhoenixContextExecutor.java:76)
> > org.apache.phoenix.query.ConnectionQueryServicesImpl.init(
> > ConnectionQueryServicesImpl.java:2384)
> > org.apache.phoenix.jdbc.PhoenixDriver.getConnectionQueryServices(
> > PhoenixDriver.java:255)
> > org.apache.phoenix.jdbc.PhoenixEmbeddedDriver.createConnection(
> > PhoenixEmbeddedDriver.java:150)
> > org.apache.phoenix.jdbc.PhoenixDriver.connect(PhoenixDriver.java:221)
> > org.apache.commons.dbcp.DriverConnectionFactory.createConnection(
> > DriverConnectionFactory.java:38)
> > org.apache.commons.dbcp.PoolableConnectionFactory.makeObject(
> > PoolableConnectionFactory.java:582)
> > org.apache.commons.dbcp.BasicDataSource.validateConnectionFactory(
> > BasicDataSource.java:1556)
> > org.apache.commons.dbcp.BasicDataSource.createPoolableConnectionFactor
> > y(BasicDataSource.java:1545)
> > org.apache.commons.dbcp.BasicDataSource.createDataSource(
> > BasicDataSource.java:1388)
> > org.apache.commons.dbcp.BasicDataSource.getConnection(
> > BasicDataSource.java:1044)
> > org.apache.calcite.adapter.jdbc.JdbcUtils$DialectPool.
> > get(JdbcUtils.java:73)
> > org.apache.calcite.adapter.jdbc.JdbcSchema.createDialect(
> > JdbcSchema.java:138)
> > org.apache.drill.exec.store.jdbc.JdbcStoragePlugin.(
> > JdbcStoragePlugin.java:103)
> > sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> > sun.reflect.NativeConstructorAccessorImpl.newInstance(
> > NativeConstructorAccessorImpl.java:62)
>

Re: Apache Phoenix integration

2018-02-02 Thread Flavio Pompermaier 
This is a very great news! But what do you mean exactly with "Apache Drill
is off of it's fork"? When is this deeper integration going to be
released/integrated?
Will Phoenix (eventually) become just a Drill storage plugin, with huge
benefits for both?

On 2 Feb 2018 20:20, "James Taylor" <jamestay...@apache.org> wrote:

> Good idea - a joint meetup would be great.
>
> On Fri, Feb 2, 2018 at 11:15 AM, Saurabh Mahapatra <
> saurabhmahapatr...@gmail.com> wrote:
>
>> This is absolutely great news! Thanks for sharing, James!
>>
>> We should have this as a presentation in one of our weekly Drill hangout
>> sessions.
>>
>> It’s about time for us to do a meetup. A joint meetup perhaps?
>>
>> Saurabh
>>
>> Sent from my iPhone
>>
>>
>>
>> > On Feb 2, 2018, at 11:13 AM, James Taylor <jamestay...@apache.org>
>> wrote:
>> >
>> > There's also a much deeper integration between Phoenix + Drill (code
>> named
>> > Drillix) underway that should be possible to complete now that Apache
>> Drill
>> > is off of it's fork and on a later version of Apache Calcite. I'm hoping
>> > that the output of this will be a Phoenix adapter in Drill. See
>> > presentation here[1] and WIP code here[2].
>> >
>> > Thanks,
>> > James
>> >
>> > [1] http://phoenix.apache.org/presentations/Drillix.pdf
>> > [2] https://github.com/jacques-n/drill/tree/phoenix_plugin
>> >
>> >> On Fri, Feb 2, 2018 at 10:21 AM, Kunal Khatua <kkha...@mapr.com>
>> wrote:
>> >>
>> >> That's great, Flavio!
>> >>
>> >> You can create a Google doc for review and share it on the user list.
>> >>
>> >> @Bridget handles the documentation on the Apache website, so she can do
>> >> the final touches and help it find a home on the website.
>> >>
>> >> -Original Message-
>> >> From: Flavio Pompermaier [mailto:pomperma...@okkam.it]
>> >> Sent: Friday, February 02, 2018 9:04 AM
>> >> To: user@drill.apache.org
>> >> Cc: James Taylor <jamestay...@apache.org>
>> >> Subject: Re: Apache Phoenix integration
>> >>
>> >> Eventually I made it to integrate Phoenix with Drill! I debugged
>> remotely
>> >> the drill-embedded via Eclipse and I discovered that the problem was
>> that
>> >> you need some extra jars to make it work!
>> >> Where can I write some documentation about debugging remotely Drill
>> from
>> >> Eclipse and integration with Drill?
>> >>
>> >> On Fri, Feb 2, 2018 at 5:28 PM, Flavio Pompermaier <
>> pomperma...@okkam.it>
>> >> wrote:
>> >>
>> >>> What is the fastest way to debug the JDBC plugin from Eclipse? I don't
>> >>> see anything in the logs that could help...
>> >>> Is it possible to connect directly to the external embedded drill
>> >>> running on my machine if I enable jmx?
>> >>> it seems that the JDBC connection is established correctly but Drill
>> >>> throws an Exception (that is not well unwrapped by Jersey):
>> >>>
>> >>> 2018-02-02 16:54:04,520 [qtp159619134-56] INFO
>> >>> o.a.p.q.ConnectionQueryServicesImpl
>> >>> - HConnection established. Stacktrace for informational purposes:
>> >>> hconnection-0x1b9fe9f8
>> >>> java.lang.Thread.getStackTrace(Thread.java:1552)
>> >>> org.apache.phoenix.util.LogUtil.getCallerStackTrace(LogUtil.java:55)
>> >>> org.apache.phoenix.query.ConnectionQueryServicesImpl.openConnection(
>> >>> ConnectionQueryServicesImpl.java:410)
>> >>> org.apache.phoenix.query.ConnectionQueryServicesImpl.access$400(
>> >>> ConnectionQueryServicesImpl.java:256)
>> >>> org.apache.phoenix.query.ConnectionQueryServicesImpl$12.call(
>> >>> ConnectionQueryServicesImpl.java:2408)
>> >>> org.apache.phoenix.query.ConnectionQueryServicesImpl$12.call(
>> >>> ConnectionQueryServicesImpl.java:2384)
>> >>> org.apache.phoenix.util.PhoenixContextExecutor.call(
>> >>> PhoenixContextExecutor.java:76)
>> >>> org.apache.phoenix.query.ConnectionQueryServicesImpl.init(
>> >>> ConnectionQueryServicesImpl.java:2384)
>> >>> org.apache.phoenix.jdbc.PhoenixDriver.getConnectionQueryServices(
>> >>> PhoenixDrive

Re: Apache Phoenix integration

2018-02-02 Thread James Taylor 
Good idea - a joint meetup would be great.

On Fri, Feb 2, 2018 at 11:15 AM, Saurabh Mahapatra <
saurabhmahapatr...@gmail.com> wrote:

> This is absolutely great news! Thanks for sharing, James!
>
> We should have this as a presentation in one of our weekly Drill hangout
> sessions.
>
> It’s about time for us to do a meetup. A joint meetup perhaps?
>
> Saurabh
>
> Sent from my iPhone
>
>
>
> > On Feb 2, 2018, at 11:13 AM, James Taylor <jamestay...@apache.org>
> wrote:
> >
> > There's also a much deeper integration between Phoenix + Drill (code
> named
> > Drillix) underway that should be possible to complete now that Apache
> Drill
> > is off of it's fork and on a later version of Apache Calcite. I'm hoping
> > that the output of this will be a Phoenix adapter in Drill. See
> > presentation here[1] and WIP code here[2].
> >
> > Thanks,
> > James
> >
> > [1] http://phoenix.apache.org/presentations/Drillix.pdf
> > [2] https://github.com/jacques-n/drill/tree/phoenix_plugin
> >
> >> On Fri, Feb 2, 2018 at 10:21 AM, Kunal Khatua <kkha...@mapr.com> wrote:
> >>
> >> That's great, Flavio!
> >>
> >> You can create a Google doc for review and share it on the user list.
> >>
> >> @Bridget handles the documentation on the Apache website, so she can do
> >> the final touches and help it find a home on the website.
> >>
> >> -----Original Message-
> >> From: Flavio Pompermaier [mailto:pomperma...@okkam.it]
> >> Sent: Friday, February 02, 2018 9:04 AM
> >> To: user@drill.apache.org
> >> Cc: James Taylor <jamestay...@apache.org>
> >> Subject: Re: Apache Phoenix integration
> >>
> >> Eventually I made it to integrate Phoenix with Drill! I debugged
> remotely
> >> the drill-embedded via Eclipse and I discovered that the problem was
> that
> >> you need some extra jars to make it work!
> >> Where can I write some documentation about debugging remotely Drill from
> >> Eclipse and integration with Drill?
> >>
> >> On Fri, Feb 2, 2018 at 5:28 PM, Flavio Pompermaier <
> pomperma...@okkam.it>
> >> wrote:
> >>
> >>> What is the fastest way to debug the JDBC plugin from Eclipse? I don't
> >>> see anything in the logs that could help...
> >>> Is it possible to connect directly to the external embedded drill
> >>> running on my machine if I enable jmx?
> >>> it seems that the JDBC connection is established correctly but Drill
> >>> throws an Exception (that is not well unwrapped by Jersey):
> >>>
> >>> 2018-02-02 16:54:04,520 [qtp159619134-56] INFO
> >>> o.a.p.q.ConnectionQueryServicesImpl
> >>> - HConnection established. Stacktrace for informational purposes:
> >>> hconnection-0x1b9fe9f8
> >>> java.lang.Thread.getStackTrace(Thread.java:1552)
> >>> org.apache.phoenix.util.LogUtil.getCallerStackTrace(LogUtil.java:55)
> >>> org.apache.phoenix.query.ConnectionQueryServicesImpl.openConnection(
> >>> ConnectionQueryServicesImpl.java:410)
> >>> org.apache.phoenix.query.ConnectionQueryServicesImpl.access$400(
> >>> ConnectionQueryServicesImpl.java:256)
> >>> org.apache.phoenix.query.ConnectionQueryServicesImpl$12.call(
> >>> ConnectionQueryServicesImpl.java:2408)
> >>> org.apache.phoenix.query.ConnectionQueryServicesImpl$12.call(
> >>> ConnectionQueryServicesImpl.java:2384)
> >>> org.apache.phoenix.util.PhoenixContextExecutor.call(
> >>> PhoenixContextExecutor.java:76)
> >>> org.apache.phoenix.query.ConnectionQueryServicesImpl.init(
> >>> ConnectionQueryServicesImpl.java:2384)
> >>> org.apache.phoenix.jdbc.PhoenixDriver.getConnectionQueryServices(
> >>> PhoenixDriver.java:255)
> >>> org.apache.phoenix.jdbc.PhoenixEmbeddedDriver.createConnection(
> >>> PhoenixEmbeddedDriver.java:150)
> >>> org.apache.phoenix.jdbc.PhoenixDriver.connect(PhoenixDriver.java:221)
> >>> org.apache.commons.dbcp.DriverConnectionFactory.createConnection(
> >>> DriverConnectionFactory.java:38)
> >>> org.apache.commons.dbcp.PoolableConnectionFactory.makeObject(
> >>> PoolableConnectionFactory.java:582)
> >>> org.apache.commons.dbcp.BasicDataSource.validateConnectionFactory(
> >>> BasicDataSource.java:1556)
> >>> org.apache.commons.dbcp.BasicDataSource.createPoolableConnectionFactor
> >>> y(Basi

Re: Apache Phoenix integration

2018-02-02 Thread Saurabh Mahapatra 
This is absolutely great news! Thanks for sharing, James!

We should have this as a presentation in one of our weekly Drill hangout 
sessions.

It’s about time for us to do a meetup. A joint meetup perhaps?

Saurabh

Sent from my iPhone



> On Feb 2, 2018, at 11:13 AM, James Taylor <jamestay...@apache.org> wrote:
> 
> There's also a much deeper integration between Phoenix + Drill (code named
> Drillix) underway that should be possible to complete now that Apache Drill
> is off of it's fork and on a later version of Apache Calcite. I'm hoping
> that the output of this will be a Phoenix adapter in Drill. See
> presentation here[1] and WIP code here[2].
> 
> Thanks,
> James
> 
> [1] http://phoenix.apache.org/presentations/Drillix.pdf
> [2] https://github.com/jacques-n/drill/tree/phoenix_plugin
> 
>> On Fri, Feb 2, 2018 at 10:21 AM, Kunal Khatua <kkha...@mapr.com> wrote:
>> 
>> That's great, Flavio!
>> 
>> You can create a Google doc for review and share it on the user list.
>> 
>> @Bridget handles the documentation on the Apache website, so she can do
>> the final touches and help it find a home on the website.
>> 
>> -Original Message-
>> From: Flavio Pompermaier [mailto:pomperma...@okkam.it]
>> Sent: Friday, February 02, 2018 9:04 AM
>> To: user@drill.apache.org
>> Cc: James Taylor <jamestay...@apache.org>
>> Subject: Re: Apache Phoenix integration
>> 
>> Eventually I made it to integrate Phoenix with Drill! I debugged remotely
>> the drill-embedded via Eclipse and I discovered that the problem was that
>> you need some extra jars to make it work!
>> Where can I write some documentation about debugging remotely Drill from
>> Eclipse and integration with Drill?
>> 
>> On Fri, Feb 2, 2018 at 5:28 PM, Flavio Pompermaier <pomperma...@okkam.it>
>> wrote:
>> 
>>> What is the fastest way to debug the JDBC plugin from Eclipse? I don't
>>> see anything in the logs that could help...
>>> Is it possible to connect directly to the external embedded drill
>>> running on my machine if I enable jmx?
>>> it seems that the JDBC connection is established correctly but Drill
>>> throws an Exception (that is not well unwrapped by Jersey):
>>> 
>>> 2018-02-02 16:54:04,520 [qtp159619134-56] INFO
>>> o.a.p.q.ConnectionQueryServicesImpl
>>> - HConnection established. Stacktrace for informational purposes:
>>> hconnection-0x1b9fe9f8
>>> java.lang.Thread.getStackTrace(Thread.java:1552)
>>> org.apache.phoenix.util.LogUtil.getCallerStackTrace(LogUtil.java:55)
>>> org.apache.phoenix.query.ConnectionQueryServicesImpl.openConnection(
>>> ConnectionQueryServicesImpl.java:410)
>>> org.apache.phoenix.query.ConnectionQueryServicesImpl.access$400(
>>> ConnectionQueryServicesImpl.java:256)
>>> org.apache.phoenix.query.ConnectionQueryServicesImpl$12.call(
>>> ConnectionQueryServicesImpl.java:2408)
>>> org.apache.phoenix.query.ConnectionQueryServicesImpl$12.call(
>>> ConnectionQueryServicesImpl.java:2384)
>>> org.apache.phoenix.util.PhoenixContextExecutor.call(
>>> PhoenixContextExecutor.java:76)
>>> org.apache.phoenix.query.ConnectionQueryServicesImpl.init(
>>> ConnectionQueryServicesImpl.java:2384)
>>> org.apache.phoenix.jdbc.PhoenixDriver.getConnectionQueryServices(
>>> PhoenixDriver.java:255)
>>> org.apache.phoenix.jdbc.PhoenixEmbeddedDriver.createConnection(
>>> PhoenixEmbeddedDriver.java:150)
>>> org.apache.phoenix.jdbc.PhoenixDriver.connect(PhoenixDriver.java:221)
>>> org.apache.commons.dbcp.DriverConnectionFactory.createConnection(
>>> DriverConnectionFactory.java:38)
>>> org.apache.commons.dbcp.PoolableConnectionFactory.makeObject(
>>> PoolableConnectionFactory.java:582)
>>> org.apache.commons.dbcp.BasicDataSource.validateConnectionFactory(
>>> BasicDataSource.java:1556)
>>> org.apache.commons.dbcp.BasicDataSource.createPoolableConnectionFactor
>>> y(BasicDataSource.java:1545)
>>> org.apache.commons.dbcp.BasicDataSource.createDataSource(
>>> BasicDataSource.java:1388)
>>> org.apache.commons.dbcp.BasicDataSource.getConnection(
>>> BasicDataSource.java:1044)
>>> org.apache.calcite.adapter.jdbc.JdbcUtils$DialectPool.
>>> get(JdbcUtils.java:73)
>>> org.apache.calcite.adapter.jdbc.JdbcSchema.createDialect(
>>> JdbcSchema.java:138)
>>> org.apache.drill.exec.store.jdbc.JdbcStoragePlugin.(
>>> JdbcStoragePlugin.java:103)
>>> sun.reflect.NativeConstruct

Re: Apache Phoenix integration

2018-02-02 Thread James Taylor 
There's also a much deeper integration between Phoenix + Drill (code named
Drillix) underway that should be possible to complete now that Apache Drill
is off of it's fork and on a later version of Apache Calcite. I'm hoping
that the output of this will be a Phoenix adapter in Drill. See
presentation here[1] and WIP code here[2].

Thanks,
James

[1] http://phoenix.apache.org/presentations/Drillix.pdf
[2] https://github.com/jacques-n/drill/tree/phoenix_plugin

On Fri, Feb 2, 2018 at 10:21 AM, Kunal Khatua <kkha...@mapr.com> wrote:

> That's great, Flavio!
>
> You can create a Google doc for review and share it on the user list.
>
> @Bridget handles the documentation on the Apache website, so she can do
> the final touches and help it find a home on the website.
>
> -Original Message-
> From: Flavio Pompermaier [mailto:pomperma...@okkam.it]
> Sent: Friday, February 02, 2018 9:04 AM
> To: user@drill.apache.org
> Cc: James Taylor <jamestay...@apache.org>
> Subject: Re: Apache Phoenix integration
>
> Eventually I made it to integrate Phoenix with Drill! I debugged remotely
> the drill-embedded via Eclipse and I discovered that the problem was that
> you need some extra jars to make it work!
> Where can I write some documentation about debugging remotely Drill from
> Eclipse and integration with Drill?
>
> On Fri, Feb 2, 2018 at 5:28 PM, Flavio Pompermaier <pomperma...@okkam.it>
> wrote:
>
> > What is the fastest way to debug the JDBC plugin from Eclipse? I don't
> > see anything in the logs that could help...
> > Is it possible to connect directly to the external embedded drill
> > running on my machine if I enable jmx?
> > it seems that the JDBC connection is established correctly but Drill
> > throws an Exception (that is not well unwrapped by Jersey):
> >
> > 2018-02-02 16:54:04,520 [qtp159619134-56] INFO
> > o.a.p.q.ConnectionQueryServicesImpl
> > - HConnection established. Stacktrace for informational purposes:
> > hconnection-0x1b9fe9f8
> > java.lang.Thread.getStackTrace(Thread.java:1552)
> > org.apache.phoenix.util.LogUtil.getCallerStackTrace(LogUtil.java:55)
> > org.apache.phoenix.query.ConnectionQueryServicesImpl.openConnection(
> > ConnectionQueryServicesImpl.java:410)
> > org.apache.phoenix.query.ConnectionQueryServicesImpl.access$400(
> > ConnectionQueryServicesImpl.java:256)
> > org.apache.phoenix.query.ConnectionQueryServicesImpl$12.call(
> > ConnectionQueryServicesImpl.java:2408)
> > org.apache.phoenix.query.ConnectionQueryServicesImpl$12.call(
> > ConnectionQueryServicesImpl.java:2384)
> > org.apache.phoenix.util.PhoenixContextExecutor.call(
> > PhoenixContextExecutor.java:76)
> > org.apache.phoenix.query.ConnectionQueryServicesImpl.init(
> > ConnectionQueryServicesImpl.java:2384)
> > org.apache.phoenix.jdbc.PhoenixDriver.getConnectionQueryServices(
> > PhoenixDriver.java:255)
> > org.apache.phoenix.jdbc.PhoenixEmbeddedDriver.createConnection(
> > PhoenixEmbeddedDriver.java:150)
> > org.apache.phoenix.jdbc.PhoenixDriver.connect(PhoenixDriver.java:221)
> > org.apache.commons.dbcp.DriverConnectionFactory.createConnection(
> > DriverConnectionFactory.java:38)
> > org.apache.commons.dbcp.PoolableConnectionFactory.makeObject(
> > PoolableConnectionFactory.java:582)
> > org.apache.commons.dbcp.BasicDataSource.validateConnectionFactory(
> > BasicDataSource.java:1556)
> > org.apache.commons.dbcp.BasicDataSource.createPoolableConnectionFactor
> > y(BasicDataSource.java:1545)
> > org.apache.commons.dbcp.BasicDataSource.createDataSource(
> > BasicDataSource.java:1388)
> > org.apache.commons.dbcp.BasicDataSource.getConnection(
> > BasicDataSource.java:1044)
> > org.apache.calcite.adapter.jdbc.JdbcUtils$DialectPool.
> > get(JdbcUtils.java:73)
> > org.apache.calcite.adapter.jdbc.JdbcSchema.createDialect(
> > JdbcSchema.java:138)
> > org.apache.drill.exec.store.jdbc.JdbcStoragePlugin.(
> > JdbcStoragePlugin.java:103)
> > sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> > sun.reflect.NativeConstructorAccessorImpl.newInstance(
> > NativeConstructorAccessorImpl.java:62)
> > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(
> > DelegatingConstructorAccessorImpl.java:45)
> > java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> > org.apache.drill.exec.store.StoragePluginRegistryImpl.create(
> > StoragePluginRegistryImpl.java:346)
> > org.apache.drill.exec.store.StoragePluginRegistryImpl.createOrUpdate(
> > StoragePluginRegistryImpl.java:239)
> > org.apache.drill.exec.server.rest.PluginConfigWr

RE: Apache Phoenix integration

2018-02-02 Thread Kunal Khatua 
That's great, Flavio!

You can create a Google doc for review and share it on the user list.

@Bridget handles the documentation on the Apache website, so she can do the 
final touches and help it find a home on the website. 

-Original Message-
From: Flavio Pompermaier [mailto:pomperma...@okkam.it] 
Sent: Friday, February 02, 2018 9:04 AM
To: user@drill.apache.org
Cc: James Taylor <jamestay...@apache.org>
Subject: Re: Apache Phoenix integration

Eventually I made it to integrate Phoenix with Drill! I debugged remotely the 
drill-embedded via Eclipse and I discovered that the problem was that you need 
some extra jars to make it work!
Where can I write some documentation about debugging remotely Drill from 
Eclipse and integration with Drill?

On Fri, Feb 2, 2018 at 5:28 PM, Flavio Pompermaier <pomperma...@okkam.it>
wrote:

> What is the fastest way to debug the JDBC plugin from Eclipse? I don't 
> see anything in the logs that could help...
> Is it possible to connect directly to the external embedded drill 
> running on my machine if I enable jmx?
> it seems that the JDBC connection is established correctly but Drill 
> throws an Exception (that is not well unwrapped by Jersey):
>
> 2018-02-02 16:54:04,520 [qtp159619134-56] INFO  
> o.a.p.q.ConnectionQueryServicesImpl
> - HConnection established. Stacktrace for informational purposes:
> hconnection-0x1b9fe9f8 
> java.lang.Thread.getStackTrace(Thread.java:1552)
> org.apache.phoenix.util.LogUtil.getCallerStackTrace(LogUtil.java:55)
> org.apache.phoenix.query.ConnectionQueryServicesImpl.openConnection(
> ConnectionQueryServicesImpl.java:410)
> org.apache.phoenix.query.ConnectionQueryServicesImpl.access$400(
> ConnectionQueryServicesImpl.java:256)
> org.apache.phoenix.query.ConnectionQueryServicesImpl$12.call(
> ConnectionQueryServicesImpl.java:2408)
> org.apache.phoenix.query.ConnectionQueryServicesImpl$12.call(
> ConnectionQueryServicesImpl.java:2384)
> org.apache.phoenix.util.PhoenixContextExecutor.call(
> PhoenixContextExecutor.java:76)
> org.apache.phoenix.query.ConnectionQueryServicesImpl.init(
> ConnectionQueryServicesImpl.java:2384)
> org.apache.phoenix.jdbc.PhoenixDriver.getConnectionQueryServices(
> PhoenixDriver.java:255)
> org.apache.phoenix.jdbc.PhoenixEmbeddedDriver.createConnection(
> PhoenixEmbeddedDriver.java:150)
> org.apache.phoenix.jdbc.PhoenixDriver.connect(PhoenixDriver.java:221)
> org.apache.commons.dbcp.DriverConnectionFactory.createConnection(
> DriverConnectionFactory.java:38)
> org.apache.commons.dbcp.PoolableConnectionFactory.makeObject(
> PoolableConnectionFactory.java:582)
> org.apache.commons.dbcp.BasicDataSource.validateConnectionFactory(
> BasicDataSource.java:1556)
> org.apache.commons.dbcp.BasicDataSource.createPoolableConnectionFactor
> y(BasicDataSource.java:1545)
> org.apache.commons.dbcp.BasicDataSource.createDataSource(
> BasicDataSource.java:1388)
> org.apache.commons.dbcp.BasicDataSource.getConnection(
> BasicDataSource.java:1044)
> org.apache.calcite.adapter.jdbc.JdbcUtils$DialectPool.
> get(JdbcUtils.java:73)
> org.apache.calcite.adapter.jdbc.JdbcSchema.createDialect(
> JdbcSchema.java:138)
> org.apache.drill.exec.store.jdbc.JdbcStoragePlugin.(
> JdbcStoragePlugin.java:103)
> sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) 
> sun.reflect.NativeConstructorAccessorImpl.newInstance(
> NativeConstructorAccessorImpl.java:62)
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(
> DelegatingConstructorAccessorImpl.java:45)
> java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> org.apache.drill.exec.store.StoragePluginRegistryImpl.create(
> StoragePluginRegistryImpl.java:346)
> org.apache.drill.exec.store.StoragePluginRegistryImpl.createOrUpdate(
> StoragePluginRegistryImpl.java:239)
> org.apache.drill.exec.server.rest.PluginConfigWrapper.
> createOrUpdateInStorage(PluginConfigWrapper.java:57)
> org.apache.drill.exec.server.rest.StorageResources.
> createOrUpdatePluginJSON(StorageResources.java:162)
> org.apache.drill.exec.server.rest.StorageResources.createOrUpdatePlugi
> n(
> StorageResources.java:177)
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:
> 62)
> sun.reflect.DelegatingMethodAccessorImpl.invoke(
> DelegatingMethodAccessorImpl.java:43)
> java.lang.reflect.Method.invoke(Method.java:498)
> org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHan
> dle
> rFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java:81)
> org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodD
> isp
> atcher$1.run(AbstractJavaResourceMethodDispatcher.java:151)
> org.glassfish.jersey.server.model.internal.AbstractJav

Re: Apache Phoenix integration

2018-02-02 Thread Flavio Pompermaier 
Eventually I made it to integrate Phoenix with Drill! I debugged remotely
the drill-embedded via Eclipse and I discovered that the problem was that
you need some extra jars to make it work!
Where can I write some documentation about debugging remotely Drill from
Eclipse and integration with Drill?

On Fri, Feb 2, 2018 at 5:28 PM, Flavio Pompermaier 
wrote:

> What is the fastest way to debug the JDBC plugin from Eclipse? I don't see
> anything in the logs that could help...
> Is it possible to connect directly to the external embedded drill running
> on my machine if I enable jmx?
> it seems that the JDBC connection is established correctly but Drill
> throws an Exception (that is not well unwrapped by Jersey):
>
> 2018-02-02 16:54:04,520 [qtp159619134-56] INFO  
> o.a.p.q.ConnectionQueryServicesImpl
> - HConnection established. Stacktrace for informational purposes:
> hconnection-0x1b9fe9f8 java.lang.Thread.getStackTrace(Thread.java:1552)
> org.apache.phoenix.util.LogUtil.getCallerStackTrace(LogUtil.java:55)
> org.apache.phoenix.query.ConnectionQueryServicesImpl.openConnection(
> ConnectionQueryServicesImpl.java:410)
> org.apache.phoenix.query.ConnectionQueryServicesImpl.access$400(
> ConnectionQueryServicesImpl.java:256)
> org.apache.phoenix.query.ConnectionQueryServicesImpl$12.call(
> ConnectionQueryServicesImpl.java:2408)
> org.apache.phoenix.query.ConnectionQueryServicesImpl$12.call(
> ConnectionQueryServicesImpl.java:2384)
> org.apache.phoenix.util.PhoenixContextExecutor.call(
> PhoenixContextExecutor.java:76)
> org.apache.phoenix.query.ConnectionQueryServicesImpl.init(
> ConnectionQueryServicesImpl.java:2384)
> org.apache.phoenix.jdbc.PhoenixDriver.getConnectionQueryServices(
> PhoenixDriver.java:255)
> org.apache.phoenix.jdbc.PhoenixEmbeddedDriver.createConnection(
> PhoenixEmbeddedDriver.java:150)
> org.apache.phoenix.jdbc.PhoenixDriver.connect(PhoenixDriver.java:221)
> org.apache.commons.dbcp.DriverConnectionFactory.createConnection(
> DriverConnectionFactory.java:38)
> org.apache.commons.dbcp.PoolableConnectionFactory.makeObject(
> PoolableConnectionFactory.java:582)
> org.apache.commons.dbcp.BasicDataSource.validateConnectionFactory(
> BasicDataSource.java:1556)
> org.apache.commons.dbcp.BasicDataSource.createPoolableConnectionFactor
> y(BasicDataSource.java:1545)
> org.apache.commons.dbcp.BasicDataSource.createDataSource(
> BasicDataSource.java:1388)
> org.apache.commons.dbcp.BasicDataSource.getConnection(
> BasicDataSource.java:1044)
> org.apache.calcite.adapter.jdbc.JdbcUtils$DialectPool.
> get(JdbcUtils.java:73)
> org.apache.calcite.adapter.jdbc.JdbcSchema.createDialect(
> JdbcSchema.java:138)
> org.apache.drill.exec.store.jdbc.JdbcStoragePlugin.(
> JdbcStoragePlugin.java:103)
> sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> sun.reflect.NativeConstructorAccessorImpl.newInstance(
> NativeConstructorAccessorImpl.java:62)
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(
> DelegatingConstructorAccessorImpl.java:45)
> java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> org.apache.drill.exec.store.StoragePluginRegistryImpl.create(
> StoragePluginRegistryImpl.java:346)
> org.apache.drill.exec.store.StoragePluginRegistryImpl.createOrUpdate(
> StoragePluginRegistryImpl.java:239)
> org.apache.drill.exec.server.rest.PluginConfigWrapper.
> createOrUpdateInStorage(PluginConfigWrapper.java:57)
> org.apache.drill.exec.server.rest.StorageResources.
> createOrUpdatePluginJSON(StorageResources.java:162)
> org.apache.drill.exec.server.rest.StorageResources.createOrUpdatePlugin(
> StorageResources.java:177)
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:
> 62)
> sun.reflect.DelegatingMethodAccessorImpl.invoke(
> DelegatingMethodAccessorImpl.java:43)
> java.lang.reflect.Method.invoke(Method.java:498)
> org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandle
> rFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java:81)
> org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDisp
> atcher$1.run(AbstractJavaResourceMethodDispatcher.java:151)
> org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDisp
> atcher.invoke(AbstractJavaResourceMethodDispatcher.java:171)
> org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherPr
> ovider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherPr
> ovider.java:195)
> org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDisp
> atcher.dispatch(AbstractJavaResourceMethodDispatcher.java:104)
> org.glassfish.jersey.server.model.ResourceMethodInvoker.
> invoke(ResourceMethodInvoker.java:387)
> org.glassfish.jersey.server.model.ResourceMethodInvoker.
> apply(ResourceMethodInvoker.java:331)
> org.glassfish.jersey.server.model.ResourceMethodInvoker.
> apply(ResourceMethodInvoker.java:103)
>

Re: Apache Phoenix integration

2018-02-02 Thread Flavio Pompermaier 
What is the fastest way to debug the JDBC plugin from Eclipse? I don't see
anything in the logs that could help...
Is it possible to connect directly to the external embedded drill running
on my machine if I enable jmx?
it seems that the JDBC connection is established correctly but Drill throws
an Exception (that is not well unwrapped by Jersey):

2018-02-02 16:54:04,520 [qtp159619134-56] INFO
o.a.p.q.ConnectionQueryServicesImpl - HConnection established. Stacktrace
for informational purposes: hconnection-0x1b9fe9f8
java.lang.Thread.getStackTrace(Thread.java:1552)
org.apache.phoenix.util.LogUtil.getCallerStackTrace(LogUtil.java:55)
org.apache.phoenix.query.ConnectionQueryServicesImpl.openConnection(ConnectionQueryServicesImpl.java:410)
org.apache.phoenix.query.ConnectionQueryServicesImpl.access$400(ConnectionQueryServicesImpl.java:256)
org.apache.phoenix.query.ConnectionQueryServicesImpl$12.call(ConnectionQueryServicesImpl.java:2408)
org.apache.phoenix.query.ConnectionQueryServicesImpl$12.call(ConnectionQueryServicesImpl.java:2384)
org.apache.phoenix.util.PhoenixContextExecutor.call(PhoenixContextExecutor.java:76)
org.apache.phoenix.query.ConnectionQueryServicesImpl.init(ConnectionQueryServicesImpl.java:2384)
org.apache.phoenix.jdbc.PhoenixDriver.getConnectionQueryServices(PhoenixDriver.java:255)
org.apache.phoenix.jdbc.PhoenixEmbeddedDriver.createConnection(PhoenixEmbeddedDriver.java:150)
org.apache.phoenix.jdbc.PhoenixDriver.connect(PhoenixDriver.java:221)
org.apache.commons.dbcp.DriverConnectionFactory.createConnection(DriverConnectionFactory.java:38)
org.apache.commons.dbcp.PoolableConnectionFactory.makeObject(PoolableConnectionFactory.java:582)
org.apache.commons.dbcp.BasicDataSource.validateConnectionFactory(BasicDataSource.java:1556)
org.apache.commons.dbcp.BasicDataSource.createPoolableConnectionFactory(BasicDataSource.java:1545)
org.apache.commons.dbcp.BasicDataSource.createDataSource(BasicDataSource.java:1388)
org.apache.commons.dbcp.BasicDataSource.getConnection(BasicDataSource.java:1044)
org.apache.calcite.adapter.jdbc.JdbcUtils$DialectPool.get(JdbcUtils.java:73)
org.apache.calcite.adapter.jdbc.JdbcSchema.createDialect(JdbcSchema.java:138)
org.apache.drill.exec.store.jdbc.JdbcStoragePlugin.(JdbcStoragePlugin.java:103)
sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
java.lang.reflect.Constructor.newInstance(Constructor.java:423)
org.apache.drill.exec.store.StoragePluginRegistryImpl.create(StoragePluginRegistryImpl.java:346)
org.apache.drill.exec.store.StoragePluginRegistryImpl.createOrUpdate(StoragePluginRegistryImpl.java:239)
org.apache.drill.exec.server.rest.PluginConfigWrapper.createOrUpdateInStorage(PluginConfigWrapper.java:57)
org.apache.drill.exec.server.rest.StorageResources.createOrUpdatePluginJSON(StorageResources.java:162)
org.apache.drill.exec.server.rest.StorageResources.createOrUpdatePlugin(StorageResources.java:177)
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
java.lang.reflect.Method.invoke(Method.java:498)
org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java:81)
org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:151)
org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:171)
org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:195)
org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:104)
org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:387)
org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:331)
org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:103)
org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:269)
org.glassfish.jersey.internal.Errors$1.call(Errors.java:271)
org.glassfish.jersey.internal.Errors$1.call(Errors.java:267)
org.glassfish.jersey.internal.Errors.process(Errors.java:315)
org.glassfish.jersey.internal.Errors.process(Errors.java:297)
org.glassfish.jersey.internal.Errors.process(Errors.java:267)
org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:297)
org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:252)
org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1023)

Re: Apache Phoenix integration

2018-02-02 Thread Flavio Pompermaier 
Hi Saurabh,
I'd like to use Drill because in my use case I have multiple sources (HDFS,
HBase, Phoenix, Oracle, Mysql) and I'd like to allow the user to see the
data.
Without writing a plugin for each storage system I'd like to leverage
Apache Drill as a broker towards all of them.

Right now I've tried to puth the phoenix-core.jar into the jar/3rdparty
folder but when I try to create the Phoenix storage I get the following
error:
Please retry: error (unable to create/ update storage)

I'm trying to debug but I don't see anything wrong..


On Fri, Feb 2, 2018 at 1:42 AM, Saurabh Mahapatra <
saurabhmahapatr...@gmail.com> wrote:

> Hi Flavio,
>
> I was curious what your use case is. Phoenix provides SQL access with
> secondary indexes on HBase. Why would you want to use Drill?
>
> Thanks,
> Saurabh
>
> On Thu, Feb 1, 2018 at 3:08 PM, Kunal Khatua  wrote:
>
> > The JDBC storage plugin allows Drill to leverage any SQL system that has
> > JDBC drivers, so it should work.
> >
> > That said, the JDBC storage plugin is a community developed storage
> > plugin, so it might not be fully tested.
> >
> > If you are looking to simply have the Phoenix JDBC driver bundled into
> the
> > Drill 3rd party directory, I (think) the reason it is not bundled is
> > because we're not actively tracking compatibility of different JDBC
> drivers
> > with Drill. There could also be license issues, which is why we can't
> > bundle drivers for platforms like Oracle, etc.
> >
> > ~ Kunal
> >
> > -Original Message-
> > From: Flavio Pompermaier [mailto:pomperma...@okkam.it]
> > Sent: Thursday, February 01, 2018 9:39 AM
> > To: user@drill.apache.org
> > Subject: Apache Phoenix integration
> >
> > Hi all,
> > looking at https://urldefense.proofpoint.com/v2/url?u=https-3A__drill.
> > apache.org_docs_apache-2Ddrill-2Dcontribution-2Dideas_=DwIBaQ=
> > cskdkSMqhcnjZxdQVpwTXg=-cT6otg6lpT_XkmYy7yg3A=daBPP_
> > PaenGs721v8erkPwwO543kYzW-gKUv-c1jVWM=KMq_T0hogIOIv_
> > ttrCLCxBeEt-AUKKxpcQt3PpeNSbs=
> > it seems that Apache Phoenix is not integrated with Drill right now. In
> > theory the integration should be quite straightforward using the JDBC
> > Phoenix connector, isn't it?
> > Right now there are just a few problem of class loading when putting the
> > phoenix jar into jars/3rdparty but it shouldn't be too hard to shade
> > conflicting classes away.
> >
> > Has anyone attempted to connect to to Apache before?
> >
> > Best,
> > Flavio
> >
>



-- 
Flavio Pompermaier
Development Department

OKKAM S.r.l.
Tel. +(39) 0461 041809

RE: Apache Phoenix integration

2018-02-01 Thread Kunal Khatua 
The JDBC storage plugin allows Drill to leverage any SQL system that has JDBC 
drivers, so it should work.

That said, the JDBC storage plugin is a community developed storage plugin, so 
it might not be fully tested. 

If you are looking to simply have the Phoenix JDBC driver bundled into the 
Drill 3rd party directory, I (think) the reason it is not bundled is because 
we're not actively tracking compatibility of different JDBC drivers with Drill. 
There could also be license issues, which is why we can't bundle drivers for 
platforms like Oracle, etc. 

~ Kunal 

-Original Message-
From: Flavio Pompermaier [mailto:pomperma...@okkam.it] 
Sent: Thursday, February 01, 2018 9:39 AM
To: user@drill.apache.org
Subject: Apache Phoenix integration

Hi all,
looking at 
https://urldefense.proofpoint.com/v2/url?u=https-3A__drill.apache.org_docs_apache-2Ddrill-2Dcontribution-2Dideas_=DwIBaQ=cskdkSMqhcnjZxdQVpwTXg=-cT6otg6lpT_XkmYy7yg3A=daBPP_PaenGs721v8erkPwwO543kYzW-gKUv-c1jVWM=KMq_T0hogIOIv_ttrCLCxBeEt-AUKKxpcQt3PpeNSbs=
it seems that Apache Phoenix is not integrated with Drill right now. In theory 
the integration should be quite straightforward using the JDBC Phoenix 
connector, isn't it?
Right now there are just a few problem of class loading when putting the 
phoenix jar into jars/3rdparty but it shouldn't be too hard to shade 
conflicting classes away.

Has anyone attempted to connect to to Apache before?

Best,
Flavio