Good idea - a joint meetup would be great. On Fri, Feb 2, 2018 at 11:15 AM, Saurabh Mahapatra < saurabhmahapatr...@gmail.com> wrote:
> This is absolutely great news! Thanks for sharing, James! > > We should have this as a presentation in one of our weekly Drill hangout > sessions. > > It’s about time for us to do a meetup. A joint meetup perhaps? > > Saurabh > > Sent from my iPhone > > > > > On Feb 2, 2018, at 11:13 AM, James Taylor <jamestay...@apache.org> > wrote: > > > > There's also a much deeper integration between Phoenix + Drill (code > named > > Drillix) underway that should be possible to complete now that Apache > Drill > > is off of it's fork and on a later version of Apache Calcite. I'm hoping > > that the output of this will be a Phoenix adapter in Drill. See > > presentation here[1] and WIP code here[2]. > > > > Thanks, > > James > > > > [1] http://phoenix.apache.org/presentations/Drillix.pdf > > [2] https://github.com/jacques-n/drill/tree/phoenix_plugin > > > >> On Fri, Feb 2, 2018 at 10:21 AM, Kunal Khatua <kkha...@mapr.com> wrote: > >> > >> That's great, Flavio! > >> > >> You can create a Google doc for review and share it on the user list. > >> > >> @Bridget handles the documentation on the Apache website, so she can do > >> the final touches and help it find a home on the website. > >> > >> -----Original Message----- > >> From: Flavio Pompermaier [mailto:pomperma...@okkam.it] > >> Sent: Friday, February 02, 2018 9:04 AM > >> To: u...@drill.apache.org > >> Cc: James Taylor <jamestay...@apache.org> > >> Subject: Re: Apache Phoenix integration > >> > >> Eventually I made it to integrate Phoenix with Drill! I debugged > remotely > >> the drill-embedded via Eclipse and I discovered that the problem was > that > >> you need some extra jars to make it work! > >> Where can I write some documentation about debugging remotely Drill from > >> Eclipse and integration with Drill? > >> > >> On Fri, Feb 2, 2018 at 5:28 PM, Flavio Pompermaier < > pomperma...@okkam.it> > >> wrote: > >> > >>> What is the fastest way to debug the JDBC plugin from Eclipse? I don't > >>> see anything in the logs that could help... > >>> Is it possible to connect directly to the external embedded drill > >>> running on my machine if I enable jmx? > >>> it seems that the JDBC connection is established correctly but Drill > >>> throws an Exception (that is not well unwrapped by Jersey): > >>> > >>> 2018-02-02 16:54:04,520 [qtp159619134-56] INFO > >>> o.a.p.q.ConnectionQueryServicesImpl > >>> - HConnection established. Stacktrace for informational purposes: > >>> hconnection-0x1b9fe9f8 > >>> java.lang.Thread.getStackTrace(Thread.java:1552) > >>> org.apache.phoenix.util.LogUtil.getCallerStackTrace(LogUtil.java:55) > >>> org.apache.phoenix.query.ConnectionQueryServicesImpl.openConnection( > >>> ConnectionQueryServicesImpl.java:410) > >>> org.apache.phoenix.query.ConnectionQueryServicesImpl.access$400( > >>> ConnectionQueryServicesImpl.java:256) > >>> org.apache.phoenix.query.ConnectionQueryServicesImpl$12.call( > >>> ConnectionQueryServicesImpl.java:2408) > >>> org.apache.phoenix.query.ConnectionQueryServicesImpl$12.call( > >>> ConnectionQueryServicesImpl.java:2384) > >>> org.apache.phoenix.util.PhoenixContextExecutor.call( > >>> PhoenixContextExecutor.java:76) > >>> org.apache.phoenix.query.ConnectionQueryServicesImpl.init( > >>> ConnectionQueryServicesImpl.java:2384) > >>> org.apache.phoenix.jdbc.PhoenixDriver.getConnectionQueryServices( > >>> PhoenixDriver.java:255) > >>> org.apache.phoenix.jdbc.PhoenixEmbeddedDriver.createConnection( > >>> PhoenixEmbeddedDriver.java:150) > >>> org.apache.phoenix.jdbc.PhoenixDriver.connect(PhoenixDriver.java:221) > >>> org.apache.commons.dbcp.DriverConnectionFactory.createConnection( > >>> DriverConnectionFactory.java:38) > >>> org.apache.commons.dbcp.PoolableConnectionFactory.makeObject( > >>> PoolableConnectionFactory.java:582) > >>> org.apache.commons.dbcp.BasicDataSource.validateConnectionFactory( > >>> BasicDataSource.java:1556) > >>> org.apache.commons.dbcp.BasicDataSource.createPoolableConnectionFactor > >>> y(BasicDataSource.java:1545) > >>> org.apache.commons.dbcp.BasicDataSource.createDataSource( > >>> BasicDataSource.java:1388) > >>> org.apache.commons.dbcp.BasicDataSource.getConnection( > >>> BasicDataSource.java:1044) > >>> org.apache.calcite.adapter.jdbc.JdbcUtils$DialectPool. > >>> get(JdbcUtils.java:73) > >>> org.apache.calcite.adapter.jdbc.JdbcSchema.createDialect( > >>> JdbcSchema.java:138) > >>> org.apache.drill.exec.store.jdbc.JdbcStoragePlugin.<init>( > >>> JdbcStoragePlugin.java:103) > >>> sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) > >>> sun.reflect.NativeConstructorAccessorImpl.newInstance( > >>> NativeConstructorAccessorImpl.java:62) > >>> sun.reflect.DelegatingConstructorAccessorImpl.newInstance( > >>> DelegatingConstructorAccessorImpl.java:45) > >>> java.lang.reflect.Constructor.newInstance(Constructor.java:423) > >>> org.apache.drill.exec.store.StoragePluginRegistryImpl.create( > >>> StoragePluginRegistryImpl.java:346) > >>> org.apache.drill.exec.store.StoragePluginRegistryImpl.createOrUpdate( > >>> StoragePluginRegistryImpl.java:239) > >>> org.apache.drill.exec.server.rest.PluginConfigWrapper. > >>> createOrUpdateInStorage(PluginConfigWrapper.java:57) > >>> org.apache.drill.exec.server.rest.StorageResources. > >>> createOrUpdatePluginJSON(StorageResources.java:162) > >>> org.apache.drill.exec.server.rest.StorageResources.createOrUpdatePlugi > >>> n( > >>> StorageResources.java:177) > >>> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > >>> sun.reflect.NativeMethodAccessorImpl.invoke( > >> NativeMethodAccessorImpl.java: > >>> 62) > >>> sun.reflect.DelegatingMethodAccessorImpl.invoke( > >>> DelegatingMethodAccessorImpl.java:43) > >>> java.lang.reflect.Method.invoke(Method.java:498) > >>> org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHan > >>> dle > >>> rFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java:81) > >>> org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodD > >>> isp > >>> atcher$1.run(AbstractJavaResourceMethodDispatcher.java:151) > >>> org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodD > >>> isp > >>> atcher.invoke(AbstractJavaResourceMethodDispatcher.java:171) > >>> org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatche > >>> rPr ovider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherPr > >>> ovider.java:195) > >>> org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodD > >>> isp > >>> atcher.dispatch(AbstractJavaResourceMethodDispatcher.java:104) > >>> org.glassfish.jersey.server.model.ResourceMethodInvoker. > >>> invoke(ResourceMethodInvoker.java:387) > >>> org.glassfish.jersey.server.model.ResourceMethodInvoker. > >>> apply(ResourceMethodInvoker.java:331) > >>> org.glassfish.jersey.server.model.ResourceMethodInvoker. > >>> apply(ResourceMethodInvoker.java:103) > >>> org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:269 > >>> ) > >>> org.glassfish.jersey.internal.Errors$1.call(Errors.java:271) > >>> org.glassfish.jersey.internal.Errors$1.call(Errors.java:267) > >>> org.glassfish.jersey.internal.Errors.process(Errors.java:315) > >>> org.glassfish.jersey.internal.Errors.process(Errors.java:297) > >>> org.glassfish.jersey.internal.Errors.process(Errors.java:267) > >>> org.glassfish.jersey.process.internal.RequestScope. > >>> runInScope(RequestScope.java:297) > >>> org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:2 > >>> 52) org.glassfish.jersey.server.ApplicationHandler.handle( > >>> ApplicationHandler.java:1023) > >>> org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:37 > >>> 2) org.glassfish.jersey.servlet.ServletContainer.service( > >>> ServletContainer.java:382) > >>> org.glassfish.jersey.servlet.ServletContainer.service( > >>> ServletContainer.java:345) > >>> org.glassfish.jersey.servlet.ServletContainer.service( > >>> ServletContainer.java:220) > >>> org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:738) > >>> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java: > >>> 551) org.eclipse.jetty.server.session.SessionHandler. > >>> doHandle(SessionHandler.java:219) > >>> org.eclipse.jetty.server.handler.ContextHandler. > >>> doHandle(ContextHandler.java:1111) > >>> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:4 > >>> 78) org.eclipse.jetty.server.session.SessionHandler. > >>> doScope(SessionHandler.java:183) > >>> org.eclipse.jetty.server.handler.ContextHandler. > >>> doScope(ContextHandler.java:1045) > >>> org.eclipse.jetty.server.handler.ScopedHandler.handle( > >>> ScopedHandler.java:141) > >>> org.eclipse.jetty.server.handler.HandlerWrapper.handle( > >>> HandlerWrapper.java:97) > >>> org.eclipse.jetty.server.Server.handle(Server.java:462) > >>> org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:279) > >>> org.eclipse.jetty.server.HttpConnection.onFillable( > >>> HttpConnection.java:232) > >>> org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java: > >>> 534) org.eclipse.jetty.util.thread.QueuedThreadPool.runJob( > >>> QueuedThreadPool.java:607) > >>> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run( > >>> QueuedThreadPool.java:536) > >>> java.lang.Thread.run(Thread.java:745) > >>> > >>> > >>> On Fri, Feb 2, 2018 at 10:21 AM, Flavio Pompermaier > >>> <pomperma...@okkam.it> > >>> wrote: > >>> > >>>> Hi Saurabh, > >>>> I'd like to use Drill because in my use case I have multiple sources > >>>> (HDFS, HBase, Phoenix, Oracle, Mysql) and I'd like to allow the user > >>>> to see the data. > >>>> Without writing a plugin for each storage system I'd like to leverage > >>>> Apache Drill as a broker towards all of them. > >>>> > >>>> Right now I've tried to puth the phoenix-core.jar into the > >>>> jar/3rdparty folder but when I try to create the Phoenix storage I > >>>> get the following > >>>> error: > >>>> Please retry: error (unable to create/ update storage) > >>>> > >>>> I'm trying to debug but I don't see anything wrong.. > >>>> > >>>> > >>>> On Fri, Feb 2, 2018 at 1:42 AM, Saurabh Mahapatra < > >>>> saurabhmahapatr...@gmail.com> wrote: > >>>> > >>>>> Hi Flavio, > >>>>> > >>>>> I was curious what your use case is. Phoenix provides SQL access > >>>>> with secondary indexes on HBase. Why would you want to use Drill? > >>>>> > >>>>> Thanks, > >>>>> Saurabh > >>>>> > >>>>>> On Thu, Feb 1, 2018 at 3:08 PM, Kunal Khatua <kkha...@mapr.com> > wrote: > >>>>>> > >>>>>> The JDBC storage plugin allows Drill to leverage any SQL system > >>>>>> that > >>>>> has > >>>>>> JDBC drivers, so it should work. > >>>>>> > >>>>>> That said, the JDBC storage plugin is a community developed > >>>>>> storage plugin, so it might not be fully tested. > >>>>>> > >>>>>> If you are looking to simply have the Phoenix JDBC driver bundled > >>>>>> into > >>>>> the > >>>>>> Drill 3rd party directory, I (think) the reason it is not bundled > >>>>>> is because we're not actively tracking compatibility of different > >>>>>> JDBC > >>>>> drivers > >>>>>> with Drill. There could also be license issues, which is why we > >>>>>> can't bundle drivers for platforms like Oracle, etc. > >>>>>> > >>>>>> ~ Kunal > >>>>>> > >>>>>> -----Original Message----- > >>>>>> From: Flavio Pompermaier [mailto:pomperma...@okkam.it] > >>>>>> Sent: Thursday, February 01, 2018 9:39 AM > >>>>>> To: u...@drill.apache.org > >>>>>> Subject: Apache Phoenix integration > >>>>>> > >>>>>> Hi all, > >>>>>> looking at https://urldefense.proofpoint. > >> com/v2/url?u=https-3A__drill. > >>>>>> apache.org_docs_apache-2Ddrill-2Dcontribution-2Dideas_&d=DwIBaQ&c= > >>>>>> cskdkSMqhcnjZxdQVpwTXg&r=-cT6otg6lpT_XkmYy7yg3A&m=daBPP_ > >>>>>> PaenGs721v8erkPwwO543kYzW-gKUv-c1jVWM&s=KMq_T0hogIOIv_ > >>>>>> ttrCLCxBeEt-AUKKxpcQt3PpeNSbs&e= > >>>>>> it seems that Apache Phoenix is not integrated with Drill right > >>>>>> now. In theory the integration should be quite straightforward > >>>>>> using the JDBC Phoenix connector, isn't it? > >>>>>> Right now there are just a few problem of class loading when > >>>>>> putting > >>>>> the > >>>>>> phoenix jar into jars/3rdparty but it shouldn't be too hard to > >>>>>> shade conflicting classes away. > >>>>>> > >>>>>> Has anyone attempted to connect to to Apache before? > >>>>>> > >>>>>> Best, > >>>>>> Flavio > >>>>>> > >>>>> > >>>> > >>>> > >>>> > >>>> -- > >>>> Flavio Pompermaier > >>>> Development Department > >>>> > >>>> OKKAM S.r.l. > >>>> Tel. +(39) 0461 041809 <+39%200461%20041809> > >>>> > >>> > >>> > >>> > >> > >> > >> -- > >> Flavio Pompermaier > >> Development Department > >> > >> OKKAM S.r.l. > >> Tel. +(39) 0461 041809 > >> >
