This is absolutely great news! Thanks for sharing, James! We should have this as a presentation in one of our weekly Drill hangout sessions.
It’s about time for us to do a meetup. A joint meetup perhaps? Saurabh Sent from my iPhone > On Feb 2, 2018, at 11:13 AM, James Taylor <jamestay...@apache.org> wrote: > > There's also a much deeper integration between Phoenix + Drill (code named > Drillix) underway that should be possible to complete now that Apache Drill > is off of it's fork and on a later version of Apache Calcite. I'm hoping > that the output of this will be a Phoenix adapter in Drill. See > presentation here[1] and WIP code here[2]. > > Thanks, > James > > [1] http://phoenix.apache.org/presentations/Drillix.pdf > [2] https://github.com/jacques-n/drill/tree/phoenix_plugin > >> On Fri, Feb 2, 2018 at 10:21 AM, Kunal Khatua <kkha...@mapr.com> wrote: >> >> That's great, Flavio! >> >> You can create a Google doc for review and share it on the user list. >> >> @Bridget handles the documentation on the Apache website, so she can do >> the final touches and help it find a home on the website. >> >> -----Original Message----- >> From: Flavio Pompermaier [mailto:pomperma...@okkam.it] >> Sent: Friday, February 02, 2018 9:04 AM >> To: user@drill.apache.org >> Cc: James Taylor <jamestay...@apache.org> >> Subject: Re: Apache Phoenix integration >> >> Eventually I made it to integrate Phoenix with Drill! I debugged remotely >> the drill-embedded via Eclipse and I discovered that the problem was that >> you need some extra jars to make it work! >> Where can I write some documentation about debugging remotely Drill from >> Eclipse and integration with Drill? >> >> On Fri, Feb 2, 2018 at 5:28 PM, Flavio Pompermaier <pomperma...@okkam.it> >> wrote: >> >>> What is the fastest way to debug the JDBC plugin from Eclipse? I don't >>> see anything in the logs that could help... >>> Is it possible to connect directly to the external embedded drill >>> running on my machine if I enable jmx? >>> it seems that the JDBC connection is established correctly but Drill >>> throws an Exception (that is not well unwrapped by Jersey): >>> >>> 2018-02-02 16:54:04,520 [qtp159619134-56] INFO >>> o.a.p.q.ConnectionQueryServicesImpl >>> - HConnection established. Stacktrace for informational purposes: >>> hconnection-0x1b9fe9f8 >>> java.lang.Thread.getStackTrace(Thread.java:1552) >>> org.apache.phoenix.util.LogUtil.getCallerStackTrace(LogUtil.java:55) >>> org.apache.phoenix.query.ConnectionQueryServicesImpl.openConnection( >>> ConnectionQueryServicesImpl.java:410) >>> org.apache.phoenix.query.ConnectionQueryServicesImpl.access$400( >>> ConnectionQueryServicesImpl.java:256) >>> org.apache.phoenix.query.ConnectionQueryServicesImpl$12.call( >>> ConnectionQueryServicesImpl.java:2408) >>> org.apache.phoenix.query.ConnectionQueryServicesImpl$12.call( >>> ConnectionQueryServicesImpl.java:2384) >>> org.apache.phoenix.util.PhoenixContextExecutor.call( >>> PhoenixContextExecutor.java:76) >>> org.apache.phoenix.query.ConnectionQueryServicesImpl.init( >>> ConnectionQueryServicesImpl.java:2384) >>> org.apache.phoenix.jdbc.PhoenixDriver.getConnectionQueryServices( >>> PhoenixDriver.java:255) >>> org.apache.phoenix.jdbc.PhoenixEmbeddedDriver.createConnection( >>> PhoenixEmbeddedDriver.java:150) >>> org.apache.phoenix.jdbc.PhoenixDriver.connect(PhoenixDriver.java:221) >>> org.apache.commons.dbcp.DriverConnectionFactory.createConnection( >>> DriverConnectionFactory.java:38) >>> org.apache.commons.dbcp.PoolableConnectionFactory.makeObject( >>> PoolableConnectionFactory.java:582) >>> org.apache.commons.dbcp.BasicDataSource.validateConnectionFactory( >>> BasicDataSource.java:1556) >>> org.apache.commons.dbcp.BasicDataSource.createPoolableConnectionFactor >>> y(BasicDataSource.java:1545) >>> org.apache.commons.dbcp.BasicDataSource.createDataSource( >>> BasicDataSource.java:1388) >>> org.apache.commons.dbcp.BasicDataSource.getConnection( >>> BasicDataSource.java:1044) >>> org.apache.calcite.adapter.jdbc.JdbcUtils$DialectPool. >>> get(JdbcUtils.java:73) >>> org.apache.calcite.adapter.jdbc.JdbcSchema.createDialect( >>> JdbcSchema.java:138) >>> org.apache.drill.exec.store.jdbc.JdbcStoragePlugin.<init>( >>> JdbcStoragePlugin.java:103) >>> sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) >>> sun.reflect.NativeConstructorAccessorImpl.newInstance( >>> NativeConstructorAccessorImpl.java:62) >>> sun.reflect.DelegatingConstructorAccessorImpl.newInstance( >>> DelegatingConstructorAccessorImpl.java:45) >>> java.lang.reflect.Constructor.newInstance(Constructor.java:423) >>> org.apache.drill.exec.store.StoragePluginRegistryImpl.create( >>> StoragePluginRegistryImpl.java:346) >>> org.apache.drill.exec.store.StoragePluginRegistryImpl.createOrUpdate( >>> StoragePluginRegistryImpl.java:239) >>> org.apache.drill.exec.server.rest.PluginConfigWrapper. >>> createOrUpdateInStorage(PluginConfigWrapper.java:57) >>> org.apache.drill.exec.server.rest.StorageResources. >>> createOrUpdatePluginJSON(StorageResources.java:162) >>> org.apache.drill.exec.server.rest.StorageResources.createOrUpdatePlugi >>> n( >>> StorageResources.java:177) >>> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >>> sun.reflect.NativeMethodAccessorImpl.invoke( >> NativeMethodAccessorImpl.java: >>> 62) >>> sun.reflect.DelegatingMethodAccessorImpl.invoke( >>> DelegatingMethodAccessorImpl.java:43) >>> java.lang.reflect.Method.invoke(Method.java:498) >>> org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHan >>> dle >>> rFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java:81) >>> org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodD >>> isp >>> atcher$1.run(AbstractJavaResourceMethodDispatcher.java:151) >>> org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodD >>> isp >>> atcher.invoke(AbstractJavaResourceMethodDispatcher.java:171) >>> org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatche >>> rPr ovider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherPr >>> ovider.java:195) >>> org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodD >>> isp >>> atcher.dispatch(AbstractJavaResourceMethodDispatcher.java:104) >>> org.glassfish.jersey.server.model.ResourceMethodInvoker. >>> invoke(ResourceMethodInvoker.java:387) >>> org.glassfish.jersey.server.model.ResourceMethodInvoker. >>> apply(ResourceMethodInvoker.java:331) >>> org.glassfish.jersey.server.model.ResourceMethodInvoker. >>> apply(ResourceMethodInvoker.java:103) >>> org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:269 >>> ) >>> org.glassfish.jersey.internal.Errors$1.call(Errors.java:271) >>> org.glassfish.jersey.internal.Errors$1.call(Errors.java:267) >>> org.glassfish.jersey.internal.Errors.process(Errors.java:315) >>> org.glassfish.jersey.internal.Errors.process(Errors.java:297) >>> org.glassfish.jersey.internal.Errors.process(Errors.java:267) >>> org.glassfish.jersey.process.internal.RequestScope. >>> runInScope(RequestScope.java:297) >>> org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:2 >>> 52) org.glassfish.jersey.server.ApplicationHandler.handle( >>> ApplicationHandler.java:1023) >>> org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:37 >>> 2) org.glassfish.jersey.servlet.ServletContainer.service( >>> ServletContainer.java:382) >>> org.glassfish.jersey.servlet.ServletContainer.service( >>> ServletContainer.java:345) >>> org.glassfish.jersey.servlet.ServletContainer.service( >>> ServletContainer.java:220) >>> org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:738) >>> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java: >>> 551) org.eclipse.jetty.server.session.SessionHandler. >>> doHandle(SessionHandler.java:219) >>> org.eclipse.jetty.server.handler.ContextHandler. >>> doHandle(ContextHandler.java:1111) >>> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:4 >>> 78) org.eclipse.jetty.server.session.SessionHandler. >>> doScope(SessionHandler.java:183) >>> org.eclipse.jetty.server.handler.ContextHandler. >>> doScope(ContextHandler.java:1045) >>> org.eclipse.jetty.server.handler.ScopedHandler.handle( >>> ScopedHandler.java:141) >>> org.eclipse.jetty.server.handler.HandlerWrapper.handle( >>> HandlerWrapper.java:97) >>> org.eclipse.jetty.server.Server.handle(Server.java:462) >>> org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:279) >>> org.eclipse.jetty.server.HttpConnection.onFillable( >>> HttpConnection.java:232) >>> org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java: >>> 534) org.eclipse.jetty.util.thread.QueuedThreadPool.runJob( >>> QueuedThreadPool.java:607) >>> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run( >>> QueuedThreadPool.java:536) >>> java.lang.Thread.run(Thread.java:745) >>> >>> >>> On Fri, Feb 2, 2018 at 10:21 AM, Flavio Pompermaier >>> <pomperma...@okkam.it> >>> wrote: >>> >>>> Hi Saurabh, >>>> I'd like to use Drill because in my use case I have multiple sources >>>> (HDFS, HBase, Phoenix, Oracle, Mysql) and I'd like to allow the user >>>> to see the data. >>>> Without writing a plugin for each storage system I'd like to leverage >>>> Apache Drill as a broker towards all of them. >>>> >>>> Right now I've tried to puth the phoenix-core.jar into the >>>> jar/3rdparty folder but when I try to create the Phoenix storage I >>>> get the following >>>> error: >>>> Please retry: error (unable to create/ update storage) >>>> >>>> I'm trying to debug but I don't see anything wrong.. >>>> >>>> >>>> On Fri, Feb 2, 2018 at 1:42 AM, Saurabh Mahapatra < >>>> saurabhmahapatr...@gmail.com> wrote: >>>> >>>>> Hi Flavio, >>>>> >>>>> I was curious what your use case is. Phoenix provides SQL access >>>>> with secondary indexes on HBase. Why would you want to use Drill? >>>>> >>>>> Thanks, >>>>> Saurabh >>>>> >>>>>> On Thu, Feb 1, 2018 at 3:08 PM, Kunal Khatua <kkha...@mapr.com> wrote: >>>>>> >>>>>> The JDBC storage plugin allows Drill to leverage any SQL system >>>>>> that >>>>> has >>>>>> JDBC drivers, so it should work. >>>>>> >>>>>> That said, the JDBC storage plugin is a community developed >>>>>> storage plugin, so it might not be fully tested. >>>>>> >>>>>> If you are looking to simply have the Phoenix JDBC driver bundled >>>>>> into >>>>> the >>>>>> Drill 3rd party directory, I (think) the reason it is not bundled >>>>>> is because we're not actively tracking compatibility of different >>>>>> JDBC >>>>> drivers >>>>>> with Drill. There could also be license issues, which is why we >>>>>> can't bundle drivers for platforms like Oracle, etc. >>>>>> >>>>>> ~ Kunal >>>>>> >>>>>> -----Original Message----- >>>>>> From: Flavio Pompermaier [mailto:pomperma...@okkam.it] >>>>>> Sent: Thursday, February 01, 2018 9:39 AM >>>>>> To: user@drill.apache.org >>>>>> Subject: Apache Phoenix integration >>>>>> >>>>>> Hi all, >>>>>> looking at https://urldefense.proofpoint. >> com/v2/url?u=https-3A__drill. >>>>>> apache.org_docs_apache-2Ddrill-2Dcontribution-2Dideas_&d=DwIBaQ&c= >>>>>> cskdkSMqhcnjZxdQVpwTXg&r=-cT6otg6lpT_XkmYy7yg3A&m=daBPP_ >>>>>> PaenGs721v8erkPwwO543kYzW-gKUv-c1jVWM&s=KMq_T0hogIOIv_ >>>>>> ttrCLCxBeEt-AUKKxpcQt3PpeNSbs&e= >>>>>> it seems that Apache Phoenix is not integrated with Drill right >>>>>> now. In theory the integration should be quite straightforward >>>>>> using the JDBC Phoenix connector, isn't it? >>>>>> Right now there are just a few problem of class loading when >>>>>> putting >>>>> the >>>>>> phoenix jar into jars/3rdparty but it shouldn't be too hard to >>>>>> shade conflicting classes away. >>>>>> >>>>>> Has anyone attempted to connect to to Apache before? >>>>>> >>>>>> Best, >>>>>> Flavio >>>>>> >>>>> >>>> >>>> >>>> >>>> -- >>>> Flavio Pompermaier >>>> Development Department >>>> >>>> OKKAM S.r.l. >>>> Tel. +(39) 0461 041809 <+39%200461%20041809> >>>> >>> >>> >>> >> >> >> -- >> Flavio Pompermaier >> Development Department >> >> OKKAM S.r.l. >> Tel. +(39) 0461 041809 >>
