This is a very great news! But what do you mean exactly with "Apache Drill is off of it's fork"? When is this deeper integration going to be released/integrated? Will Phoenix (eventually) become just a Drill storage plugin, with huge benefits for both?
On 2 Feb 2018 20:20, "James Taylor" <jamestay...@apache.org> wrote: > Good idea - a joint meetup would be great. > > On Fri, Feb 2, 2018 at 11:15 AM, Saurabh Mahapatra < > saurabhmahapatr...@gmail.com> wrote: > >> This is absolutely great news! Thanks for sharing, James! >> >> We should have this as a presentation in one of our weekly Drill hangout >> sessions. >> >> It’s about time for us to do a meetup. A joint meetup perhaps? >> >> Saurabh >> >> Sent from my iPhone >> >> >> >> > On Feb 2, 2018, at 11:13 AM, James Taylor <jamestay...@apache.org> >> wrote: >> > >> > There's also a much deeper integration between Phoenix + Drill (code >> named >> > Drillix) underway that should be possible to complete now that Apache >> Drill >> > is off of it's fork and on a later version of Apache Calcite. I'm hoping >> > that the output of this will be a Phoenix adapter in Drill. See >> > presentation here[1] and WIP code here[2]. >> > >> > Thanks, >> > James >> > >> > [1] http://phoenix.apache.org/presentations/Drillix.pdf >> > [2] https://github.com/jacques-n/drill/tree/phoenix_plugin >> > >> >> On Fri, Feb 2, 2018 at 10:21 AM, Kunal Khatua <kkha...@mapr.com> >> wrote: >> >> >> >> That's great, Flavio! >> >> >> >> You can create a Google doc for review and share it on the user list. >> >> >> >> @Bridget handles the documentation on the Apache website, so she can do >> >> the final touches and help it find a home on the website. >> >> >> >> -----Original Message----- >> >> From: Flavio Pompermaier [mailto:pomperma...@okkam.it] >> >> Sent: Friday, February 02, 2018 9:04 AM >> >> To: user@drill.apache.org >> >> Cc: James Taylor <jamestay...@apache.org> >> >> Subject: Re: Apache Phoenix integration >> >> >> >> Eventually I made it to integrate Phoenix with Drill! I debugged >> remotely >> >> the drill-embedded via Eclipse and I discovered that the problem was >> that >> >> you need some extra jars to make it work! >> >> Where can I write some documentation about debugging remotely Drill >> from >> >> Eclipse and integration with Drill? >> >> >> >> On Fri, Feb 2, 2018 at 5:28 PM, Flavio Pompermaier < >> pomperma...@okkam.it> >> >> wrote: >> >> >> >>> What is the fastest way to debug the JDBC plugin from Eclipse? I don't >> >>> see anything in the logs that could help... >> >>> Is it possible to connect directly to the external embedded drill >> >>> running on my machine if I enable jmx? >> >>> it seems that the JDBC connection is established correctly but Drill >> >>> throws an Exception (that is not well unwrapped by Jersey): >> >>> >> >>> 2018-02-02 16:54:04,520 [qtp159619134-56] INFO >> >>> o.a.p.q.ConnectionQueryServicesImpl >> >>> - HConnection established. Stacktrace for informational purposes: >> >>> hconnection-0x1b9fe9f8 >> >>> java.lang.Thread.getStackTrace(Thread.java:1552) >> >>> org.apache.phoenix.util.LogUtil.getCallerStackTrace(LogUtil.java:55) >> >>> org.apache.phoenix.query.ConnectionQueryServicesImpl.openConnection( >> >>> ConnectionQueryServicesImpl.java:410) >> >>> org.apache.phoenix.query.ConnectionQueryServicesImpl.access$400( >> >>> ConnectionQueryServicesImpl.java:256) >> >>> org.apache.phoenix.query.ConnectionQueryServicesImpl$12.call( >> >>> ConnectionQueryServicesImpl.java:2408) >> >>> org.apache.phoenix.query.ConnectionQueryServicesImpl$12.call( >> >>> ConnectionQueryServicesImpl.java:2384) >> >>> org.apache.phoenix.util.PhoenixContextExecutor.call( >> >>> PhoenixContextExecutor.java:76) >> >>> org.apache.phoenix.query.ConnectionQueryServicesImpl.init( >> >>> ConnectionQueryServicesImpl.java:2384) >> >>> org.apache.phoenix.jdbc.PhoenixDriver.getConnectionQueryServices( >> >>> PhoenixDriver.java:255) >> >>> org.apache.phoenix.jdbc.PhoenixEmbeddedDriver.createConnection( >> >>> PhoenixEmbeddedDriver.java:150) >> >>> org.apache.phoenix.jdbc.PhoenixDriver.connect(PhoenixDriver.java:221) >> >>> org.apache.commons.dbcp.DriverConnectionFactory.createConnection( >> >>> DriverConnectionFactory.java:38) >> >>> org.apache.commons.dbcp.PoolableConnectionFactory.makeObject( >> >>> PoolableConnectionFactory.java:582) >> >>> org.apache.commons.dbcp.BasicDataSource.validateConnectionFactory( >> >>> BasicDataSource.java:1556) >> >>> org.apache.commons.dbcp.BasicDataSource.createPoolableConnec >> tionFactor >> >>> y(BasicDataSource.java:1545) >> >>> org.apache.commons.dbcp.BasicDataSource.createDataSource( >> >>> BasicDataSource.java:1388) >> >>> org.apache.commons.dbcp.BasicDataSource.getConnection( >> >>> BasicDataSource.java:1044) >> >>> org.apache.calcite.adapter.jdbc.JdbcUtils$DialectPool. >> >>> get(JdbcUtils.java:73) >> >>> org.apache.calcite.adapter.jdbc.JdbcSchema.createDialect( >> >>> JdbcSchema.java:138) >> >>> org.apache.drill.exec.store.jdbc.JdbcStoragePlugin.<init>( >> >>> JdbcStoragePlugin.java:103) >> >>> sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) >> >>> sun.reflect.NativeConstructorAccessorImpl.newInstance( >> >>> NativeConstructorAccessorImpl.java:62) >> >>> sun.reflect.DelegatingConstructorAccessorImpl.newInstance( >> >>> DelegatingConstructorAccessorImpl.java:45) >> >>> java.lang.reflect.Constructor.newInstance(Constructor.java:423) >> >>> org.apache.drill.exec.store.StoragePluginRegistryImpl.create( >> >>> StoragePluginRegistryImpl.java:346) >> >>> org.apache.drill.exec.store.StoragePluginRegistryImpl.createOrUpdate( >> >>> StoragePluginRegistryImpl.java:239) >> >>> org.apache.drill.exec.server.rest.PluginConfigWrapper. >> >>> createOrUpdateInStorage(PluginConfigWrapper.java:57) >> >>> org.apache.drill.exec.server.rest.StorageResources. >> >>> createOrUpdatePluginJSON(StorageResources.java:162) >> >>> org.apache.drill.exec.server.rest.StorageResources.createOrU >> pdatePlugi >> >>> n( >> >>> StorageResources.java:177) >> >>> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> >>> sun.reflect.NativeMethodAccessorImpl.invoke( >> >> NativeMethodAccessorImpl.java: >> >>> 62) >> >>> sun.reflect.DelegatingMethodAccessorImpl.invoke( >> >>> DelegatingMethodAccessorImpl.java:43) >> >>> java.lang.reflect.Method.invoke(Method.java:498) >> >>> org.glassfish.jersey.server.model.internal.ResourceMethodInv >> ocationHan >> >>> dle >> >>> rFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java:81) >> >>> org.glassfish.jersey.server.model.internal.AbstractJavaResou >> rceMethodD >> >>> isp >> >>> atcher$1.run(AbstractJavaResourceMethodDispatcher.java:151) >> >>> org.glassfish.jersey.server.model.internal.AbstractJavaResou >> rceMethodD >> >>> isp >> >>> atcher.invoke(AbstractJavaResourceMethodDispatcher.java:171) >> >>> org.glassfish.jersey.server.model.internal.JavaResourceMetho >> dDispatche >> >>> rPr ovider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherPr >> >>> ovider.java:195) >> >>> org.glassfish.jersey.server.model.internal.AbstractJavaResou >> rceMethodD >> >>> isp >> >>> atcher.dispatch(AbstractJavaResourceMethodDispatcher.java:104) >> >>> org.glassfish.jersey.server.model.ResourceMethodInvoker. >> >>> invoke(ResourceMethodInvoker.java:387) >> >>> org.glassfish.jersey.server.model.ResourceMethodInvoker. >> >>> apply(ResourceMethodInvoker.java:331) >> >>> org.glassfish.jersey.server.model.ResourceMethodInvoker. >> >>> apply(ResourceMethodInvoker.java:103) >> >>> org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntim >> e.java:269 >> >>> ) >> >>> org.glassfish.jersey.internal.Errors$1.call(Errors.java:271) >> >>> org.glassfish.jersey.internal.Errors$1.call(Errors.java:267) >> >>> org.glassfish.jersey.internal.Errors.process(Errors.java:315) >> >>> org.glassfish.jersey.internal.Errors.process(Errors.java:297) >> >>> org.glassfish.jersey.internal.Errors.process(Errors.java:267) >> >>> org.glassfish.jersey.process.internal.RequestScope. >> >>> runInScope(RequestScope.java:297) >> >>> org.glassfish.jersey.server.ServerRuntime.process(ServerRunt >> ime.java:2 >> >>> 52) org.glassfish.jersey.server.ApplicationHandler.handle( >> >>> ApplicationHandler.java:1023) >> >>> org.glassfish.jersey.servlet.WebComponent.service(WebCompone >> nt.java:37 >> >>> 2) org.glassfish.jersey.servlet.ServletContainer.service( >> >>> ServletContainer.java:382) >> >>> org.glassfish.jersey.servlet.ServletContainer.service( >> >>> ServletContainer.java:345) >> >>> org.glassfish.jersey.servlet.ServletContainer.service( >> >>> ServletContainer.java:220) >> >>> org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder >> .java:738) >> >>> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHan >> dler.java: >> >>> 551) org.eclipse.jetty.server.session.SessionHandler. >> >>> doHandle(SessionHandler.java:219) >> >>> org.eclipse.jetty.server.handler.ContextHandler. >> >>> doHandle(ContextHandler.java:1111) >> >>> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHand >> ler.java:4 >> >>> 78) org.eclipse.jetty.server.session.SessionHandler. >> >>> doScope(SessionHandler.java:183) >> >>> org.eclipse.jetty.server.handler.ContextHandler. >> >>> doScope(ContextHandler.java:1045) >> >>> org.eclipse.jetty.server.handler.ScopedHandler.handle( >> >>> ScopedHandler.java:141) >> >>> org.eclipse.jetty.server.handler.HandlerWrapper.handle( >> >>> HandlerWrapper.java:97) >> >>> org.eclipse.jetty.server.Server.handle(Server.java:462) >> >>> org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:279) >> >>> org.eclipse.jetty.server.HttpConnection.onFillable( >> >>> HttpConnection.java:232) >> >>> org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnec >> tion.java: >> >>> 534) org.eclipse.jetty.util.thread.QueuedThreadPool.runJob( >> >>> QueuedThreadPool.java:607) >> >>> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run( >> >>> QueuedThreadPool.java:536) >> >>> java.lang.Thread.run(Thread.java:745) >> >>> >> >>> >> >>> On Fri, Feb 2, 2018 at 10:21 AM, Flavio Pompermaier >> >>> <pomperma...@okkam.it> >> >>> wrote: >> >>> >> >>>> Hi Saurabh, >> >>>> I'd like to use Drill because in my use case I have multiple sources >> >>>> (HDFS, HBase, Phoenix, Oracle, Mysql) and I'd like to allow the user >> >>>> to see the data. >> >>>> Without writing a plugin for each storage system I'd like to leverage >> >>>> Apache Drill as a broker towards all of them. >> >>>> >> >>>> Right now I've tried to puth the phoenix-core.jar into the >> >>>> jar/3rdparty folder but when I try to create the Phoenix storage I >> >>>> get the following >> >>>> error: >> >>>> Please retry: error (unable to create/ update storage) >> >>>> >> >>>> I'm trying to debug but I don't see anything wrong.. >> >>>> >> >>>> >> >>>> On Fri, Feb 2, 2018 at 1:42 AM, Saurabh Mahapatra < >> >>>> saurabhmahapatr...@gmail.com> wrote: >> >>>> >> >>>>> Hi Flavio, >> >>>>> >> >>>>> I was curious what your use case is. Phoenix provides SQL access >> >>>>> with secondary indexes on HBase. Why would you want to use Drill? >> >>>>> >> >>>>> Thanks, >> >>>>> Saurabh >> >>>>> >> >>>>>> On Thu, Feb 1, 2018 at 3:08 PM, Kunal Khatua <kkha...@mapr.com> >> wrote: >> >>>>>> >> >>>>>> The JDBC storage plugin allows Drill to leverage any SQL system >> >>>>>> that >> >>>>> has >> >>>>>> JDBC drivers, so it should work. >> >>>>>> >> >>>>>> That said, the JDBC storage plugin is a community developed >> >>>>>> storage plugin, so it might not be fully tested. >> >>>>>> >> >>>>>> If you are looking to simply have the Phoenix JDBC driver bundled >> >>>>>> into >> >>>>> the >> >>>>>> Drill 3rd party directory, I (think) the reason it is not bundled >> >>>>>> is because we're not actively tracking compatibility of different >> >>>>>> JDBC >> >>>>> drivers >> >>>>>> with Drill. There could also be license issues, which is why we >> >>>>>> can't bundle drivers for platforms like Oracle, etc. >> >>>>>> >> >>>>>> ~ Kunal >> >>>>>> >> >>>>>> -----Original Message----- >> >>>>>> From: Flavio Pompermaier [mailto:pomperma...@okkam.it] >> >>>>>> Sent: Thursday, February 01, 2018 9:39 AM >> >>>>>> To: user@drill.apache.org >> >>>>>> Subject: Apache Phoenix integration >> >>>>>> >> >>>>>> Hi all, >> >>>>>> looking at https://urldefense.proofpoint. >> >> com/v2/url?u=https-3A__drill. >> >>>>>> apache.org_docs_apache-2Ddrill-2Dcontribution-2Dideas_&d=DwIBaQ&c= >> >>>>>> cskdkSMqhcnjZxdQVpwTXg&r=-cT6otg6lpT_XkmYy7yg3A&m=daBPP_ >> >>>>>> PaenGs721v8erkPwwO543kYzW-gKUv-c1jVWM&s=KMq_T0hogIOIv_ >> >>>>>> ttrCLCxBeEt-AUKKxpcQt3PpeNSbs&e= >> >>>>>> it seems that Apache Phoenix is not integrated with Drill right >> >>>>>> now. In theory the integration should be quite straightforward >> >>>>>> using the JDBC Phoenix connector, isn't it? >> >>>>>> Right now there are just a few problem of class loading when >> >>>>>> putting >> >>>>> the >> >>>>>> phoenix jar into jars/3rdparty but it shouldn't be too hard to >> >>>>>> shade conflicting classes away. >> >>>>>> >> >>>>>> Has anyone attempted to connect to to Apache before? >> >>>>>> >> >>>>>> Best, >> >>>>>> Flavio >> >>>>>> >> >>>>> >> >>>> >> >>>> >> >>>> >> >>>> -- >> >>>> Flavio Pompermaier >> >>>> Development Department >> >>>> >> >>>> OKKAM S.r.l. >> >>>> Tel. +(39) 0461 041809 <+39%200461%20041809> <+39%200461%20041809> >> >>>> >> >>> >> >>> >> >>> >> >> >> >> >> >> -- >> >> Flavio Pompermaier >> >> Development Department >> >> >> >> OKKAM S.r.l. >> >> Tel. +(39) 0461 041809 <+39%200461%20041809> >> >> >> > >
