I filed a JIRA for this work [1]. Please add questions and comments here. I already started on some PoC code and will be creating a PR in the next 1-2 weeks.
[1] https://issues.apache.org/jira/browse/ARROW-7744 On Wed, Jan 29, 2020 at 10:44 AM Andy Grove <andygrov...@gmail.com> wrote: > Thanks for all the information. This is very helpful. > > Andy. > > On Wed, Jan 29, 2020 at 9:24 AM Jacques Nadeau <jacq...@apache.org> wrote: > >> At Dremio we have two things at the moment: >> >> A JDBC driver that is built on Arrow and served as the inspiration for >> some >> of the design choices in flight [1] >> A preview flight connector that doesn't yet expose JDBC [2] >> >> These the former is built on Avatica (part of the Apache Calcite project) >> so the thinking has been that a Avatica on top of Flight client might make >> the most sense. I don't believe anyone has started thinking through how to >> do that though. >> >> [1] https://github.com/dremio/dremio-oss/tree/master/client/jdbc >> [2] https://github.com/dremio-hub/dremio-flight-connector >> >> On Wed, Jan 29, 2020 at 8:18 AM Micah Kornfield <emkornfi...@gmail.com> >> wrote: >> >> > Sorry as a follow-up Uwe wrote a blog post [1] about using the JDBC to >> > Arrow code via python. Other than that I'm not aware of anything. >> > >> > Thanks, >> > Micah >> > >> > [1] >> > >> > >> https://uwekorn.com/2019/11/17/fast-jdbc-access-in-python-using-pyarrow-jvm.html >> > >> > On Wed, Jan 29, 2020 at 8:07 AM Micah Kornfield <emkornfi...@gmail.com> >> > wrote: >> > >> > > Hi Andy, >> > > As far as I know there is no jdbc driver. What exists today is a >> wrapper >> > > to convert jdbc resultset to arrow. >> > > >> > > I believe dremio might have open sourced some code in their repo >> which >> > > wraps arrow inside of a jdbc driver. not sure if it uses flight or >> > another >> > > protocol for actual transport >> > > >> > > I think a driver on top of arrow data would be a good contribution. >> > > >> > > Thanks, >> > > Micah >> > > >> > > On Wednesday, January 29, 2020, Andy Grove <andygrov...@gmail.com> >> > wrote: >> > > >> > >> We're pretty close to having a Flight server implemented in Rust, >> > >> executing >> > >> SQL queries with DataFusion and I'm testing with a Java Flight >> client. >> > >> >> > >> Once this is working, I want to build a Flight JDBC driver. I see >> that >> > we >> > >> already have a JDBC driver in the project but I'm struggling to find >> > >> documentation about it. I searched all the markdown files in the Java >> > >> project and there are no mentions of JDBC. A quick Google search >> didn't >> > >> show anything immediately either. >> > >> >> > >> I want to get involved in the Java implementation and this is a good >> > place >> > >> for me to start and maybe my first task should be documenting the >> > current >> > >> JDBC driver. Can anyone tell me what the goals are for the current >> > driver? >> > >> Are there some docs somewhere that I missed? >> > >> >> > >> Thanks, >> > >> >> > >> Andy. >> > >> >> > > >> > >> >
