Atul Dambalkar commented on ARROW-1780:

I have put together some very basic interface for the JDBC Adapter - so far by 
forking Arrow 
(https://github.com/atuldambalkar/arrow/tree/master/java/adapter/jdbc). I had a 
brief discussion with Uwe earlier on this on Slack, so wanted to get some more 
views on this and also not to redo or overstep. At this time, I have one API in 
the adapter which can return Arrow Vector objects after executing SQL query on 
the given JDBC connection object - VectorSchemaRoot sqlToArrow(Connection 
connection, String query).

One more possible interface could be to fetch a certain number of records from 
all the tables from the SQL database and build Arrow objects for that. The API 
can of-course be implemented lazily and only when the data for a particular 
table is requested.

> JDBC Adapter for Apache Arrow
> -----------------------------
>                 Key: ARROW-1780
>                 URL: https://issues.apache.org/jira/browse/ARROW-1780
>             Project: Apache Arrow
>          Issue Type: New Feature
>            Reporter: Atul Dambalkar
>            Priority: Major
> At a high level the JDBC Adapter will allow upstream apps to query RDBMS data 
> over JDBC and get the JDBC objects converted to Arrow objects/structures. The 
> upstream utility can then work with Arrow objects/structures with usual 
> performance benefits. The utility will be very much similar to C++ 
> implementation of "Convert a vector of row-wise data into an Arrow table" as 
> described here - 
> https://arrow.apache.org/docs/cpp/md_tutorials_row_wise_conversion.html
> The utility will read data from RDBMS and covert the data into Arrow 
> objects/structures. So from that perspective this will Read data from RDBMS, 
> If the utility can push Arrow objects to RDBMS is something need to be 
> discussed and will be out of scope for this utility for now. 

This message was sent by Atlassian JIRA

Reply via email to