[
https://issues.apache.org/jira/browse/ARROW-1780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16423060#comment-16423060
]
ASF GitHub Bot commented on ARROW-1780:
---------------------------------------
laurentgo commented on a change in pull request #1759: ARROW-1780 - [WIP] JDBC
Adapter to convert Relational Data objects to Arrow Data Format Vector Objects
URL: https://github.com/apache/arrow/pull/1759#discussion_r178635982
##########
File path:
java/adapter/jdbc/src/main/java/org/apache/arrow/adapter/jdbc/JdbcToArrow.java
##########
@@ -0,0 +1,116 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.arrow.adapter.jdbc;
+
+import org.apache.arrow.memory.RootAllocator;
+import org.apache.arrow.vector.VectorSchemaRoot;
+
+import java.sql.*;
+
+/**
+ * Utility class to convert JDBC objects to columnar Arrow format objects.
+ *
+ * This utility uses following data mapping to map JDBC/SQL datatype to Arrow
data types.
+ *
+ * CHAR --> ArrowType.Utf8
+ * NCHAR --> ArrowType.Utf8
+ * VARCHAR --> ArrowType.Utf8
+ * NVARCHAR --> ArrowType.Utf8
+ * LONGVARCHAR --> ArrowType.Utf8
+ * LONGNVARCHAR --> ArrowType.Utf8
+ * NUMERIC --> ArrowType.Decimal(precision, scale)
+ * DECIMAL --> ArrowType.Decimal(precision, scale)
+ * BIT --> ArrowType.Bool
+ * TINYINT --> ArrowType.Int(8, signed)
+ * SMALLINT --> ArrowType.Int(16, signed)
+ * INTEGER --> ArrowType.Int(32, signed)
+ * BIGINT --> ArrowType.Int(64, signed)
+ * REAL --> ArrowType.FloatingPoint(FloatingPointPrecision.SINGLE)
+ * FLOAT --> ArrowType.FloatingPoint(FloatingPointPrecision.SINGLE)
+ * DOUBLE --> ArrowType.FloatingPoint(FloatingPointPrecision.DOUBLE)
+ * BINARY --> ArrowType.Binary
+ * VARBINARY --> ArrowType.Binary
+ * LONGVARBINARY --> ArrowType.Binary
+ * DATE --> ArrowType.Date(DateUnit.MILLISECOND)
+ * TIME --> ArrowType.Time(TimeUnit.MILLISECOND, 32)
+ * TIMESTAMP --> ArrowType.Timestamp(TimeUnit.MILLISECOND, timezone=null)
+ * CLOB --> ArrowType.Utf8
+ * BLOB --> ArrowType.Binary
+ *
+ * @since 0.10.0
+ * @see ArrowDataFetcher
+ */
+public class JdbcToArrow {
+
+ /**
+ * For the given SQL query, execute and fetch the data from Relational DB
and convert it to Arrow objects.
+ *
+ * @param connection Database connection to be used. This method will not
close the passed connection object. Since hte caller has passed
+ * the connection object it's the responsibility of the
caller to close or return the connection to the pool.
+ * @param query The DB Query to fetch the data.
+ * @return
+ * @throws SQLException Propagate any SQL Exceptions to the caller after
closing any resources opened such as ResultSet and Statment objects.
+ */
+ public static VectorSchemaRoot sqlToArrow(Connection connection, String
query) throws Exception {
+
+ assert connection != null: "JDBC conncetion object can not be null";
+ assert query != null && query.length() > 0: "SQL query can not be null
or empty";
+
+ RootAllocator rootAllocator = new RootAllocator(Integer.MAX_VALUE);
Review comment:
If there's no way to automatically free buffers/close the allocator, you
probably want to modify the existing function to take one as an input.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> JDBC Adapter for Apache Arrow
> -----------------------------
>
> Key: ARROW-1780
> URL: https://issues.apache.org/jira/browse/ARROW-1780
> Project: Apache Arrow
> Issue Type: New Feature
> Reporter: Atul Dambalkar
> Assignee: Atul Dambalkar
> Priority: Major
> Labels: pull-request-available
> Fix For: 0.10.0
>
>
> At a high level the JDBC Adapter will allow upstream apps to query RDBMS data
> over JDBC and get the JDBC objects converted to Arrow objects/structures. The
> upstream utility can then work with Arrow objects/structures with usual
> performance benefits. The utility will be very much similar to C++
> implementation of "Convert a vector of row-wise data into an Arrow table" as
> described here -
> https://arrow.apache.org/docs/cpp/md_tutorials_row_wise_conversion.html
> The utility will read data from RDBMS and covert the data into Arrow
> objects/structures. So from that perspective this will Read data from RDBMS,
> If the utility can push Arrow objects to RDBMS is something need to be
> discussed and will be out of scope for this utility for now.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
