lidavidm commented on code in PR #229:
URL: https://github.com/apache/arrow-cookbook/pull/229#discussion_r920547587
##########
java/source/demo/pom.xml:
##########
@@ -21,9 +21,17 @@
<properties>
<maven.compiler.source>8</maven.compiler.source>
<maven.compiler.target>8</maven.compiler.target>
- <arrow.version>8.0.0</arrow.version>
+ <arrow.version>9.0.0.dev191</arrow.version><!-- Prepare for 9.0.0
release -->
Review Comment:
I don't think we want to make the cookbook depend on nightly arrow…so we
should hold off until after the release?
##########
java/source/io.rst:
##########
@@ -444,6 +444,176 @@ Reading Parquet File
Please check :doc:`Dataset <./dataset>`
+Reading JDBC ResultSets
+***********************
+
+The `Arrow Java JDBC module <https://arrow.apache.org/docs/java/jdbc.html>`_
+help us to convert JDBC ResultSets objects into columnar arrow format objects
+through Arrow VectorSchemaRoots.
+
+ResultSet to VectorSchemaRoot Conversion
+----------------------------------------
+
+The main class to help us to convert ResultSet to VectorSchemaRoot is
+`JdbcToArrow
<https://arrow.apache.org/docs/java/reference/org/apache/arrow/adapter/jdbc/JdbcToArrow.html>`_
+
+.. testcode::
+
+ import org.apache.arrow.adapter.jdbc.ArrowVectorIterator;
+ import org.apache.arrow.adapter.jdbc.JdbcToArrow;
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.VectorSchemaRoot;
+ import org.h2.jdbcx.JdbcConnectionPool;
+
+ import java.io.IOException;
+ import java.sql.ResultSet;
+ import java.sql.SQLException;
+
+ try (BufferAllocator allocator = new RootAllocator()) {
+ JdbcConnectionPool pool = JdbcConnectionPool.create(
+
"jdbc:h2:zip:./thirdpartydeps/database/jdbc-cookbook.zip!/test",
+ "sa", "");
+ ResultSet resultSet =
pool.getConnection().createStatement().executeQuery(
Review Comment:
ResultSet needs to be closed, right?
##########
java/source/io.rst:
##########
@@ -444,6 +444,176 @@ Reading Parquet File
Please check :doc:`Dataset <./dataset>`
+Reading JDBC ResultSets
+***********************
+
+The `Arrow Java JDBC module <https://arrow.apache.org/docs/java/jdbc.html>`_
+help us to convert JDBC ResultSets objects into columnar arrow format objects
+through Arrow VectorSchemaRoots.
+
+ResultSet to VectorSchemaRoot Conversion
+----------------------------------------
+
+The main class to help us to convert ResultSet to VectorSchemaRoot is
+`JdbcToArrow
<https://arrow.apache.org/docs/java/reference/org/apache/arrow/adapter/jdbc/JdbcToArrow.html>`_
+
+.. testcode::
+
+ import org.apache.arrow.adapter.jdbc.ArrowVectorIterator;
+ import org.apache.arrow.adapter.jdbc.JdbcToArrow;
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.VectorSchemaRoot;
+ import org.h2.jdbcx.JdbcConnectionPool;
+
+ import java.io.IOException;
+ import java.sql.ResultSet;
+ import java.sql.SQLException;
+
+ try (BufferAllocator allocator = new RootAllocator()) {
+ JdbcConnectionPool pool = JdbcConnectionPool.create(
+
"jdbc:h2:zip:./thirdpartydeps/database/jdbc-cookbook.zip!/test",
+ "sa", "");
+ ResultSet resultSet =
pool.getConnection().createStatement().executeQuery(
+ "SELECT int_field1, bool_field2, double_field7 FROM TABLE1");
+ try (ArrowVectorIterator iterator =
JdbcToArrow.sqlToArrowVectorIterator(
+ resultSet, allocator)) {
+ while (iterator.hasNext()) {
+ VectorSchemaRoot root = iterator.next();
+ System.out.print(root.contentToTSVString());
+ }
+ }
+ } catch (SQLException throwables) {
+ throwables.printStackTrace();
+ } catch (IOException e) {
+ e.printStackTrace();
Review Comment:
```suggestion
} catch (SQLException | IOException e) {
e.printStackTrace();
```
and ditto below
##########
java/source/io.rst:
##########
@@ -444,6 +444,176 @@ Reading Parquet File
Please check :doc:`Dataset <./dataset>`
+Reading JDBC ResultSets
+***********************
+
+The `Arrow Java JDBC module <https://arrow.apache.org/docs/java/jdbc.html>`_
+help us to convert JDBC ResultSets objects into columnar arrow format objects
+through Arrow VectorSchemaRoots.
+
+ResultSet to VectorSchemaRoot Conversion
+----------------------------------------
+
+The main class to help us to convert ResultSet to VectorSchemaRoot is
+`JdbcToArrow
<https://arrow.apache.org/docs/java/reference/org/apache/arrow/adapter/jdbc/JdbcToArrow.html>`_
+
+.. testcode::
+
+ import org.apache.arrow.adapter.jdbc.ArrowVectorIterator;
+ import org.apache.arrow.adapter.jdbc.JdbcToArrow;
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.VectorSchemaRoot;
+ import org.h2.jdbcx.JdbcConnectionPool;
+
+ import java.io.IOException;
+ import java.sql.ResultSet;
+ import java.sql.SQLException;
+
+ try (BufferAllocator allocator = new RootAllocator()) {
+ JdbcConnectionPool pool = JdbcConnectionPool.create(
+
"jdbc:h2:zip:./thirdpartydeps/database/jdbc-cookbook.zip!/test",
+ "sa", "");
+ ResultSet resultSet =
pool.getConnection().createStatement().executeQuery(
+ "SELECT int_field1, bool_field2, double_field7 FROM TABLE1");
+ try (ArrowVectorIterator iterator =
JdbcToArrow.sqlToArrowVectorIterator(
+ resultSet, allocator)) {
+ while (iterator.hasNext()) {
+ VectorSchemaRoot root = iterator.next();
+ System.out.print(root.contentToTSVString());
+ }
+ }
+ } catch (SQLException throwables) {
+ throwables.printStackTrace();
+ } catch (IOException e) {
+ e.printStackTrace();
+ }
+
+.. testoutput::
+
+ INT_FIELD1 BOOL_FIELD2 DOUBLE_FIELD7
+ 101 true 5.6478356785345E10
+ 102 true 5.6478356785345E10
+ 103 true 5.6478356785345E10
+
+JdbcToArrow could be created also with a custom configurations needed with the
+support of `JdbcToArrowConfig
<https://arrow.apache.org/docs/java/reference/org/apache/arrow/adapter/jdbc/JdbcToArrowConfig.html>`_
+, it is useful at the moment to work with array columns.
Review Comment:
We talk about arrays and row counts here, what about decimals and other
things Todd added?
##########
java/source/io.rst:
##########
@@ -444,6 +444,176 @@ Reading Parquet File
Please check :doc:`Dataset <./dataset>`
+Reading JDBC ResultSets
+***********************
+
+The `Arrow Java JDBC module <https://arrow.apache.org/docs/java/jdbc.html>`_
+help us to convert JDBC ResultSets objects into columnar arrow format objects
+through Arrow VectorSchemaRoots.
+
+ResultSet to VectorSchemaRoot Conversion
+----------------------------------------
+
+The main class to help us to convert ResultSet to VectorSchemaRoot is
+`JdbcToArrow
<https://arrow.apache.org/docs/java/reference/org/apache/arrow/adapter/jdbc/JdbcToArrow.html>`_
+
+.. testcode::
+
+ import org.apache.arrow.adapter.jdbc.ArrowVectorIterator;
+ import org.apache.arrow.adapter.jdbc.JdbcToArrow;
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.VectorSchemaRoot;
+ import org.h2.jdbcx.JdbcConnectionPool;
+
+ import java.io.IOException;
+ import java.sql.ResultSet;
+ import java.sql.SQLException;
+
+ try (BufferAllocator allocator = new RootAllocator()) {
+ JdbcConnectionPool pool = JdbcConnectionPool.create(
+
"jdbc:h2:zip:./thirdpartydeps/database/jdbc-cookbook.zip!/test",
+ "sa", "");
+ ResultSet resultSet =
pool.getConnection().createStatement().executeQuery(
+ "SELECT int_field1, bool_field2, double_field7 FROM TABLE1");
+ try (ArrowVectorIterator iterator =
JdbcToArrow.sqlToArrowVectorIterator(
+ resultSet, allocator)) {
+ while (iterator.hasNext()) {
+ VectorSchemaRoot root = iterator.next();
+ System.out.print(root.contentToTSVString());
+ }
+ }
+ } catch (SQLException throwables) {
+ throwables.printStackTrace();
+ } catch (IOException e) {
+ e.printStackTrace();
+ }
+
+.. testoutput::
+
+ INT_FIELD1 BOOL_FIELD2 DOUBLE_FIELD7
+ 101 true 5.6478356785345E10
+ 102 true 5.6478356785345E10
+ 103 true 5.6478356785345E10
+
+JdbcToArrow could be created also with a custom configurations needed with the
+support of `JdbcToArrowConfig
<https://arrow.apache.org/docs/java/reference/org/apache/arrow/adapter/jdbc/JdbcToArrowConfig.html>`_
+, it is useful at the moment to work with array columns.
+
+.. testcode::
+
+ import org.apache.arrow.adapter.jdbc.ArrowVectorIterator;
+ import org.apache.arrow.adapter.jdbc.JdbcFieldInfo;
+ import org.apache.arrow.adapter.jdbc.JdbcToArrow;
+ import org.apache.arrow.adapter.jdbc.JdbcToArrowConfig;
+ import org.apache.arrow.adapter.jdbc.JdbcToArrowConfigBuilder;
+ import org.apache.arrow.adapter.jdbc.JdbcToArrowUtils;
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.VectorSchemaRoot;
+ import org.h2.jdbcx.JdbcConnectionPool;
+
+ import java.io.IOException;
+ import java.sql.ResultSet;
+ import java.sql.SQLException;
+ import java.sql.Types;
+ import java.util.HashMap;
+
+ try (BufferAllocator allocator = new RootAllocator()) {
+ JdbcConnectionPool pool = JdbcConnectionPool.create(
+
"jdbc:h2:zip:./thirdpartydeps/database/jdbc-cookbook.zip!/test",
Review Comment:
is an in-memory database not possible? Apache Derby lets us do that (that's
what I've been using for testing in ADBC, and it's what Flight SQL uses)
##########
java/source/io.rst:
##########
@@ -444,6 +444,176 @@ Reading Parquet File
Please check :doc:`Dataset <./dataset>`
+Reading JDBC ResultSets
+***********************
+
+The `Arrow Java JDBC module <https://arrow.apache.org/docs/java/jdbc.html>`_
+help us to convert JDBC ResultSets objects into columnar arrow format objects
+through Arrow VectorSchemaRoots.
+
+ResultSet to VectorSchemaRoot Conversion
+----------------------------------------
+
+The main class to help us to convert ResultSet to VectorSchemaRoot is
+`JdbcToArrow
<https://arrow.apache.org/docs/java/reference/org/apache/arrow/adapter/jdbc/JdbcToArrow.html>`_
+
+.. testcode::
+
+ import org.apache.arrow.adapter.jdbc.ArrowVectorIterator;
+ import org.apache.arrow.adapter.jdbc.JdbcToArrow;
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.VectorSchemaRoot;
+ import org.h2.jdbcx.JdbcConnectionPool;
+
+ import java.io.IOException;
+ import java.sql.ResultSet;
+ import java.sql.SQLException;
+
+ try (BufferAllocator allocator = new RootAllocator()) {
+ JdbcConnectionPool pool = JdbcConnectionPool.create(
+
"jdbc:h2:zip:./thirdpartydeps/database/jdbc-cookbook.zip!/test",
+ "sa", "");
+ ResultSet resultSet =
pool.getConnection().createStatement().executeQuery(
+ "SELECT int_field1, bool_field2, double_field7 FROM TABLE1");
+ try (ArrowVectorIterator iterator =
JdbcToArrow.sqlToArrowVectorIterator(
+ resultSet, allocator)) {
+ while (iterator.hasNext()) {
+ VectorSchemaRoot root = iterator.next();
+ System.out.print(root.contentToTSVString());
+ }
+ }
+ } catch (SQLException throwables) {
+ throwables.printStackTrace();
+ } catch (IOException e) {
+ e.printStackTrace();
+ }
+
+.. testoutput::
+
+ INT_FIELD1 BOOL_FIELD2 DOUBLE_FIELD7
+ 101 true 5.6478356785345E10
+ 102 true 5.6478356785345E10
+ 103 true 5.6478356785345E10
+
+JdbcToArrow could be created also with a custom configurations needed with the
+support of `JdbcToArrowConfig
<https://arrow.apache.org/docs/java/reference/org/apache/arrow/adapter/jdbc/JdbcToArrowConfig.html>`_
+, it is useful at the moment to work with array columns.
Review Comment:
```suggestion
JdbcToArrow accepts configuration through `JdbcToArrowConfig
<https://arrow.apache.org/docs/java/reference/org/apache/arrow/adapter/jdbc/JdbcToArrowConfig.html>`_.
For example, the type of the elements of array columns can be specified.
```
##########
java/source/io.rst:
##########
@@ -444,6 +444,176 @@ Reading Parquet File
Please check :doc:`Dataset <./dataset>`
+Reading JDBC ResultSets
+***********************
+
+The `Arrow Java JDBC module <https://arrow.apache.org/docs/java/jdbc.html>`_
+help us to convert JDBC ResultSets objects into columnar arrow format objects
+through Arrow VectorSchemaRoots.
Review Comment:
```suggestion
The `Arrow Java JDBC module <https://arrow.apache.org/docs/java/jdbc.html>`_
converts JDBC ResultSets into Arrow VectorSchemaRoots.
```
##########
java/source/demo/pom.xml:
##########
@@ -21,9 +21,17 @@
<properties>
<maven.compiler.source>8</maven.compiler.source>
<maven.compiler.target>8</maven.compiler.target>
- <arrow.version>8.0.0</arrow.version>
+ <arrow.version>9.0.0.dev191</arrow.version><!-- Prepare for 9.0.0
release -->
Review Comment:
either that or we do need to set up stable/nightly cookbooks, CC @amol-?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
