aiguofer opened a new issue, #37021:
URL: https://github.com/apache/arrow/issues/37021
### Describe the usage question you have. Please include as many useful
details as possible.
# Error description
We have a Java FlightServer for Arrow Flight SQL that connects to various
types of datawarehouses through JDBC and converts the results into arrow to
send back using `sqlToArrowVectorIterator`. Everything has been working pretty
well, but we just ran into some issues with `TIMESTAMP_TZ` columns on Snowflake.
We were getting the following exception (redacted):
```
java.lang.RuntimeException: Error occurred while creating iterator.
at
org.apache.arrow.adapter.jdbc.ArrowVectorIterator.create(ArrowVectorIterator.java:87)
at
org.apache.arrow.adapter.jdbc.JdbcToArrow.sqlToArrowVectorIterator(JdbcToArrow.java:101)
...
Caused by: java.lang.NullPointerException: Cannot invoke
"org.apache.arrow.vector.types.pojo.ArrowType.getTypeID()" because "arrowType"
is null
at
org.apache.arrow.adapter.jdbc.JdbcToArrowUtils.getConsumer(JdbcToArrowUtils.java:437)
at
org.apache.arrow.adapter.jdbc.ArrowVectorIterator.initialize(ArrowVectorIterator.java:142)
at
org.apache.arrow.adapter.jdbc.ArrowVectorIterator.createVectorSchemaRoot(ArrowVectorIterator.java:134)
```
After digging around, I noticed we were getting back a JDBC type
`TIMESTAMP_WITH_TIMEZONE` (2014) which wasn't mapped by the default converter
`JdbcToArrowUtils.getArrowTypeFromJdbcType`. I created my own converter that
mapped that type the same way it maps `TIMESTAMP` to get me unblocked (figured
I'd deal with the TZ issue later). Now I don't get the exception, but the
`TIMESTAMP_WITH_TIMEZONE` is excluded from the results. For example here's how
I set up the data in Snowflake:
<img width="997" alt="image"
src="https://github.com/apache/arrow/assets/780934/b80bcf88-87df-4174-8d83-fef982b8d0e0";>
And here's what I get when I query through Dbeaver using the AFS JDBC driver
through our service:
<img width="598" alt="image"
src="https://github.com/apache/arrow/assets/780934/4e4abef7-7c20-4738-b7de-d1a8cafd1fa0";>
# Debugging attempts
My attempts at debugging this have yielded very strange results:
- When I look in the debugger on our service, the `VectorSchemaRoot` looks
correct:
<img width="882" alt="image"
src="https://github.com/apache/arrow/assets/780934/b622e406-42fa-4942-88f6-f8da502c60ed";>
- If I try to select just the NTZ column it works fine:
<img width="525" alt="image"
src="https://github.com/apache/arrow/assets/780934/25b16e83-9837-4460-8f6a-bf73e79c161b";>
- If I try to select just the TZ column it seems to try to make a
`acceptPutPreparedStatementUpdate` call, which we haven't implemented so it
fails.
My guess is that this is a bug on the JDBC driver, but as far as that's
concerned the 2 vectors are of the same type, so I don't see why it would just
drop one column.
Any help would be greatly appreciated!
### Component(s)
Java
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
