aiguofer opened a new issue, #39355:
URL: https://github.com/apache/arrow/issues/39355
### Describe the enhancement requested
Currently, when an error occurs while consuming a JDBC ResultSet, it's
impossible to debug what might have happened. For example, here's a redacted
stack trace:
```
java.lang.RuntimeException: Error occurred while getting next schema root.
at
org.apache.arrow.adapter.jdbc.ArrowVectorIterator.next(ArrowVectorIterator.java:181)
...
Caused by: java.lang.RuntimeException: Error occurred while consuming data.
at
org.apache.arrow.adapter.jdbc.ArrowVectorIterator.consumeData(ArrowVectorIterator.java:117)
at
org.apache.arrow.adapter.jdbc.ArrowVectorIterator.load(ArrowVectorIterator.java:159)
at
org.apache.arrow.adapter.jdbc.ArrowVectorIterator.next(ArrowVectorIterator.java:177)
... 10 common frames omitted
Caused by: java.lang.UnsupportedOperationException: Decimal size greater
than 16 bytes: 17
at
org.apache.arrow.vector.util.DecimalUtility.writeByteArrayToArrowBufHelper(DecimalUtility.java:175)
at
org.apache.arrow.vector.util.DecimalUtility.writeBigDecimalToArrowBuf(DecimalUtility.java:136)
at org.apache.arrow.vector.DecimalVector.set(DecimalVector.java:365)
at
org.apache.arrow.adapter.jdbc.consumer.DecimalConsumer.set(DecimalConsumer.java:79)
at
org.apache.arrow.adapter.jdbc.consumer.DecimalConsumer$NullableDecimalConsumer.consume(DecimalConsumer.java:101)
at
org.apache.arrow.adapter.jdbc.consumer.CompositeJdbcConsumer.consume(CompositeJdbcConsumer.java:46)
at
org.apache.arrow.adapter.jdbc.ArrowVectorIterator.consumeData(ArrowVectorIterator.java:106)
... 12 common frames omitted
```
These types of errors are likely due to an incorrect mapping from `JdbcType`
to `ArrowType`, but at this point there's no insight into that mapping. My
proposal is to introduce a `JdbcConsumerException` that stores a
`JdbcFieldInfo` for the ResultSet column and `ArrowType` for the vector. The
easiest way to do this would be in a `try/catch` inside the
`CompositeJdbcConsumer` and then bubble it up in `ArrowVectorIterator`. Users
would then be able to extract that information from the exception to make more
informative logs.
It would be nice to also store the actual value that caused the exception,
but that could potentially expose PII so maybe not the best idea.
I'll make a PR and can discuss further.
### Component(s)
Java
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
