aiguofer commented on issue #37021: URL: https://github.com/apache/arrow/issues/37021#issuecomment-1666026512
Nvm, I figured it out. The issue is we were still using `jdbcToArrowSchema(ResultSetMetaData rsmd, Calendar calendar)`, which uses the default converter in the `createPreparedStatement` request. It seems that when `jdbcToArrowSchema` can't map a specific type, it simply excludes that `Field` in the `Schema` that it generates. Then, when you do `getStreamPreparedStatement` the conversion fails. By passing the same custom `JdbcToArrowConfig` with our custom converter to both `jdbcToArrowSchema` and to `sqlToArrowVectorIterator` we're now getting the right results!! Now I just have to make sure we set the TZ correctly, which might require querying snowflake for the current session timezone... this might be tough, especially if a user changes the session timezone while talking to our service -.- Would it make sense to make a bug report for the behavior discrepancy between `jdbcToArrowSchema` and `sqlToArrowVectorIterator` with regards to unmapped types? or to add support for `TIMESTAMP_WITH_TIMEZONE` in the default converter? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
