mbutrovich commented on issue #7208: URL: https://github.com/apache/arrow-rs/issues/7208#issuecomment-2691444722
I've actually been debugging a similar issue for DataFusion Comet, and will open a related issue shortly. The issue may stem from the fact that Spark still defaults to writing INT96 for timestamps. In my issue, we read back a Parquet file written with large timestamp values from a Parquet file, and arrow-rs coerces them into a Timestamp(TimeUnit::Nanoseconds, None) by default which cannot represent as large of a date range as an INT96. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
