pitrou commented on issue #41667: URL: https://github.com/apache/arrow/issues/41667#issuecomment-2515006755
So, the problem is that the Arrow data is invalid (a non-nullable field has nulls), and the Parquet writer doesn't notice the inconsistency, ending up writing invalid data. This was already reported in #31329. I'm not sure we want to do anything in the Parquet writer to avoid this. However, it would be nice if validation of such incorrect data actually failed: see #31387 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
