jupiter commented on issue #5606: URL: https://github.com/apache/arrow-rs/issues/5606#issuecomment-2043496776
It was discussed, but I don't think that was the conclusion. The creator's issue was resolved by rewriting a file. In order to operate with precious Parquet files from huge data lakes (e.g. DataFusion probably would want to support files produced by other systems), I'm of the opinion that it should tolerate this like most of the other implementations do (e.g. DuckDB, parquet-tools, and probably many more). I'm all for correctness, but in this particular case you need to consider the intention and purpose. There is no way that an optional key can be intentional. Being compatible with a vast amount of data is the purpose of Parquet integration. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
