alamb commented on PR #4884: URL: https://github.com/apache/arrow-rs/pull/4884#issuecomment-1743443830
> There's an email thread and discussions on this: https://www.mail-archive.com/[email protected]/msg14529.html ``` Despite several attempts by the parquet-cpp developers, we were not able to reach the point where LZ4-compressed Parquet files are bidirectionally compatible between parquet-cpp and parquet-mr. Other implementations are having, or have had, similar issues. My conclusion is that the Parquet spec doesn't allow independent reimplementation of the LZ4 compression format required by parquet-mr. Therefore, LZ4 compression should be removed from the spec (possibly replaced with another enum value for a properly-specified, interoperable, LZ4-backed compression scheme). ``` That is about as good as a rationale for removing LZ4 as I have heard -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
