jorisvandenbossche commented on issue #36882: URL: https://github.com/apache/arrow/issues/36882#issuecomment-1753143306
I reopened this issue, and temporarily added "Blocker" to it for the upcoming release, until we clarify for ourselves if we actually want this change or not. From https://github.com/apache/arrow/pull/38070#discussion_r1348626768, @mapleFU said: > 🤔As we mentioned in https://issues.apache.org/jira/browse/PARQUET-2222 . I think export RLE is not a so good idea... > Rle is default enabled on Page V2, however, write page v2 is not recommended. > > Page V2 format is great, however, most parquet implementions didn't get agreement on it. As a result, page v2 is unstable now. > > RLE Boolean is default enabled on page v2, I think it's ok there, but I don't think is a good idea to default enable it. > Just as this https://blog.getdaft.io/p/working-with-the-apache-parquet-file blog saying. Parquet V2 is a ambigious naming. Although we (arrow and arrow-rs ) is using format 2.x and some properties on it. Most of implementions can still decode page v1. > > So I think if user know what he/she is doing, RLE is ok to export to user, however I think here we can just hide it until PARQUET-2222 has a conclusion about this -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
