tustvold commented on issue #6778: URL: https://github.com/apache/arrow-rs/issues/6778#issuecomment-2495140615
> I think parquet-rs should then not write offset indexes if page statistics are not enabled. There are valid use-cases where the offset index is beneficial, but the page statistics might not be desired. For example, if using some external index or statistics. The offset index is critical to being able to efficiently perform pushdown. I think we can add an option to disable offset index generation, but we should make this an explicit option and stick a big warning that it may severely degrade read performance -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
