Hi everyone, Thanks again to everyone who joined and contributed. Here is a brief update from the meeting.
We reviewed each field in the existing footer, discussing whether it should be preserved and which module it should belong to. We also established our benchmarking goals: we need microbenchmarks focused on the footer itself, both to measure footer size and parsing performance. We will start with the microbenchmarks and compare three footer implementations: 1. The current footer. 2. The current footer with all proposed optimizations, including removing unused or duplicate fields and adding a jump table to support O(1) access to column metadata. 3. The new modular footer. To facilitate asynchronous collaboration, we would like to create a separate Google Group for coordinating the implementation work. The discussions will be highly specific and may include work-in-progress implementation details, which could generate too many emails for the main mailing list. The Google Group is: https://groups.google.com/g/apache-parquet-footer-working-group The new group is also public and everyone is welcome to join if interested. However, if anyone objects to using a separate group, we can conduct all collaboration on the main mailing list instead. Please let me know if you have any concerns. Best, Jiayi
