emkornfield commented on code in PR #542:
URL: https://github.com/apache/parquet-format/pull/542#discussion_r2636111539
##########
src/main/thrift/parquet.thrift:
##########
@@ -958,6 +958,24 @@ union ColumnCryptoMetaData {
struct ColumnChunk {
/** File where column data is stored. If not set, assumed to be same file as
* metadata. This path is relative to the current file.
+ *
+ * As of December 2025, the only known use-case for this field is writing
summary
+ * parquet files (i.e. "_metadata" files). These files consolidate footers
from
+ * multiple parquet files to allow for efficient reading of footers to
avoid file
+ * listing costs and prune out files that do not need to be read based on
statistics.
+ * This is legacy feature as modern table formats (e.g. Iceberg, Hudi and
Delta Lake)
+ * are more scalable and serve effectively the same purpose.
+ *
+ * There is no other known use-case for this field. Specifically, there are
no known
Review Comment:
Yep, usage makes sense.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
