emkornfield commented on code in PR #542:
URL: https://github.com/apache/parquet-format/pull/542#discussion_r2637249685
##########
src/main/thrift/parquet.thrift:
##########
@@ -958,6 +958,22 @@ union ColumnCryptoMetaData {
struct ColumnChunk {
/** File where column data is stored. If not set, assumed to be same file as
* metadata. This path is relative to the current file.
+ *
+ * As of December 2025, the only known use-case for this field is writing
summary
+ * parquet files (i.e. "_metadata" files). These files consolidate footers
from
+ * multiple parquet files to allow for efficient reading of footers to
avoid file
+ * listing costs and prune out files that do not need to be read based on
statistics.
+ * This is legacy feature as modern table formats (e.g. Iceberg, Hudi and
Delta Lake)
+ * are more scalable and serve effectively the same purpose.
+ *
+ * There is no other known usage of this field. Specifically, there are no
known
+ * readers that will read externally stored column data if this field is
populated
+ * within a standard parquet file. Making use of the field for this purpose
is currently
+ * not considered part of the Parquet specification.
+ *
+ * Any new use of this field must go through the normal Parquet feature
+ * addition process.
Review Comment:
Yes, I think this is now implied by the first paragraph, I'll remove.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
