alamb commented on code in PR #542:
URL: https://github.com/apache/parquet-format/pull/542#discussion_r2615876120
##########
src/main/thrift/parquet.thrift:
##########
@@ -958,6 +958,22 @@ union ColumnCryptoMetaData {
struct ColumnChunk {
/** File where column data is stored. If not set, assumed to be same file as
* metadata. This path is relative to the current file.
+ *
+ * As of December 2025, there are no known released Parquet implementations
+ * that make use of this field when reading columns. Readers should check
+ * that the field is empty before retrieving a column from within the file,
+ * and error if it isn't.
+ *
+ * Writers should not populate this field.
+ *
+ * Any new use of this field must go through the normal Parquet feature
+ * addition process. CONTRIBUTING.md in the parquet-format repository
+ * provides details on the process.
+ *
+ * One known use-case for this field is to batch parquet footers together
Review Comment:
I found this somewhat confusing as above it says there are no
implementations that make use of this implementation but then this paragraph
explains a usecase.
Maybe we could say something like
> As of December 2025, there are no known open source Parquet
implementations that support
> reading external data via this field. Some query engines (which ones??)
make use of this field
> to batch can batch parquet footers together into a single file that serve
as an index, but this is
> not common.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
