pitrou commented on code in PR #242:
URL: https://github.com/apache/parquet-format/pull/242#discussion_r1609951594
##########
src/main/thrift/parquet.thrift:
##########
@@ -835,6 +864,65 @@ struct ColumnMetaData {
16: optional SizeStatistics size_statistics;
}
+struct ColumnChunkMetaDataV3 {
Review Comment:
Ok, let's step back because I think I am not following you.
Here is the file organization as proposed here:
1) `FileMetaDataV3` points to one `FileColumnMetadataV3` per column
2) `FileColumnMetadataV3` points to one `ColumnChunkV3` per row group
I'm not sure what you're proposing exactly when you mean "can skip encoding
columns that do not have values"? This is currently not possible using Parquet,
as at least the corresponding definition levels would be expected (even if they
are all 0).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
