etseidl commented on PR #1177: URL: https://github.com/apache/parquet-mr/pull/1177#issuecomment-1771735094
Thanks @wgtmac, this looks great! I'm not sure if this is in scope for this PR, but it would be nice if the CLI was aware of the changes. Specifically, it would be great if the `column-index` command could write out the unencoded sizes and histograms. The former is pretty straightforward, but I'm not entirely sure what the right approach is for the histograms. Also, `rewrite` currently ignores the new statistics. For a straight copy, all that's needed is to add `chunk.getSizeStatistics()` to the arg list of `ColumnChunkMetadata.get()` [here](https://github.com/wgtmac/parquet-mr/blob/45e8ee427e8766489e4a84246f452178ceefb75a/parquet-hadoop/src/main/java/org/apache/parquet/hadoop/ParquetFileWriter.java#L1358). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@parquet.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
