pitrou commented on code in PR #1291:
URL: https://github.com/apache/parquet-mr/pull/1291#discussion_r1515793712
##########
parquet-column/src/main/java/org/apache/parquet/column/values/factory/DefaultV2ValuesWriterFactory.java:
##########
@@ -102,19 +125,39 @@ private ValuesWriter
getBinaryValuesWriter(ColumnDescriptor path) {
}
private ValuesWriter getInt32ValuesWriter(ColumnDescriptor path) {
- ValuesWriter fallbackWriter = new DeltaBinaryPackingValuesWriterForInteger(
- parquetProperties.getInitialSlabSize(),
- parquetProperties.getPageSizeThreshold(),
- parquetProperties.getAllocator());
+ final ValuesWriter fallbackWriter;
+ // Ideally we should only enable BYTE_STREAM_SPLIT if compression is
enabled for this column.
+ // However, this information is not available here.
Review Comment:
@wgtmac I have not found where the compression settings are stored in
parquet-mr. I'm probably missing something!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
