wypoon commented on code in PR #6799:
URL: https://github.com/apache/iceberg/pull/6799#discussion_r1332295801
##########
core/src/main/java/org/apache/iceberg/ManifestWriter.java:
##########
@@ -229,18 +252,34 @@ protected ManifestEntry<DataFile>
prepare(ManifestEntry<DataFile> entry) {
@Override
protected FileAppender<ManifestEntry<DataFile>> newAppender(
PartitionSpec spec, OutputFile file) {
+ return newAppender(spec, file, null, null);
+ }
+
+ @Override
+ protected FileAppender<ManifestEntry<DataFile>> newAppender(
+ PartitionSpec spec, OutputFile file, String compressionCodec, Integer
compressionLevel) {
Schema manifestSchema = V2Metadata.entrySchema(spec.partitionType());
try {
- return Avro.write(file)
- .schema(manifestSchema)
- .named("manifest_entry")
- .meta("schema", SchemaParser.toJson(spec.schema()))
- .meta("partition-spec", PartitionSpecParser.toJsonFields(spec))
- .meta("partition-spec-id", String.valueOf(spec.specId()))
- .meta("format-version", "2")
- .meta("content", "data")
- .overwrite()
- .build();
+ Avro.WriteBuilder builder =
+ Avro.write(file)
+ .schema(manifestSchema)
+ .named("manifest_entry")
+ .meta("schema", SchemaParser.toJson(spec.schema()))
+ .meta("partition-spec", PartitionSpecParser.toJsonFields(spec))
+ .meta("partition-spec-id", String.valueOf(spec.specId()))
+ .meta("format-version", "2")
+ .meta("content", "data")
+ .overwrite();
+
+ if (compressionCodec != null) {
+ builder.set(TableProperties.AVRO_COMPRESSION, compressionCodec);
+ }
+
+ if (compressionLevel != null) {
+ builder.set(TableProperties.AVRO_COMPRESSION_LEVEL,
compressionLevel.toString());
Review Comment:
On the other hand, I agree that making compressionLevel a String is more
efficient due to how the implementation works. One can make an argument for
either case, it just depends on what you prioritize.
I don't have a strong opinion on this. For now, I'll keep this as it is.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
