marton-bod commented on code in PR #3131:
URL: https://github.com/apache/hive/pull/3131#discussion_r846065462
##########
iceberg/iceberg-handler/src/main/java/org/apache/iceberg/mr/hive/HiveIcebergOutputFormat.java:
##########
@@ -83,9 +83,20 @@ private static HiveIcebergRecordWriter writer(JobConf jc) {
.operationId(operationId)
.build();
String tableName = jc.get(Catalogs.NAME);
- HiveFileWriterFactory hfwf = new HiveFileWriterFactory(table, fileFormat,
schema,
- null, fileFormat, null, null, null, null);
- return new HiveIcebergRecordWriter(schema, spec, fileFormat,
- hfwf, outputFileFactory, io, targetFileSize, taskAttemptID, tableName);
+ HiveFileWriterFactory writerFactory = new HiveFileWriterFactory(table,
fileFormat, schema, null, fileFormat,
+ null, null, null, getPositionDeleteRowSchema(schema, fileFormat));
+ if (HiveIcebergStorageHandler.isDelete(jc, tableName)) {
+ return new HiveIcebergDeleteWriter(schema, spec, fileFormat,
writerFactory, outputFileFactory, io, targetFileSize,
+ taskAttemptID, tableName);
+ } else {
+ return new HiveIcebergRecordWriter(schema, spec, fileFormat,
writerFactory, outputFileFactory, io, targetFileSize,
+ taskAttemptID, tableName);
+ }
+ }
+
+ private static Schema getPositionDeleteRowSchema(Schema schema, FileFormat
fileFormat) {
+ // TODO: remove this Avro-specific logic once we have Avro writer function
ready
Review Comment:
Actually we no longer need an exception for Avro, we can use the
`DataWriter` class from the iceberg-avro module
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
