jack-moseley commented on code in PR #3633:
URL: https://github.com/apache/gobblin/pull/3633#discussion_r1098074596
##########
gobblin-hive-registration/src/main/java/org/apache/gobblin/hive/writer/HiveMetadataWriter.java:
##########
@@ -232,12 +222,69 @@ public void write(GobblinMetadataChangeEvent gmce,
Map<String, Collection<HiveSp
break;
}
default: {
- log.error("unsupported operation {}",
gmce.getOperationType().toString());
+ log.error("unsupported operation {}", opType);
return;
}
}
}
+ /**
+ * Helper function to gracefully handle errors when creating a hive table.
Existence is not an error.
Review Comment:
Maybe "Non-existence is not an error" is a bit more accurate.
##########
gobblin-iceberg/src/test/java/org/apache/gobblin/iceberg/writer/HiveMetadataWriterTest.java:
##########
@@ -256,6 +258,54 @@ public void testHiveWriteRewriteFileGMCE() throws
IOException {
});
}
+ @Test(dependsOnMethods = {"testHiveWriteRewriteFileGMCE"},
groups={"hiveMetadataWriterTest"})
+ public void testHiveWriteDeleteFileGMCE() throws IOException, TException {
+ gmce.setTopicPartitionOffsetsRange(null);
+ Map<String, String> registrationState = gmce.getRegistrationProperties();
+ registrationState.put("additional.hive.database.names", dedupedDbName);
+ registrationState.put(HiveMetaStoreBasedRegister.SCHEMA_SOURCE_DB, dbName);
+ gmce.setRegistrationProperties(registrationState);
+ gmce.setSchemaSource(SchemaSource.NONE);
+ gmce.setOldFilePrefixes(Lists.newArrayList(dailyDataFile.toString()));
+ gmce.setOperationType(OperationType.drop_files);
+ gobblinMCEWriter.writeEnvelope(new RecordEnvelope<>(gmce,
Review Comment:
Could you add a couple more comments in the tests just to make it easy to
understand what is being tested, here are we testing dropping a partition that
doesn't exist?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
