[
https://issues.apache.org/jira/browse/GOBBLIN-1775?focusedWorklogId=843958&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-843958
]
ASF GitHub Bot logged work on GOBBLIN-1775:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 07/Feb/23 01:10
Start Date: 07/Feb/23 01:10
Worklog Time Spent: 10m
Work Description: jack-moseley commented on code in PR #3633:
URL: https://github.com/apache/gobblin/pull/3633#discussion_r1098074596
##########
gobblin-hive-registration/src/main/java/org/apache/gobblin/hive/writer/HiveMetadataWriter.java:
##########
@@ -232,12 +222,69 @@ public void write(GobblinMetadataChangeEvent gmce,
Map<String, Collection<HiveSp
break;
}
default: {
- log.error("unsupported operation {}",
gmce.getOperationType().toString());
+ log.error("unsupported operation {}", opType);
return;
}
}
}
+ /**
+ * Helper function to gracefully handle errors when creating a hive table.
Existence is not an error.
Review Comment:
Maybe "Non-existence is not an error" is a bit more accurate.
##########
gobblin-iceberg/src/test/java/org/apache/gobblin/iceberg/writer/HiveMetadataWriterTest.java:
##########
@@ -256,6 +258,54 @@ public void testHiveWriteRewriteFileGMCE() throws
IOException {
});
}
+ @Test(dependsOnMethods = {"testHiveWriteRewriteFileGMCE"},
groups={"hiveMetadataWriterTest"})
+ public void testHiveWriteDeleteFileGMCE() throws IOException, TException {
+ gmce.setTopicPartitionOffsetsRange(null);
+ Map<String, String> registrationState = gmce.getRegistrationProperties();
+ registrationState.put("additional.hive.database.names", dedupedDbName);
+ registrationState.put(HiveMetaStoreBasedRegister.SCHEMA_SOURCE_DB, dbName);
+ gmce.setRegistrationProperties(registrationState);
+ gmce.setSchemaSource(SchemaSource.NONE);
+ gmce.setOldFilePrefixes(Lists.newArrayList(dailyDataFile.toString()));
+ gmce.setOperationType(OperationType.drop_files);
+ gobblinMCEWriter.writeEnvelope(new RecordEnvelope<>(gmce,
Review Comment:
Could you add a couple more comments in the tests just to make it easy to
understand what is being tested, here are we testing dropping a partition that
doesn't exist?
Issue Time Tracking
-------------------
Worklog Id: (was: 843958)
Time Spent: 0.5h (was: 20m)
> Add logic in GMIP to not fail the container when we cannot create table
> -----------------------------------------------------------------------
>
> Key: GOBBLIN-1775
> URL: https://issues.apache.org/jira/browse/GOBBLIN-1775
> Project: Apache Gobblin
> Issue Type: Improvement
> Reporter: Matthew Ho
> Priority: Major
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
