meethngala commented on code in PR #3663:
URL: https://github.com/apache/gobblin/pull/3663#discussion_r1146630615
##########
gobblin-data-management/src/test/java/org/apache/gobblin/data/management/copy/iceberg/IcebergDatasetTest.java:
##########
@@ -419,14 +437,21 @@ private static void
verifyAncestorPermissions(List<CopyEntityDeserializer.FileOw
}
}
+ private static void verifyPostPublishStep(String json) {
+ String expectedCommitStep =
"org.apache.gobblin.data.management.copy.iceberg.IcebergRegisterStep";
+ String actualCommitStep = new Gson().fromJson(json, JsonObject.class)
+
.getAsJsonObject("object-data").getAsJsonObject("step").getAsJsonPrimitive("object-type").getAsString();
+ Assert.assertEquals(actualCommitStep, expectedCommitStep);
+ }
+
/**
* Sadly, this is needed to avoid losing `FileSystem` mock to replacement
from the `FileSystem.get` `static`
* Without this, so to lose the mock, we'd be unable to set up any source
paths as existing.
*/
protected static class TrickIcebergDataset extends IcebergDataset {
- public TrickIcebergDataset(String db, String table, IcebergTable
icebergTbl, Properties properties,
+ public TrickIcebergDataset(String db, String table, IcebergTable
icebergTbl, IcebergTable targetIcebergTbl, Properties properties,
Review Comment:
makes sense! Renamed it in my latest commit
##########
gobblin-data-management/src/main/java/org/apache/gobblin/data/management/copy/iceberg/IcebergDataset.java:
##########
@@ -153,7 +153,9 @@ Collection<CopyEntity> generateCopyEntities(FileSystem
targetFs, CopyConfigurati
fileEntity.setSourceData(getSourceDataset(this.sourceFs));
fileEntity.setDestinationData(getDestinationDataset(targetFs));
copyEntities.add(fileEntity);
+
}
+ copyEntities.add(addPostPublishStep(this.srcIcebergTable,
this.existingDestinationIcebergTable));
Review Comment:
yup... I agree! Updated the method name in my latest commit
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
