phet commented on code in PR #3673:
URL: https://github.com/apache/gobblin/pull/3673#discussion_r1164668823
##########
gobblin-data-management/src/main/java/org/apache/gobblin/data/management/copy/iceberg/IcebergDataset.java:
##########
@@ -316,8 +316,8 @@ protected DatasetDescriptor
getDestinationDataset(FileSystem targetFs) {
return this.destIcebergTable.getDatasetDescriptor(targetFs);
}
- private PostPublishStep createPostPublishStep(IcebergTable srcIcebergTable,
IcebergTable dstIcebergTable) {
- IcebergRegisterStep icebergRegisterStep = new
IcebergRegisterStep(srcIcebergTable, dstIcebergTable);
+ private PostPublishStep createPostPublishStep(String dbName, String
inputTableName, Properties properties) {
Review Comment:
ah yes, I see the challenge arising from the hadoop props. these are likely
to have a predictable prefix too, so we could retain only specific ones... but
that might take trial-an-error, so, for now, I suggest adding a TODO
documenting our preference not to serialize absolutely every prop... and yet
the challenge w/ the hadoop config.
as we return to this, we should also investigate whether `Configuration` may
itself already be serializable
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
