ZihanLi58 commented on code in PR #3672:
URL: https://github.com/apache/gobblin/pull/3672#discussion_r1164651755
##########
gobblin-iceberg/src/main/java/org/apache/gobblin/iceberg/writer/IcebergMetadataWriter.java:
##########
@@ -801,12 +813,14 @@ public void flush(String dbName, String tableName) throws
IOException {
Transaction transaction = tableMetadata.transaction.get();
Map<String, String> props = tableMetadata.newProperties.or(
Maps.newHashMap(tableMetadata.lastProperties.or(getIcebergTable(tid).properties())));
- String topic = props.get(TOPIC_NAME_KEY);
+ //Set data offset range
+ setDatasetOffsetRange(tableMetadata, props);
+ String topicName = getTopicName(tid, tableMetadata);
Review Comment:
We are still fallback to the previous topic name calculated from NativeName
here in case datasetOffsetRange does not exist. Also GTE require this
datasetName to set the HDFS dataset path information
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
