Alowator commented on code in PR #9125:
URL: https://github.com/apache/hudi/pull/9125#discussion_r1255392984
##########
hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/util/StreamerUtil.java:
##########
@@ -183,6 +183,7 @@ public static TypedProperties
flinkConf2TypedProperties(Configuration conf) {
properties.put(option.key(), option.defaultValue());
}
}
+ properties.put(HoodieTableConfig.TYPE.key(),
conf.getString(FlinkOptions.TABLE_TYPE));
Review Comment:
Its foreign here, to fox broken testWriteMultiWriterPartialOverlapping you
just need to remove HoodieTableConfig.TYPE property in test
##########
hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/sink/meta/CkpMetadata.java:
##########
@@ -193,6 +193,18 @@ public String lastPendingInstant() {
return null;
}
+ @VisibleForTesting
+ public String lastCompleteInstant() {
Review Comment:
@Nullable
##########
hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/client/SparkRDDTableServiceClient.java:
##########
@@ -320,9 +320,11 @@ private void updateTableMetadata(HoodieTable table,
HoodieCommitMetadata commitM
@Override
protected void preCommit(HoodieCommitMetadata metadata) {
- // Create a Hoodie table after startTxn which encapsulated the commits and
files visible.
- // Important to create this after the lock to ensure the latest commits
show up in the timeline without need for reload
- HoodieTable table = createTable(config, hadoopConf);
- resolveWriteConflict(table, metadata,
this.pendingInflightAndRequestedInstants);
+ if
(config.getWriteConcurrencyMode().supportsOptimisticConcurrencyControl()) {
+ // Create a Hoodie table after startTxn which encapsulated the commits
and files visible.
+ // Important to create this after the lock to ensure the latest commits
show up in the timeline without need for reload
+ HoodieTable table = createTable(config, hadoopConf);
+ resolveWriteConflict(table, metadata,
this.pendingInflightAndRequestedInstants);
+ }
Review Comment:
Maybe extract this to Base class to avoid code duplication, and override in
Java class (not Flink and Spark) ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
