YuweiXiao commented on code in PR #6493:
URL: https://github.com/apache/hudi/pull/6493#discussion_r955729960
##########
hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/sink/StreamWriteOperatorCoordinator.java:
##########
@@ -396,7 +396,11 @@ private void initInstant(String instant) {
reset();
} else {
LOG.info("Recommit instant {}", instant);
- commitInstant(instant);
+ boolean success = commitInstant(instant);
+ if (success) {
+ LOG.info("instant {} ReSync Hive", instant);
+ syncHive();
+ }
Review Comment:
Hi Danny, want to bring up another topic here. If the commit is not success,
e.g., the last batch has no data, we will reuse the instant of the last batch,
and in the meanwhile, we will also start a new instant. This then leads to an
inconsistent ckp_meta.
We encounter this issue this our inner branch. I could fire up a fix for
this if necessary.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
