Liulietong commented on a change in pull request #2584:
URL: https://github.com/apache/hudi/pull/2584#discussion_r583434679
##########
File path:
hudi-common/src/main/java/org/apache/hudi/common/table/log/HoodieLogFormatReader.java
##########
@@ -104,7 +104,7 @@ public boolean hasNext() {
throw new HoodieIOException("unable to initialize read with log file
", io);
}
LOG.info("Moving to the next reader for logfile " +
currentReader.getLogFile());
- return this.currentReader.hasNext();
+ return this.currentReader.hasNext() || hasNext();
Review comment:
1. When 'spark.speculation' is enabled, two tasks trying to append one
log file, the second one will create a new logFile because it can't get lease
of file. The second task will leave one zero-size log file when the first task
succeed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
