geruh commented on code in PR #14824:
URL: https://github.com/apache/iceberg/pull/14824#discussion_r2616629285
##########
core/src/main/java/org/apache/iceberg/rest/ScanTaskIterable.java:
##########
@@ -240,10 +239,14 @@ public void close() {
}
private boolean isDone() {
- return taskQueue.isEmpty()
+ // Reorder the conditions to make sure TaskQueue is empty is checked
last.
+ // It may happen that a worker is about to add a new task to the queue,
but before
+ // that happens, taskQueue.isEmpty() is checked then it completes fast
before the
+ // activeWorker is decremented. This would lead to a false negative.
+ return activeWorkers.get() == 0
&& planTasks.isEmpty()
- && activeWorkers.get() == 0
- && initialFileScanTasks.isEmpty();
+ && initialFileScanTasks.isEmpty()
+ && taskQueue.isEmpty();
Review Comment:
Thanks for the context! I think the poison pill approach in this PR works
well!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
