devmadhuu commented on code in PR #12000:
URL: https://github.com/apache/ozone/pull/12000#discussion_r4560487204
##########
hadoop-ozone/recon/src/test/java/org/apache/hadoop/ozone/recon/tasks/TestNSSummaryTask.java:
##########
@@ -403,6 +403,25 @@ public void testIndependentSubTaskSeekPositions() throws
IOException {
.getNSSummary(BUCKET_THREE_OBJECT_ID).getSizeOfFiles());
}
+ @Test
+ public void testRetryWithReturnedSeekPositionsDoesNotReapply() throws
IOException {
Review Comment:
Can you also add a test, that commits an initial portion of batch, injects a
failure before the final commit, and retries using the returned position.
##########
hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/tasks/NSSummaryTaskWithOBS.java:
##########
Review Comment:
This can throw `RuntimeException`, so better to preserve `seekPos`. This
flush method only catches `IOException`. So for example:
1. Process three file-creation events, A, B, and C.
2. Successfully save A and B during an intermediate flush.
3. The final flush for C throws a `RuntimeException` before committing.
4. The caller falls back to position 0.
5. Retry succeeds, applying A and B again, then C.
6. Recon reports five files instead of three. Because retry succeeds, this
path does not request a rebuild.
And this case applies to all tasks not just OBS, check Legacy and FSO
handlers also, they also have flush outside of try catch which will not
preserve seekPos in case of `RuntimeException`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]