chihsuan commented on code in PR #11214:
URL: https://github.com/apache/ozone/pull/11214#discussion_r3991465081
##########
hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/RDBSnapshotProvider.java:
##########
@@ -129,6 +134,22 @@ public DBCheckpoint downloadDBSnapshotFromLeader(String
leaderNodeID)
LOG.info("DB snapshot transfer is complete.");
return getCheckpointFromUntarredDb(unTarredDb);
}
+
+ // The next request is built from the files already in the candidate
dir, so a part which brings
+ // no new file leaves the leader with the same request to answer and the
transfer cannot progress.
+ int numFilesAfterPart = HAUtils.getExistingFiles(candidateDir).size();
+ if (numFilesAfterPart > numFiles) {
+ numNoProgressParts = 0;
+ } else if (++numNoProgressParts >= MAX_NO_PROGRESS_PARTS) {
+ throw new IOException(String.format("DB snapshot transfer from leader
%s aborted after %d parts: the last"
+ + " %d parts brought no new file into %s and the transfer is still
incomplete. A single file on the"
+ + " leader is likely larger than its per-request SST size limit"
+ + " (OM: ozone.om.ratis.snapshot.max.total.sst.size).",
+ leaderNodeID, numParts, numNoProgressParts, candidateDir));
Review Comment:
nit: Missing exclusions could also cause the same issue. Would it make sense
to say an oversized SST is one possible cause rather than the likely cause?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
