GeorgeJahad commented on code in PR #5215:
URL: https://github.com/apache/ozone/pull/5215#discussion_r1311061619
##########
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OMDBCheckpointServlet.java:
##########
@@ -172,20 +179,41 @@ hardLinkFiles, toExcludeFiles,
includeSnapshotData(request),
}
}
- // Format list from follower to match data on leader.
+ /**
+ * Format the list of excluded sst files from follower to match data
+ * on leader.
+ * @param toExcludeList - list of excluded sst files from follower
+ * @param checkpointLocation - location of checkpoint for this tarball
+ * @param sstBackupDir - location info about sstBackupDir
+ * @return A Map of src and dest paths for the entries in the toExcludeList.
+ * Formatted in a manner analogous to the copyFiles data structure
+ * described above. Because this structure only points to sst files,
+ * the implementation ignores the compactionLog dir, (which doesn't
+ * include sst files.)
+ */
@VisibleForTesting
- public static Set<Path> normalizeExcludeList(List<String> toExcludeList,
- Path checkpointLocation) {
- Set<Path> paths = new HashSet<>();
+ public static Map<Path, Path> normalizeExcludeList(
+ List<String> toExcludeList,
+ Path checkpointLocation,
+ DirectoryData sstBackupDir) {
+ Map<Path, Path> paths = new HashMap<>();
for (String s : toExcludeList) {
- if (!s.startsWith(OM_SNAPSHOT_DIR)) {
+ Path metaDirPath = getMetaDirPath(checkpointLocation);
+ Path destPath = Paths.get(metaDirPath.toString(), s);
Review Comment:
> Is it possible that destPath doesn't exist?
Good question. To clarify, the OMDBCheckpointServlet is being run on the
leader to create files for the follower. The srcPath is the path on the
leader, the destPath is the path on the follower. Since the destPath is by
definition the path that will be created on the follower, it will always exist
(because this servlet creates the tarball for the follower.)
It is possible for the srcPath not to exist. That happens when a file on
the follower, (from a previous tarball,) no longer exists on the leader,
(because of compaction.)
The servlet should, (and does,) check for the existence of the srcPath,
before trying to access it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
