GeorgeJahad commented on code in PR #3980:
URL: https://github.com/apache/ozone/pull/3980#discussion_r1144093843
##########
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OmSnapshotManager.java:
##########
@@ -279,6 +307,92 @@ private void verifySnapshotInfoForSnapDiff(final
SnapshotInfo fromSnapshot,
}
}
+ /**
+ * Create file of links to add to tarball.
+ * Format of entries are either:
+ * dir1/fileTo fileFrom
+ * for files in active db or:
+ * dir1/fileTo dir2/fileFrom
+ * for files in another directory, (either another snapshot dir or
+ * sst compaction backup directory)
+ * @param truncateLength - Length of initial path to trim in file path.
+ * @param hardLinkFiles - Map of link->file paths.
+ * @return Path to the file of links created.
+ */
+ @SuppressFBWarnings({"NP_NULL_ON_SOME_PATH_FROM_RETURN_VALUE"})
+ static Path createHardLinkList(int truncateLength,
+ Map<Path, Path> hardLinkFiles)
+ throws IOException {
+ Path data = Files.createTempFile("data", "txt");
+ StringBuilder sb = new StringBuilder();
+ for (Map.Entry<Path, Path> entry : hardLinkFiles.entrySet()) {
+ String fixedFile = truncateFileName(truncateLength, entry.getValue());
+ // If this file is from the active db, strip the path.
+ if (fixedFile.startsWith(OM_CHECKPOINT_DIR)) {
+ fixedFile = Paths.get(fixedFile).getFileName().toString();
+ }
+ sb.append(truncateFileName(truncateLength, entry.getKey()))
Review Comment:
I spent some time trying to remove the truncateLength and the path
stripping, but was unable to do so in any way that increased the clarity of the
code. It turns out that each serves an important purpose.
The "truncate length" is required to remove the leader specific
superdirectories from the path. So for example it changes this path:
```
/home/gbj/incoming/build-master/hadoop-ozone/integration-test/target/test-dir/MiniOzoneClusterImpl-967d81d4-9ebf-4d14-b384-7912f7535144/omNode-1/db.snapshots/checkpointState/om.db-fd5616df-4c39-47ad-a5d6-37d40fa378f2/CURRENT
```
to a relative path for the destination
```
db.snapshots/checkpointState/om.db-fd5616df-4c39-47ad-a5d6-37d40fa378f2/CURRENT
```
This is necessary because there is no guarantee that the follower uses the
same path to the db, (and in fact in the ha integration tests, all three nodes
run on one machine so the path specifically is different between the different
nodes. So it must be truncated and replaced on the follower.)
"Path stripping" refers to removing the path from the file to just leave the
file name. This is needed for the files in the active fs. Those files come
from the checkpoint directory on the leader, not from the directory leaders
active fs.
So both the truncateLength and path stripping are required in some form, and
I wasn't able to come up with any simpler way. So I just left them unchanged.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
