sreejasahithi commented on code in PR #10547:
URL: https://github.com/apache/ozone/pull/10547#discussion_r3564653227
##########
hadoop-ozone/cli-debug/src/main/java/org/apache/hadoop/ozone/debug/datanode/container/analyze/AnalyzeSubcommand.java:
##########
@@ -34,65 +50,205 @@
*/
@Command(
name = "analyze",
- description = "Analyze container consistency between on-disk container " +
- "directories on this DataNode and SCM metadata. Must be run
locally on a DataNode.")
+ description = {
+ "Analyze container consistency between on-disk container directories
on this DataNode and SCM metadata.",
+ "Must be run locally on a DataNode.",
+ "",
+ "Reports:",
+ " Duplicate container directories: same containerID found on more
than one volume.",
+ " Orphan containers (requires --scm-db): present on disk but not
present in SCM metadata.",
+ " Containers marked DELETED in SCM but present on disk (requires
--scm-db).",
+ "",
+ "Each reported occurrence includes container directory path(s), size
and an on-disk metadata status:",
+ " MISSING_METADATA: metadata/{containerId}.container does not exist.",
+ " INVALID_METADATA: metadata file exists but cannot be parsed, or the
containerID in the",
+ " file does not match the directory name.",
+ " VALID: metadata file is present, parses correctly, and its
containerID matches the directory name."
+ })
public class AnalyzeSubcommand extends AbstractSubcommand implements
Callable<Void> {
+ @Deprecated
@CommandLine.Option(names = {"--count"},
- defaultValue = "20",
- description = "Number of containers to display")
- private int count;
+ hidden = true,
+ description = "Number of containers to display")
+ private Integer count;
+
+ @CommandLine.Mixin
+ private ListLimitOptions listOptions;
+
+ private static final Logger LOG =
LoggerFactory.getLogger(AnalyzeSubcommand.class);
+
+ @CommandLine.Option(names = {"--scm-db"},
+ description = "Path to an offline scm.db directory, or its parent
metadata directory.")
+ private File scmDb;
@Override
public Void call() throws Exception {
- if (count < 1) {
- throw new IOException("Count must be an integer greater than 0.");
- }
+ validateOptions();
OzoneConfiguration conf = getOzoneConf();
ContainerScanResult scanResult = ContainerDirectoryScanner.scan(conf);
Map<Long, List<ContainerDiskOccurrence>> enrichedDuplicates =
ContainerDirectoryScanner.enrichDuplicates(scanResult.getDuplicates());
- // TODO: SCM metadata lookup from --scm-db when provided.
- // TODO: For each id in scanResult.getSingles().keySet() classified
NOT_IN_SCM or DELETED:
- // enrichOccurrence(id, scanResult.getSingles().get(id)) and report.
- // TODO: For each id in enrichedDuplicates.keySet() classified NOT_IN_SCM
or DELETED:
- // enrichedDuplicates.get(id) is already enriched — just report.
+ if (scmDb != null && checkClusterIdConsistency(conf)) {
+ findOrphanAndDeletedButPresentContainers(conf, scanResult,
enrichedDuplicates);
+ } else if (scmDb == null) {
+ out().println("To identify orphan containers (wrt SCM) and containers
that are marked as DELETED in SCM but"
+ + " exist in the datanode's current directory, provide the SCM
database path using the --scm-db option."
+ );
+ }
Review Comment:
When user passes --scm-db /wrong/path
checkClusterIdConsistency() runs before orphan/deleted analysis and
resolveScmDbDirectory() throws IOException("SCM database directory not found:
...")
That is caught, printed to stderr, and checkClusterIdConsistency() returns
false so findOrphanAndDeletedButPresentContainers() is not called and code
continues to printDuplicates()
This way atleast user will be able to see if duplicates are present even if
the scm db path provided was wrong because to find duplicates we do not need
scm db.
```
bash-5.1$ ozone debug datanode container analyze --scm-db
/path/to/scm/metadata
SCM database directory not found: /path/to/scm/metadata
Number of containers with duplicate container directories on this DataNode: 1
Container 2 (2 occurrences):
path=/data/hdds0/hdds/CID-f6a3b7ac-f3fc-40bd-a181-1a68264cacaa/current/containerDir0/2
status=VALID size=512566 bytes
path=/data/hdds1/hdds/CID-f6a3b7ac-f3fc-40bd-a181-1a68264cacaa/current/containerDir0/2
status=VALID size=512566 bytes
bash-5.1$
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
