adoroszlai commented on code in PR #7597: URL: https://github.com/apache/ozone/pull/7597#discussion_r1894113820
########## hadoop-hdds/docs/content/design/tools.md: ########## @@ -0,0 +1,136 @@ +--- +title: Improved Layout of Ozone Tools +summary: Proposal for an improved layout of Ozone tools CLI +date: 2024-12-18 +author: Ethan Rose +--- +<!-- + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. See accompanying LICENSE file. +--> + +# Improved Layout of Ozone Tools + +The `ozone-tools` module currently contains utilites that can be used to troubleshoot and fix low level issues in an Ozone cluster through the `ozone debug` and `ozone repair` subcommands. These tools have traditionally been updated in an ad-hoc fashion and not well documented. As we continue to add more tools and their complexity increases, we should standardize the layout and function of these tools to make sure they are safe and easy to use. + +## Current Layout + +This is the current layout of the first two levels of the `ozone-tools` command line: +- ozone debug + - print-log-dag + - find-missing-padding + - read-replicas + - recover + - prefix + - version + - chunkinfo + - container + - ldb + - ratislogparser +- ozone repair + - cert-recover + - ldb + - quota + +## New Layout + +Since debug and repair tools are not meant to be scripted, we should be able to change the CLI layout without impact to compatibility. the following is a proposal to logically regroup all existing commands while leaving room for new additions. + +### Debug + +Invoked as `ozone debug`, these commands make no modifications to Ozone and can be run while the Ozone cluster is live. They can be broadly grouped into three categories: +- Commands that run locally on all components. + - These include generic things like `ldb` and ratis log parsing. +- Commands that run locally on a specific component. + - This include things like repairing quota or snapshot chain on the OM. +- Commands that run over the network. + - This include things like reading all replicas and checking their integrity. + +Commands from different categories should not exist under the same `ozone debug` subcommand. For example, `ozone debug ldb` should not have a mix of subcommands that work on all DB instances and some that only work on OM. All subcommands should clearly document where they are expected to run in their help message. We can use these guidelines to place subcommands based on their type: +- Commands that run locally or commands that run over the network may be top level subcommands of `ozone debug`. +- Commands that run locally on a specific component should be under a dedicated subcommand, like `ozone debug om`. + +We can then apply these rules to group the existing commands: + +- `ozone debug ldb` + This command which provides RocksDB parsing for all components will remain in its original location. +- `ozone debug version` + Implemented in [HDDS-11740](https://issues.apache.org/jira/browse/HDDS-11740), this command prints the internal component versions of the Ozone instance installed on the current host. It will remain in its original location +- `ozone debug om` + A subcommand for all debug operations that run locally on an OM instance. + - `ozone debug om compaction-dag` + This will be the new location of `ozone debug print-log-dag`. Since it outputs a file locally to the leader OM, it should be modified to run locally as well. The current version goes over the network but does not return the result over the network. The name has been updated to better reflect its output. The command only works on the OM since we do not track RocksDB compactions for other components. + - `ozone debug om prefix` + The new location of `ozone debug prefix`. +- `ozone debug datanode` + A subcommand for all debug operations that run locally on a datanode instance. + - `ozone debug datanode container` + The new location of `ozone debug container` and all of its subcommands. These commands are used to inspect container replicas locally on datanodes. Currently `container` is the only debug subcommand on the datanodes. +- `ozone debug replicas` + Currently all online debug tools fit under this category, where they get a listing from the OM and perform checks on the replica information retrieved. The help message for this subcommand should explicitly state that it runs over the network against a running cluster. + - `ozone debug replicas chunk-info` + This is the new location of `ozone debug chunkinfo` + - `ozone debug replicas verify [--checksums] [--padding] [--blocks]` + This is the combination of `ozone debug find-missing-padding`, `ozone debug read-replicas`, and the [proposed block metadata check tool](https://github.com/apache/ozone/pull/7548). The command takes paths in the namespace and performs checks on the objects based on the flags given: + - No flags: Error, at least one check must be specified. + - `--blocks` checks block existence of all replicas as proposed in [HDDS-11891](https://issues.apache.org/jira/browse/HDDS-11891). + - `--padding` Checks padding for EC keys only. No-op for Ratis keys. + - `--checksums` Pulls data and verifies the checksums on the client side for all replicas. This is a thorough check but it will run very slowly. +- `ozone debug ratis` + This subcomand is where we can put Ratis related commands that are specific to Ozone. For bringing the Ratis shell into Ozone, see the [`ozone ratis`](#ratis) subcommand. + - `ozone debug ratis parse <--role={om,scm,datanode}>` + Currently our only Ozone-specific subcommand, this can be used to parse any Ratis on-disk state and apply the Ozone specific schema to it if needed. The command should be able to figure out whether it is looking at a log segment, raft-meta, etc on it's own, but the user will need to specify what type of Ozone role the Ratis information came from so it can apply the proper schema. + +### Repair + +Invoked as `ozone repair`, these commands make modifications to an Ozone instance. For now, there is no plan to support repair operations online or over the network, but this could be implemented in the future. All commands will support a `--dry-run` flag to show what they will do without making any modifications. Subcommand grouping follows similar guidelines to `ozone debug`. + +- `ozone repair om` + These commands make repairs to an individual OM instance. + - `ozone repair om quota` + The new location of `ozone repair ldb quota` Review Comment: `ozone repair ldb quota` does not exist currently, but I found this: ``` Usage: ozone repair quota [COMMAND] Operational tool to repair quota in OM DB. Commands: status CLI to get the status of last trigger quota repair if available. start CLI to trigger quota repair. ``` which is an online command (requires running OM). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
