adoroszlai commented on code in PR #6608: URL: https://github.com/apache/ozone/pull/6608#discussion_r1584205604
########## hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/debug/FSORepairCLI.java: ########## @@ -0,0 +1,78 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.hadoop.ozone.debug; Review Comment: Can it be moved to `org.apache.hadoop.ozone.repair` package? ########## hadoop-ozone/tools/pom.xml: ########## @@ -121,6 +121,53 @@ https://maven.apache.org/xsd/maven-4.0.0.xsd";> <maxHeap>2048</maxHeap> </configuration> </plugin> + <plugin> + <artifactId>maven-enforcer-plugin</artifactId> + <executions> + <execution> + <id>depcheck</id> + <phase></phase> + </execution> + <execution> + <id>banned-rocksdb-imports</id> + <phase>process-sources</phase> + <goals> + <goal>enforce</goal> + </goals> + <configuration> + <rules> + <RestrictImports> + <includeTestCode>false</includeTestCode> + <reason>Use managed RocksObjects under org.apache.hadoop.hdds.utils.db.managed instead.</reason> + <!-- By default, ban all the classes in org.rocksdb --> + <bannedImport>org.rocksdb.**</bannedImport> + <allowedImports> + <allowedImport>org.rocksdb.AbstractEventListener</allowedImport> + <allowedImport>org.rocksdb.Checkpoint</allowedImport> + <allowedImport>org.rocksdb.ColumnFamilyDescriptor</allowedImport> + <allowedImport>org.rocksdb.ColumnFamilyHandle</allowedImport> + <allowedImport>org.rocksdb.ColumnFamilyOptions</allowedImport> + <allowedImport>org.rocksdb.CompactionJobInfo</allowedImport> + <allowedImport>org.rocksdb.CompressionType</allowedImport> + <allowedImport>org.rocksdb.DBOptions</allowedImport> + <allowedImport>org.rocksdb.FlushOptions</allowedImport> + <allowedImport>org.rocksdb.Holder</allowedImport> + <allowedImport>org.rocksdb.LiveFileMetaData</allowedImport> + <allowedImport>org.rocksdb.Options</allowedImport> + <allowedImport>org.rocksdb.RocksDB</allowedImport> + <allowedImport>org.rocksdb.RocksDBException</allowedImport> + <allowedImport>org.rocksdb.SstFileReader</allowedImport> + <allowedImport>org.rocksdb.TableProperties</allowedImport> + <allowedImport>org.rocksdb.ReadOptions</allowedImport> + <allowedImport>org.rocksdb.SstFileReaderIterator</allowedImport> + </allowedImports> + <exclusion>org.apache.hadoop.hdds.utils.db.managed.*</exclusion> + </RestrictImports> + </rules> + </configuration> + </execution> + </executions> + </plugin> Review Comment: Import restriction is inherited from the root POM. Please don't copy it, especially not from `rocksdb-checkpoint-differ`. If you need to allow additional RocksDB classes: - if they are safe, add them to the root POM - otherwise add managed wrappers and use those ########## hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/debug/FSORepairCLI.java: ########## @@ -0,0 +1,78 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.hadoop.ozone.debug; + +import org.apache.hadoop.hdds.cli.SubcommandWithParent; +import org.kohsuke.MetaInfServices; +import picocli.CommandLine; + +import java.util.concurrent.Callable; + +/** + * Parser for scm.db file. + */ [email protected]( + name = "fso-repair", + description = "Identify a disconnected FSO tree, and optionally mark " + Review Comment: I think it should be a subcommand of `ozone repair`, and `fso-tree` would be a better name. ########## hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/debug/FSORepairCLI.java: ########## @@ -0,0 +1,78 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.hadoop.ozone.debug; + +import org.apache.hadoop.hdds.cli.SubcommandWithParent; +import org.kohsuke.MetaInfServices; +import picocli.CommandLine; + +import java.util.concurrent.Callable; + +/** + * Parser for scm.db file. + */ [email protected]( + name = "fso-repair", + description = "Identify a disconnected FSO tree, and optionally mark " + + "unreachable entries for deletion. OM should be " + + "stopped while this tool is run. Information will be logged at " + + "INFO and DEBUG levels." +) +@MetaInfServices(SubcommandWithParent.class) +public class FSORepairCLI implements Callable<Void>, SubcommandWithParent { + + @CommandLine.Option(names = {"--db"}, + required = true, + description = "Path to OM RocksDB") + private String dbPath; + + @CommandLine.Option(names = {"--read-mode-only", "-r"}, + required = true, + description = + "Mode to run the tool in. Read-mode will just log information about unreachable files or directories;" + + "otherwise the tool will move those files and directories to the deleted tables.", + defaultValue = "true") + private boolean readModeOnly; + + @CommandLine.ParentCommand + private OzoneDebug parent; + + @Override + public Void call() throws Exception { + + try { + // TODO case insensitive enum options. + FSORepairTool repairTool = new FSORepairTool(dbPath, readModeOnly); + repairTool.run(); + } catch (Exception ex) { + throw new IllegalArgumentException("FSO repair failed: " + ex.getMessage()); + } + + System.out.printf("FSO %s finished. See client logs for results.%n", Review Comment: Logging for CLI is off by default. The tool should print all important pieces of information to std.out, and errors/warnings to std.err. This allows users to see the results while avoiding chatty logs from some internal classes. Print more detailed messages if `--verbose` is used. ########## hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/debug/FSORepairCLI.java: ########## @@ -0,0 +1,78 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.hadoop.ozone.debug; + +import org.apache.hadoop.hdds.cli.SubcommandWithParent; +import org.kohsuke.MetaInfServices; +import picocli.CommandLine; + +import java.util.concurrent.Callable; + +/** + * Parser for scm.db file. + */ [email protected]( + name = "fso-repair", + description = "Identify a disconnected FSO tree, and optionally mark " + + "unreachable entries for deletion. OM should be " + + "stopped while this tool is run. Information will be logged at " + + "INFO and DEBUG levels." +) +@MetaInfServices(SubcommandWithParent.class) +public class FSORepairCLI implements Callable<Void>, SubcommandWithParent { + + @CommandLine.Option(names = {"--db"}, + required = true, + description = "Path to OM RocksDB") + private String dbPath; + + @CommandLine.Option(names = {"--read-mode-only", "-r"}, + required = true, + description = + "Mode to run the tool in. Read-mode will just log information about unreachable files or directories;" + + "otherwise the tool will move those files and directories to the deleted tables.", + defaultValue = "true") Review Comment: `--dry-run` is a common name for such options. However, if the default value is `true`, users always have to use the option as negated (`--no-...`), which is cumbersome. I suggest changing this to `--fix` (or similar), setting the default value to `false`, and updating the description accordingly. This way functionality is the same (read-only by default), but the option is more user-friendly. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
