adoroszlai commented on a change in pull request #2198: URL: https://github.com/apache/ozone/pull/2198#discussion_r631985263
########## File path: hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/freon/OzoneClientRemover.java ########## @@ -0,0 +1,127 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with this + * work for additional information regarding copyright ownership. The ASF + * licenses this file to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * <p> + * http://www.apache.org/licenses/LICENSE-2.0 + * <p> + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations under + * the License. + */ +package org.apache.hadoop.ozone.freon; + +import com.codahale.metrics.Timer; +import org.apache.hadoop.hdds.cli.HddsVersionProvider; +import org.apache.hadoop.hdds.conf.OzoneConfiguration; +import org.apache.hadoop.ozone.client.OzoneBucket; +import org.apache.hadoop.ozone.client.OzoneClient; +import org.apache.hadoop.ozone.client.OzoneVolume; +import picocli.CommandLine.Command; +import picocli.CommandLine.Option; + +import java.util.concurrent.Callable; + +/** + * Data remover tool test om performance. + */ +@Command(name = "ocr", + aliases = "ozone-client-remover", + description = "Remove keys with the help of the ozone clients.", + versionProvider = HddsVersionProvider.class, + mixinStandardHelpOptions = true, + showDefaultValues = true) +public class OzoneClientRemover extends BaseFreonGenerator + implements Callable<Void> { + + @Option(names = {"-v", "--volume"}, + description = "Name of the bucket which contains the test data. Will be" + + " created if missing.", + defaultValue = "vol1") + private String volumeName; + + @Option(names = {"-b", "--bucket"}, + description = "Name of the bucket which contains the test data. Will be" + + " ignored when \"--remove-bucket\" is set.", + defaultValue = "bucket1") + private String bucketName; + + @Option( + names = "--om-service-id", + description = "OM Service ID" + ) + private String omServiceID = null; + + @Option(names = {"--remove-bucket"}, + description = "If turned on, Remover will be used to remove buckets. " + + "Can only set one option between \"--remove-key\" " + + "and \"--remove-bucket\"") + private boolean isRemoveBucket; + + @Option(names = {"--remove-key"}, + description = "If turned on, Remover will be used to remove keys. " + + "Can only set one option between \"--remove-key\" " + + "and \"--remove-bucket\"") + private boolean isRemoveKey; Review comment: By default both of these flags are `false`. If invoked without either, the program will spin forever, not removing anything: ``` $ docker-compose exec -T scm ozone freon ocr 2021-05-13 17:38:51,277 [main] INFO impl.MetricsConfig: Loaded properties from hadoop-metrics2.properties 2021-05-13 17:38:51,371 [main] INFO impl.MetricsSystemImpl: Scheduled Metric snapshot period at 10 second(s). 2021-05-13 17:38:51,372 [main] INFO impl.MetricsSystemImpl: ozone-freon metrics system started 2021-05-13 17:38:51,475 [main] INFO freon.BaseFreonGenerator: Executing test with prefix e90lalfdcs 2021-05-13 17:38:51,484 [Thread-3] INFO freon.ProgressBar: Progress: 0.00 % (0 out of 1000) 2021-05-13 17:38:52,488 [Thread-3] INFO freon.ProgressBar: Progress: 0.00 % (0 out of 1000) 2021-05-13 17:38:53,490 [Thread-3] INFO freon.ProgressBar: Progress: 0.00 % (0 out of 1000) 2021-05-13 17:38:54,492 [Thread-3] INFO freon.ProgressBar: Progress: 0.00 % (0 out of 1000) 2021-05-13 17:38:55,493 [Thread-3] INFO freon.ProgressBar: Progress: 0.00 % (0 out of 1000) ^C ``` `checkOption()` should also verify that one of these options is selected. I think this, and the fact that `--bucket` is `ignored when "--remove-bucket" is set`, indicates that key remover and bucket remover should be separate, just like `OzoneClientKeyGenerator` and `OmBucketGenerator`. ########## File path: hadoop-ozone/dist/src/main/smoketest/freon/remove.robot ########## @@ -0,0 +1,43 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +Documentation Test freon data remove commands +Resource ../lib/os.robot +Test Timeout 5 minutes + +*** Variables *** +${PREFIX} ${EMPTY} + +*** Test Cases *** +Ozone Client Remover (Error) + ${result} = Execute and checkrc ozone freon ocr ${OM_HA_PARAM} -t=1 -n=1 -p ocr${PREFIX} --remove-key --remove-bucket 255 + Should contain ${result} Invalid Option + +Ozone Client Key Generator For Remover + ${result} = Execute ozone freon ockg ${OM_HA_PARAM} -t=1 -n=1 -p ocr${PREFIX} + Should contain ${result} Successful executions: 1 Review comment: Ideally this should be a `[setup]` step, not a separate test case. http://robotframework.org/robotframework/3.2.1/RobotFrameworkUserGuide.html#test-setup-and-teardown ########## File path: hadoop-ozone/dist/src/main/smoketest/freon/remove.robot ########## @@ -0,0 +1,43 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +*** Settings *** +Documentation Test freon data remove commands +Resource ../lib/os.robot +Test Timeout 5 minutes + +*** Variables *** +${PREFIX} ${EMPTY} + +*** Test Cases *** +Ozone Client Remover (Error) + ${result} = Execute and checkrc ozone freon ocr ${OM_HA_PARAM} -t=1 -n=1 -p ocr${PREFIX} --remove-key --remove-bucket 255 + Should contain ${result} Invalid Option + +Ozone Client Key Generator For Remover + ${result} = Execute ozone freon ockg ${OM_HA_PARAM} -t=1 -n=1 -p ocr${PREFIX} + Should contain ${result} Successful executions: 1 + +Ozone Client Remover (KEY) + ${result} = Execute ozone freon ocr ${OM_HA_PARAM} -t=1 -n=1 -p ocr${PREFIX} --remove-key + Should contain ${result} Successful executions: 1 + +OM Bucket Generator For Remover + ${result} = Execute ozone freon ombg ${OM_HA_PARAM} -t=1 -n=1 -p ocr${PREFIX} + Should contain ${result} Successful executions: 1 Review comment: Same here regarding `[setup]`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
