adoroszlai commented on code in PR #8654:
URL: https://github.com/apache/ozone/pull/8654#discussion_r2162963412
##########
hadoop-ozone/dist/src/main/smoketest/ozone-fi/byteman_faults_sample.robot:
##########
@@ -0,0 +1,32 @@
+*** Variables ***
+${RULE} /opt/hadoop/share/ozone/byteman/skip-put-block.btm
+${vol} vol1
+${buck} buck1
+${key} key1
+${SECURITY_ENABLED} true
+
+*** Settings ***
+Resource BytemanKeywords.robot
+Resource os.robot
+Resource ../lib/os.robot
Review Comment:
```suggestion
Resource ../lib/os.robot
```
##########
hadoop-ozone/dist/src/main/compose/ozonesecure-ha/byteman.yaml:
##########
@@ -0,0 +1,47 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+x-byteman-config:
+ &byteman-config
+ environment:
+ BYTEMAN_HOME: /opt/byteman/
+ OZONE_OPTS:
-javaagent:/opt/byteman.jar=listener:true,interface:0.0.0.0,port:${BYTEMAN_PORT}
Review Comment:
`OZONE_OPTS` is applied to all `ozone` commands. `ozone sh`, `ozone admin`,
etc. will show error due to `Address already in use`. We'll need to add a way
to set this only for server processes (HDDS-13325).
Also got `illegal agent argument : interface:0.0.0.0`. Byteman is listening
only on localhost (`127.0.0.1:9091`), thus cannot connect to other container's
listener.
##########
hadoop-ozone/dist/src/main/smoketest/ozone-fi/BytemanKeywords.robot:
##########
@@ -0,0 +1,52 @@
+*** Settings ***
+Library ../lib/BytemanLibrary.py
+Library String
+
+*** Variables ***
+${BYTEMAN_PORT} 9091
+${DATANODE1_BYTEMAN_HOST_PORT} datanode1:${BYTEMAN_PORT}
+${DATANODE2_BYTEMAN_HOST_PORT} datanode2:${BYTEMAN_PORT}
+${DATANODE3_BYTEMAN_HOST_PORT} datanode3:${BYTEMAN_PORT}
+${OM1_BYTEMAN_HOST_PORT} om1:${BYTEMAN_PORT}
+${OM2_BYTEMAN_HOST_PORT} om2:${BYTEMAN_PORT}
+${OM3_BYTEMAN_HOST_PORT} om3:${BYTEMAN_PORT}
+${RECON_BYTEMAN_HOST_PORT} recon:${BYTEMAN_PORT}
+${SCM1_BYTEMAN_HOST_PORT} scm1.org:${BYTEMAN_PORT}
+${SCM2_BYTEMAN_HOST_PORT} scm2.org:${BYTEMAN_PORT}
+${SCM3_BYTEMAN_HOST_PORT} scm3:${BYTEMAN_PORT}
+${HTTPFS_BYTEMAN_HOST_PORT} httpfs:${BYTEMAN_PORT}
+${S3G_BYTEMAN_HOST_PORT} s3g:${BYTEMAN_PORT}
+
+*** Keywords ***
+Setup Byteman For Component
+ [Arguments] ${component} ${host_port}
+ ${host} ${port} = Split String ${host_port} :
+ Connect To Byteman Agent ${component} ${host} ${port}
+
+Setup All Byteman Agents
+ Log Inside Setup All Byteman Agents
+ Setup Byteman For Component datanode1 ${DATANODE1_BYTEMAN_HOST_PORT}
+ Setup Byteman For Component datanode2 ${DATANODE2_BYTEMAN_HOST_PORT}
+ Setup Byteman For Component datanode3 ${DATANODE3_BYTEMAN_HOST_PORT}
+ Setup Byteman For Component om1 ${OM1_BYTEMAN_HOST_PORT}
+ Setup Byteman For Component om2 ${OM2_BYTEMAN_HOST_PORT}
+ Setup Byteman For Component om3 ${OM3_BYTEMAN_HOST_PORT}
+ Setup Byteman For Component recon ${RECON_BYTEMAN_HOST_PORT}
+ Setup Byteman For Component scm1 ${SCM1_BYTEMAN_HOST_PORT}
+ Setup Byteman For Component scm2 ${SCM2_BYTEMAN_HOST_PORT}
+ Setup Byteman For Component scm3 ${SCM3_BYTEMAN_HOST_PORT}
+ Setup Byteman For Component https ${HTTPFS_BYTEMAN_HOST_PORT}
+ Setup Byteman For Component s3g ${S3G_BYTEMAN_HOST_PORT}
Review Comment:
I think this is unnecessary.
1. `Connect To Byteman Agent` just stores the `component => host:port`
mapping. This can be skipped, `Inject Fault` and `Remove Fault` can use
hostname directly, and take port from the environment variable
`%{BYTEMAN_PORT}`.
2. `Setup Byteman For Component` splits its `${host_port}` argument, so
creating all these variables in `host:port` format is not needed.
##########
hadoop-ozone/dist/src/main/smoketest/ozone-fi/byteman_faults_sample.robot:
##########
@@ -0,0 +1,32 @@
+*** Variables ***
+${RULE} /opt/hadoop/share/ozone/byteman/skip-put-block.btm
+${vol} vol1
+${buck} buck1
+${key} key1
+${SECURITY_ENABLED} true
+
+*** Settings ***
+Resource BytemanKeywords.robot
+Resource os.robot
+Resource ../lib/os.robot
+Resource ../commonlib.robot
+Resource ../ozone-lib/shell.robot
+Suite Setup Setup Suite
+Suite Teardown Teardown Suite
+
+
+*** Keywords ***
+Setup Suite
+ Setup All Byteman Agents
+ Inject Fault Into Component datanode1 ${RULE}
+ Log To Console Kinit
+ Execute And Ignore Error kinit -kt
/etc/security/keytabs/testuser.keytab testuser/om
Review Comment:
Please use `Kinit test user` keyword.
##########
hadoop-ozone/dist/src/main/smoketest/lib/BytemanLibrary.py:
##########
@@ -0,0 +1,63 @@
+#!/usr/bin/env python3
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional consolermation
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+import subprocess
+import json
+from robot.api import logger
+
+class BytemanLibrary:
+ def __init__(self):
+ self.byteman_clients = {}
+
+ def connect_to_byteman_agent(self, component_name, host="localhost",
port=9091):
+ """Connect to Byteman agent on specific component"""
+ self.byteman_clients[component_name] = {'host': host, 'port': port,
+ 'base_url':
f"http://{host}:{port}"}
+ logger.console(f"Connected to Byteman agent for {component_name} at
{host}:{port}")
+
+ def add_byteman_rule(self, component_name, rule_file):
+ """Add Byteman rule into specific component"""
+ client = self.byteman_clients[component_name]
+
+ # Use bmsubmit command to load rule
+ cmd = ["bmsubmit", "-h", component_name, "-p", str(client['port']),
"-l", rule_file]
+ result = subprocess.run(cmd, capture_output=True, text=True)
+ if result.returncode != 0:
+ raise RuntimeError(f"Failed to load rule: {result.stderr}")
+ logger.console(f"Loaded Byteman rule {rule_file} into
{component_name}")
+
+ def remove_byteman_rule(self, component_name, rule_file):
+ """Remove Byteman rule for specific component"""
+ client = self.byteman_clients[component_name]
+
+ cmd = ["bmsubmit", "-h", component_name, "-p", str(client['port']),
"-u", rule_file]
+ logger.console(cmd)
+ result = subprocess.run(cmd, capture_output=True, text=True)
+ if result.returncode != 0:
+ raise RuntimeError(f"Failed to unload rule: {result.stderr}")
+ logger.console(f"Unloaded Byteman rule {rule_file} from
{component_name}")
Review Comment:
Please reduce duplication between these two.
##########
hadoop-ozone/dist/src/main/smoketest/ozone-fi/byteman_faults_sample.robot:
##########
@@ -0,0 +1,32 @@
+*** Variables ***
+${RULE} /opt/hadoop/share/ozone/byteman/skip-put-block.btm
+${vol} vol1
+${buck} buck1
+${key} key1
+${SECURITY_ENABLED} true
+
+*** Settings ***
+Resource BytemanKeywords.robot
+Resource os.robot
+Resource ../lib/os.robot
+Resource ../commonlib.robot
+Resource ../ozone-lib/shell.robot
+Suite Setup Setup Suite
+Suite Teardown Teardown Suite
+
+
+*** Keywords ***
+Setup Suite
+ Setup All Byteman Agents
+ Inject Fault Into Component datanode1 ${RULE}
+ Log To Console Kinit
Review Comment:
Robot `log.html` shows all keyword execution in detail, no need to log to
the console like that.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
