[
https://issues.apache.org/jira/browse/HDDS-1424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16834920#comment-16834920
]
Hudson commented on HDDS-1424:
------------------------------
SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #16512 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/16512/])
HDDS-1424. Support multi-container robot test execution (elek: rev
1a696cc7619cf2ef86ecc9bacada96f4ad919d47)
* (add) hadoop-ozone/dist/src/main/compose/ozoneperf/test.sh
* (add) hadoop-ozone/dist/src/main/smoketest/ozonefs/hadoopo3fs.robot
* (edit) hadoop-ozone/dist/dev-support/bin/dist-layout-stitching
* (add) hadoop-ozone/dist/src/main/compose/ozonesecure/test.sh
* (edit) hadoop-ozone/dist/pom.xml
* (add) hadoop-ozone/dist/src/main/compose/test-single.sh
* (add) hadoop-ozone/dist/src/main/compose/ozonefs/test.sh
* (edit) hadoop-ozone/dist/src/main/smoketest/README.md
* (add) hadoop-ozone/dist/src/main/smoketest/kinit.robot
* (add) hadoop-ozone/dist/src/main/compose/test-all.sh
* (add) hadoop-ozone/dist/src/main/compose/ozones3/test.sh
* (add) hadoop-ozone/dist/src/main/compose/ozone/test.sh
* (add) hadoop-ozone/dist/src/main/compose/testlib.sh
* (edit) hadoop-ozone/dist/src/main/smoketest/test.sh
> Support multi-container robot test execution
> --------------------------------------------
>
> Key: HDDS-1424
> URL: https://issues.apache.org/jira/browse/HDDS-1424
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
> Reporter: Elek, Marton
> Assignee: Elek, Marton
> Priority: Major
> Labels: pull-request-available
> Time Spent: 3.5h
> Remaining Estimate: 0h
>
> The ./smoketest folder in the distribution package contains robotframework
> based test scripts to test the main behaviour of Ozone.
> The tests have two layers:
> 1. robot test definitions to execute commands and assert the results (on a
> given host machine)
> 2. ./smoketest/test.sh which starts/stops the docker-compose based
> environments AND execute the selected robot tests inside the right hosts
> The second one (test.sh) has some serious limitations:
> 1. all the tests are executed inside the same container (om):
> https://github.com/apache/hadoop/blob/5f951ea2e39ae4dfe554942baeec05849cd7d3c2/hadoop-ozone/dist/src/main/smoketest/test.sh#L89
> Some of the tests (ozonesecure-mr, ozonefs) may require the flexibility to
> execute different robot tests in different containers.
> 2. The definition of the global test set is complex and hard to understood.
> The current code is:
> {code}
> TESTS=("basic")
> execute_tests ozone "${TESTS[@]}"
> TESTS=("auditparser")
> execute_tests ozone "${TESTS[@]}"
> TESTS=("ozonefs")
> execute_tests ozonefs "${TESTS[@]}"
> TESTS=("basic")
> execute_tests ozone-hdfs "${TESTS[@]}"
> TESTS=("s3")
> execute_tests ozones3 "${TESTS[@]}"
> TESTS=("security")
> execute_tests ozonesecure .
> {code}
> For example for ozonesecure the TESTS is not used. And the usage of bash
> lists require additional complexity in the execute_tests function.
> I propose here a very lightweight refactor. Instead of including both the
> test definitions AND the helper methods in test.sh I would separate them.
> Let's put a test.sh to each of the compose directories. The separated test.sh
> can include common methods from a main shell script. For example:
> {code}
> source "$COMPOSE_DIR/../testlib.sh"
> start_docker_env
> execute_robot_test scm basic/basic.robot
> execute_robot_test scm s3
> stop_docker_env
> generate_report
> {code}
> This is a more clean and more flexible definition. It's easy to execute just
> this test (as it's saved to the compose/ozones3 directory. And it's more
> flexible.
> Other example, where multiple containers are used to execute tests:
> {code}
> source "$COMPOSE_DIR/../testlib.sh"
> start_docker_env
> execute_robot_test scm ozonefs/ozonefs.robot
> export OZONE_HOME=/opt/ozone
> execute_robot_test hadoop32 ozonefs/hadoopo3fs.robot
> execute_robot_test hadoop31 ozonefs/hadoopo3fs.robot
> stop_docker_env
> generate_report
> {code}
> With this separation the definition of the helper methods (eg.
> execute_robot_test or stop_docker_env) would also be simplified.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
