zstan commented on code in PR #12637: URL: https://github.com/apache/ignite/pull/12637#discussion_r3510577931
########## modules/ducktests/README.md: ########## @@ -1,311 +1,305 @@ -# Overview -The `ignitetest` framework provides basic functionality and services -to write integration tests for Apache Ignite. This framework bases on -the `ducktape` test framework, for information about it check the links: -- https://github.com/confluentinc/ducktape - source code of the `ducktape`. -- https://ducktape.readthedocs.io/en/latest/index.html - documentation to the `ducktape`. - -Structure of the `tests` directory is: -- `./ignitetest/services` contains basic services functionality. -- `./ignitetest/utils` contains utils for testing. -- `./ignitetest/tests` contains tests. -- `./checks` contains unit tests of utils, tests' decorators etc. - -Some tests (like the CDC replication ones) require modules maintained in the -separate [ignite-extensions](https://github.com/apache/ignite-extensions) repository. - -To run these tests the `ignite-extensions` working directory should be checked out -to the same filesystem level next to the `ignite` one. So it should be the below structure -of directories: -- `./ignite` working directory checked out from the ignite repository (mentioned as `${IGNITE_HOME}` below) -- `./ignite-extensions` working directory checked out from the ignite-extensions repository - -The needed extension module should be built before tests run. - -For example for the CDC replication tests the `cdc-ext` module should be built as: -``` -cd ${IGNITE_HOME}\..\ignite-extensions -mvn clean package -pl :ignite-cdc-ext -Pskip-docs -DskipTests -``` +# Apache Ignite Integration Test Framework - Ducktests + +The `ignitetest` framework provides basic functionality and services to write integration tests for Apache Ignite. This framework is built on top of the **ducktape** test framework. +* For core concepts, see the [ducktape source code](https://github.com/confluentinc/ducktape). +* For framework details, see the [ducktape documentation](https://ducktape.readthedocs.io/en/latest/index.html). + +### Repository Structure +All paths below are relative to `${IGNITE_HOME}/modules/ducktests/tests`: +* `./ignitetest/services`: Contains basic services and cluster orchestration logic. +* `./ignitetest/utils`: Contains testing helper utilities. +* `./ignitetest/tests`: Contains the actual integration test scenarios. +* `./checks`: Contains internal framework unit tests and style decorators. + +--- -# Local run -Docker is used to emulate distributed environment. Single container represents -a running node. +## Quick Start (Local Docker Run) -## Requirements -To just start tests locally the only requirement is preinstalled `docker`. -For development process requirements are `python` >= 3.7. +Docker is used to emulate a distributed multi-node cluster environment where each individual container acts as a running cluster node. -## Run tests -- Change a current directory to`${IGNITE_HOME}` -- Build Apache IGNITE invoking `${IGNITE_HOME}/scripts/build-module.sh ducktests` -- (Optionally) Build the needed extension modules in the `${IGNITE_HOME}\..\ignite-extensions` directory -- Change a current directory to `${IGNITE_HOME}/modules/ducktests/tests` -- Run tests in docker containers using a following command: +### 1. Prerequisites +* **Docker** installed and running on your host system. +* **Python >= 3.8** installed on your host system (required only for local environment scripts and development). + +### 2. Prepare the Environment & Code +Execute these preparation steps from the root directory of your project: +```bash +# 1. Change your current directory to the Ignite root +cd ${IGNITE_HOME} + +# 2. Build the Apache Ignite ducktests modules +./scripts/build-module.sh ducktests + +# 3. Navigate into the ducktests directory +cd modules/ducktests/tests ``` -./docker/run_tests.sh + +### 3. Preparing the Local Environment +Run the following commands from your host system's shell inside `${IGNITE_HOME}/modules/ducktests/tests`: +```bash +# Create and activate an isolated development virtual environment +python3 -m venv ~/.virtualenvs/ignite-ducktests-dev +source ~/.virtualenvs/ignite-ducktests-dev/bin/activate + +# Install core framework testing requirements and editable dependencies +pip install -r docker/requirements-dev.txt +pip install -e . ``` -- For detailed help and instructions, use a following command: + +> If your environment is configured to look only at internal or restricted artifact registries, `pip install` may fail to find specific package versions. +> +> To resolve this, append the public PyPI mirror to your installation command: + +```bash +pip install -r docker/requirements-dev.txt --extra-index-url https://pypi.org/simple ``` -./docker/run_tests.sh --help + +### 4. Run a Smoke Test +Run the test runner script by pointing it directly to a specific smoke test target. The script will automatically build the required container images, bring up the necessary nodes, and run the test scenario: + +```bash +./docker/run_tests.sh -t ./ignitetest/tests/smoke_test.py::SmokeServicesTest.test_ignite_start_stop -n 3 --global-json '{"cluster_size": 2}' Review Comment: want to note that this step i found an issue with 'network defence = on' ``` RUN pip3 install -r /root/requirements.txt ---> Running in 723776310f1d WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='pypi.org', port=443): Read timed out. (read timeout=15)")': /simple/filelock/ ``` with 'off' seems it works fine -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
