URL: https://github.com/freeipa/freeipa/pull/293 Author: martbab Title: #293: Run out-of-tree tests in Travis CI Action: synchronized
To pull the PR as Git branch: git remote add ghfreeipa https://github.com/freeipa/freeipa git fetch ghfreeipa pull/293/head:pr293 git checkout pr293
From d7af288c0f0cba47d300d9483e8ff78ba43e7e87 Mon Sep 17 00:00:00 2001 From: Martin Babinsky <mbabi...@redhat.com> Date: Wed, 30 Nov 2016 10:30:04 +0100 Subject: [PATCH 1/2] Configuration file for ipa-docker-test-runner Prepare a configuration file for https://github.com/martbab/ipa-docker-test-runner. The latest freeipa-fedora-test-runner Docker image (F25 as of time of writing this message) will be used to run tests. Some of them will be purposefuly excluded from the test suite, namely: * test_integration and test_webui: for obvious reasons, CI tests require complicated multi-host setup which is currently not achievable in Travis CI * test_ipapython/test_keyring: Docker can not cope with storing and retrieving secrets from Kernel keyring, that is a known issue * test_xmlrpc/test_dns_plugin.py:test_dns_soa: There are 2-3 non-deterministic failures in this suite in Travis CI, this suite was disabled until the root cause is discovered and fixed/workarounded --- .test_runner_config.yaml | 50 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 50 insertions(+) create mode 100644 .test_runner_config.yaml diff --git a/.test_runner_config.yaml b/.test_runner_config.yaml new file mode 100644 index 0000000..f1205e8 --- /dev/null +++ b/.test_runner_config.yaml @@ -0,0 +1,50 @@ +container: + detach: true + hostname: master.ipa.test + working_dir: /freeipa +host: + binds: + - /sys/fs/cgroup:/sys/fs/cgroup:ro + - /dev/urandom:/dev/random:ro + privileged: true + security_opt: + - label:disable + tmpfs: + - /tmp + - /run +server: + domain: ipa.test + password: Secret123 + realm: IPA.TEST +steps: + build: + - make V=0 ${make_target} + builddep: + - rm -rf /var/cache/dnf/* + - "dnf makecache fast || :" + - dnf builddep -y ${builddep_opts} --spec freeipa.spec.in --best --allowerasing + cleanup: + - chown -R ${uid}:${gid} ${container_working_dir} + configure: + - ./autogen.sh + install_packages: + - dnf install -y ${container_working_dir}/dist/rpms/*.rpm --best --allowerasing + install_server: + - ipa-server-install -U --domain ${server_domain} --realm ${server_realm} -p ${server_password} + -a ${server_password} --setup-dns --auto-forwarders + - ipa-kra-install -p ${server_password} + lint: + - make V=0 lint + prepare_tests: + - echo ${server_password} | kinit admin && ipa ping + - cp -r /etc/ipa/* ~/.ipa/ + - echo ${server_password} > ~/.ipa/.dmpw + - echo 'wait_for_dns=5' >> ~/.ipa/default.conf + run_tests: + - ipa-run-tests ${tests_ignore} -k-test_dns_soa ${tests_verbose} ${path} +tests: + ignore: + - test_integration + - test_webui + - test_ipapython/test_keyring.py + verbose: true From 8eb8773aeedeabfaf8e51b62ef8894f6e41748f2 Mon Sep 17 00:00:00 2001 From: Martin Babinsky <mbabi...@redhat.com> Date: Wed, 30 Nov 2016 10:37:46 +0100 Subject: [PATCH 2/2] Use ipa-docker-test-runner to run tests in Travis CI https://github.com/martbab/ipa-docker-test-runner is now used to run the following tasks in Travis CI: * pull in a FreeIPA test runner Docker image * configure/make lint/make rpms * install rpms * install FreeIPA server and KRA * run out-of-tree tests For performance reasons (last two steps are very time-consuming) the available tests were split roughly in half and are run as two separate jobs to speed up the process. AD trust is not installed as part of tests since the enabled compat plugin causes false negative errors. --- .travis.yml | 36 ++++++++++++++++++++++++++++++++---- 1 file changed, 32 insertions(+), 4 deletions(-) diff --git a/.travis.yml b/.travis.yml index 44a2ee6..a05d1d7 100644 --- a/.travis.yml +++ b/.travis.yml @@ -1,16 +1,44 @@ services: - docker +env: + global: + - TEST_RUNNER_IMAGE="martbab/freeipa-fedora-test-runner:master-latest" + matrix: + - TESTS_TO_RUN="test_xmlrpc/test_[a-k]*.py" + - > + TESTS_TO_RUN="test_cmdline + test_install + test_ipalib + test_ipapython + test_ipaserver + test_pkcs10 + test_xmlrpc/test_[l-z]*.py" before_install: - pip install pep8 + - > + pip3 install + git+https://github.com/martbab/ipa-docker-test-runner@release-0-2-0 script: - > if [[ "$TRAVIS_EVENT_TYPE" == "pull_request" ]]; then - git diff origin/${TRAVIS_BRANCH} -U0 | pep8 --diff; + git diff origin/${TRAVIS_BRANCH} -U0 | pep8 --diff &> pep8_errors.log; fi + - "pushd ipatests; test_set=`ls -d -1 $TESTS_TO_RUN 2> /dev/null`; popd" + # use travis_wait so that long running tasks (tests) which produce no + # output do not cause premature termination of the build + - "docker pull ${TEST_RUNNER_IMAGE}" - > - docker run -v $PWD:/freeipa -w /freeipa - martbab/freeipa-fedora-builder:${TRAVIS_BRANCH}-latest - /bin/bash -c 'dnf builddep -y -D "with_lint 1" --spec freeipa.spec.in && autoreconf -i && ./configure && make lint && make rpms' + travis_wait 50 + ipa-docker-test-runner -l ci_results_${TRAVIS_BRANCH}.log + -c .test_runner_config.yaml + --container-image ${TEST_RUNNER_IMAGE} + --git-repo ${TRAVIS_BUILD_DIR} + run-tests $test_set +after_failure: + - echo "Test failures:" + - tail -n 5000 ci_results_${TRAVIS_BRANCH}.log + - echo "Pep 8 errors:" + - cat pep8_errors.log
-- Manage your subscription for the Freeipa-devel mailing list: https://www.redhat.com/mailman/listinfo/freeipa-devel Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code
