URL: https://github.com/freeipa/freeipa/pull/293
Author: martbab
 Title: #293: Run out-of-tree tests in Travis CI
Action: synchronized

To pull the PR as Git branch:
git remote add ghfreeipa https://github.com/freeipa/freeipa
git fetch ghfreeipa pull/293/head:pr293
git checkout pr293
From d7af288c0f0cba47d300d9483e8ff78ba43e7e87 Mon Sep 17 00:00:00 2001
From: Martin Babinsky <mbabi...@redhat.com>
Date: Wed, 30 Nov 2016 10:30:04 +0100
Subject: [PATCH 1/2] Configuration file for ipa-docker-test-runner

Prepare a configuration file for
https://github.com/martbab/ipa-docker-test-runner. The latest
freeipa-fedora-test-runner Docker image (F25 as of time of writing this
message) will be used to run tests. Some of them will be purposefuly excluded
from the test suite, namely:

* test_integration and test_webui: for obvious reasons, CI tests require
  complicated multi-host setup which is currently not achievable in Travis CI
* test_ipapython/test_keyring: Docker can not cope with storing and retrieving
  secrets from Kernel keyring, that is a known issue
* test_xmlrpc/test_dns_plugin.py:test_dns_soa: There are 2-3 non-deterministic
  failures in this suite in Travis CI, this suite was disabled until the root
  cause is discovered and fixed/workarounded
---
 .test_runner_config.yaml | 50 ++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 50 insertions(+)
 create mode 100644 .test_runner_config.yaml

diff --git a/.test_runner_config.yaml b/.test_runner_config.yaml
new file mode 100644
index 0000000..f1205e8
--- /dev/null
+++ b/.test_runner_config.yaml
@@ -0,0 +1,50 @@
+container:
+  detach: true
+  hostname: master.ipa.test
+  working_dir: /freeipa
+host:
+  binds:
+  - /sys/fs/cgroup:/sys/fs/cgroup:ro
+  - /dev/urandom:/dev/random:ro
+  privileged: true
+  security_opt:
+  - label:disable
+  tmpfs:
+  - /tmp
+  - /run
+server:
+  domain: ipa.test
+  password: Secret123
+  realm: IPA.TEST
+steps:
+  build:
+  - make V=0 ${make_target}
+  builddep:
+  - rm -rf /var/cache/dnf/*
+  - "dnf makecache fast || :"
+  - dnf builddep -y ${builddep_opts} --spec freeipa.spec.in --best --allowerasing
+  cleanup:
+  - chown -R ${uid}:${gid} ${container_working_dir}
+  configure:
+  - ./autogen.sh
+  install_packages:
+  - dnf install -y ${container_working_dir}/dist/rpms/*.rpm --best --allowerasing
+  install_server:
+  - ipa-server-install -U --domain ${server_domain} --realm ${server_realm} -p ${server_password}
+    -a ${server_password} --setup-dns --auto-forwarders
+  - ipa-kra-install -p ${server_password}
+  lint:
+  - make V=0 lint
+  prepare_tests:
+  - echo ${server_password} | kinit admin && ipa ping
+  - cp -r /etc/ipa/* ~/.ipa/
+  - echo ${server_password} > ~/.ipa/.dmpw
+  - echo 'wait_for_dns=5' >> ~/.ipa/default.conf
+  run_tests:
+  - ipa-run-tests ${tests_ignore} -k-test_dns_soa ${tests_verbose} ${path}
+tests:
+  ignore:
+  - test_integration
+  - test_webui
+  - test_ipapython/test_keyring.py
+  verbose: true

From d5e72844a36f5147038d64947235369fece4a5c8 Mon Sep 17 00:00:00 2001
From: Martin Babinsky <mbabi...@redhat.com>
Date: Wed, 30 Nov 2016 10:37:46 +0100
Subject: [PATCH 2/2] Use ipa-docker-test-runner to run tests in Travis CI

https://github.com/martbab/ipa-docker-test-runner is now used to run the
following tasks in Travis CI:

* pull in a FreeIPA test runner Docker image
* configure/make lint/make rpms
* install rpms
* install FreeIPA server and KRA
* run out-of-tree tests

For performance reasons (last two steps are very time-consuming) the available
tests were split roughly in half and are run as two separate jobs to speed up
the process.

AD trust is not installed as part of tests since the enabled compat plugin
causes false negative errors.
---
 .travis.yml | 36 ++++++++++++++++++++++++++++++++----
 1 file changed, 32 insertions(+), 4 deletions(-)

diff --git a/.travis.yml b/.travis.yml
index 44a2ee6..e870213 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -1,16 +1,44 @@
 services:
     - docker
 
+env:
+    global:
+        - TEST_RUNNER_IMAGE="martbab/freeipa-fedora-test-runner:master-latest"
+    matrix:
+        - TESTS_TO_RUN="test_xmlrpc/test_[a-k]*.py"
+        - >
+            TESTS_TO_RUN="test_cmdline
+            test_install
+            test_ipalib
+            test_ipapython
+            test_ipaserver
+            test_pkcs10
+            test_xmlrpc/test_[l-z]*.py"
 before_install:
     - pip install pep8
+    - >
+      pip3 install
+      git+https://github.com/freeipa/ipa-docker-test-runner@release-0-2-0
 
 script:
     - >
         if [[ "$TRAVIS_EVENT_TYPE" == "pull_request" ]];
         then
-            git diff origin/${TRAVIS_BRANCH} -U0 | pep8 --diff;
+            git diff origin/${TRAVIS_BRANCH} -U0 | pep8 --diff &> pep8_errors.log;
         fi
+    - "pushd ipatests; test_set=`ls -d -1 $TESTS_TO_RUN 2> /dev/null`; popd"
+    # use travis_wait so that long running tasks (tests) which produce no
+    # output do not cause premature termination of the build
+    - "docker pull ${TEST_RUNNER_IMAGE}"
     - >
-        docker run -v $PWD:/freeipa -w /freeipa
-        martbab/freeipa-fedora-builder:${TRAVIS_BRANCH}-latest
-        /bin/bash -c 'dnf builddep -y -D "with_lint 1" --spec freeipa.spec.in && autoreconf -i && ./configure && make lint && make rpms'
+        travis_wait 50
+        ipa-docker-test-runner -l ci_results_${TRAVIS_BRANCH}.log
+        -c .test_runner_config.yaml
+        --container-image ${TEST_RUNNER_IMAGE}
+        --git-repo ${TRAVIS_BUILD_DIR}
+        run-tests $test_set
+after_failure:
+  - echo "Test runner output:"
+  - tail -n 5000 ci_results_${TRAVIS_BRANCH}.log
+  - echo "PEP-8 errors:"
+  - cat pep8_errors.log
-- 
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

Reply via email to