URL: https://github.com/freeipa/freeipa/pull/357
Author: martbab
Title: #357: Travis ci improvements
Action: opened
PR body:
"""
Test PR against my own fork to test Travis CI improvements.
"""
To pull the PR as Git branch:
git remote add ghfreeipa https://github.com/freeipa/freeipa
git fetch ghfreeipa pull/357/head:pr357
git checkout pr357
From 60ac73eb37920a850db7bd4faa2682a1f57491f3 Mon Sep 17 00:00:00 2001
From: Martin Babinsky <mbabi...@redhat.com>
Date: Tue, 20 Dec 2016 10:11:20 +0100
Subject: [PATCH 1/6] Bump up ipa-docker-test-runner version
---
.travis.yml | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/.travis.yml b/.travis.yml
index e870213..c32c5d7 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -18,7 +18,7 @@ before_install:
- pip install pep8
- >
pip3 install
- git+https://github.com/freeipa/ipa-docker-test-runner@release-0-2-0
+ git+https://github.com/freeipa/ipa-docker-test-runner@release-0-2-1
script:
- >
From 0f43576fc0832088a8a32ca109b8a25e733829ce Mon Sep 17 00:00:00 2001
From: Martin Babinsky <mbabi...@redhat.com>
Date: Tue, 20 Dec 2016 15:47:31 +0100
Subject: [PATCH 2/6] travis: mark FreeIPA as python project
---
.travis.yml | 1 +
1 file changed, 1 insertion(+)
diff --git a/.travis.yml b/.travis.yml
index c32c5d7..2855bf2 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -1,3 +1,4 @@
+language: python
services:
- docker
From 79a11702911eedb3731619fbd4f3cc08ed14ac06 Mon Sep 17 00:00:00 2001
From: Martin Babinsky <mbabi...@redhat.com>
Date: Tue, 20 Dec 2016 15:55:55 +0100
Subject: [PATCH 3/6] Travis CI: a separate script to run test tasks
this script is intended only for use in Travis CI and contains
configuration of the test run requested:
* it can run linter step separately by specifying TASK_TO_RUN="lint"
environment variable in .travis.yml. In this case it also runs
pep8 checker on the commits in PR.
* other steps are run in developer mode in order to skip pylint run
and speed up the task
* in all cases the CI result log is populated and can be displayed
if the job fails
---
.travis_run_task.sh | 30 ++++++++++++++++++++++++++++++
1 file changed, 30 insertions(+)
create mode 100644 .travis_run_task.sh
diff --git a/.travis_run_task.sh b/.travis_run_task.sh
new file mode 100644
index 0000000..5f8d77e
--- /dev/null
+++ b/.travis_run_task.sh
@@ -0,0 +1,30 @@
+#!/bin/bash
+
+# NOTE: this script is intended to run in Travis CI only
+
+test_set=""
+developer_mode_opt="--developer-mode"
+
+if [[ "TASK_TO_RUN" == "lint" && "$TRAVIS_EVENT_TYPE" == "pull_request" ]]
+then
+ git diff origin/${TRAVIS_BRANCH} -U0 | pep8 --diff &> $PEP8_ERROR_LOG ||:
+
+ # disable developer mode in lint task, otherwise we get an error
+ developer_mode_opt=""
+fi
+
+if [[ -z "$TESTS_TO_RUN" ]]
+then
+ pushd ipatests
+ test_set=`ls -d -1 $TESTS_TO_RUN 2> /dev/null`
+ popd
+fi
+
+docker pull ${TEST_RUNNER_IMAGE}
+
+travis_wait 50 ipa-docker-test-runner -l $CI_RESULTS_LOG \
+ -c $TEST_RUNNER_IMAGE \
+ $developer_mode_opt \
+ --container-image ${TEST_RUNNER_IMAGE} \
+ --git-repo ${TRAVIS_BUILD_DIR} \
+ $TASK_TO_RUN $test_set
From f45923efeb1ff21b8c59798c59f31ef01943c56d Mon Sep 17 00:00:00 2001
From: Martin Babinsky <mbabi...@redhat.com>
Date: Tue, 20 Dec 2016 16:01:00 +0100
Subject: [PATCH 4/6] Put the commands informing and displaying build logs on
single line
This prevents Travis log collector to add separate expansion marks to
the echo output and the actuall log output.
---
.travis.yml | 6 ++----
1 file changed, 2 insertions(+), 4 deletions(-)
diff --git a/.travis.yml b/.travis.yml
index 2855bf2..f11e250 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -39,7 +39,5 @@ script:
--git-repo ${TRAVIS_BUILD_DIR}
run-tests $test_set
after_failure:
- - echo "Test runner output:"
- - tail -n 5000 ci_results_${TRAVIS_BRANCH}.log
- - echo "PEP-8 errors:"
- - cat pep8_errors.log
+ - echo "Test runner output:"; tail -n 5000 ci_results_${TRAVIS_BRANCH}.log
+ - echo "PEP-8 errors:"; cat pep8_errors.log
From 065430670084f0c16106cb8ef080feb71804f51a Mon Sep 17 00:00:00 2001
From: Martin Babinsky <mbabi...@redhat.com>
Date: Tue, 20 Dec 2016 16:03:25 +0100
Subject: [PATCH 5/6] Travis: offload test execution to a separate script
---
.travis.yml | 17 +----------------
1 file changed, 1 insertion(+), 16 deletions(-)
diff --git a/.travis.yml b/.travis.yml
index f11e250..e6c2d40 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -22,22 +22,7 @@ before_install:
git+https://github.com/freeipa/ipa-docker-test-runner@release-0-2-1
script:
- - >
- if [[ "$TRAVIS_EVENT_TYPE" == "pull_request" ]];
- then
- git diff origin/${TRAVIS_BRANCH} -U0 | pep8 --diff &> pep8_errors.log;
- fi
- - "pushd ipatests; test_set=`ls -d -1 $TESTS_TO_RUN 2> /dev/null`; popd"
- # use travis_wait so that long running tasks (tests) which produce no
- # output do not cause premature termination of the build
- - "docker pull ${TEST_RUNNER_IMAGE}"
- - >
- travis_wait 50
- ipa-docker-test-runner -l ci_results_${TRAVIS_BRANCH}.log
- -c .test_runner_config.yaml
- --container-image ${TEST_RUNNER_IMAGE}
- --git-repo ${TRAVIS_BUILD_DIR}
- run-tests $test_set
+ - ./.travis_run_task.sh
after_failure:
- echo "Test runner output:"; tail -n 5000 ci_results_${TRAVIS_BRANCH}.log
- echo "PEP-8 errors:"; cat pep8_errors.log
From 9f1c3191a4f8d6cfc4183b0cd8c8bef9865aa6a6 Mon Sep 17 00:00:00 2001
From: Martin Babinsky <mbabi...@redhat.com>
Date: Tue, 20 Dec 2016 16:04:02 +0100
Subject: [PATCH 6/6] split out lint to a separate Travis job
In order to speed our Travis CI gating even further, the lint step has
been split to a separate job that can be run in parallel with the test
runs. The test runs are in turn launched in developer mode to speed them
up.
---
.travis.yml | 10 ++++++++--
1 file changed, 8 insertions(+), 2 deletions(-)
diff --git a/.travis.yml b/.travis.yml
index e6c2d40..bc52d06 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -5,9 +5,15 @@ services:
env:
global:
- TEST_RUNNER_IMAGE="martbab/freeipa-fedora-test-runner:master-latest"
+ TEST_RUNNER_CONFIG=".test_runner_config.yaml"
+ PEP8_ERROR_LOG="pep8_errors.log"
+ CI_RESULTS_LOG="ci_results_${TRAVIS_BRANCH}.log"
matrix:
- - TESTS_TO_RUN="test_xmlrpc/test_[a-k]*.py"
- - >
+ - TASK_TO_RUN="lint"
+ - TASK_TO_RUN="run-tests"
+ TESTS_TO_RUN="test_xmlrpc/test_[a-k]*.py"
+ - TASK_TO_RUN="run-tests"
+ >
TESTS_TO_RUN="test_cmdline
test_install
test_ipalib
--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code
