URL: https://github.com/freeipa/freeipa/pull/361
Author: martbab
Title: #361: This PR implements a number of improvements for our Travis CI:
Action: synchronized
To pull the PR as Git branch:
git remote add ghfreeipa https://github.com/freeipa/freeipa
git fetch ghfreeipa pull/361/head:pr361
git checkout pr361
From a59ecbc489393ad9d509bd4718ffb87e3197c355 Mon Sep 17 00:00:00 2001
From: Martin Babinsky <mbabi...@redhat.com>
Date: Tue, 20 Dec 2016 10:11:20 +0100
Subject: [PATCH 01/10] Bump up ipa-docker-test-runner version
---
.travis.yml | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/.travis.yml b/.travis.yml
index e870213..c32c5d7 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -18,7 +18,7 @@ before_install:
- pip install pep8
- >
pip3 install
- git+https://github.com/freeipa/ipa-docker-test-runner@release-0-2-0
+ git+https://github.com/freeipa/ipa-docker-test-runner@release-0-2-1
script:
- >
From ab0c72c08bf222c3903c6681d562284169aa2f02 Mon Sep 17 00:00:00 2001
From: Martin Babinsky <mbabi...@redhat.com>
Date: Tue, 20 Dec 2016 15:47:31 +0100
Subject: [PATCH 02/10] travis: mark FreeIPA as python project
---
.travis.yml | 1 +
1 file changed, 1 insertion(+)
diff --git a/.travis.yml b/.travis.yml
index c32c5d7..2855bf2 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -1,3 +1,4 @@
+language: python
services:
- docker
From 0a8de3a9758459c1aab64fa475771694e3c869ff Mon Sep 17 00:00:00 2001
From: Martin Babinsky <mbabi...@redhat.com>
Date: Tue, 20 Dec 2016 16:01:00 +0100
Subject: [PATCH 03/10] Put the commands informing and displaying build logs on
single line
This prevents Travis log collector to add separate expansion marks to
the echo output and the actuall log output.
---
.travis.yml | 6 ++----
1 file changed, 2 insertions(+), 4 deletions(-)
diff --git a/.travis.yml b/.travis.yml
index 2855bf2..28f481f 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -39,7 +39,5 @@ script:
--git-repo ${TRAVIS_BUILD_DIR}
run-tests $test_set
after_failure:
- - echo "Test runner output:"
- - tail -n 5000 ci_results_${TRAVIS_BRANCH}.log
- - echo "PEP-8 errors:"
- - cat pep8_errors.log
+ - echo "Test runner output:"; tail -n 5000 ci_results_${TRAVIS_BRANCH}.log
+ - echo "PEP-8 errors:"; cat pep8_errors.log
From 8172ea91f1e23cfe16e5d6962a67c51e7a778af7 Mon Sep 17 00:00:00 2001
From: Martin Babinsky <mbabi...@redhat.com>
Date: Tue, 20 Dec 2016 15:55:55 +0100
Subject: [PATCH 04/10] Travis CI: a separate script to run test tasks
this script is intended only for use in Travis CI and contains
configuration of the test run requested:
* it can run linter step separately by specifying TASK_TO_RUN="lint"
environment variable in .travis.yml. In this case it also runs
pep8 checker on the commits in PR.
* other steps are run in developer mode in order to skip pylint run
and speed up the task
* in all cases the CI result log is populated and can be displayed
if the job fails
---
.travis_run_task.sh | 34 ++++++++++++++++++++++++++++++++++
1 file changed, 34 insertions(+)
create mode 100755 .travis_run_task.sh
diff --git a/.travis_run_task.sh b/.travis_run_task.sh
new file mode 100755
index 0000000..2163a9b
--- /dev/null
+++ b/.travis_run_task.sh
@@ -0,0 +1,34 @@
+#!/bin/bash
+
+# NOTE: this script is intended to run in Travis CI only
+set -ev
+
+test_set=""
+developer_mode_opt="--developer-mode"
+
+if [[ "$TASK_TO_RUN" == "lint" ]]
+then
+ if [[ "$TRAVIS_EVENT_TYPE" == "pull_request" ]]
+ then
+ git diff origin/$TRAVIS_BRANCH -U0 | pep8 --diff &> $PEP8_ERROR_LOG ||:
+ fi
+
+ # disable developer mode for lint task, otherwise we get an error
+ developer_mode_opt=""
+fi
+
+if [[ -n "$TESTS_TO_RUN" ]]
+then
+ pushd ipatests
+ test_set=`ls -d -1 $TESTS_TO_RUN 2> /dev/null | tr '\n' ' '`
+ popd
+fi
+
+docker pull $TEST_RUNNER_IMAGE
+
+ipa-docker-test-runner -l $CI_RESULTS_LOG \
+ -c $TEST_RUNNER_CONFIG \
+ $developer_mode_opt \
+ --container-image $TEST_RUNNER_IMAGE \
+ --git-repo $TRAVIS_BUILD_DIR \
+ $TASK_TO_RUN $test_set
From 549b439956f063350ff8b31cc7829a4e973bc312 Mon Sep 17 00:00:00 2001
From: Martin Babinsky <mbabi...@redhat.com>
Date: Tue, 20 Dec 2016 16:03:25 +0100
Subject: [PATCH 05/10] Travis: offload test execution to a separate script
---
.travis.yml | 17 +----------------
1 file changed, 1 insertion(+), 16 deletions(-)
diff --git a/.travis.yml b/.travis.yml
index 28f481f..8692dd7 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -22,22 +22,7 @@ before_install:
git+https://github.com/freeipa/ipa-docker-test-runner@release-0-2-1
script:
- - >
- if [[ "$TRAVIS_EVENT_TYPE" == "pull_request" ]];
- then
- git diff origin/${TRAVIS_BRANCH} -U0 | pep8 --diff &> pep8_errors.log;
- fi
- - "pushd ipatests; test_set=`ls -d -1 $TESTS_TO_RUN 2> /dev/null`; popd"
- # use travis_wait so that long running tasks (tests) which produce no
- # output do not cause premature termination of the build
- - "docker pull ${TEST_RUNNER_IMAGE}"
- - >
- travis_wait 50
- ipa-docker-test-runner -l ci_results_${TRAVIS_BRANCH}.log
- -c .test_runner_config.yaml
- --container-image ${TEST_RUNNER_IMAGE}
- --git-repo ${TRAVIS_BUILD_DIR}
- run-tests $test_set
+ - travis_wait 50 ./.travis_run_task.sh
after_failure:
- echo "Test runner output:"; tail -n 5000 ci_results_${TRAVIS_BRANCH}.log
- echo "PEP-8 errors:"; cat pep8_errors.log
From 478386e2bfb4dec7db002bad799b2f439c10c02f Mon Sep 17 00:00:00 2001
From: Martin Babinsky <mbabi...@redhat.com>
Date: Tue, 20 Dec 2016 16:04:02 +0100
Subject: [PATCH 06/10] split out lint to a separate Travis job
In order to speed our Travis CI gating even further, the lint step has
been split to a separate job that can be run in parallel with the test
runs. The test runs are in turn launched in developer mode to speed them
up.
---
.travis.yml | 15 ++++++++++-----
1 file changed, 10 insertions(+), 5 deletions(-)
diff --git a/.travis.yml b/.travis.yml
index 8692dd7..2bcd843 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -5,10 +5,15 @@ services:
env:
global:
- TEST_RUNNER_IMAGE="martbab/freeipa-fedora-test-runner:master-latest"
+ TEST_RUNNER_CONFIG=".test_runner_config.yaml"
+ PEP8_ERROR_LOG="pep8_errors.log"
+ CI_RESULTS_LOG="ci_results_${TRAVIS_BRANCH}.log"
matrix:
- - TESTS_TO_RUN="test_xmlrpc/test_[a-k]*.py"
- - >
- TESTS_TO_RUN="test_cmdline
+ - TASK_TO_RUN="lint"
+ - TASK_TO_RUN="run-tests"
+ TESTS_TO_RUN="test_xmlrpc/test_[a-k]*.py"
+ - TASK_TO_RUN="run-tests"
+ TESTS_TO_RUN="test_cmdline
test_install
test_ipalib
test_ipapython
@@ -24,5 +29,5 @@ before_install:
script:
- travis_wait 50 ./.travis_run_task.sh
after_failure:
- - echo "Test runner output:"; tail -n 5000 ci_results_${TRAVIS_BRANCH}.log
- - echo "PEP-8 errors:"; cat pep8_errors.log
+ - echo "Test runner output:"; tail -n 5000 $CI_RESULTS_LOG
+ - echo "PEP-8 errors:"; cat $PEP8_ERROR_LOG
From b58a03e9ab6f86ae333544b418cf6763abe3c3c7 Mon Sep 17 00:00:00 2001
From: Martin Babinsky <mbabi...@redhat.com>
Date: Wed, 21 Dec 2016 10:30:49 +0100
Subject: [PATCH 07/10] introduce install step to .travis.yml and cache pip
installs
---
.travis.yml | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/.travis.yml b/.travis.yml
index 2bcd843..2e60e29 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -2,6 +2,7 @@ language: python
services:
- docker
+cache: pip
env:
global:
- TEST_RUNNER_IMAGE="martbab/freeipa-fedora-test-runner:master-latest"
@@ -20,7 +21,7 @@ env:
test_ipaserver
test_pkcs10
test_xmlrpc/test_[l-z]*.py"
-before_install:
+install:
- pip install pep8
- >
pip3 install
From f4e146134132e9b23a4b5af1af5e269fa2e96ccd Mon Sep 17 00:00:00 2001
From: Martin Babinsky <mbabi...@redhat.com>
Date: Wed, 21 Dec 2016 10:16:47 +0100
Subject: [PATCH 08/10] Travis CI: use specific Python version during build
This is a preparatory work for the future requirement of running
Python2/3 jobs simultaneously.
---
.travis.yml | 2 ++
.travis_run_task.sh | 3 ++-
2 files changed, 4 insertions(+), 1 deletion(-)
diff --git a/.travis.yml b/.travis.yml
index 2e60e29..4f36b82 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -2,6 +2,8 @@ language: python
services:
- docker
+python:
+ - "2.7"
cache: pip
env:
global:
diff --git a/.travis_run_task.sh b/.travis_run_task.sh
index 2163a9b..4eb78c9 100755
--- a/.travis_run_task.sh
+++ b/.travis_run_task.sh
@@ -1,8 +1,8 @@
#!/bin/bash
# NOTE: this script is intended to run in Travis CI only
-set -ev
+PYTHON="/usr/bin/python${TRAVIS_PYTHON_VERSION}"
test_set=""
developer_mode_opt="--developer-mode"
@@ -29,6 +29,7 @@ docker pull $TEST_RUNNER_IMAGE
ipa-docker-test-runner -l $CI_RESULTS_LOG \
-c $TEST_RUNNER_CONFIG \
$developer_mode_opt \
+ --container-environment "PYTHON=$PYTHON" \
--container-image $TEST_RUNNER_IMAGE \
--git-repo $TRAVIS_BUILD_DIR \
$TASK_TO_RUN $test_set
From 08cc71dbcf59bb2d38e8b7cc806f13e31014d7b9 Mon Sep 17 00:00:00 2001
From: Martin Babinsky <mbabi...@redhat.com>
Date: Tue, 3 Jan 2017 12:47:48 +0100
Subject: [PATCH 09/10] Add license headers to the files used by Travis CI
---
.test_runner_config.yaml | 6 ++++++
.travis_run_task.sh | 4 +++-
2 files changed, 9 insertions(+), 1 deletion(-)
diff --git a/.test_runner_config.yaml b/.test_runner_config.yaml
index f1205e8..2aece9a 100644
--- a/.test_runner_config.yaml
+++ b/.test_runner_config.yaml
@@ -1,3 +1,9 @@
+#
+# Copyright (C) 2017 FreeIPA Contributors see COPYING for license
+#
+
+# Configuration file for the test runner used in Travis CI
+
container:
detach: true
hostname: master.ipa.test
diff --git a/.travis_run_task.sh b/.travis_run_task.sh
index 4eb78c9..a9144b5 100755
--- a/.travis_run_task.sh
+++ b/.travis_run_task.sh
@@ -1,5 +1,7 @@
#!/bin/bash
-
+#
+# Copyright (C) 2017 FreeIPA Contributors see COPYING for license
+#
# NOTE: this script is intended to run in Travis CI only
PYTHON="/usr/bin/python${TRAVIS_PYTHON_VERSION}"
From 39a784e42553943bd18a4cd4cf3e3514087689ad Mon Sep 17 00:00:00 2001
From: Martin Babinsky <mbabi...@redhat.com>
Date: Thu, 5 Jan 2017 10:10:06 +0100
Subject: [PATCH 10/10] Trim the test runner log to show only pytest
failures/errors
If we get to the `run-tests` phase, we no longer care about messages
from previous steps so we can truncate the log output to only show
pytest failures/errors, reducing log size.
As a fallback, the previous behaviour to print last 5000 log lines was
kept in the case the CI tests fail during setup.
---
.travis.yml | 3 ++-
.travis_run_task.sh | 16 ++++++++++++++++
2 files changed, 18 insertions(+), 1 deletion(-)
diff --git a/.travis.yml b/.travis.yml
index 4f36b82..6301974 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -11,6 +11,7 @@ env:
TEST_RUNNER_CONFIG=".test_runner_config.yaml"
PEP8_ERROR_LOG="pep8_errors.log"
CI_RESULTS_LOG="ci_results_${TRAVIS_BRANCH}.log"
+ CI_BACKLOG_SIZE=5000
matrix:
- TASK_TO_RUN="lint"
- TASK_TO_RUN="run-tests"
@@ -32,5 +33,5 @@ install:
script:
- travis_wait 50 ./.travis_run_task.sh
after_failure:
- - echo "Test runner output:"; tail -n 5000 $CI_RESULTS_LOG
+ - echo "Test runner output:"; tail -n $CI_BACKLOG_SIZE $CI_RESULTS_LOG
- echo "PEP-8 errors:"; cat $PEP8_ERROR_LOG
diff --git a/.travis_run_task.sh b/.travis_run_task.sh
index a9144b5..9fd1c1c 100755
--- a/.travis_run_task.sh
+++ b/.travis_run_task.sh
@@ -8,6 +8,17 @@ PYTHON="/usr/bin/python${TRAVIS_PYTHON_VERSION}"
test_set=""
developer_mode_opt="--developer-mode"
+function truncate_log_to_test_failures() {
+ # chop off everything in the CI_RESULTS_LOG preceding pytest error output
+ # if there are pytest errors in the log
+ error_fail_regexp='\(=== ERRORS ===\)\|\(=== FAILURES ===\)'
+
+ if grep -e "$error_fail_regexp" $CI_RESULTS_LOG > /dev/null
+ then
+ sed -i "/$error_fail_regexp/,\$!d" $CI_RESULTS_LOG
+ fi
+}
+
if [[ "$TASK_TO_RUN" == "lint" ]]
then
if [[ "$TRAVIS_EVENT_TYPE" == "pull_request" ]]
@@ -35,3 +46,8 @@ ipa-docker-test-runner -l $CI_RESULTS_LOG \
--container-image $TEST_RUNNER_IMAGE \
--git-repo $TRAVIS_BUILD_DIR \
$TASK_TO_RUN $test_set
+
+if $?
+then
+ truncate_log_to_test_failures
+fi
--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code
