The branch, master has been updated
via d5759794d6d add .gitlab-ci-coverage.yml for a scheduled build
via 7387da74e6f script/autobuild.py: split samba-ad-dc-4* tests into two
via 6f5546ae64f script/autobuild.py: split samba-nopython out of
samba-minimal-smbd again
via 95a9c3b2952 script/autobuild.py: split samba-no-opath into two tests
via 3189807278b script/autobuild.py: skip lcov step for samba-fips
via 45522798238 .gitlab-ci-main.yml: specify the image only by
SAMBA_CI_JOB_IMAGE
via 6999e080ce0 .gitlab-ci*.yml: only use gitlab.org shared runners if
possible
via a0a1988afb3 bootstrap/.gitlab-ci.yml: make sure we force gitlab.com
runners for now
via 23a54f4731c .gitlab-ci-main.yml: build coverity using
--with-cluster-support
via c378d4dde0a .gitlab-ci.yml: move the content to .gitlab-ci-main.yml
via a786ff99677 .gitlab-ci.yml: move samba-ad-back{1,2} and
samba-schemaupgrade to shared runners
via 63853b823c6 script/autobuild.py: split samba-ad-dc-backup into
samba-ad-back{1,2}
via c2a725b224b script/autobuild.py: move ad_dc_backup to samba-ad-dc-6
via 2e8b58bc2df .gitlab-ci.yml: let private runners also make use of
pre-builds
via 307edf82023 script/autobuild.py: split out
samba-{nt4,h5l,no-opath}-build
via e466bac9a8a .gitlab-ci.yml: be more resilient to intrastructure
failures
via 101237b4e6d .gitlab-ci.yml: specify explicit job timeouts
via 4df7f2b6914 .gitlab-ci.yml: split out samba-{def,mit}-build into
the build_first stage
via dc16294342d .gitlab-ci.yml: print out information of the available
cpus
via 37619d399b7 script/autobuild.py: split out "samba-{def,mit}-build"
via 95849d3d6a8 script/autobuild.py: add support for dependencies
via 404cd173e9e script/autobuild.py: store the directory for the
running builder in self.builder_dir
via e315ce40c24 script/autobuild.py: defer cp and git clone
via ba6f6a3cce1 script/autobuild.py: pass --with-selftest-prefix via
make instead of configure
via 2933c02718c script/autobuild.py: split out a CLEAN_SOURCE_TREE_CMD
via 0e7a7440f7c script/autobuild.py: change the task definitions into
an dictionary
via 467cfaf852e testprogs:blackbox: create temporary files under
$PREFIX/SELFTEST_TMPDIR
via d260d2c59bf s4:client:tests: create temporary files under
$PREFIX/SELFTEST_TMPDIR
via 3d618689e25 python:tests:samba_tool: create temporary files under
$SELFTEST_TMPDIR
via cef28acbd2b s3:script:tests: create temporary files under
$PREFIX/SELFTEST_TMPDIR
from 5b0d3b209a7 lib/audit_logging/test: fix typos
https://git.samba.org/?p=samba.git;a=shortlog;h=master
- Log -----------------------------------------------------------------
commit d5759794d6d384de02bbb5df9d46b3a8675813d0
Author: Stefan Metzmacher <me...@samba.org>
Date: Tue Dec 29 15:15:13 2020 +0100
add .gitlab-ci-coverage.yml for a scheduled build
This will be used by the https://gitlab.com/samba-team/samba
configuration, while https://gitlab.com/samba-team/devel/samba
will still use .gitlab-ci.yml (via the legacy .gitlab-ci-private.yml).
The key point is the usage of the more powerful n1-standard-2
runners for testing.
Signed-off-by: Stefan Metzmacher <me...@samba.org>
Reviewed-by: Andrew Bartlett <abart...@samba.org>
Reviewed-by: Andreas Schneider <a...@samba.org>
Autobuild-User(master): Stefan Metzmacher <me...@samba.org>
Autobuild-Date(master): Tue Apr 13 09:33:14 UTC 2021 on sn-devel-184
commit 7387da74e6f0e33de5f80b9a5e59f268541f52cd
Author: Stefan Metzmacher <me...@samba.org>
Date: Wed Dec 30 02:01:30 2020 +0100
script/autobuild.py: split samba-ad-dc-4* tests into two
As single job they used more than 1h.
Signed-off-by: Stefan Metzmacher <me...@samba.org>
Reviewed-by: Andrew Bartlett <abart...@samba.org>
Reviewed-by: Andreas Schneider <a...@samba.org>
commit 6f5546ae64f39f3d1a252c5095c09b92a09a70e6
Author: Stefan Metzmacher <me...@samba.org>
Date: Wed Dec 30 01:58:48 2020 +0100
script/autobuild.py: split samba-nopython out of samba-minimal-smbd again
This was using more than 1h as a single job.
Signed-off-by: Stefan Metzmacher <me...@samba.org>
Reviewed-by: Andrew Bartlett <abart...@samba.org>
Reviewed-by: Andreas Schneider <a...@samba.org>
commit 95a9c3b29529e60643eeae2ef99e8b0ef7d43710
Author: Stefan Metzmacher <me...@samba.org>
Date: Wed Dec 30 01:33:00 2020 +0100
script/autobuild.py: split samba-no-opath into two tests
This was is basically a combination of 'samba-nt4' and
'samba-fileserver'.
As a single job it used more than 1h only for testing,
while the samba-no-nopath-build uses ~ 10mins (with a filled ccache).
Now we have two test jobs with ~ 30mins.
Signed-off-by: Stefan Metzmacher <me...@samba.org>
Reviewed-by: Andrew Bartlett <abart...@samba.org>
Reviewed-by: Andreas Schneider <a...@samba.org>
commit 3189807278b0cb10007831ad84fa55c2cb14b125
Author: Stefan Metzmacher <me...@samba.org>
Date: Wed Dec 30 12:11:55 2020 +0100
script/autobuild.py: skip lcov step for samba-fips
This doesn't really work and only generates an empty samba-fips.info
file.
Someone familiar with gcov/lcov should look at this and fix it.
Signed-off-by: Stefan Metzmacher <me...@samba.org>
Reviewed-by: Andrew Bartlett <abart...@samba.org>
Reviewed-by: Andreas Schneider <a...@samba.org>
commit 455227982382d33225d579689c9e70688b19e551
Author: Stefan Metzmacher <me...@samba.org>
Date: Fri Apr 9 00:14:36 2021 +0200
.gitlab-ci-main.yml: specify the image only by SAMBA_CI_JOB_IMAGE
That way we can construct the url just in one place,
we can also add SAMBA_CI_JOB_IMAGE to the ccache identifier.
Signed-off-by: Stefan Metzmacher <me...@samba.org>
Reviewed-by: Andrew Bartlett <abart...@samba.org>
Reviewed-by: Andreas Schneider <a...@samba.org>
commit 6999e080ce0d5a0ab470daede88d7febafb49637
Author: Stefan Metzmacher <me...@samba.org>
Date: Tue Dec 29 10:21:51 2020 +0100
.gitlab-ci*.yml: only use gitlab.org shared runners if possible
We no longer fallback to our private runner, lets see how that works
out...
Signed-off-by: Stefan Metzmacher <me...@samba.org>
Reviewed-by: Andrew Bartlett <abart...@samba.org>
Reviewed-by: Andreas Schneider <a...@samba.org>
commit a0a1988afb3aa4b0064919ad02a1ef7f38f44d80
Author: Stefan Metzmacher <me...@samba.org>
Date: Thu Apr 8 23:12:39 2021 +0200
bootstrap/.gitlab-ci.yml: make sure we force gitlab.com runners for now
We've just added our own runners with 'docker' and 'gce'.
Signed-off-by: Stefan Metzmacher <me...@samba.org>
Reviewed-by: Andrew Bartlett <abart...@samba.org>
Reviewed-by: Andreas Schneider <a...@samba.org>
commit 23a54f4731c12333c2a62ed80c5abbf3ca054492
Author: Stefan Metzmacher <me...@samba.org>
Date: Wed Dec 30 12:02:05 2020 +0100
.gitlab-ci-main.yml: build coverity using --with-cluster-support
Signed-off-by: Stefan Metzmacher <me...@samba.org>
Reviewed-by: Andrew Bartlett <abart...@samba.org>
Reviewed-by: Andreas Schneider <a...@samba.org>
commit c378d4dde0a550c70a8d75bbb0e2b002b26477df
Author: Stefan Metzmacher <me...@samba.org>
Date: Tue Dec 29 14:45:55 2020 +0100
.gitlab-ci.yml: move the content to .gitlab-ci-main.yml
We introduce an indirection from
gitlab-ci.yml via .gitlab-ci-default.yml to .gitlab-ci-main.yml
We do that in order to introduce a .gitlab-ci-coverage.yml later
as that will have to use different settings in future.
Signed-off-by: Stefan Metzmacher <me...@samba.org>
Reviewed-by: Andrew Bartlett <abart...@samba.org>
Reviewed-by: Andreas Schneider <a...@samba.org>
commit a786ff99677bd22dc0499a01027f003bf8bfae27
Author: Stefan Metzmacher <me...@samba.org>
Date: Tue Dec 29 01:19:31 2020 +0100
.gitlab-ci.yml: move samba-ad-back{1,2} and samba-schemaupgrade to shared
runners
This seems to work quite reliable now.
Signed-off-by: Stefan Metzmacher <me...@samba.org>
Reviewed-by: Andrew Bartlett <abart...@samba.org>
Reviewed-by: Andreas Schneider <a...@samba.org>
commit 63853b823c6a263c69c06f91163bcada4f3646c9
Author: Stefan Metzmacher <me...@samba.org>
Date: Mon Dec 28 15:56:57 2020 +0100
script/autobuild.py: split samba-ad-dc-backup into samba-ad-back{1,2}
This will make it possible to run them in parallel (hopefully on shared
gitlab runners).
Signed-off-by: Stefan Metzmacher <me...@samba.org>
Reviewed-by: Andrew Bartlett <abart...@samba.org>
Reviewed-by: Andreas Schneider <a...@samba.org>
commit c2a725b224b8a0abb6123b15e558e0bc6727aefb
Author: Stefan Metzmacher <me...@samba.org>
Date: Mon Dec 28 15:38:52 2020 +0100
script/autobuild.py: move ad_dc_backup to samba-ad-dc-6
Signed-off-by: Stefan Metzmacher <me...@samba.org>
Reviewed-by: Andrew Bartlett <abart...@samba.org>
Reviewed-by: Andreas Schneider <a...@samba.org>
commit 2e8b58bc2df6929bbaf2037cf7a21467bd542ca7
Author: Stefan Metzmacher <me...@samba.org>
Date: Mon Dec 28 15:47:27 2020 +0100
.gitlab-ci.yml: let private runners also make use of pre-builds
Signed-off-by: Stefan Metzmacher <me...@samba.org>
Reviewed-by: Andrew Bartlett <abart...@samba.org>
Reviewed-by: Andreas Schneider <a...@samba.org>
commit 307edf82023545cb803b7e23bd43bfe631bbf2a0
Author: Stefan Metzmacher <me...@samba.org>
Date: Mon Dec 28 17:12:39 2020 +0100
script/autobuild.py: split out samba-{nt4,h5l,no-opath}-build
These will be used to move the build stages from private gitlab runner
jobs to shared runners.
Signed-off-by: Stefan Metzmacher <me...@samba.org>
Reviewed-by: Andrew Bartlett <abart...@samba.org>
Reviewed-by: Andreas Schneider <a...@samba.org>
commit e466bac9a8addf5c35d287874078a940a0e72916
Author: Stefan Metzmacher <me...@samba.org>
Date: Wed Dec 23 16:49:19 2020 +0100
.gitlab-ci.yml: be more resilient to intrastructure failures
Signed-off-by: Stefan Metzmacher <me...@samba.org>
Reviewed-by: Andrew Bartlett <abart...@samba.org>
Reviewed-by: Andreas Schneider <a...@samba.org>
commit 101237b4e6dd1401da41f26af02f1741c014b1ac
Author: Stefan Metzmacher <me...@samba.org>
Date: Wed Dec 23 16:40:43 2020 +0100
.gitlab-ci.yml: specify explicit job timeouts
Signed-off-by: Stefan Metzmacher <me...@samba.org>
Reviewed-by: Andrew Bartlett <abart...@samba.org>
Reviewed-by: Andreas Schneider <a...@samba.org>
commit 4df7f2b6914645ca61edf05e8ef1fd4fd66cf1cb
Author: Stefan Metzmacher <me...@samba.org>
Date: Fri Nov 20 12:25:33 2020 +0100
.gitlab-ci.yml: split out samba-{def,mit}-build into the build_first stage
It's enough to have 2 jobs in build_first, so we can move 'samba-fips'
out of it again.
Signed-off-by: Stefan Metzmacher <me...@samba.org>
Reviewed-by: Andrew Bartlett <abart...@samba.org>
Reviewed-by: Andreas Schneider <a...@samba.org>
commit dc16294342d4a92f744816caaa0689db996cb658
Author: Stefan Metzmacher <me...@samba.org>
Date: Fri Nov 20 23:20:25 2020 +0100
.gitlab-ci.yml: print out information of the available cpus
Signed-off-by: Stefan Metzmacher <me...@samba.org>
Reviewed-by: Andrew Bartlett <abart...@samba.org>
Reviewed-by: Andreas Schneider <a...@samba.org>
commit 37619d399b7d8b5e3f6f15be913a73bba9abeb1d
Author: Stefan Metzmacher <me...@samba.org>
Date: Thu Nov 19 15:23:00 2020 +0100
script/autobuild.py: split out "samba-{def,mit}-build"
This means we avoid a lot of cpu usage for the build.
Signed-off-by: Stefan Metzmacher <me...@samba.org>
Reviewed-by: Andrew Bartlett <abart...@samba.org>
Reviewed-by: Andreas Schneider <a...@samba.org>
commit 95849d3d6a8abf09f0059e87a9051d07604666e8
Author: Stefan Metzmacher <me...@samba.org>
Date: Thu Dec 17 11:53:21 2020 +0100
script/autobuild.py: add support for dependencies
This will make it possible to split build and test stages
in the next steps.
Signed-off-by: Stefan Metzmacher <me...@samba.org>
Reviewed-by: Andrew Bartlett <abart...@samba.org>
Reviewed-by: Andreas Schneider <a...@samba.org>
commit 404cd173e9e166c5b7c57c6cfc3e0366d7672893
Author: Stefan Metzmacher <me...@samba.org>
Date: Thu Dec 17 11:53:21 2020 +0100
script/autobuild.py: store the directory for the running builder in
self.builder_dir
For now it keeps being the same as self.test_source_dir, but that will
change soon.
Signed-off-by: Stefan Metzmacher <me...@samba.org>
Reviewed-by: Andrew Bartlett <abart...@samba.org>
Reviewed-by: Andreas Schneider <a...@samba.org>
commit e315ce40c249c6cd24c07f53b1eb8a8a8f77a175
Author: Stefan Metzmacher <me...@samba.org>
Date: Thu Nov 19 15:11:39 2020 +0100
script/autobuild.py: defer cp and git clone
This will make further reconstruction easier.
Use 'git show -w' to see the minimal diff.
Signed-off-by: Stefan Metzmacher <me...@samba.org>
Reviewed-by: Andrew Bartlett <abart...@samba.org>
Reviewed-by: Andreas Schneider <a...@samba.org>
commit ba6f6a3cce1b55026ebbfae3ceccb5ed5c3e00c5
Author: Stefan Metzmacher <me...@samba.org>
Date: Thu Nov 19 15:01:04 2020 +0100
script/autobuild.py: pass --with-selftest-prefix via make instead of
configure
We do start for almost all jobs, just samba-ctdb keeps passing it via
configure in order to have a regression test for it.
Signed-off-by: Stefan Metzmacher <me...@samba.org>
Reviewed-by: Andrew Bartlett <abart...@samba.org>
Reviewed-by: Andreas Schneider <a...@samba.org>
commit 2933c02718c81d858e73dca08e5237efc36a3e28
Author: Stefan Metzmacher <me...@samba.org>
Date: Thu Nov 19 14:41:16 2020 +0100
script/autobuild.py: split out a CLEAN_SOURCE_TREE_CMD
This works for all cases even if the builder runs in a subdirectory.
Signed-off-by: Stefan Metzmacher <me...@samba.org>
Reviewed-by: Andrew Bartlett <abart...@samba.org>
Reviewed-by: Andreas Schneider <a...@samba.org>
commit 0e7a7440f7cc44a8e18942048527b3152e26e0eb
Author: Stefan Metzmacher <me...@samba.org>
Date: Thu Nov 19 11:49:25 2020 +0100
script/autobuild.py: change the task definitions into an dictionary
The will make it easier to add more meta data properties to task
definitions.
Use 'git show -w' to see the minimal diff.
Signed-off-by: Stefan Metzmacher <me...@samba.org>
Reviewed-by: Andrew Bartlett <abart...@samba.org>
Reviewed-by: Andreas Schneider <a...@samba.org>
commit 467cfaf852e67a283c453b71007a10ee7f9cf0d4
Author: Stefan Metzmacher <me...@samba.org>
Date: Sun Nov 22 20:20:56 2020 +0100
testprogs:blackbox: create temporary files under $PREFIX/SELFTEST_TMPDIR
Tests should not create files in the build nor the source directory!
Signed-off-by: Stefan Metzmacher <me...@samba.org>
Reviewed-by: Andrew Bartlett <abart...@samba.org>
Reviewed-by: Andreas Schneider <a...@samba.org>
commit d260d2c59bf7a7026e3c4fd07209c32f6764210d
Author: Stefan Metzmacher <me...@samba.org>
Date: Mon Nov 23 09:58:30 2020 +0100
s4:client:tests: create temporary files under $PREFIX/SELFTEST_TMPDIR
Tests should not create files in the build nor the source directory!
Signed-off-by: Stefan Metzmacher <me...@samba.org>
Reviewed-by: Andrew Bartlett <abart...@samba.org>
Reviewed-by: Andreas Schneider <a...@samba.org>
commit 3d618689e254429cd2c36132af0c0b3904c1d7e4
Author: Stefan Metzmacher <me...@samba.org>
Date: Sun Nov 22 20:14:19 2020 +0100
python:tests:samba_tool: create temporary files under $SELFTEST_TMPDIR
Tests should not create files in the build nor the source directory!
Signed-off-by: Stefan Metzmacher <me...@samba.org>
Reviewed-by: Andrew Bartlett <abart...@samba.org>
Reviewed-by: Andreas Schneider <a...@samba.org>
commit cef28acbd2b3295a5d79575d6598d5d1fb92f005
Author: Stefan Metzmacher <me...@samba.org>
Date: Thu Nov 19 17:10:02 2020 +0000
s3:script:tests: create temporary files under $PREFIX/SELFTEST_TMPDIR
Tests should not create files in the build nor the source directory!
They should cope with read only access to them.
Signed-off-by: Stefan Metzmacher <me...@samba.org>
Reviewed-by: Andrew Bartlett <abart...@samba.org>
Reviewed-by: Andreas Schneider <a...@samba.org>
-----------------------------------------------------------------------
Summary of changes:
.gitlab-ci-coverage-runners.yml | 10 +
.gitlab-ci-coverage.yml | 12 +
.gitlab-ci-default-runners.yml | 52 +
.gitlab-ci-default.yml | 9 +
.gitlab-ci.yml => .gitlab-ci-main.yml | 323 ++++--
.gitlab-ci-private.yml | 3 +
.gitlab-ci.yml | 367 +------
bootstrap/.gitlab-ci.yml | 4 +
bootstrap/sha1sum.txt | 2 +-
python/samba/tests/samba_tool/computer_edit.sh | 2 +-
python/samba/tests/samba_tool/contact_edit.sh | 2 +-
python/samba/tests/samba_tool/group_edit.sh | 2 +-
python/samba/tests/samba_tool/user_edit.sh | 2 +-
script/autobuild.py | 1031 +++++++++++++-------
source3/script/tests/test_aio_outstanding.sh | 3 +
source3/script/tests/test_close_denied_share.sh | 2 +
.../script/tests/test_durable_handle_reconnect.sh | 2 +
source3/script/tests/test_forceuser_validusers.sh | 1 +
source3/script/tests/test_large_acl.sh | 2 +
source3/script/tests/test_local_s3.sh | 5 +-
source3/script/tests/test_netfileenum.sh | 2 +
source3/script/tests/test_open_eintr.sh | 2 +
source3/script/tests/test_preserve_case.sh | 2 +
source3/script/tests/test_registry_upgrade.sh | 16 +-
source3/script/tests/test_shadow_copy_torture.sh | 2 +
.../script/tests/test_smb1_shadow_copy_torture.sh | 2 +
source3/script/tests/test_smbclient_mget.sh | 2 +
source3/script/tests/test_smbclient_s3.sh | 9 +-
source3/script/tests/test_smbget.sh | 2 +-
source3/script/tests/test_zero_data.sh | 2 +
source4/client/tests/test_cifsdd.sh | 16 +-
source4/client/tests/test_smbclient.sh | 32 +-
testprogs/blackbox/test_pdbtest.sh | 6 +-
testprogs/blackbox/test_pkinit_pac_heimdal.sh | 2 +-
34 files changed, 1070 insertions(+), 863 deletions(-)
create mode 100644 .gitlab-ci-coverage-runners.yml
create mode 100644 .gitlab-ci-coverage.yml
create mode 100644 .gitlab-ci-default-runners.yml
create mode 100644 .gitlab-ci-default.yml
copy .gitlab-ci.yml => .gitlab-ci-main.yml (51%)
Changeset truncated at 500 lines:
diff --git a/.gitlab-ci-coverage-runners.yml b/.gitlab-ci-coverage-runners.yml
new file mode 100644
index 00000000000..0f6b2ec1581
--- /dev/null
+++ b/.gitlab-ci-coverage-runners.yml
@@ -0,0 +1,10 @@
+include:
+ - /.gitlab-ci-default-runners.yml
+
+.shared_runner_test:
+ # We need the more powerful n1-standard-2 runners
+ # in order to handle the lcov overhead.
+ #
+ # See .gitlab-ci-default-runners.yml for more details
+ tags:
+ - gitlab-org-docker
diff --git a/.gitlab-ci-coverage.yml b/.gitlab-ci-coverage.yml
new file mode 100644
index 00000000000..8a9ded8f6ca
--- /dev/null
+++ b/.gitlab-ci-coverage.yml
@@ -0,0 +1,12 @@
+# This is just used for the scheduled pipelines in the
+# https://gitlab.com/samba-team/samba configuration
+#
+
+variables:
+ SAMBA_CI_FLAVOR: "coverage"
+ # "--enable-coverage" or ""
+ SAMBA_CI_AUTOBUILD_ENABLE_COVERAGE: "--enable-coverage"
+
+include:
+ - /.gitlab-ci-coverage-runners.yml
+ - /.gitlab-ci-main.yml
diff --git a/.gitlab-ci-default-runners.yml b/.gitlab-ci-default-runners.yml
new file mode 100644
index 00000000000..2dea6e82c49
--- /dev/null
+++ b/.gitlab-ci-default-runners.yml
@@ -0,0 +1,52 @@
+# From https://docs.gitlab.com/ee/user/gitlab_com/#shared-runners:
+#
+# ...
+#
+# All your CI/CD jobs run on n1-standard-1 instances with 3.75GB of RAM,
CoreOS
+# and the latest Docker Engine installed. Instances provide 1 vCPU and 25GB
of
+# HDD disk space. The default region of the VMs is US East1. Each instance is
+# used only for one job, this ensures any sensitive data left on the system
can’t
+# be accessed by other people their CI jobs.
+#
+# The gitlab-shared-runners-manager-X.gitlab.com fleet of runners are
dedicated
+# for GitLab projects as well as community forks of them. They use a slightly
+# larger machine type (n1-standard-2) and have a bigger SSD disk size. They
don’t
+# run untagged jobs and unlike the general fleet of shared runners, the
instances
+# are re-used up to 40 times.
+#
+# ...
+#
+# The n1-standard-1 runners seem to be tagged with 'docker' together with
'gce'.
+#
+# The more powerful n1-standard-2 runners seem to be tagged with
+# 'gitlab-org-docker' or some with just 'gitlab-org'.
+#
+#
+# Our current private runner 'docker', 'samba-ci-private', 'shared' and
+# 'ubuntu1804'. It runs with an ubuntu1804 kernel and privides an ext4
filesystem
+# and similar RAM as the n1-standard-2 runners.
+#
+
+.shared_runner_build:
+ # We use n1-standard-1 shared runners by default.
+ #
+ # There are currently 5 shared runners with 'docker' and 'gce',
+ # while there are only 2 provising 'docker' together with 'shared'.
+ #
+ # We used to fallback to our private runner if the docker+shared runners
+ # were busy, but now that we use the 5 docker+gce runners, we try to only
+ # use shared runners without a fallback to our private runner!
+ # Lets see how that will work out.
+ tags:
+ - docker
+ - gce
+
+.shared_runner_test:
+ # Currently we're fine using the n1-standard-1 runners also for testing
+ extends: .shared_runner_build
+
+.private_runner_test:
+ # We use our private runner only for special tests
+ tags:
+ - docker
+ - samba-ci-private
diff --git a/.gitlab-ci-default.yml b/.gitlab-ci-default.yml
new file mode 100644
index 00000000000..d0831017d9b
--- /dev/null
+++ b/.gitlab-ci-default.yml
@@ -0,0 +1,9 @@
+variables:
+ SAMBA_CI_FLAVOR: "default"
+ # "--enable-coverage" or ""
+ # See .gitlab-ci-coverage.yml
+ SAMBA_CI_AUTOBUILD_ENABLE_COVERAGE: ""
+
+include:
+ - /.gitlab-ci-default-runners.yml
+ - /.gitlab-ci-main.yml
diff --git a/.gitlab-ci.yml b/.gitlab-ci-main.yml
similarity index 51%
copy from .gitlab-ci.yml
copy to .gitlab-ci-main.yml
index eafd454d445..cbc812648e8 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci-main.yml
@@ -3,23 +3,33 @@
# Stages explained
#
# images: Build the images with the bootstrap script
-# build_first: Build one thing first to find silly errors (fast job)
+# build_first: Build a few things first to find silly errors (fast job)
# (don't pay for 35 machines until something compiles)
# build: The main parallel job
# (keep these to 1hour as we are billed per hour)
+# test_only: Tests using the build from prior stages, these typically
+# have an explicit dependency defined to a specific build job,
+# which means that start as soon as the build job finished.
+# test_private: Like test_only, but running on private runners
# report: Code coverage reporting
stages:
- images
- build_first
- build
+ - test_only
+ - test_private
- report
variables:
+ # We want to be resilient to runner failures
+ ARTIFACT_DOWNLOAD_ATTEMPTS: "3"
+ EXECUTOR_JOB_SECTION_ATTEMPTS: "3"
+ GET_SOURCES_ATTEMPTS: "3"
+ RESTORE_CACHE_ATTEMPTS: "3"
+ #
GIT_STRATEGY: fetch
GIT_DEPTH: "3"
- # "--enable-coverage" or ""
- SAMBA_CI_AUTOBUILD_ENABLE_COVERAGE: ""
#
# we run autobuild.py inside a samba CI docker image located on gitlab's
registry
# overwrite this variable if you want use your own image registry.
@@ -32,7 +42,7 @@ variables:
# Set this to the contents of bootstrap/sha1sum.txt
# which is generated by bootstrap/template.py --render
#
- SAMBA_CI_CONTAINER_TAG: c4c00eb35cae36d8d6e752ee01fd943432993721
+ SAMBA_CI_CONTAINER_TAG: f25e1fa0a828f428ade0e4f37d8a646ff1e207d4
#
# We use the ubuntu1804 image as default as
# it matches what we have on sn-devel-184.
@@ -43,43 +53,49 @@ variables:
# Please see the samba-o3 sections at the end of this file!
# We should run that for each available image
#
- SAMBA_CI_CONTAINER_IMAGE_ubuntu1604:
${SAMBA_CI_CONTAINER_REGISTRY}/samba-ci-ubuntu1604:${SAMBA_CI_CONTAINER_TAG}
- SAMBA_CI_CONTAINER_IMAGE_ubuntu1804:
${SAMBA_CI_CONTAINER_REGISTRY}/samba-ci-ubuntu1804:${SAMBA_CI_CONTAINER_TAG}
- SAMBA_CI_CONTAINER_IMAGE_ubuntu2004:
${SAMBA_CI_CONTAINER_REGISTRY}/samba-ci-ubuntu2004:${SAMBA_CI_CONTAINER_TAG}
- SAMBA_CI_CONTAINER_IMAGE_debian9:
${SAMBA_CI_CONTAINER_REGISTRY}/samba-ci-debian9:${SAMBA_CI_CONTAINER_TAG}
- SAMBA_CI_CONTAINER_IMAGE_debian10:
${SAMBA_CI_CONTAINER_REGISTRY}/samba-ci-debian10:${SAMBA_CI_CONTAINER_TAG}
- SAMBA_CI_CONTAINER_IMAGE_opensuse151:
${SAMBA_CI_CONTAINER_REGISTRY}/samba-ci-opensuse151:${SAMBA_CI_CONTAINER_TAG}
- SAMBA_CI_CONTAINER_IMAGE_opensuse152:
${SAMBA_CI_CONTAINER_REGISTRY}/samba-ci-opensuse152:${SAMBA_CI_CONTAINER_TAG}
- SAMBA_CI_CONTAINER_IMAGE_fedora32:
${SAMBA_CI_CONTAINER_REGISTRY}/samba-ci-fedora32:${SAMBA_CI_CONTAINER_TAG}
- SAMBA_CI_CONTAINER_IMAGE_fedora33:
${SAMBA_CI_CONTAINER_REGISTRY}/samba-ci-fedora33:${SAMBA_CI_CONTAINER_TAG}
- SAMBA_CI_CONTAINER_IMAGE_centos7:
${SAMBA_CI_CONTAINER_REGISTRY}/samba-ci-centos7:${SAMBA_CI_CONTAINER_TAG}
- SAMBA_CI_CONTAINER_IMAGE_centos8:
${SAMBA_CI_CONTAINER_REGISTRY}/samba-ci-centos8:${SAMBA_CI_CONTAINER_TAG}
+ SAMBA_CI_CONTAINER_IMAGE_ubuntu1604: ubuntu1604
+ SAMBA_CI_CONTAINER_IMAGE_ubuntu1804: ubuntu1804
+ SAMBA_CI_CONTAINER_IMAGE_ubuntu2004: ubuntu2004
+ SAMBA_CI_CONTAINER_IMAGE_debian9: debian9
+ SAMBA_CI_CONTAINER_IMAGE_debian10: debian10
+ SAMBA_CI_CONTAINER_IMAGE_opensuse151: opensuse151
+ SAMBA_CI_CONTAINER_IMAGE_opensuse152: opensuse152
+ SAMBA_CI_CONTAINER_IMAGE_fedora32: fedora32
+ SAMBA_CI_CONTAINER_IMAGE_fedora33: fedora33
+ SAMBA_CI_CONTAINER_IMAGE_centos7: centos7
+ SAMBA_CI_CONTAINER_IMAGE_centos8: centos8
include:
# The image creation details are specified in a separate file
# See bootstrap/README.md for details
- 'bootstrap/.gitlab-ci.yml'
+.shared_runner_build_image:
+ extends: .shared_runner_build
+ variables:
+ SAMBA_CI_JOB_IMAGE: ${SAMBA_CI_CONTAINER_IMAGE}
+ image:
${SAMBA_CI_CONTAINER_REGISTRY}/samba-ci-${SAMBA_CI_JOB_IMAGE}:${SAMBA_CI_CONTAINER_TAG}
+
.shared_template:
+ extends: .shared_runner_build_image
# All Samba jobs are interruptible, this avoids burning CPU when a
# newer branch is pushed.
interruptible: true
+ timeout: 2h
variables:
AUTOBUILD_JOB_NAME: $CI_JOB_NAME
- image:
${SAMBA_CI_CONTAINER_REGISTRY}/samba-ci-${SAMBA_CI_CONTAINER_IMAGE}:${SAMBA_CI_CONTAINER_TAG}
stage: build
- tags:
- - docker
- - shared
cache:
- key: ccache.${CI_JOB_NAME}
+ key: ccache.${CI_JOB_NAME}.${SAMBA_CI_JOB_IMAGE}.${SAMBA_CI_FLAVOR}
paths:
- ccache
before_script:
- uname -a
- lsb_release -a
- cat /etc/os-release
+ - lscpu
+ - cat /proc/cpuinfo
- mount
- df -h
- cat /proc/swaps
@@ -89,11 +105,14 @@ include:
# See bootstrap/.gitlab-ci.yml how to generate a new image
- echo "SAMBA_CI_CONTAINER_REGISTRY[${SAMBA_CI_CONTAINER_REGISTRY}]"
- echo "SAMBA_CI_CONTAINER_TAG[${SAMBA_CI_CONTAINER_TAG}]"
+ - echo "SAMBA_CI_JOB_IMAGE[${SAMBA_CI_JOB_IMAGE}]"
+ - echo "CI_JOB_IMAGE[${CI_JOB_IMAGE}]"
- bootstrap/template.py --sha1sum > /tmp/sha1sum-template.txt
- diff -u bootstrap/sha1sum.txt /tmp/sha1sum-template.txt
- echo "${SAMBA_CI_CONTAINER_TAG}" > /tmp/sha1sum-tag.txt
- diff -u bootstrap/sha1sum.txt /tmp/sha1sum-tag.txt
- diff -u bootstrap/sha1sum.txt /sha1sum.txt
+ - echo "${CI_COMMIT_SHA} ${CI_COMMIT_TITLE}" > /tmp/commit.txt
- export CCACHE_BASEDIR="${PWD}"
- export CCACHE_DIR="${PWD}/ccache" && mkdir -pv "$CCACHE_DIR"
- export CC="ccache cc"
@@ -118,6 +137,14 @@ include:
when:
- runner_system_failure
- stuck_or_timeout_failure
+ - api_failure
+ - runner_unsupported
+ - stale_schedule
+ - job_execution_timeout
+ - archived_failure
+ - scheduler_failure
+ - data_integrity_failure
+
script:
# gitlab predefines CI_JOB_NAME for each job. The gitlab job usually
matches the
# autobuild name, which means we can define a default template that runs
most autobuild jobs
@@ -139,6 +166,108 @@ others:
- script/autobuild.py samba-xc $SAMBA_CI_AUTOBUILD_ENABLE_COVERAGE
--verbose --nocleanup --keeplogs --tail --full-testbase
/tmp/samba-testbase/samba-xc
- script/autobuild.py docs-xml $SAMBA_CI_AUTOBUILD_ENABLE_COVERAGE
--verbose --nocleanup --keeplogs --tail --full-testbase
/tmp/samba-testbase/docs-xml
+.shared_template_build_only:
+ extends: .shared_template
+ timeout: 45m
+ artifacts:
+ expire_in: 1 week
+ paths:
+ - "*.stdout"
+ - "*.stderr"
+ - "*.info"
+ - system-info.txt
+ - samba-testbase.tar.gz
+ script:
+ # gitlab predefines CI_JOB_NAME for each job. The gitlab job usually
matches the
+ # autobuild name, which means we can define a default template that runs
most autobuild jobs
+ - script/autobuild.py $AUTOBUILD_JOB_NAME
$SAMBA_CI_AUTOBUILD_ENABLE_COVERAGE --verbose --nocleanup --keeplogs --tail
--full-testbase /tmp/samba-testbase
+ # On success we need to pack everything into an artifacts file
+ # which needs to be in the git checkout.
+ # As tar doesn't handle hardlink of read-only files,
+ # we remember the acls and add write permissions
+ # before creating the archive. The consumer will apply
+ # the acls again.
+ - cp -a /sha1sum.txt /tmp/samba-testbase/image-sha1sum.txt
+ - cp -a /tmp/commit.txt /tmp/samba-testbase/commit.txt
+ - ln -s /tmp/samba-testbase/${AUTOBUILD_JOB_NAME}/
/tmp/samba-testbase/build_subdir_link
+ - pushd /tmp && getfacl -R samba-testbase > samba-testbase.acl.dump && popd
+ - chmod -R +w /tmp/samba-testbase
+ - mv /tmp/samba-testbase.acl.dump /tmp/samba-testbase/
+ - tar cfz samba-testbase.tar.gz /tmp/samba-testbase
+ - ls -la samba-testbase.tar.gz
+ - sha1sum samba-testbase.tar.gz
+
+.shared_template_test_only:
+ extends:
+ - .shared_template
+ - .shared_runner_test
+ stage: test_only
+ script:
+ # We unpack the artifacts file created by the .shared_template_build_only
+ # run we depend on
+ - ls -la samba-testbase.tar.gz
+ - sha1sum samba-testbase.tar.gz
+ - tar xfz samba-testbase.tar.gz -C /
+ - diff -u /tmp/samba-testbase/image-sha1sum.txt /sha1sum.txt
+ - diff -u /tmp/samba-testbase/commit.txt /tmp/commit.txt
+ - mv /tmp/samba-testbase/samba-testbase.acl.dump
/tmp/samba-testbase.acl.dump
+ - pushd /tmp && setfacl --restore=/tmp/samba-testbase.acl.dump && popd
+ - ls -la /tmp/samba-testbase/
+ - ls -la /tmp/samba-testbase/build_subdir_link
+ - ls -la /tmp/samba-testbase/build_subdir_link/
+ - if [ -n "$SAMBA_CI_AUTOBUILD_ENABLE_COVERAGE" ]; then find
/tmp/samba-testbase/build_subdir_link/ -type d -printf "'%p'\n" | xargs chmod
u+w; fi
+ - ls -la /tmp/samba-testbase/build_subdir_link/
+ # gitlab predefines CI_JOB_NAME for each job. The gitlab job usually
matches the
+ # autobuild name, which means we can define a default template that runs
most autobuild jobs
+ - script/autobuild.py $AUTOBUILD_JOB_NAME
$SAMBA_CI_AUTOBUILD_ENABLE_COVERAGE --skip-dependencies --verbose --nocleanup
--keeplogs --tail --full-testbase /tmp/samba-testbase
+
+samba-def-build:
+ extends: .shared_template_build_only
+ stage: build_first
+
+.needs_samba-def-build:
+ extends: .shared_template_test_only
+ needs:
+ - job: samba-def-build
+ artifacts: true
+
+samba-mit-build:
+ extends: .shared_template_build_only
+ stage: build_first
+
+.needs_samba-mit-build:
+ extends: .shared_template_test_only
+ needs:
+ - job: samba-mit-build
+ artifacts: true
+
+samba-h5l-build:
+ extends: .shared_template_build_only
+
+.needs_samba-h5l-build:
+ extends: .shared_template_test_only
+ needs:
+ - job: samba-h5l-build
+ artifacts: true
+
+samba-nt4-build:
+ extends: .shared_template_build_only
+
+.needs_samba-nt4-build:
+ extends: .shared_template_test_only
+ needs:
+ - job: samba-nt4-build
+ artifacts: true
+
+samba-no-opath-build:
+ extends: .shared_template_build_only
+
+.needs_samba-no-opath-build:
+ extends: .shared_template_test_only
+ needs:
+ - job: samba-no-opath-build
+ artifacts: true
+
samba:
extends: .shared_template
@@ -148,30 +277,46 @@ samba-mitkrb5:
samba-minimal-smbd:
extends: .shared_template
-samba-admem:
+samba-nopython:
extends: .shared_template
+samba-admem:
+ extends: .needs_samba-def-build
+
samba-ad-dc-2:
- extends: .shared_template
+ extends: .needs_samba-def-build
samba-ad-dc-3:
- extends: .shared_template
+ extends: .needs_samba-def-build
-samba-ad-dc-4:
- extends: .shared_template
+samba-ad-dc-4a:
+ extends: .needs_samba-def-build
+
+samba-ad-dc-4b:
+ extends: .needs_samba-def-build
samba-ad-dc-5:
- extends: .shared_template
+ extends: .needs_samba-def-build
samba-ad-dc-6:
- extends: .shared_template
+ extends: .needs_samba-def-build
+
+samba-ad-back1:
+ extends: .needs_samba-def-build
+
+samba-ad-back2:
+ extends: .needs_samba-def-build
+
+samba-schemaupgrade:
+ extends: .needs_samba-def-build
samba-libs:
extends: .shared_template
samba-fuzz:
extends: .shared_template
- image:
${SAMBA_CI_CONTAINER_REGISTRY}/samba-ci-ubuntu1604:${SAMBA_CI_CONTAINER_TAG}
+ variables:
+ SAMBA_CI_JOB_IMAGE: ${SAMBA_CI_CONTAINER_IMAGE_ubuntu1604}
ctdb:
extends: .shared_template
@@ -180,26 +325,27 @@ samba-ctdb:
extends: .shared_template
samba-ad-dc-ntvfs:
- extends: .shared_template
+ extends: .needs_samba-def-build
samba-admem-mit:
- extends: .shared_template
+ extends: .needs_samba-mit-build
-samba-ad-dc-4-mitkrb5:
- extends: .shared_template
+samba-ad-dc-4a-mitkrb5:
+ extends: .needs_samba-mit-build
+
+samba-ad-dc-4b-mitkrb5:
+ extends: .needs_samba-mit-build
# This task is run first to ensure we compile before we start the
# main run as it is the fastest full compile of Samba.
samba-fips:
- stage: build_first
extends: .shared_template
- image: $SAMBA_CI_CONTAINER_IMAGE_fedora33
+ variables:
+ SAMBA_CI_JOB_IMAGE: ${SAMBA_CI_CONTAINER_IMAGE_fedora33}
-.private_template:
- extends: .shared_template
- tags:
- - docker
- - samba-ci-private
+.private_test_only:
+ extends: .private_runner_test
+ stage: test_private
only:
variables:
# These jobs are only run if the gitlab repo has private runners
available.
@@ -208,34 +354,53 @@ samba-fips:
# settings -> CI/CD -> Environment variables
- $SUPPORT_PRIVATE_TEST == "yes"
-samba-ad-dc-backup:
- extends: .private_template
+.needs_samba-def-build-private:
+ extends:
+ - .needs_samba-def-build
+ - .private_test_only
+
+.needs_samba-mit-build-private:
+ extends:
+ - .needs_samba-mit-build
+ - .private_test_only
+
+.needs_samba-h5l-build-private:
+ extends:
+ - .needs_samba-h5l-build
+ - .private_test_only
+
+.needs_samba-nt4-build-private:
+ extends:
+ - .needs_samba-nt4-build
+ - .private_test_only
+
+.needs_samba-no-opath-build-private:
+ extends:
+ - .needs_samba-no-opath-build
+ - .private_test_only
samba-fileserver:
- extends: .private_template
+ extends: .needs_samba-h5l-build-private
samba-ad-dc-1:
- extends: .private_template
+ extends: .needs_samba-def-build-private
samba-nt4:
- extends: .private_template
--
Samba Shared Repository
