This is an automated email from the ASF dual-hosted git repository.
aw pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/yetus.git
The following commit(s) were added to refs/heads/master by this push:
new 8c3c818 YETUS-683. Publish apache/yetus docker images on
hub.docker.com
8c3c818 is described below
commit 8c3c818285ee4e03bf31c56f678924ca37cfaf6e
Author: Allen Wittenauer <[email protected]>
AuthorDate: Sun Nov 11 22:26:26 2018 -0800
YETUS-683. Publish apache/yetus docker images on hub.docker.com
Signed-off-by: Allen Wittenauer <[email protected]>
---
.../documentation/in-progress/precommit-basic.md | 6 +-
.../documentation/in-progress/precommit-robots.md | 59 ++++++++++++++-
asf-site-src/source/downloads.html.md.erb | 2 +-
asf-site-src/source/yetus-docker-image.md | 86 ++++++++++++++++++++++
4 files changed, 147 insertions(+), 6 deletions(-)
diff --git a/asf-site-src/source/documentation/in-progress/precommit-basic.md
b/asf-site-src/source/documentation/in-progress/precommit-basic.md
index 5e24d28..63d25f2 100644
--- a/asf-site-src/source/documentation/in-progress/precommit-basic.md
+++ b/asf-site-src/source/documentation/in-progress/precommit-basic.md
@@ -84,7 +84,7 @@ Build Tools:
Automation and Isolation:
* [Circle CI](https://www.circleci.com)
-* [Docker](https://www.docker.com) version 1.6.0+
+* [Docker](https://www.docker.com) version 1.7.0+
* [Gitlab CI](https://www.gitlab.com)
* [Jenkins](https://www.jenkins-ci.org)
* [Travis CI](https://www.travis-ci.com)
@@ -329,10 +329,10 @@ NOTE: JAVA\_HOME is always appended to the list of JDKs
in MultiJDK mode. If JA
`test-patch` also has a mode to utilize Docker:
```bash
-$ test-patch. (other options) --docker
+$ test-patch (other options) --docker
```
-This will do some preliminary setup and then re-execute itself inside a Docker
container. For more information on how to provide a custom Dockerfile and
other Docker-specific features, see [precommit-docker](../precommit-docker) for
more information.
+This command will do some preliminary setup and then re-execute itself inside
a Docker container. For more information on how to provide a custom Dockerfile
and other Docker-specific features, see the specific [precommit Docker
support](../precommit-docker) page and the [Apache Yetus Docker Hub
Images](/yetus-docker-image) page for more information on the convenience
Docker images.
# In Closing
diff --git a/asf-site-src/source/documentation/in-progress/precommit-robots.md
b/asf-site-src/source/documentation/in-progress/precommit-robots.md
index b0fab18..faa487c 100644
--- a/asf-site-src/source/documentation/in-progress/precommit-robots.md
+++ b/asf-site-src/source/documentation/in-progress/precommit-robots.md
@@ -38,6 +38,23 @@ TRIGGER: ${CIRCLECI}=true
Circle CI support in `test-patch` is limited to github.com. Artifacts (the
`--patch-dir` directory) location needs to be handled set on the command line.
Linking to the logs is not currently supported.
+To use the pre-built Apache Yetus Docker image from docker hub as the build
environment, use the following snippet in the `.circleci/config.yaml` file,
substituting the tag for the version of Apache Yetus that should be used and
replacing the JAVA_HOME with the appropriate version as bundled mentioned in
the Dockerfile:
+
+```yaml
+jobs:
+ build:
+ docker:
+ - image: apache/yetus:0.9.0
+
+ environment:
+ JAVA_HOME: /usr/lib/jvm/java-8-openjdk-amd64
+
+ ...
+```
+
+See also
+ * See also the source tree's `.circleci/config.yaml` for some tips and
tricks.
+
Gitlab CI
=========
@@ -54,6 +71,21 @@ Artifacts, patch logs, etc are configured to go to a
yetus-out directory in the
```
+To use the pre-built Apache Yetus Docker image from docker hub as the build
environment, use the following snippet in the `.gitlab-ci.yml` file,
substituting the tag for the version of Apache Yetus that should be used and
replacing the JAVA_HOME with the appropriate version as bundled mentioned in
the Dockerfile:
+
+```yaml
+job:
+ image: apache/yetus:0.9.0
+ allow_failure: true
+ variables:
+ JAVA_HOME: /usr/lib/jvm/java-8-openjdk-amd64
+
+ ...
+```
+
+See also
+ * See also the source tree's `.gitlab-ci.yml` for some tips and tricks.
+
Jenkins
=======
@@ -66,9 +98,29 @@ There is some support for a few well known environment
variables:
* `${GIT_URL}` will trigger the same extra handling if 'github' or 'gitlab'
appear in the string.
* If `${ghprbPullId}` is set, then test-patch will configure itself for a
Github-style PR.
+To use the pre-built Apache Yetus Docker image from docker hub as the build
environment, use the following snippet in the `Jenkinsfile`, substituting the
tag for the version of Apache Yetus that should be used and replacing the
JAVA_HOME with the appropriate version as bundled mentioned in the Dockerfile:
+
+```groovy
+pipeline {
+ agent {
+ docker {
+ image 'apache/yetus:0.9.0'
+ args '-v /var/run/docker.sock:/var/run/docker.sock'
+ }
+ }
+
+ environment {
+ JAVA_HOME = '/usr/lib/jvm/java-8-openjdk-amd64'
+ }
+
+ ...
+
+}
+
+```
See also
- * See also the source tree's Jenkinsfile for some tips and tricks.
+ * See also the source tree's `Jenkinsfile` for some tips and tricks.
* [precommit-admin](precommit-admin), for special utilities built for
Jenkins.
* [GitHub Branch Source
Plugin](https://wiki.jenkins.io/display/JENKINS/GitHub+Branch+Source+Plugin)
* [GitHub Pull Request Builder
Plugin](https://wiki.jenkins.io/display/JENKINS/GitHub+pull+request+builder+plugin)
@@ -83,9 +135,12 @@ Travis CI support will update the local checked out source
repository to include
If `${ARTIFACTS_PATH}` is configured, then `--patch-dir` is set to the first
listed directory path. However, links to the location logs must still be
configured manually.
-Personalities will override the auto-detected Github repository information.
It may be necessary to manually configure it in your .travis.yml file.
+Personalities will override the auto-detected Github repository information.
It may be necessary to manually configure it in your `.travis.yml` file.
+As of this writing, it is not possible to make the Travis CI build environment
use the Apache Yetus pre-built docker images without using ` docker run` in the
before_install phase. Therefore, using the image is the same as described in
the [Apache Yetus Docker Hub Images](/yetus-docker-image) page.
+See also
+ * See also the source tree's `.travis.yml` for some tips and tricks.
Manual Configuration
====================
diff --git a/asf-site-src/source/downloads.html.md.erb
b/asf-site-src/source/downloads.html.md.erb
index e3c0f11..27c362e 100644
--- a/asf-site-src/source/downloads.html.md.erb
+++ b/asf-site-src/source/downloads.html.md.erb
@@ -31,7 +31,7 @@ should verify the integrity of these artifacts via [the ASF
procedure](https://w
using the [KEYS file for our
project](https://dist.apache.org/repos/dist/release/yetus/KEYS).
In addition to the artifacts below, we publish convenience artifacts in some
-third-party repositories (e.g. Maven Central).
+third-party repositories (e.g. Maven Central) and [Docker
Hub](../yetus-docker-image).
## Releases
diff --git a/asf-site-src/source/yetus-docker-image.md
b/asf-site-src/source/yetus-docker-image.md
new file mode 100644
index 0000000..963cbf5
--- /dev/null
+++ b/asf-site-src/source/yetus-docker-image.md
@@ -0,0 +1,86 @@
+<!---
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+-->
+
+Convenience Docker Hub Images
+===========================
+
+While not official releases, the Apache Yetus project makes available two
types of images on hub.docker.com:
+
+ * apache/yetus-base
+
+ This image is the same as the 'built-in' Dockerfile when `--docker` is
used without `--dockerfile` on the precommit command line. It includes all of
the pre-requisites as needed by the various Apache Yetus components. It is
located in `precommit/src/main/shell/test-patch-docker/`.
+
+ * apache/yetus
+
+ This image is the same as apache/yetus-base but includes a pre-built
version of Apache Yetus as part of the base OS image. In other words, qbt,
releasedocmaker, shelldocs, test-patch, etc., are in /usr/bin and available in
the default path. It is generated from the Dockerfile located in the root of
the source and is built with the options provided in the hooks directory.
+
+Both images should be suitable to be used as a building block or even directly
if your build environment needs no other dependencies. These images are
especially useful for various CI systems that require a Docker image to be used.
+
+Images are tagged such that 'master' represents the last successful Docker
image build of the master branch. Images based off of the official source
releases are tagged with a matching version number (e.g., 0.9.0). There is no
'latest' tagged image. It is recommended that users choose a stable tag so as
not to be surprised by incompatible changes.
+
+File Access
+========
+
+All of the executables that Apache Yetus provides requires access to one or
more directories. These directories should be provided on the command line via
the volume flags to docker run. For example, to run shelldocs against
test-patch.sh:
+
+```bash
+docker run \
+ --rm \
+ -v /tmp/out:/output \
+ -v /src/precommit/src/main/shell:/input:ro \
+ apache/yetus:0.9.0 \
+ shelldocs --output /output/test-patch.md --input /input/test-patch.sh
+```
+
+In this example, we mount two volumes: one for input (marked read-only), one
for output. After execution, shelldocs has processed test-patch.sh and
generated a test-patch.md file in /tmp/out on our local machine.
+
+Precommit also works, assuming that Apache Yetus image has all of your
project's dependencies. For example:
+
+```bash
+docker run \
+ --rm \
+ -v /tmp/patchdir:/patchdir \
+ -v /src:/basedir \
+ apache/yetus:0.9.0 \
+ qbt \
+ --plugins=all \
+ --basedir=/basedir \
+ --patch-dir=/patchdir \
+ --project=yetus
+```
+If your project needs additional dependencies, it is trivial to build off of
the Apache Yetus image:
+
+```Dockerfile
+FROM apache/yetus:0.9.0
+RUN apt-get -q update && apt-get -q install -y \
+ clang \
+ libssl-dev \
+ valgrind \
+ zlib1g-dev
+```
+ ```bash
+docker build -t project/build:0.9.0 -f .
+```
+
+This example builds a docker image based off of Apache Yetus 0.9.0 but with
the additions of clang, some development libraries, and valgrind. Now
project/build:0.9.0 can be used instead of apache/yetus:0.9.0 since it has all
of Apache Yetus and the additions needed by our project.
+
+A Note About Precommit
+================
+
+test-patch and friends have direct support for Docker outside of the
convenience images. That information is covered in-depth in
the[precommit-docker](../precommit-docker) section.
