[
https://issues.apache.org/jira/browse/BEAM-4495?focusedWorklogId=138618&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-138618
]
ASF GitHub Bot logged work on BEAM-4495:
----------------------------------------
Author: ASF GitHub Bot
Created on: 28/Aug/18 02:12
Start Date: 28/Aug/18 02:12
Worklog Time Spent: 10m
Work Description: udim opened a new pull request #6282: [BEAM-4495]
Website pre-commit job
URL: https://github.com/apache/beam/pull/6282
This PR rolls up https://github.com/apache/beam/pull/5809 and runs website
builds and tests inside a Docker container.
Description of #5809:
This adds a new pre-commit job for the website sources in the apache/beam
repo. This is a step in the migration effort to move website sources into the
main repo and improve automation reliability. See:
https://s.apache.org/beam-site-automation
Note that this change doesn't cover automatic staging of the repository.
I'll work on that in a follow-up PR.
Running the jobs in a container has these advantages:
- Consistency of environment: minimal differences between local dev machine
and Jenkins.
- Using an existing Ruby container minimizes automation effort and build
time.
- No need to install Jekyll and other website dependencies locally.
I investigated the following ways of running precommit and postcommit
website jobs using Docker.
1. https://plugins.jenkins.io/docker-custom-build-environment
This plugin has been installed in builds.apache.org's Jenkins service. I
couldn't test it locally however, and I wouldn't expect developers working on
Jenkins jobs to go through the hassle of Installing and configuring Jenkins on
their local machine or VM. I tried using the dockerized-jenkins (in this repo),
and while I managed to configure and launch the jobs with the plugin, I still
came across the issue of running
[docker-in-docker](https://jpetazzo.github.io/2015/09/03/do-not-use-docker-in-docker-for-ci/).
I opted for the solution of running jobs as sibling containers (of the Jenkins
container), but the plugin wants to bind mount the repo and /tmp directories to
the job container and that doesn't seem to work.
1. https://github.com/bmuschko/gradle-docker-plugin
I tried this plugin hoping to make managing Docker containers simpler, but
it seems to be aimed more at building and publishing images. For instance I was
missing the -u flag for docker create.
1. Calling docker CLI directly.
This is what I settled on. It should work very similarly on dev and Jenkins
machines.
------------------------
Follow this checklist to help us incorporate your contribution quickly and
easily:
- [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in
ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA
issue, if applicable. This will automatically link the pull request to the
issue.
- [ ] If this contribution is large, please file an Apache [Individual
Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
It will help us expedite review of your Pull Request if you tag someone
(e.g. `@username`) to look at it.
Post-Commit Tests Status (on master branch)
------------------------------------------------------------------------------------------------
Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark
--- | --- | --- | --- | --- | --- | --- | ---
Go | [](https://builds.apache.org/job/beam_PostCommit_Go_GradleBuild/lastCompletedBuild/)
| --- | --- | --- | --- | --- | ---
Java | [](https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex_Gradle/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Gradle/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump_Gradle/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza_Gradle/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark_Gradle/lastCompletedBuild/)
Python | [](https://builds.apache.org/job/beam_PostCommit_Python_Verify/lastCompletedBuild/)
| --- | [](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/)
</br> [](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/)
| --- | --- | --- | ---
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 138618)
Time Spent: 4h 40m (was: 4.5h)
> Create website pre-commits for apache/beam repository
> -----------------------------------------------------
>
> Key: BEAM-4495
> URL: https://issues.apache.org/jira/browse/BEAM-4495
> Project: Beam
> Issue Type: Sub-task
> Components: testing, website
> Reporter: Scott Wegner
> Assignee: Udi Meiri
> Priority: Major
> Labels: beam-site-automation-reliability
> Time Spent: 4h 40m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
