[
https://issues.apache.org/jira/browse/BEAM-4445?focusedWorklogId=116082&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-116082
]
ASF GitHub Bot logged work on BEAM-4445:
----------------------------------------
Author: ASF GitHub Bot
Created on: 26/Jun/18 18:16
Start Date: 26/Jun/18 18:16
Worklog Time Spent: 10m
Work Description: Ardagan commented on a change in pull request #5757:
[BEAM-4445] Split pre-commit tests into separate jobs based on trigger
condition.
URL: https://github.com/apache/beam/pull/5757#discussion_r198246956
##########
File path: .test-infra/jenkins/PrecommitBuilder.groovy
##########
@@ -0,0 +1,123 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+import common_job_properties
+
+/** This class defines PreCommitBuilder.build() helper for defining pre-comit
jobs. */
+class PrecommitBuilder {
+ /** scope 'this' parameter from top-level script; used for binding Job DSL
methods. */
+ Object scope
+
+ /** Base name for each post-commit suite job, i.e. 'Go'. */
+ String nameBase
+
+ /** The Gradle task to execute. */
+ String gradleTask
+
+ /** Overall job timeout. */
+ int timeoutMins = 90
+
+ /** If defined, set of path expressions used to trigger the job on commit. */
+ List<String> triggerPathPatterns = []
+
+ /**
+ * Define a set of pre-commit jobs.
+ *
+ * @param additionalCustomization Job DSL closure with additional
customization to apply to the job.
+ */
+ void build(Closure additionalCustomization = {}) {
+ defineCronJob additionalCustomization
+ defineCommitJob additionalCustomization
+ definePhraseJob additionalCustomization
+ }
+
+ /** Create a pre-commit job which runs on a daily schedule. */
+ private void defineCronJob(Closure additionalCustomization) {
+ def job = createBaseJob 'Cron'
+ job.with {
+ description buildDescription('on a daily schedule.')
+ common_job_properties.setAutoJob delegate
+ }
+ job.with additionalCustomization
+ }
+
+ /** Create a pre-commit job which runs on every commit to a PR. */
+ private void defineCommitJob(Closure additionalCustomization) {
+ def job = createBaseJob 'Commit', true
+ def defaultPathTriggers = [
+ '^build.gradle$',
+ '^build_rules.gradle$',
+ '^gradle.properties$',
+ '^gradlew$',
+ '^gradle.bat$',
+ '^settings.gradle$'
+ ]
+ triggerPathPatterns.addAll(defaultPathTriggers)
+ job.with {
+ description buildDescription('for each commit push.')
+ concurrentBuild()
+ common_job_properties.setPullRequestBuildTrigger(delegate,
+ githubUiHint(),
+ '',
+ false,
+ triggerPathPatterns)
+ }
+ job.with additionalCustomization
+ }
+
+ private void definePhraseJob(Closure additionalCustomization) {
+ def job = createBaseJob 'Phrase'
+ job.with {
+ description buildDescription("on trigger phrase
'${buildTriggerPhrase()}.)")
+ concurrentBuild()
+ common_job_properties.setPullRequestBuildTrigger delegate,
githubUiHint(), buildTriggerPhrase()
+ }
+ job.with additionalCustomization
+ }
+
+ private Object createBaseJob(nameSuffix, usesRegionFilter = false) {
+ return scope.job("beam_PreCommit_${nameBase}_${nameSuffix}") {
+ common_job_properties.setTopLevelMainJobProperties(delegate,
+ 'master',
+ timeoutMins,
+ !usesRegionFilter) // needed for included regions PR triggering; see
[JENKINS-23606]
+ steps {
+ gradle {
+ rootBuildScriptDir(common_job_properties.checkoutDir)
+ tasks(gradleTask)
+ common_job_properties.setGradleSwitches(delegate)
+ }
+ }
+ }
+ }
+
+ /** The magic phrase used to trigger the job when posted as a PR comment. */
+ private String buildTriggerPhrase() {
+ return "Run ${nameBase} PreCommit"
+ }
+
+ /** A human-readable description which will be used as the base of all suite
jobs. */
+ private buildDescription(String triggerDescription) {
+ return "Runs ${nameBase} PreCommit tests ${triggerDescription}"
+ }
+
+ /** The Jenkins job name to display in GitHub. */
Review comment:
I believe this comment is redundant, method name is descriptive enough.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 116082)
Time Spent: 2.5h (was: 2h 20m)
> Filter pre-commit triggering based on touched files
> ---------------------------------------------------
>
> Key: BEAM-4445
> URL: https://issues.apache.org/jira/browse/BEAM-4445
> Project: Beam
> Issue Type: Sub-task
> Components: build-system, testing
> Reporter: Scott Wegner
> Assignee: Scott Wegner
> Priority: Minor
> Labels: beam-site-automation-reliability
> Fix For: Not applicable
>
> Time Spent: 2.5h
> Remaining Estimate: 0h
>
> This is discussed in the [Beam-Site Automation
> Reliability|https://s.apache.org/beam-site-automation] design, under
> "Pre-Commit Job Filtering"
> The proposal is to filter pre-commit job triggered on PR's based on which
> files are touched. The impact is that most PRs will only run one set of
> relevant tests, rather than all three. This will decrease test overhead and
> the impact of flaky tests.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
