This is an automated email from the ASF dual-hosted git repository.
markliu pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/master by this push:
new 86b5b19 [BEAM-6908] New Jenkins branch for Python35 benchmark
new d9c2de8 Merge pull request #8393 from markflyhigh/py35-benchmark
86b5b19 is described below
commit 86b5b191ee50d215bed846bebaf10a56fe8453bc
Author: Mark Liu <mark...@markliu-macbookpro2.roam.corp.google.com>
AuthorDate: Wed Apr 24 14:08:31 2019 -0700
[BEAM-6908] New Jenkins branch for Python35 benchmark
---
.test-infra/jenkins/README.md | 1 +
.../jenkins/job_Performancetests_Python35.groovy | 63 ++++++++++++++++++++++
2 files changed, 64 insertions(+)
diff --git a/.test-infra/jenkins/README.md b/.test-infra/jenkins/README.md
index f478d31..775569e 100644
--- a/.test-infra/jenkins/README.md
+++ b/.test-infra/jenkins/README.md
@@ -84,6 +84,7 @@ Beam Jenkins overview page:
[link](https://builds.apache.org/view/A-D/view/Beam/
| beam_PerformanceTests_ManyFiles_TextIOIT |
[cron](https://builds.apache.org/job/beam_PerformanceTests_ManyFiles_TextIOIT/),
[hdfs_cron](https://builds.apache.org/job/beam_PerformanceTests_ManyFiles_TextIOIT_HDFS/)
| `Run Java ManyFilesTextIO Performance Test` | [](https://builds.apache.org/job/beam_PerformanceTests_ManyFiles_TextIOIT)
[,
[hdfs_cron](https://builds.apache.org/job/beam_PerformanceTests_ParquetIOIT_HDFS/)
| `Run Java ParquetIO Performance Test` | [](https://builds.apache.org/job/beam_PerformanceTests_ParquetIOIT)
[ | `Run
Python Performance Test` | [](https://builds.apache.org/job/beam_PerformanceTests_Python)
|
+| beam_PerformanceTests_Python35 |
[cron](https://builds.apache.org/job/beam_PerformanceTests_Python35/) | `Run
Python35 Performance Test` | [](https://builds.apache.org/job/beam_PerformanceTests_Python35)
|
| beam_PerformanceTests_Spark |
[cron](https://builds.apache.org/job/beam_PerformanceTests_Spark/) | `Run Spark
Performance Test` | [](https://builds.apache.org/job/beam_PerformanceTests_Spark)
|
| beam_PerformanceTests_TFRecordIOIT |
[cron](https://builds.apache.org/job/beam_PerformanceTests_TFRecordIOIT/) |
`Run Java JdbcIO Performance Test` | [](https://builds.apache.org/job/beam_PerformanceTests_TFRecordIOIT)
|
| beam_PerformanceTests_TextIOIT |
[cron](https://builds.apache.org/job/beam_PerformanceTests_TextIOIT/),
[hdfs_cron](https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/)
| `Run Java TextIO Performance Test` | [](https://builds.apache.org/job/beam_PerformanceTests_TextIOIT)
[](https://builds.apa
[...]
diff --git a/.test-infra/jenkins/job_Performancetests_Python35.groovy
b/.test-infra/jenkins/job_Performancetests_Python35.groovy
new file mode 100644
index 0000000..842df97
--- /dev/null
+++ b/.test-infra/jenkins/job_Performancetests_Python35.groovy
@@ -0,0 +1,63 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+import CommonJobProperties as commonJobProperties
+
+// This job runs the Beam Python35 performance benchmark on PerfKit
Benchmarker.
+job('beam_PerformanceTests_Python35'){
+ // Set default Beam job properties.
+ commonJobProperties.setTopLevelMainJobProperties(delegate)
+
+ // Run job in postcommit every 6 hours, don't trigger every push.
+ commonJobProperties.setAutoJob(
+ delegate,
+ 'H */6 * * *')
+
+ // Allows triggering this build against pull requests.
+ commonJobProperties.enablePhraseTriggeringFromPullRequest(
+ delegate,
+ 'Python35 SDK Performance Test',
+ 'Run Python35 Performance Test')
+
+ def pipelineArgs = [
+ project: 'apache-beam-testing',
+ staging_location: 'gs://temp-storage-for-end-to-end-tests/staging-it',
+ temp_location: 'gs://temp-storage-for-end-to-end-tests/temp-it',
+ output: 'gs://temp-storage-for-end-to-end-tests/py-it-cloud/output'
+ ]
+ def pipelineArgList = []
+ pipelineArgs.each({
+ key, value -> pipelineArgList.add("--$key=$value")
+ })
+ def pipelineArgsJoined = pipelineArgList.join(',')
+
+ def argMap = [
+ beam_sdk : 'python',
+ benchmarks : 'beam_integration_benchmark',
+ bigquery_table : 'beam_performance.wordcount_py35_pkb_results',
+ beam_it_class :
'apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it',
+ beam_it_module : 'sdks/python/test-suites/dataflow/py3',
+ beam_prebuilt : 'true', // skip beam prebuild
+ beam_python_sdk_location :
'test-suites/dataflow/py3/build/apache-beam.tar.gz',
+ beam_runner : 'TestDataflowRunner',
+ beam_it_timeout : '1200',
+ beam_it_args : pipelineArgsJoined,
+ ]
+
+ commonJobProperties.buildPerformanceTest(delegate, argMap)
+}
