[
https://issues.apache.org/jira/browse/BEAM-3060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16322778#comment-16322778
]
ASF GitHub Bot commented on BEAM-3060:
--------------------------------------
chamikaramj closed pull request #4378: [BEAM-3060] split one job into several
jobs, one for each IO.
URL: https://github.com/apache/beam/pull/4378
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git
a/.test-infra/jenkins/job_beam_PerformanceTests_FileBasedIO_IT.groovy
b/.test-infra/jenkins/job_beam_PerformanceTests_FileBasedIO_IT.groovy
index b41af717168..667b11d2072 100644
--- a/.test-infra/jenkins/job_beam_PerformanceTests_FileBasedIO_IT.groovy
+++ b/.test-infra/jenkins/job_beam_PerformanceTests_FileBasedIO_IT.groovy
@@ -18,60 +18,102 @@
import common_job_properties
-// This job runs the file-based IOs performance tests on PerfKit Benchmarker.
-job('beam_PerformanceTests_FileBasedIO_IT') {
- description('Runs PerfKit tests for file-based IOs.')
+def testsConfigurations = [
+ [
+ jobName : 'beam_PerformanceTests_TextIOIT',
+ jobDescription : 'Runs PerfKit tests for TextIOIT',
+ itClass : 'org.apache.beam.sdk.io.text.TextIOIT',
+ bqTable : 'beam_performance.textioit_pkb_results',
+ prCommitStatusName: 'Java TextIO Performance Test',
+ prTriggerPhase : 'Run Java TextIO Performance Test',
- // Set default Beam job properties.
- common_job_properties.setTopLevelMainJobProperties(delegate)
+ ],
+ [
+ jobName :
'beam_PerformanceTests_Compressed_TextIOIT',
+ jobDescription : 'Runs PerfKit tests for TextIOIT with
GZIP compression',
+ itClass : 'org.apache.beam.sdk.io.text.TextIOIT',
+ bqTable :
'beam_performance.compressed_textioit_pkb_results',
+ prCommitStatusName : 'Java CompressedTextIO Performance Test',
+ prTriggerPhase : 'Run Java CompressedTextIO Performance
Test',
+ extraPipelineArgs: [
+ compressionType: 'GZIP'
+ ]
+ ],
+ [
+ jobName : 'beam_PerformanceTests_AvroIOIT',
+ jobDescription : 'Runs PerfKit tests for AvroIOIT',
+ itClass : 'org.apache.beam.sdk.io.avro.AvroIOIT',
+ bqTable : 'beam_performance.avroioit_pkb_results',
+ prCommitStatusName: 'Java AvroIO Performance Test',
+ prTriggerPhase : 'Run Java AvroIO Performance Test',
+ ],
+ [
+ jobName : 'beam_PerformanceTests_TFRecordIOIT',
+ jobDescription : 'Runs PerfKit tests for
beam_PerformanceTests_TFRecordIOIT',
+ itClass :
'org.apache.beam.sdk.io.tfrecord.TFRecordIOIT',
+ bqTable :
'beam_performance.tfrecordioit_pkb_results',
+ prCommitStatusName: 'Java TFRecordIO Performance Test',
+ prTriggerPhase : 'Run Java TFRecordIO Performance Test',
+ ],
+]
+
+for (testConfiguration in testsConfigurations) {
+ create_filebasedio_performance_test_job(testConfiguration)
+}
- // Allows triggering this build against pull requests.
- common_job_properties.enablePhraseTriggeringFromPullRequest(
- delegate,
- 'Java FileBasedIOs Performance Test',
- 'Run Java FileBasedIOs Performance Test')
- // Run job in postcommit every 6 hours, don't trigger every push, and
- // don't email individual committers.
- common_job_properties.setPostCommit(
- delegate,
- '0 */6 * * *',
- false,
- '[email protected]',
- false)
+private void create_filebasedio_performance_test_job(testConfiguration) {
- def pipelineArgs = [
- project: 'apache-beam-testing',
- tempRoot: 'gs://temp-storage-for-perf-tests',
- numberOfRecords: '1000000',
- filenamePrefix:
'gs://temp-storage-for-perf-tests/filebased/${BUILD_ID}/TESTIOIT',
- ]
- def pipelineArgList = []
- pipelineArgs.each({
- key, value -> pipelineArgList.add("\"--$key=$value\"")
- })
- def pipelineArgsJoined = "[" + pipelineArgList.join(',') + "]"
+ // This job runs the file-based IOs performance tests on PerfKit
Benchmarker.
+ job(testConfiguration.jobName) {
+ description(testConfiguration.jobDescription)
+ // Set default Beam job properties.
+ common_job_properties.setTopLevelMainJobProperties(delegate)
- def itClasses = [
- "org.apache.beam.sdk.io.text.TextIOIT",
- "org.apache.beam.sdk.io.avro.AvroIOIT",
- "org.apache.beam.sdk.io.tfrecord.TFRecordIOIT",
- ]
+ // Allows triggering this build against pull requests.
+ common_job_properties.enablePhraseTriggeringFromPullRequest(
+ delegate,
+ testConfiguration.prCommitStatusName,
+ testConfiguration.prTriggerPhase)
+
+ // Run job in postcommit every 6 hours, don't trigger every push, and
+ // don't email individual committers.
+ common_job_properties.setPostCommit(
+ delegate,
+ '0 */6 * * *',
+ false,
+ '[email protected]',
+ false)
+
+ def pipelineArgs = [
+ project : 'apache-beam-testing',
+ tempRoot : 'gs://temp-storage-for-perf-tests',
+ numberOfRecords: '1000000',
+ filenamePrefix :
"gs://temp-storage-for-perf-tests/${testConfiguration.jobName}/\${BUILD_ID}/",
+ ]
+ if (testConfiguration.containsKey('extraPipelineArgs')) {
+ pipelineArgs << testConfiguration.extraPipelineArgs
+ }
+
+ def pipelineArgList = []
+ pipelineArgs.each({
+ key, value -> pipelineArgList.add("\"--$key=$value\"")
+ })
+ def pipelineArgsJoined = "[" + pipelineArgList.join(',') + "]"
- itClasses.each {
def argMap = [
- benchmarks: 'beam_integration_benchmark',
- beam_it_timeout: '1200',
- beam_it_profile: 'io-it',
- beam_prebuilt: 'true',
- beam_sdk: 'java',
- beam_it_module: 'sdks/java/io/file-based-io-tests',
- beam_it_class: "${it}",
- beam_it_options: pipelineArgsJoined,
- beam_extra_mvn_properties: '["filesystem=gcs"]',
- bigquery_table: 'beam_performance.filebasedioit_pkb_results',
+ benchmarks : 'beam_integration_benchmark',
+ beam_it_timeout : '1200',
+ beam_it_profile : 'io-it',
+ beam_prebuilt : 'true',
+ beam_sdk : 'java',
+ beam_it_module : 'sdks/java/io/file-based-io-tests',
+ beam_it_class : testConfiguration.itClass,
+ beam_it_options : pipelineArgsJoined,
+ beam_extra_mvn_properties: '["filesystem=gcs"]',
+ bigquery_table : testConfiguration.bqTable,
]
common_job_properties.buildPerformanceTest(delegate, argMap)
}
-}
+}
\ No newline at end of file
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
> Add performance tests for commonly used file-based I/O PTransforms
> ------------------------------------------------------------------
>
> Key: BEAM-3060
> URL: https://issues.apache.org/jira/browse/BEAM-3060
> Project: Beam
> Issue Type: Test
> Components: sdk-java-core
> Reporter: Chamikara Jayalath
> Assignee: Szymon Nieradka
>
> We recently added a performance testing framework [1] that can be used to do
> following.
> (1) Execute Beam tests using PerfkitBenchmarker
> (2) Manage Kubernetes-based deployments of data stores.
> (3) Easily publish benchmark results.
> I think it will be useful to add performance tests for commonly used
> file-based I/O PTransforms using this framework. I suggest looking into
> following formats initially.
> (1) AvroIO
> (2) TextIO
> (3) Compressed text using TextIO
> (4) TFRecordIO
> It should be possibly to run these tests for various Beam runners (Direct,
> Dataflow, Flink, Spark, etc.) and file-systems (GCS, local, HDFS, etc.)
> easily.
> In the initial version, tests can be made manually triggerable for PRs
> through Jenkins. Later, we could make some of these tests run periodically
> and publish benchmark results (to BigQuery) through PerfkitBenchmarker.
> [1] https://beam.apache.org/documentation/io/testing/
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
