[
https://issues.apache.org/jira/browse/BEAM-10674?focusedWorklogId=472913&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-472913
]
ASF GitHub Bot logged work on BEAM-10674:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 20/Aug/20 14:14
Start Date: 20/Aug/20 14:14
Worklog Time Spent: 10m
Work Description: kamilwu commented on a change in pull request #12542:
URL: https://github.com/apache/beam/pull/12542#discussion_r474015631
##########
File path: .test-infra/jenkins/job_LoadTests_coGBK_Python.groovy
##########
@@ -147,25 +147,30 @@ def loadTestConfigurations = { datasetName ->
autoscaling_algorithm: 'NONE'
]
],
- ].each { test -> test.pipelineOptions.putAll(additionalPipelineArgs) }
+ ]
+ .each { test -> test.pipelineOptions.putAll(additionalPipelineArgs) }
+ .each { test -> (mode) != 'streaming' ?: addStreamingOptions(test) }
}
-def batchLoadTestJob = { scope, triggeringContext ->
- scope.description('Runs Python CoGBK load tests on Dataflow runner in batch
mode')
- commonJobProperties.setTopLevelMainJobProperties(scope, 'master', 240)
+def addStreamingOptions(test) {
+ // Use highmem workers to prevent out of memory issues.
+ test.pipelineOptions << [streaming: null,
+ worker_machine_type: 'n1-highmem-4'
Review comment:
> How is it called in Java SDK?
It's `--numberOfWorkerHarnessThreads`. Some time ago, its equivalent in
Python SDK was `--experimental worker_threads=[n]`, but the option was removed.
I'll open a discussion on dev@ to find out if it is something the community
should take care of.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 472913)
Time Spent: 3h 10m (was: 3h)
> Create Streaming coGBK Python Load Test Jenkins Job
> ---------------------------------------------------
>
> Key: BEAM-10674
> URL: https://issues.apache.org/jira/browse/BEAM-10674
> Project: Beam
> Issue Type: Sub-task
> Components: testing
> Reporter: Kamil Wasilewski
> Assignee: Kamil Wasilewski
> Priority: P2
> Time Spent: 3h 10m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
