[
https://issues.apache.org/jira/browse/BEAM-7085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16819626#comment-16819626
]
Valentyn Tymofieiev commented on BEAM-7085:
-------------------------------------------
Assumptions:
- We have 16 jenkins workers
- We have suites for Py 2.7, 3.5, 3.6, 3.7, which will run in parallel..
- PostCommit test suite will start up to 8 pipelines simultaneously,
https://github.com/apache/beam/blob/4c322107ca5ebc0ab1cc6581d957501fd3ed9cc4/build.gradle#L200.
AFAIK precommit test suite always runs 1 test at a time.
- Each pipeline stars 1 worker consuming 1 vCPU.
- Each Jenkins worker runs at most test 1 suite.
In this case the ballpark for the upper bound would be: 16 Jenkins machines * 4
Python minor versions * 8 pipelines per test suite = 512 vCPU cores.
> Several PostCommit test suites are failing recently due to insufficient CPU
> quota in apache-beam-testing
> ---------------------------------------------------------------------------------------------------------
>
> Key: BEAM-7085
> URL: https://issues.apache.org/jira/browse/BEAM-7085
> Project: Beam
> Issue Type: Improvement
> Components: test-failures, testing
> Reporter: Valentyn Tymofieiev
> Assignee: yifan zou
> Priority: Major
>
> Current usage is 632 with limit 750.
> https://pantheon.corp.google.com/iam-admin/quotas?project=apache-beam-testing&metric=CPUs&usage=USED
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
