After this PR goes in should we revisit breaking up the python tests into separate jenkins jobs by python version? One of the problems with that plan originally was that we lost the parallelism that gradle provides because we were left with only one tox task per jenkins job, and so the total time to complete all python jenkins jobs went up a lot. With pytest + xdist we should hopefully be able to keep the parallelism even with just one tox task. This could be a big win. I feel like I'm spending more time monitoring and re-queuing timed-out jenkins jobs lately than I am writing code.
On Mon, Dec 9, 2019 at 10:32 AM Udi Meiri <eh...@google.com> wrote: > This PR <https://github.com/apache/beam/pull/10322> (in review) migrates > py27-gcp to using pytest. > It reduces the testPy2Gcp task down to ~13m > <https://scans.gradle.com/s/kj7ogemnd3toe/timeline?details=ancsbov425524> > (from ~45m). This speedup will probably be lower once all 8 tasks are using > pytest. > It also adds 5 previously uncollected tests. >
