Thanks a lot for your work on pytest, Udi! Is the speedup due to running more tests in parallel or pytest spends less time orchestrating the run?
On Mon, Dec 9, 2019 at 5:59 PM Valentyn Tymofieiev <valen...@google.com> wrote: > > > On Mon, Dec 9, 2019 at 5:36 PM Udi Meiri <eh...@google.com> wrote: > >> I have given this some thought honestly don't know if splitting into >> separate jobs will help. >> - I have seen race conditions with running setuptools in parallel, so >> more isolation is better. OTOH, if 2 setuptools scripts run at the same >> time on the same machine they might still race (same homedir, same /tmp >> dir). >> - Retrying due to flakes will be faster, but if something is broken >> you'll need to write 4x the number of "run python precommit" phrases. >> > > there is also "retest this please" which will rerun all precommits. > > >> - Increased parallelism may also run into quota issues with Dataflow. >> >> What benefits do you see from splitting up the jobs? >> >> On Mon, Dec 9, 2019 at 4:17 PM Chad Dombrova <chad...@gmail.com> wrote: >> >>> After this PR goes in should we revisit breaking up the python tests >>> into separate jenkins jobs by python version? One of the problems with >>> that plan originally was that we lost the parallelism that gradle provides >>> because we were left with only one tox task per jenkins job, and so the >>> total time to complete all python jenkins jobs went up a lot. With >>> pytest + xdist we should hopefully be able to keep the parallelism even >>> with just one tox task. This could be a big win. I feel like I'm spending >>> more time monitoring and re-queuing timed-out jenkins jobs lately than I am >>> writing code. >>> >>> On Mon, Dec 9, 2019 at 10:32 AM Udi Meiri <eh...@google.com> wrote: >>> >>>> This PR <https://github.com/apache/beam/pull/10322> (in review) >>>> migrates py27-gcp to using pytest. >>>> It reduces the testPy2Gcp task down to ~13m >>>> <https://scans.gradle.com/s/kj7ogemnd3toe/timeline?details=ancsbov425524> >>>> (from ~45m). This speedup will probably be lower once all 8 tasks are using >>>> pytest. >>>> It also adds 5 previously uncollected tests. >>>> >>>
