Victor, To correct the math, Python3 runs 686 tests in 15.971s or 42.95 tests per second. Python2 runs 1495 tests in 22.595s or 66.16 tests per second. Python2 is 54% faster (but even that's just an estimate, since it assumes all tests run at the same speed, which isn't true - plus it runs them on different hardware and not all VMs are created equal ;) ).
We're aware that the 'dangling mock detector' makes the code run significantly slower (historically ~15 min instead of ~5 min) however it's helped us catch more than one bug with minimal effort. The question is why does it run even slower in python3? If it is just the infra issue of running on slower hardware, then we could probably address that. Peter -----Original Message----- From: Victor Stinner [mailto:[email protected]] Sent: May-16-16 10:57 AM To: [email protected] Subject: Re: [openstack-dev] [trove] timeouts in gate-trove-python34-db Le 16/05/2016 16:12, Peter Stachowski a écrit : > Amrith is right though - python34 is *much* slower running this check > (and somewhat in general) than python27. Maybe that doesn't translate > into real-world performance data, but it has made me a bit nervous > nonetheless. IMHO we should look more closely to the output. See my comments on the bug report: https://bugs.launchpad.net/trove/+bug/1582257 Currently, running unit tests on Python 3 runs 686 tests in 15.971s, whereas Python 2 runs 1495 tests in 22.595s. Python 3 is "faster" :-) (16 sec < 23 sec) The whole Python 3 job takes 10 minutes, but in these 10 minutes, almost 8 minutes are taken just to build binary wheel packages. On Python 2, prebuilt packages are used and so creating the virtual environment is much faster (around one minute). I proposed a change to prebuild also wheel packages on Python 3: https://review.openstack.org/#/c/316890/ Ok, but sometimes we get timeout, right? Again, if you look closely to the output, between two runs on Python 2, the timing also changes a lot: (a) Ran 1495 tests in 22.595s (b) Ran 1495 tests in 393.364s => 17x slower! The issue is not specific to Python 3. But it looks like the random slowdown is much larger on Python 3. Duration of Python 3 unit tests: (a) "Ran 686 tests in 15.971s" (b) "Ran 686 tests in 1581.090s" => 100x slower! I guess that the difference is that Python 3 unit tests are currently run sequentially (1 process), whereas Python 2 unit tests are run in parallel (8 processes). *Again* please check the code detecting dangling mocks. For example, "tox -e py34" takes 14 seconds without this code, 177 seconds with the code: 12x slower with the code. Victor __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: [email protected]?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: [email protected]?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
