On Wed, Oct 17, 2018 at 7:17 PM Udi Meiri <eh...@google.com> wrote: > On Wed, Oct 17, 2018 at 1:38 AM Robert Bradshaw <rober...@google.com> > wrote: > >> On Tue, Oct 16, 2018 at 12:48 AM Udi Meiri <eh...@google.com> wrote: >> >>> Hi, >>> >>> In light of increasing Python pre-commit times due to the added Python 3 >>> tests, >>> I thought it might be time to re-evaluate the tools used for Python >>> tests and development, and propose an alternative. >>> >>> Currently, we use nosetests, tox, and virtualenv for testing. >>> The proposal is to use Bazel, which I believe can replace the above >>> tools while adding: >>> - parallel testing: each target has its own build directory, >>> >> >> We could look at detox and/or retox again to get parallel testing (though >> not, possibly, at such a low level). We tried detox for a while, but there >> were issues debugging timeouts (specifically, it buffered the stdout while >> testing to avoid multiplexing it, but that meant little info when a test >> actually timed out on jenkins). >> >> We could alternatively look into leveraging gradle's within-project >> paralleliziaton to speed this up. It is a pain that right now every Python >> test is run sequentially. >> > I don't believe that Gradle has an easy solution. The only within-project > parallilization I can find requires using the Worker API > <https://guides.gradle.org/using-the-worker-api/?_ga=2.143780085.1705314017.1539791984-819557858.1539791984> > . > > I've tried pytest-xdist with limited success (pickling the session with > pytest-xdist's execnet dependency fails). > > >> >> >>> providing isolation from build artifacts such as from Cython >>> >> >> Each tox environment already has (I think) its own build directory. Or is >> this not what we're seeing? >> > Cython-based unit test runs leave behind artifacts that must be cleaned > up, which is why we can't run all tox environments in parallel. > We use this script to clean up: > > https://github.com/apache/beam/blob/master/sdks/python/scripts/run_tox_cleanup.sh > > > I am 90% certain that this would not be an issue with bazel, since it > stages all build dependencies in a separate build directory, so any > generated files would be placed there. >
The point of tox is to provide isolated, hermetic environments for testing. I'll try to look into why, or if, this is needed. (Perhaps it doesn't isolate the build environments by default; I do see a --parallel--safe-build flag...)
