Re: [PROPOSAL] Using Bazel and Docker for Python SDK development and tests

On Thu, Oct 18, 2018 at 9:19 PM Udi Meiri wrote: > Unless I'm missing something, we can't run cython builds in parallel with > other builds without first making cython build artifacts appear in a > separate build directory. > > AFAICT --parallel--safe-build is deprecated >

Re: [PROPOSAL] Using Bazel and Docker for Python SDK development and tests

Unless I'm missing something, we can't run cython builds in parallel with other builds without first making cython build artifacts appear in a separate build directory. AFAICT --parallel--safe-build is deprecated / on by default. I vote to move

Re: [PROPOSAL] Using Bazel and Docker for Python SDK development and tests

On Wed, Oct 17, 2018 at 10:57 PM Lukasz Cwik wrote: > Gradle works pretty well at executing separate projects in parallel. There > are a few Java projects which contain only tests with different flags which > allow us to use the Gradle project based parallelization effectively. > See >

Re: [PROPOSAL] Using Bazel and Docker for Python SDK development and tests

On Thu, Oct 18, 2018 at 10:06 AM Valentyn Tymofieiev wrote: > Thanks for starting the discussion. For Python 3, we specifically run > tests only in Python 3.5, and as was shown in > https://issues.apache.org/jira/browse/BEAM-5663, this does not provide > sufficient coverage for Python 3

Re: [PROPOSAL] Using Bazel and Docker for Python SDK development and tests

Thanks for starting the discussion. For Python 3, we specifically run tests only in Python 3.5, and as was shown in https://issues.apache.org/jira/browse/BEAM-5663, this does not provide sufficient coverage for Python 3 compatibility. To accelerate Py3 compatibility effort, we need to start

Re: [PROPOSAL] Using Bazel and Docker for Python SDK development and tests

We run the full suite of Python unit tests with (sequentially) Python 2 Python 2 + GCP deps Python 2 + Cython Python 3 There is, admittedly, a huge amount of redundancy here. On Thu, Oct 18, 2018 at 4:11 AM Kenneth Knowles wrote: > For the benefit of those not working on the Python SDK, can

Re: [PROPOSAL] Using Bazel and Docker for Python SDK development and tests

On Wed, Oct 17, 2018 at 7:17 PM Udi Meiri wrote: > On Wed, Oct 17, 2018 at 1:38 AM Robert Bradshaw > wrote: > >> On Tue, Oct 16, 2018 at 12:48 AM Udi Meiri wrote: >> >>> Hi, >>> >>> In light of increasing Python pre-commit times due to the added Python 3 >>> tests, >>> I thought it might be

Re: [PROPOSAL] Using Bazel and Docker for Python SDK development and tests

For the benefit of those not working on the Python SDK, can we know the test matrix? It might help bring perspective; certainly it would help me understand what might move to post-commit, as one example. Kenn On Wed, Oct 17, 2018 at 6:21 PM Ahmet Altay wrote: > > > On Wed, Oct 17, 2018 at 1:57

Re: [PROPOSAL] Using Bazel and Docker for Python SDK development and tests

On Wed, Oct 17, 2018 at 1:57 PM, Lukasz Cwik wrote: > Gradle works pretty well at executing separate projects in parallel. There > are a few Java projects which contain only tests with different flags which > allow us to use the Gradle project based parallelization effectively. > See

Re: [PROPOSAL] Using Bazel and Docker for Python SDK development and tests

Gradle works pretty well at executing separate projects in parallel. There are a few Java projects which contain only tests with different flags which allow us to use the Gradle project based parallelization effectively. See

Re: [PROPOSAL] Using Bazel and Docker for Python SDK development and tests

On Wed, Oct 17, 2018 at 1:38 AM Robert Bradshaw wrote: > On Tue, Oct 16, 2018 at 12:48 AM Udi Meiri wrote: > >> Hi, >> >> In light of increasing Python pre-commit times due to the added Python 3 >> tests, >> I thought it might be time to re-evaluate the tools used for Python tests >> and

Re: [PROPOSAL] Using Bazel and Docker for Python SDK development and tests

On Tue, Oct 16, 2018 at 12:48 AM Udi Meiri wrote: > Hi, > > In light of increasing Python pre-commit times due to the added Python 3 > tests, > I thought it might be time to re-evaluate the tools used for Python tests > and development, and propose an alternative. > > Currently, we use

[PROPOSAL] Using Bazel and Docker for Python SDK development and tests

Hi, In light of increasing Python pre-commit times due to the added Python 3 tests, I thought it might be time to re-evaluate the tools used for Python tests and development, and propose an alternative. Currently, we use nosetests, tox, and virtualenv for testing. The proposal is to use Bazel,