Re: Cython unit test suites running without Cythonized sources

> > IIUC, isolated_build=True and the removal of setup.py invocation in the > current virtualenv should eliminate any Cython output files in the repo, > and no need for run_tox_cleanup.sh? > Correct, that script is deleted in this commit:

Re: Cython unit test suites running without Cythonized sources

The `changedir = {envsitepackagesdir}` setting is definitely something I haven't thought of. It solves the shadowing issue without needing to split tests and packages from one another. (though I still think it's unnecessary to include tests in the published package) IIUC, isolated_build=True and

Re: Cython unit test suites running without Cythonized sources

Hi Udi, > Sorry I didn't realize you already had a solution for the shadowing issue > and BEAM-8572. > No worries at all. I haven't had much time to invest into that PR lately (most of it I did at home on my own time), but I did get past most of the major issues. You've been working on so many

Re: Cython unit test suites running without Cythonized sources

Sorry I didn't realize you already had a solution for the shadowing issue and BEAM-8572. On Tue, Dec 10, 2019 at 6:21 PM Chad Dombrova wrote: > Hi Udi, I know you're aware of my PR > , but I really encourage you > to look into pep517 and pep518. They

Re: Cython unit test suites running without Cythonized sources

Hi Udi, I know you're aware of my PR , but I really encourage you to look into pep517 and pep518. They are the new solution for all of this -- declaring build dependencies and creating isolated out-of-source builds e.g. using tox. Another thing I added

Re: Cython unit test suites running without Cythonized sources

To follow up, since I'm trying to run cython-based tests using pytest: - tox does in fact correctly install apache-beam with cythonized modules in its virtualenv. - Since our tests are under apache_beam/, local sources shadow those in the installed apache_beam package. - The original issue I

Re: Cython unit test suites running without Cythonized sources

Thank you for spending time on this to clarify it for all of us! Much appreciated. On Sun, Nov 10, 2019 at 3:45 PM Chad Dombrova wrote: > Hi all, > > >> The sdist step creates a package that should be installed into each >> tox environment. If the tox environment has cython when this apache >>

Re: Cython unit test suites running without Cythonized sources

Hi all, > The sdist step creates a package that should be installed into each > tox environment. If the tox environment has cython when this apache > beam package is installed, it should be used. Nose (or whatever) > should then run the tests. > I spent some time this weekend trying to

Re: Cython unit test suites running without Cythonized sources

On Thu, Nov 7, 2019 at 6:25 PM Chad Dombrova wrote: > > Hi, > Answers inline below, > >>> It's unclear from the nose source[1] whether it's calling build_py and >>> build_ext, or just build_ext. It's also unclear whether the result of that >>> build is actually used. When python setup.py

Re: Cython unit test suites running without Cythonized sources

Hi, Answers inline below, It's unclear from the nose source[1] whether it's calling build_py >> and build_ext, or just build_ext. It's also unclear whether the result of >> that build is actually used. When python setup.py nosetests runs, it runs >> inside of a virtualenv created by tox, and

Re: Cython unit test suites running without Cythonized sources

On Thu, Nov 7, 2019 at 1:37 PM Chad Dombrova wrote: > > On Thu, Nov 7, 2019 at 11:31 AM Robert Bradshaw > wrote: > >> Does python setup.py nosetests invoke build_ext (or, more generally, >> build)? > > > It's unclear from the nose source[1] whether it's calling build_py > and build_ext, or just

Re: Cython unit test suites running without Cythonized sources

On Thu, Nov 7, 2019 at 11:31 AM Robert Bradshaw wrote: > Does python setup.py nosetests invoke build_ext (or, more generally, > build)? It's unclear from the nose source[1] whether it's calling build_py and build_ext, or just build_ext. It's also unclear whether the result of that build is

Re: Cython unit test suites running without Cythonized sources

Does python setup.py nosetests invoke build_ext (or, more generally, build)? It's possible cython is present, but the build step is not invoked which would explain the skip for slow_coders_test. The correct test is being used in

Re: Cython unit test suites running without Cythonized sources

I believe tox is correctly installing cython and executes "python setup.py nosetests" which triggers cythonzation path inside setup.py. Some indications that cython is installed and used is the following log entries (from a recent precommit cron job [1]) - [ 1/12] Cythonizing

Re: Cython unit test suites running without Cythonized sources

Another potential solution would be to _not_ use the sdist task to build the tarball and let tox do it. Tox should install cython on supported platforms before running sdist itself (which it does by default unless you explicitly provide it with a tarball, which we are doing). This has the added