> > > I think I tend to agree with David. Here is my point of view: > > I am not sure how to implement it. cython command *does not know* about > the build directory. This is more part of the build system > (setuptools/meson etc). My point is that the cython command should be > stupid and just generate a C file with a specified name/location nothing > more. It is also consistent with current usage of it (it just generates C > files). I understand that it is the build system responsibility to do all > the magic around it. As an example, here is working integration of current > "stupid" utility present in the PR implemented by scipy dev (Ralf Gommers) > [1]: > https://github.com/scipy/scipy/compare/main...rgommers:scipy:cy-shared-memoryview?expand=1 > > The only question is how it can be integrated into setuptools too. The > most obvious and stupid solution is to generate C file (which was also > recommended way of working with Cython extension until recently), commit it > to repository and compile it as regular C extension example is already > present in test of the PR: > > from Cython.Build import cythonize > from setuptools import setup, Extension > from Cython.Compiler import Options > import numpy > > extensions = [ > Extension("*", ["**/*.pyx"], include_dirs=[numpy.get_include()]), > Extension("pkg2.MemoryView", sources=["pkg2/MemoryView.c"]) > ] > > setup( > ext_modules = cythonize(extensions, > compiler_directives={'use_shared_utility': 'pkg2'}) > ) > > > Create all necessary directories, then write the output into the target > directory. No need for a failure. > > Here again `cython` command I would keep consistent. If user wants to > generate file, directory must already exist. This is the same logic as with > compilation of regular .pyx file: > > (cython3.11) root@a3be2cc9ff83:~/dev/cython# cython /foo/bar.pyx > /root/dev/cython3.11/bin/cython: No such file or directory: > '/foo/bar.pyx' > > Moreover, if we decide to create all directories in cython command, should > we create also `__init__.py` files too? I think this is an overkill for > cython command. Of course, in the build system it is different story, build > backend could do it. > > That said, I propose following: > > 1. Introduce the cython command parameter `--generate-shared` which has > the argument name & path of generated C file. E.g. > > cython --generate-shared scipy/_lib/cyshared.c > > will generate `scipy._lib/cyshared.c` file. This approach should be easily > integrated in meson build system based on example in [1]. Also `cython` > command will fail if directory `scipy/_lib` does not exist. Also, > cython will not care whether it is importable. It will just do stupid and > simple C file generation. The rest should be done by build system > (meson/setuptools/whatever). **This will be part of this PR**. > > 2. Introduce new `cython` command parameter `--shared` which will take > fully qualified module name: > > cython --shared=scipy._lib.cyshared.memoryview cython_extention.pyx > > I think here is already agreement between us. We can do some good old > bikeshedding about parameter name. This will be **part of this PR **too. > > 3. Extend existing `cython.build.cythonize()` function. This should be > used in the setuptools: > > from Cython.Build import cythonize > from setuptools import setup, Extension > from Cython.Compiler import Options > import numpy > > extensions = [ > Extension("*", ["**/*.pyx"], include_dirs=[numpy.get_include()]), > ] > > setup( > ext_modules = cythonize(extensions, shared_module=pkg2.MemoryView) > ) > > This function should also ensures that all directories and `__init__.py` > files are generated, hence the final `.so` module is importable. Currently, > I cannot see all the details and all devils hiding in the details. > Hence, I **would postpone it in the separate issue/PR**. Users of meson > are fine (see [1]) and users setuptools still can use generated and > commited C file in repo. > > > Please let me know what do you think. This is blocking my further progress > in PR and I would like to have clear vision to avoid PR flooding with > changes. > Moreover, I would like to keep the current PR simple and focus on actual > code generation of shared module and its integration into the extentions. > Build related logic can be implemented later on. > > > [1] https://github.com/cython/cython/pull/6531#issuecomment-2551351591
_______________________________________________ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
