Another question popped in my mind. How we will deal with: > Introduce new `cython` command parameter `--shared` which will take fully qualified module name:
On Fri, 14 Feb 2025 at 16:06, matus valo <matusv...@gmail.com> wrote: > >> I think I tend to agree with David. Here is my point of view: >> >> I am not sure how to implement it. cython command *does not know* about >> the build directory. This is more part of the build system >> (setuptools/meson etc). My point is that the cython command should be >> stupid and just generate a C file with a specified name/location nothing >> more. It is also consistent with current usage of it (it just generates C >> files). I understand that it is the build system responsibility to do all >> the magic around it. As an example, here is working integration of current >> "stupid" utility present in the PR implemented by scipy dev (Ralf Gommers) >> [1]: >> https://github.com/scipy/scipy/compare/main...rgommers:scipy:cy-shared-memoryview?expand=1 >> >> The only question is how it can be integrated into setuptools too. The >> most obvious and stupid solution is to generate C file (which was also >> recommended way of working with Cython extension until recently), commit it >> to repository and compile it as regular C extension example is already >> present in test of the PR: >> >> from Cython.Build import cythonize >> from setuptools import setup, Extension >> from Cython.Compiler import Options >> import numpy >> >> extensions = [ >> Extension("*", ["**/*.pyx"], include_dirs=[numpy.get_include()]), >> Extension("pkg2.MemoryView", sources=["pkg2/MemoryView.c"]) >> ] >> >> setup( >> ext_modules = cythonize(extensions, >> compiler_directives={'use_shared_utility': 'pkg2'}) >> ) >> >> > Create all necessary directories, then write the output into the target >> directory. No need for a failure. >> >> Here again `cython` command I would keep consistent. If user wants to >> generate file, directory must already exist. This is the same logic as with >> compilation of regular .pyx file: >> >> (cython3.11) root@a3be2cc9ff83:~/dev/cython# cython /foo/bar.pyx >> /root/dev/cython3.11/bin/cython: No such file or directory: >> '/foo/bar.pyx' >> >> Moreover, if we decide to create all directories in cython command, >> should we create also `__init__.py` files too? I think this is an overkill >> for cython command. Of course, in the build system it is different story, >> build backend could do it. >> >> That said, I propose following: >> >> 1. Introduce the cython command parameter `--generate-shared` which has >> the argument name & path of generated C file. E.g. >> >> cython --generate-shared scipy/_lib/cyshared.c >> >> will generate `scipy._lib/cyshared.c` file. This approach should be >> easily integrated in meson build system based on example in [1]. Also >> `cython` command will fail if directory `scipy/_lib` does not exist. Also, >> cython will not care whether it is importable. It will just do stupid and >> simple C file generation. The rest should be done by build system >> (meson/setuptools/whatever). **This will be part of this PR**. >> >> 2. Introduce new `cython` command parameter `--shared` which will take >> fully qualified module name: >> >> cython --shared=scipy._lib.cyshared.memoryview cython_extention.pyx >> >> I think here is already agreement between us. We can do some good old >> bikeshedding about parameter name. This will be **part of this PR **too. >> >> 3. Extend existing `cython.build.cythonize()` function. This should be >> used in the setuptools: >> >> from Cython.Build import cythonize >> from setuptools import setup, Extension >> from Cython.Compiler import Options >> import numpy >> >> extensions = [ >> Extension("*", ["**/*.pyx"], include_dirs=[numpy.get_include()]), >> ] >> >> setup( >> ext_modules = cythonize(extensions, shared_module=pkg2.MemoryView) >> ) >> >> This function should also ensures that all directories and `__init__.py` >> files are generated, hence the final `.so` module is importable. Currently, >> I cannot see all the details and all devils hiding in the details. >> Hence, I **would postpone it in the separate issue/PR**. Users of meson >> are fine (see [1]) and users setuptools still can use generated and >> commited C file in repo. >> >> >> Please let me know what do you think. This is blocking my further >> progress in PR and I would like to have clear vision to avoid PR flooding >> with changes. >> Moreover, I would like to keep the current PR simple and focus on actual >> code generation of shared module and its integration into the extentions. >> Build related logic can be implemented later on. >> >> >> [1] https://github.com/cython/cython/pull/6531#issuecomment-2551351591 > >
_______________________________________________ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
