Snap! Clicked send by mistake. Another question popped in my mind. How we will deal with:
> Introduce new `cython` command parameter `--shared` which will take fully qualified module name: When we will introduce new `--shared` option we will have 3 places where we can specify shared module ffully qualified module name: 1. New Cython parameter `--shared` foo.bar 2. Cython Directive parameter: `-X use_shared_utility=foo.bar 3. Cython module comment: `# cython: use_shared_utility=foo.bar What is the priority between 1., 2. and 3.? I suppose 1. and 2. should override 3. But what we should do if 1. and 2. are set but have different values? Matus On Sat, 15 Feb 2025 at 00:00, matus valo <matusv...@gmail.com> wrote: > Another question popped in my mind. How we will deal with: > > Introduce new `cython` command parameter `--shared` which will take > fully qualified module name: > > On Fri, 14 Feb 2025 at 16:06, matus valo <matusv...@gmail.com> wrote: > >> >>> I think I tend to agree with David. Here is my point of view: >>> >>> I am not sure how to implement it. cython command *does not know* about >>> the build directory. This is more part of the build system >>> (setuptools/meson etc). My point is that the cython command should be >>> stupid and just generate a C file with a specified name/location nothing >>> more. It is also consistent with current usage of it (it just generates C >>> files). I understand that it is the build system responsibility to do all >>> the magic around it. As an example, here is working integration of current >>> "stupid" utility present in the PR implemented by scipy dev (Ralf Gommers) >>> [1]: >>> https://github.com/scipy/scipy/compare/main...rgommers:scipy:cy-shared-memoryview?expand=1 >>> >>> The only question is how it can be integrated into setuptools too. The >>> most obvious and stupid solution is to generate C file (which was also >>> recommended way of working with Cython extension until recently), commit it >>> to repository and compile it as regular C extension example is already >>> present in test of the PR: >>> >>> from Cython.Build import cythonize >>> from setuptools import setup, Extension >>> from Cython.Compiler import Options >>> import numpy >>> >>> extensions = [ >>> Extension("*", ["**/*.pyx"], include_dirs=[numpy.get_include()]), >>> Extension("pkg2.MemoryView", sources=["pkg2/MemoryView.c"]) >>> ] >>> >>> setup( >>> ext_modules = cythonize(extensions, >>> compiler_directives={'use_shared_utility': 'pkg2'}) >>> ) >>> >>> > Create all necessary directories, then write the output into the >>> target directory. No need for a failure. >>> >>> Here again `cython` command I would keep consistent. If user wants to >>> generate file, directory must already exist. This is the same logic as with >>> compilation of regular .pyx file: >>> >>> (cython3.11) root@a3be2cc9ff83:~/dev/cython# cython /foo/bar.pyx >>> /root/dev/cython3.11/bin/cython: No such file or directory: >>> '/foo/bar.pyx' >>> >>> Moreover, if we decide to create all directories in cython command, >>> should we create also `__init__.py` files too? I think this is an overkill >>> for cython command. Of course, in the build system it is different story, >>> build backend could do it. >>> >>> That said, I propose following: >>> >>> 1. Introduce the cython command parameter `--generate-shared` which has >>> the argument name & path of generated C file. E.g. >>> >>> cython --generate-shared scipy/_lib/cyshared.c >>> >>> will generate `scipy._lib/cyshared.c` file. This approach should be >>> easily integrated in meson build system based on example in [1]. Also >>> `cython` command will fail if directory `scipy/_lib` does not exist. Also, >>> cython will not care whether it is importable. It will just do stupid >>> and simple C file generation. The rest should be done by build system >>> (meson/setuptools/whatever). **This will be part of this PR**. >>> >>> 2. Introduce new `cython` command parameter `--shared` which will take >>> fully qualified module name: >>> >>> cython --shared=scipy._lib.cyshared.memoryview cython_extention.pyx >>> >>> I think here is already agreement between us. We can do some good old >>> bikeshedding about parameter name. This will be **part of this PR **too. >>> >>> 3. Extend existing `cython.build.cythonize()` function. This should be >>> used in the setuptools: >>> >>> from Cython.Build import cythonize >>> from setuptools import setup, Extension >>> from Cython.Compiler import Options >>> import numpy >>> >>> extensions = [ >>> Extension("*", ["**/*.pyx"], include_dirs=[numpy.get_include()]), >>> ] >>> >>> setup( >>> ext_modules = cythonize(extensions, shared_module=pkg2.MemoryView) >>> ) >>> >>> This function should also ensures that all directories and `__init__.py` >>> files are generated, hence the final `.so` module is importable. Currently, >>> I cannot see all the details and all devils hiding in the details. >>> Hence, I **would postpone it in the separate issue/PR**. Users of meson >>> are fine (see [1]) and users setuptools still can use generated and >>> commited C file in repo. >>> >>> >>> Please let me know what do you think. This is blocking my further >>> progress in PR and I would like to have clear vision to avoid PR flooding >>> with changes. >>> Moreover, I would like to keep the current PR simple and focus on actual >>> code generation of shared module and its integration into the extentions. >>> Build related logic can be implemented later on. >>> >>> >>> [1] https://github.com/cython/cython/pull/6531#issuecomment-2551351591 >> >>
_______________________________________________ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
