On 10/05/2022 15:04, Dan Stromberg wrote:
On Tue, May 10, 2022 at 3:15 AM Chris Angelico <ros...@gmail.com> wrote:
> It is often the case that developer write Code in Python and
then convert to a C extension module for performance regions.
>
> A C extension module has a lot of boiler plate code - for
instance the Structures required for each class, the functions for
Module initialization etc.
>
> My Idea is a simple tool that uses introspection tools to take a
Python module and to generate the relevant boiler plate for the
module - including blank functions for the module classes and for
methods. This tool would use type annotations (if given) to make
sensible choices for parameter and attribute types, including
using int and float directly rather than Internal objects
(depending on tool options).
Two things to say about this:
1) Sometimes abandoning a pure python module for a C extension for
performance is a mistake - because Pypy is probably going to be much
faster with the pure python module
Dan,
Thanks for your response, but I think PyPy will have a long way to go to
make JIT generated code more efficient than hand crafted C extension.
for instance PyPy will almost certainly be unable to determine if
integers, floats etc will need to use the Python runtime or can be
optimized to use direct C types, another example would be lists - there
are some cases where the C extension will always be a lot more efficient
using C arrays than using the C list runtime machinery.
While there might be some cases where PyPy will be more efficient, I
don't think that will be the case for all programs, and that ignores the
fact that PyPi has major issues with the CAPI.
2) I've had some luck using m4 to maintain a single source file that
is used to automatically generate both pure python and cython. This
is a little like using cpp in a C project.
For examples of #2, perhaps see:
https://stromberg.dnsalias.org/~strombrg/treap/
https://stromberg.dnsalias.org/svn/rolling_checksum_mod/trunk/
https://stromberg.dnsalias.org/~strombrg/sort-comparison/
It's often nice to keep the lines of the pure-python and cython having
a 1-1 relationship, so that tracebacks report useful line numbers
either way. However, in the treap example I've dispensed with that
because some methods were almost identical but had some boilerplate -
and m4 was able to handle that nicely at the cost of lines being 1-1.
HTH
--
Anthony Flury
*Moble*: +44 07743 282707
*Home*: +44 (0)1206 391294
*email*: anthony.fl...@btinternet.com
--
https://mail.python.org/mailman/listinfo/python-list
