[Cython] BUG: cannot override cdef by cpdef method in .pxd
Hello,
I recently learned from Robert Bradshaw that it is legal in Cython to
override cdef methods by cpdef methods. But doing this in a .pxd file
causes a compile-time error:
*foo.pyx*:
cdef class Base(object):
cdef meth(self):
print("Base.meth()")
cdef class Derived(Base):
cpdef meth(self):
print("Derived.meth()")
*foo.pxd*:
cdef class Base(object):
cdef meth(self)
cdef class Derived(Base):
cpdef meth(self)
This gives (both on 0.24.1 and on master):
Error compiling Cython file:
...
cdef class Base(object):
cdef meth(self):
return self
cdef class Derived(Base):
cpdef meth(self):
^
foo.pyx:6:10: 'meth' already defined
As second attempt, I could simply not declare the cpdef method in the
.pxd file. Just use
cdef class Derived(Base):
pass
This compiles, but leads to bugs with broken vtabs. I am omitting the
complete code here, but it's clear why this cannot work: other modules
get the vtab wrong since they don't know about the cpdef slot.
Jeroen.
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] (Possible) bug: module level global vars end up in upper scope?
Yury V. Zaytsev schrieb am 25.07.2016 um 15:27:
> I've hit the following deviation from Python 2: when a Cython module has a
> global variable, somehow, upon importing the module, the global variables
> end up in the *current* (importing) module scope, rather than *imported*
> module scope, and, while normally relatively harmless, in the case of
> namedtuples, for instance, this has caused pickling errors in our
> production code.
>
> test1.py / test2.pyx
>
> import collections
>
> Test = collections.namedtuple("Test", ["test"])
>
> $ ipython
> Python 2.7.6 (default, Jun 22 2015, 17:58:13)
> IPython 1.2.1 -- An enhanced Interactive Python.
>
> In [1]: import test1
>
> In [2]: test1.Test
> Out[2]: test1.Test
>
> In [3]: import test2
>
> In [4]: test2.Test
> Out[4]: __main__.Test
>
> I would appreciate any hints in the case that I'm deeply confused and doing
> something obviously wrong...
namedtuple has an impressively complicated and fragile implementation.
Amongst other things, it does this at the end:
result.__module__ = _sys._getframe(1).f_globals.get(
'__name__', '__main__')
Since Cython modules do not have frames (by default, for performance
reasons), the module it finds then points to the caller instead.
I wonder if we shouldn't consider the module init function a special
(enough) case here that is never performance critical, and just always
generate a frame for it. Later frame lookups would then still fail (so we'd
create somewhat of an inconsistency), but the case above looks like a
legitimate use case, and namedtuples are often (I guess in *most* cases)
created at module init time.
I created a ticket.
https://github.com/cython/cython/issues/536
As a work-around, I could only come up with a hack. You could create a
Python module, import and call into it from your Cython module, create the
namedtuple in Python, and then fix the __module__ reference of the
namedtuple class after the fact. Although I wonder when the insertion into
the module namespace happens. I couldn't find it on a quick look.
Stefan
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel
Re: [Cython] (Possible) bug: module level global vars end up in upper scope?
Hi Stefan,
On Wed, 27 Jul 2016, Stefan Behnel wrote:
I wonder if we shouldn't consider the module init function a special
(enough) case here that is never performance critical, and just always
generate a frame for it. Later frame lookups would then still fail (so we'd
create somewhat of an inconsistency), but the case above looks like a
legitimate use case, and namedtuples are often (I guess in *most* cases)
created at module init time.
I created a ticket.
https://github.com/cython/cython/issues/536
Thank you very much for the insightful analysis, makes total sense!
I agree that creating a frame for the init function sounds like a most
reasonable solution, the only drawback that I can see is the inconsistency
you mentioned, but apparently that's as good as it gets...
As a work-around, I could only come up with a hack. You could create a
Python module, import and call into it from your Cython module, create
the namedtuple in Python, and then fix the __module__ reference of the
namedtuple class after the fact. Although I wonder when the insertion
into the module namespace happens. I couldn't find it on a quick look.
Now that you've explained the root cause, I believe that there is a less
disgusting workaround one could possibly go for, what do you think?
class PyTest(namedtuple('Test', 'test')):
pass
--
Sincerely yours,
Yury V. Zaytsev
___
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel
