[issue45256] Remove the usage of the C stack in Python to Python calls
Christian Tismer added the comment: Very much appreciated approach. Too bad that things stop when people are writing extensions as usual. Or do you think we can teach them how to avoid the C stack? Anyway, good luck! -- ___ Python tracker <https://bugs.python.org/issue45256> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45256] Remove the usage of the C stack in Python to Python calls
Christian Tismer added the comment: FWIW, getting all function to avoid the C stack will most probably take a long time, if it happens at all. Especially functions which might call into Python multiple times must be re-designed heavily, turned into multiple pieces to be in tail position. Been there, long ago... :) -- ___ Python tracker <https://bugs.python.org/issue45256> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45256] Remove the usage of the C stack in Python to Python calls
Christian Tismer added the comment: FYI., in Stackless Python I built a chain of frames by a double linked list. This was the replacement for the current frame/stack mix. Debugging was still quite easy, following this frame chain. Note that it is a rather easy step to add the capability to squirrel the whole current chain away and replace it by another one. Add some header to such a chain, and you can call it "Tasklet". -- ___ Python tracker <https://bugs.python.org/issue45256> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45256] Remove the usage of the C stack in Python to Python calls
Christian Tismer added the comment: Hey guys, you know that you are about to implement the core idea of Stackless Python, right? :-D -- nosy: +Christian.Tismer ___ Python tracker <https://bugs.python.org/issue45256> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15870] PyType_FromSpec should take metaclass as an argument
Christian Tismer added the comment: Since PyPy does not use the Limited API, PySide can quite easily work around the limitations by directly working with the type object. But the usage of PyType_Modified() would make very much sense for PySide‘s new switchable features. That would work immediately without any change, because we already use that function to invalidate Python 3.10‘s type cache. -- nosy: +Christian.Tismer ___ Python tracker <https://bugs.python.org/issue15870> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45061] [C API] Detect refcount bugs on True/False in C extensions
Christian Tismer added the comment: > Apart None, True and False, do you know other objects which must not be > deleted? Yes, what I can think of is the immutable empty tuple singleton which is such a candidate to be forgotten. -- ___ Python tracker <https://bugs.python.org/issue45061> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45061] [C API] Detect refcount bugs on True/False in C extensions
Christian Tismer added the comment: Ah, that would be really much to store: every object with a refcount going to zero would have to be memorized :/ -- ___ Python tracker <https://bugs.python.org/issue45061> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45061] [C API] Detect refcount bugs on True/False in C extensions
Christian Tismer added the comment: What about an even more flexible solution? A debug option could memorize always the last object deallocated and give full info (the object's repr) before the crash would happen. -- nosy: +Christian.Tismer ___ Python tracker <https://bugs.python.org/issue45061> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30459] PyList_SET_ITEM could be safer
Christian Tismer added the comment: Congrats to that change! -- nosy: +Christian.Tismer ___ Python tracker <https://bugs.python.org/issue30459> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue35810] Object Initialization does not incref Heap-allocated Types
Christian Tismer added the comment: > How does this change affect stable ABI? Is it necessary to change the > logic in modules that use only the Py_LIMITED_API? If you use heap types, you need to adjust refcounts beginning with Python 3.8 . And since the Py_LIMITED_API uses heap types without hiding these refcount changes by an API call, this breaks the stable API. But it is easily fixable by a runtime version check, or you start over with Python 3.8 as lowest version . -- ___ Python tracker <https://bugs.python.org/issue35810> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39016] Negative Refcount in Python 3.8
Change by Christian Tismer : -- resolution: -> not a bug status: open -> closed ___ Python tracker <https://bugs.python.org/issue39016> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39016] Negative Refcount in Python 3.8
Change by Christian Tismer : -- stage: patch review -> resolved ___ Python tracker <https://bugs.python.org/issue39016> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39016] Negative Refcount in Python 3.8
Christian Tismer added the comment: Sorry, I believe I was wrong and lookup_maybe_method does not return a borrowed reference. _PyType_Lookup does and I was confused. Closing that for now. -- ___ Python tracker <https://bugs.python.org/issue39016> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39016] Negative Refcount in Python 3.8
Christian Tismer added the comment: On 11.12.19 17:37, Petr Viktorin wrote: > > Petr Viktorin added the comment: > > The goal now should be to find a reasonably small reproducer. > > I'm trying to compile PySide to see what it does, but it's a big project and > I'm a bit lost. > Christian, you said in an e-mail that you have a workaround -- unsetting the > Py_TPFLAGS_METHOD_DESCRIPTOR. Could you point me to the commit with that > workaround, so I could try exploring the code in context? Yes, it went into gerrit here: https://codereview.qt-project.org/c/pyside/pyside-setup/+/282705 The problem is that during PySide type creation the PyType_Ready function looks into the mro() method, which uses this new flag. When I temporary remove that, everything works. Cheers -- Chris -- ___ Python tracker <https://bugs.python.org/issue39016> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39016] Negative Refcount in Python 3.8
Christian Tismer added the comment: On 11.12.19 14:13, Petr Viktorin wrote: > > Petr Viktorin added the comment: > > lookup_maybe_method should not return a borrowed reference. It increfs its > return value. At second sight, this seems to be true. No idea why I was so convinced. Need to sleep and look again.. -- ___ Python tracker <https://bugs.python.org/issue39016> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39016] Negative Refcount in Python 3.8
Christian Tismer added the comment: On 10.12.19 17:57, Pablo Galindo Salgado wrote: > > Pablo Galindo Salgado added the comment: > >> No, this appears to be impossible. > > Oh, I see. Apologies then for the misunderstunding. No problem! You could as well have been right. I tried to move the patch to master and realized only then that there was nothing similar. Interesting btw., I should investigate when this divergence was introduced. Best -- Chris -- ___ Python tracker <https://bugs.python.org/issue39016> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39016] Negative Refcount in Python 3.8
Christian Tismer added the comment: On 10.12.19 17:57, Pablo Galindo Salgado wrote: > > Pablo Galindo Salgado added the comment: > >> No, this appears to be impossible. > > Oh, I see. Apologies then for the misunderstunding. Well, but I think that is weird, too! Why should that custom clause be in 3.8 but not in master? -- ___ Python tracker <https://bugs.python.org/issue39016> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39016] Negative Refcount in Python 3.8
Christian Tismer added the comment:
No, this appears to be impossible.
The function "type_mro_modified" exists as well, but there is no
"if (custom) {" branch at all!
On 10.12.19 17:21, Pablo Galindo Salgado wrote:
>
> Pablo Galindo Salgado added the comment:
>
>>From the PR:
>
> Christian, can you open the PR against master instead? We will backport the
> change to 3.8 after is merged.
>
> --
> nosy: +pablogsal
>
> ___
> Python tracker
> <https://bugs.python.org/issue39016>
> ___
>
--
___
Python tracker
<https://bugs.python.org/issue39016>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39016] Negative Refcount in Python 3.8
New submission from Christian Tismer :
By the new Py_TPFLAGS_METHOD_DESCRIPTOR flag, a new code path is
activated, and when extension types like PySide create a new
class, we observe negative refcounts.
The reason is that the code in typeobject.c fkt. type_mro_modified
calls lookup_maybe_method which returns a _borrowed_ reference.
This happens in the "if (custom) {" branch.
Removing all Py_XDECREF calls from the function fixes that.
--
components: Extension Modules
messages: 358198
nosy: Christian.Tismer
priority: critical
severity: normal
status: open
title: Negative Refcount in Python 3.8
type: crash
versions: Python 3.8
___
Python tracker
<https://bugs.python.org/issue39016>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39016] Negative Refcount in Python 3.8
Change by Christian Tismer : -- keywords: +patch pull_requests: +17029 stage: -> patch review pull_request: https://github.com/python/cpython/pull/17555 ___ Python tracker <https://bugs.python.org/issue39016> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue35810] Object Initialization does not incref Heap-allocated Types
Christian Tismer added the comment: Neil, that is the absolute super-move! When all types are heap types, then I have no longer the problem that I cannot get at slots from builtin types, with all are static. I am very much for that change, because then I can make my stable ABI implementation of PySide much cleaner. -- ___ Python tracker <https://bugs.python.org/issue35810> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue35810] Object Initialization does not incref Heap-allocated Types
Christian Tismer added the comment: Thanks for including me here! We have indeed converted everything to new style types but saw no refcount problems, yet. This will probably come after the patch. We will add an issue to the tracker for Python 3.8. -- ___ Python tracker <https://bugs.python.org/issue35810> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26979] The danger of PyType_FromSpec()
Christian Tismer added the comment: The default of PyType_FromSpec for tp_dealloc is wrong! --- After a long struggle with crashes and leaks, the situation was finally clarified: When a type is converted from a static type to a heaptype via PyType_FromSpec, and the tp_dealloc field is NULL, then subtype_dealloc is inserted. This is not correct. The reasoning in the code is that a heaptype has subtype_dealloc. But before the conversion, the type was static, and for static types the function object_dealloc was inserted. In the given type, the only change is to become a heaptype. The by default inserted object_dealloc should not be changed, since the existing code was written with object_dealloc in mind and not the consequences of replacing it with subtype_dealloc. Before this solution, I used a dummy function to prevend subpype_dealloc being inserted, but that caused leaks. It was hard to understand that the default for a static type is object_dealloc. After that, it was easy to fix that: The now correctly working workaround is to explicitly insert an object_dealloc function whenever the default for tp_dealloc is NULL. Again, in order to use this fix, it is necessary to break the Limited API, because in order to write an object_dealloc function (it is not public) you need access to type objects. -- ___ Python tracker <https://bugs.python.org/issue26979> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26979] The danger of PyType_FromSpec()
Christian Tismer added the comment: If somebody is interested to see how the transition to the Limited API was done, here is the documentation that I was forced to write :) https://github.com/pyside/pyside2-setup/blob/5.11/sources/shiboken2/libshiboken/pep384impl_doc.rst -- ___ Python tracker <https://bugs.python.org/issue26979> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26979] The danger of PyType_FromSpec()
Christian Tismer added the comment: > ... Specifically, you need a way to create class with a metaclass, from C. > > Is that right? Or was this only an example of a larger problem? Yes, you are exactly right. I needed access to very few fields. In particular: PyTypeObject.tp_init PyTypeObject.tp_basicsize(workaround using Python) .tp_dict The latter would not be necessary for Pep 384, but for my __signature__ extension, but that is another story :) > Well, basicsize might be exposed through Python, but it's still not > part of the limited API. Which is fine – all that means is you might > need to change the extension for some future version of Python. Oh? I thought it is automatically allowed to use if it is exposed by Python. I thought that the central property of basicsize would be that it is no longer a constant, but that a size is always there. Sure, we could have worked without the size and create a more complex structure that does not know anything about types. I actually started with that approach. But in the end I thought it would not hurt to assume that there is a (variable) size, as long as we are below Python 4.0 . > I'm sure your tests will tell you when the time comes, and I hope we'll > have a better solution then! Yes, I had a very hard time to convince myself that I _may_ use some assumptions, as long as I always prove that these assumptions are right. But in a way, I feel guilty and would prefer to go without any trickery. > BTW, I'm honestly very impressed how far PySide got with the limited API! Thank you! I was hoping to get ready after 2 months, when I realized that all types needed to be changed. So I pulled teeth, and it took >5 months. Btw., I don't understand how the PyQt5 guy(s) solved this. Maybe his structure is way cleaner and simpler than the PySide mess? I'll ask him. Ciao -- Chris -- ___ Python tracker <https://bugs.python.org/issue26979> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26979] The danger of PyType_FromSpec()
Christian Tismer added the comment: > 1) If I understand correctly, this problem could be solved by > per-class C storage? Something like PyModuleDef.m_size / > PyModule_GetState, but for classes? No. To obtain sizeof(PyType_Type), we simply used the Python code type.__basicsize__ It is a simple trick, but probably very few people know this path to get that info. At least, for me it took way too long ;-) > 2) Making PyType_GetSlot work for static types would not be trivial. > All types do have the same *basic* structure (the tp_* fields), but > the nb_*/sq_*/etc. fields don't need to be in the same order, and > might not exist at all. Even making it work for the tp_* fields only > would bake in some assumptions I do want to get rid of. That's my question. Do you want to hide the fact that some fields are direct and others are indirect? I don't think that will ever be changed before Python 4.0, so why put this burden on every user, then? I would simply give access to tp_* for all types and that's it. > I'd like to > understand your use case better. Do you have a specific example > here? Yes. I need access to PyType_Type's tp_new field. PyType_Type is a very central structure in Python, and it really hurts to hide every attribute away. You can see the use of this field here: https://github.com/pyside/pyside2-setup/blob/5.11/sources/shiboken2/libshiboken/basewrapper.cpp#L319 // The meta type creates a new type when the Python programmer extends a wrapped C++ class. newfunc type_new = reinterpret_cast(PyType_Type.tp_new); SbkObjectType *newType = reinterpret_cast(type_new(metatype, args, kwds)); if (!newType) return 0; Not being able to use this field led to many weird workaround attempts, which did not really work. We tried to call this function from Python code, but then there are several checks which make it impossible to use. We could maybe generate a heaptype clone of PyType_Type and grab the function from there. But this is quite obscure and cannot be the recommended way to get at the tp_new function? Maybe there also is a way to avoid the use of this function all together. But we did not want to re-implement a central function of a huge project that we only understood good enough to change it a bit. Maybe we approach it again. At that time, it was not even clear that we would succeed with the limited API. Now that we know, we can try more. -- ___ Python tracker <https://bugs.python.org/issue26979> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26979] The danger of PyType_FromSpec()
Christian Tismer added the comment: Petr, Fine! PySide's problems are solved for now, no worries. But as mentioned, there is more. Problem 1) -- There is another thing that gives problems: When creating types which have fields "behind" the normal type fields, PySide had explicit structs that contained PyHeaptypeObject explicitly, plus the extra fields. We changed that into a normal PyType declaration (opaque) and a pointer expression that dynamically computed the offset. For that, we needed sizeof(PyType_Type). Ok, that could be solved by Python code, but this "trick" is not so obvious and should be documented, or a variable should be provided that gives the heap type size, dynamically. Problem 2) -- Harder is this: For some operations, we needed access to fields of normal types, for instance the tp_new field of PyType_Type. That is not possible, although it would be easy: PyType_GetSlot forbids access to normal types, rigorously. But concerning the "normal" type fields, those which are not indirections, it would cost nothing to allow this for normal types. This uses a property of types which has not explicitly been used: * All types have the same basic structure * That means, we can allow access to all "tp_" fields with simply changing a check in PyType_GetSlot. I'm not sure if that information should be made official. May be it should be hidden and PyType_GetSlot should be made more complicated. But the proposed change comes at no cost. And the prefixes like "nb_" or "sq_" are still visible, so I don't think there was an intent to make type structures completely opaque? For PySide, it was crucial to use that information to compute the offset of certain fields of normal types, which is a bit of a Pep 384 breach. We wrote code that verifies that these assumptions are all valid, but I would be happy to remove that and revert to only using PyType_GetSlot. Please let me know if I can help with something! Cheers - Chris -- ___ Python tracker <https://bugs.python.org/issue26979> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26979] The danger of PyType_FromSpec()
Christian Tismer added the comment:
Hi Petr,
yes I have that what generated the wrong thing, but it is inside
a specific PySide repository in a big project. Before I try to extract
and simulate that, let me just show it.
All types which had been no heaptypes before were already quite complicated
types which had their own deallocators before, or a 0 if they did not
need one, or better "should not have one".
When switching to the new interface, those deallocators which were 0 were
turned into the default deallocator, which crashed most of the time.
By replacing it by a dummy function fixed the problem. Example:
static PyType_Slot PySideSignalMetaType_slots[] = {
{Py_tp_methods, (void *)Signal_methods},
{Py_tp_base, (void *)_Type},
{Py_tp_free, (void *)PyObject_GC_Del},
{Py_tp_dealloc, (void *)SbkDummyDealloc},
{0, 0}
};
static PyType_Spec PySideSignalMetaType_spec = {
"PySide2.QtCore.MetaSignal",
0,
// sizeof(PyHeapTypeObject) is filled in by PyType_FromSpecWithBases
// which calls PyType_Ready which calls inherit_special.
0,
Py_TPFLAGS_DEFAULT,
PySideSignalMetaType_slots,
};
You can find the checkins here. This branch contains everything relevant in
small steps.
https://github.com/pyside/pyside2-setup/commit/2f0658be36107097810985f2190fe0f2acfba178
The first working transformation was in the commit
"PEP 384-8-HT-4: Successful Restart using PyType_FromSpec".
I have yet no idea how to extract a minimum example that produces this problem.
Summary:
In a way, without being heaptypes, the old types were dynamic, too, and
inserting
something that was not there before was a bad idea.
(sorry, there is more...)
--
___
Python tracker
<https://bugs.python.org/issue26979>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue34093] Reproducible pyc: FLAG_REF is not stable.
Christian Tismer added the comment: Why must this become slower? To my knowledge, many projects prefer marshal over pickle for suitable simple objects because it is so very fast. I would not throw that away: Would it not be easy to add a named optional keyword argument, like "stable=True"? -- nosy: +Christian.Tismer ___ Python tracker <https://bugs.python.org/issue34093> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue33738] PyIndex_Check conflicts with PEP 384
Christian Tismer added the comment: @Victor I cannot test on Windows because I'm in vacation. But it is very likely similar to bpo-33614 . The three missing symbols which are listed in python3.def do clearly come into existence when the limited API is active. -- ___ Python tracker <https://bugs.python.org/issue33738> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue33738] PyIndex_Check conflicts with PEP 384
Change by Christian Tismer : -- pull_requests: +7207 ___ Python tracker <https://bugs.python.org/issue33738> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue33738] PyIndex_Check conflicts with PEP 384
Christian Tismer added the comment: """Excluding names from limited API can break existing code that use them with defined Py_LIMITED_API.""" How is that different? Right now, the code would break at compile time, because the macros are accessing opaque type fields. Excluding them has the same effect but is much cleaner. My current first approach is conservative, because it only makes things work that didn't work before. Core that is clearly meant as macro is obviously not meant for the limited API. And if it should be better included, I'm all open for it. Right now I want to remove quickly the breakage that was a showstopper. Maybe I misunderstood you and did exactly what you proposed? Please let us start a discussion on python-dev. I think there is more to do to make the limited API really usable in large projects. -- ___ Python tracker <https://bugs.python.org/issue33738> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue33738] PyIndex_Check conflicts with PEP 384
Christian Tismer added the comment: On 07.06.18 17:59, Eric Snow wrote: > > Eric Snow added the comment: > > @Christian, you can use the "blurb" tool to create the NEWS entry. You can > use pip to install it. See: > > > https://devguide.python.org/committing/?highlight=blurb#what-s-new-and-news-entries > > -- > nosy: +eric.snow > > ___ > Python tracker > <https://bugs.python.org/issue33738> > ___ > Thank you, Eric! Will do so. I had something to change, anyway :-) -- ___ Python tracker <https://bugs.python.org/issue33738> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue33738] PyIndex_Check conflicts with PEP 384
Change by Christian Tismer : -- pull_requests: -7096 ___ Python tracker <https://bugs.python.org/issue33738> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue33738] PyIndex_Check conflicts with PEP 384
Change by Christian Tismer : -- pull_requests: -6956 ___ Python tracker <https://bugs.python.org/issue33738> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue33738] PyIndex_Check conflicts with PEP 384
Christian Tismer added the comment: I did not understand the Misc/NEWS.d thing. What should go where, or where would a "skip news" label go? -- ___ Python tracker <https://bugs.python.org/issue33738> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue33738] PyIndex_Check conflicts with PEP 384
Change by Christian Tismer : -- pull_requests: +7100 ___ Python tracker <https://bugs.python.org/issue33738> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue33738] PyIndex_Check conflicts with PEP 384
Change by Christian Tismer : -- pull_requests: +7096 ___ Python tracker <https://bugs.python.org/issue33738> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue33738] PyIndex_Check conflicts with PEP 384
Christian Tismer added the comment: Hi Ned, we had a delivery date yesterday for PySide. The PR is almost ready and will go out today. Ciao - Chris On 07.06.18 09:43, Ned Deily wrote: > > Ned Deily added the comment: > > Christian, any progress on this? 3.7.0rc1 is planned to be tagged in 4 days, > on Monday 2018-06-11. Do you think you will be able to provide a PR before > then? > > -- > > ___ > Python tracker > <https://bugs.python.org/issue33738> > ___ > -- ___ Python tracker <https://bugs.python.org/issue33738> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26979] The danger of PyType_FromSpec()
Christian Tismer added the comment:
There is another catch that belongs in the same category.
There is line 2841 in typeobject.c
if (type->tp_dealloc == NULL) {
/* It's a heap type, so needs the heap types' dealloc.
subtype_dealloc will call the base type's tp_dealloc, if
necessary. */
type->tp_dealloc = subtype_dealloc;
}
When converting the PySide types, it became a real problem. Types with an
existing tp_dealloc behaved normal, but those with tp_dealloc=NULL
gave segfaults. I ended up supplying a default dummy_dealloc()
that is just there to occupy this slot. Then it worked.
And another source of incompatibility:
PyType_FromSpec() enforces a tp_name field to be dotted, to compute a module
name. Very annoying incompatibility that could be avoided.
A small but remarkable glitch:
Most fields of the PyType_Spec/PyType_Slot combo are copied, but some are not.
So expect funny behaviour when creating types with dynamically generated
specs without strdup. That should IMHO be changed.
--
nosy: +Christian.Tismer
___
Python tracker
<https://bugs.python.org/issue26979>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue33738] PyIndex_Check conflicts with PEP 384
Christian Tismer added the comment: Yes, sure, I will submit a patch that tries to reach as much as possible locations that have a similar problem. Of course, the code will only be turned into functions for the PEP context. Takes a day because I need to re-learn a bit how to do this :-) (last patch was 2010, I think... ) -- related personal note - does someone know what happened to Martin? -- ___ Python tracker <https://bugs.python.org/issue33738> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue33738] PyIndex_Check conflicts with PEP 384
Christian Tismer added the comment: Ok, I tried to submit a patch (not yet successful), but as it stands, there are more locations in the code where this will show up with similar problems. Should I take care of these all as much as I can, or is it better to leave it to one of you? Whatever you prefer is ok. I just want the problem to disappear in 3.7, because I hate to break the API any longer. Cheers - Chris -- ___ Python tracker <https://bugs.python.org/issue33738> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue33738] PyIndex_Check conflicts with PEP 384
Change by Christian Tismer : -- keywords: +patch pull_requests: +6956 stage: -> patch review ___ Python tracker <https://bugs.python.org/issue33738> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue33738] PyIndex_Check conflicts with PEP 384
New submission from Christian Tismer : The file number.rst on python 3.6 says """ .. c:function:: int PyIndex_Check(PyObject *o) Returns ``1`` if *o* is an index integer (has the nb_index slot of the tp_as_number structure filled in), and ``0`` otherwise. """ But in reality, this is a macro: """ #define PyIndex_Check(obj) \ ((obj)->ob_type->tp_as_number != NULL && \ (obj)->ob_type->tp_as_number->nb_index != NULL) """ But such a macro does not work with the limited API, since non-heaptypes cannot use PyType_GetSlot. The patch is trivial: Define the function instead of the macro when Py_LIMITED_API is set. I also think the documentation makes more sense when it is correct. -- components: Extension Modules messages: 318436 nosy: Christian.Tismer, larry, ned.deily priority: release blocker severity: normal status: open title: PyIndex_Check conflicts with PEP 384 type: compile error versions: Python 3.4, Python 3.5, Python 3.6, Python 3.7 ___ Python tracker <https://bugs.python.org/issue33738> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21381] Python 3.4+ interpreter built on/with OS X 10.7 deployment target segfaults on 10.8+
Christian Tismer added the comment: Ned, thank you for locating the patch that causes the problem. At least, I could now make my script work, built a patch feature into it. cheers - Chris -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21381 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21381] python build crash on Mavericht
New submission from Christian Tismer: Building python on OSX Mavericks (10.9) of Python 3.4 crashes when this is set: MACOSX_DEPLOYMENT_TARGET=10.7 This happens with OSX 10.9.2, all current updates installed, as of 2014-04-28. Demo script: You can use my attached script to validate this error. The script builds PySide for 16 configurations. Exactly on Python 3.4 / OSX 10.7 it crashes. You can easily tweak the creating loop to just build the crashing case. Cheers -- Chris -- components: Build files: pyside_builder.py messages: 217458 nosy: Christian.Tismer priority: low severity: normal stage: needs patch status: open title: python build crash on Mavericht type: crash versions: Python 3.4 Added file: http://bugs.python.org/file35081/pyside_builder.py ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21381 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21381] python build crash on Mavericht
Christian Tismer added the comment: Ned: In general, we don't test or claim to support building for a deployment target lower than the system being built on. This is not convincing, because the cpython builds are always against $ MACOSX_DEPLOYMENT_TARGET=10.6 . Other builds use the current OS deployment. See for instance homebrew. My script uses them all, from a 10.9 version. This is because I never know what users use for building, and I don't care and build just everything. By your default builds that are uploaded, you do claim that things build for 10.6 exactly, and nothing else. I just think that all targets for all versions should build. Am I mistaken here? -- priority: low - normal ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21381 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19274] make zipfile.PyZipFile more usable
Christian Tismer added the comment: ah, I just see that. The problem was that the checkin drove me nuts. It forced me to run reindent to normalize the code. I did that with my WindIde editor, but this did not help. The point was: Actually an end-of-line was missing at the end of the files. Sorry, I did not see that at all, because the indentation changes are at the end. I usually avoid this strictly. It was just the check-in rejection... -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19274 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19274] make zipfile.PyZipFile more usable
Christian Tismer added the comment: @georg: While reviewing: is it intended that the filter is only called for directories and not for individual files? Not really. I will add this, later. Just wanted to see if this makes sense and it's worth the effort to extend it. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19274 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19274] make zipfile.PyZipFile more usable
Christian Tismer added the comment: added that with tests. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19274 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19274] make zipfile.PyZipFile more usable
Changes by Christian Tismer tis...@stackless.com: -- status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19274 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19274] make zipfile.PyZipFile more usable
Christian Tismer added the comment: Hi Georg, So do you think it is ok this way? I was not sure if extending the function with an optional arg is ok, or if a method to configure PyZipFile would be better. At the moment I just needed the simple functionality. Should it maybe get a regex like compileall.py ? And a general question: What is the right constraint for a filter function? As I wrote it, returning nothing would simply be treated as False. Maybe it is better to enforce a return value which explicitly forbids to be just None, which often just means I forgot the return value ? About feature or fix: Well, I need this for python2.7, because without it, the whole purpose of PyZipFile is pretty questionable. I might argue it a fix, because it crashes on the standard library ;-) Anyway, tell me and I'l add test, docs and put it into dev. cheers - chris -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19274 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19274] make zipfile.PyZipFile more usable
Christian Tismer added the comment: @serhiy.storchaka I don't think this is needed. You can walk a tree and call writepy() for files and directories which you want. What exactly do mean by this and needed? I cannot see the connection of my initial post and your reply. Running PyZipFile on a package dir of the standard lib _does_ traverse the tree, and there is no way to stop it from doing that. That was the whole point of the issue. Please correct me if I'm missing something. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19274 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19274] make zipfile.PyZipFile more usable
Christian Tismer added the comment: Ah, I understand: The case that does not compile comes from the toplevel test folder, which I could have excluded explicitly. But it is not a complete solution: If I want to add every package from the standard lib, then I necessarily encounter enclosed test folders, for instance: Lib/unittest contains Lib/unittest/test Your hint to just not call writepy() on directories which contain files which shouldn't be zipped means that I cannot use writepy at all, because many library packages contain tests. But that is the only purpose of class PyZipFile, therefore the patch. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19274 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19274] make zipfile.PyZipFile more usable
New submission from Christian Tismer: zipfile.PyZipFile needs a filter function. Reason: When creating an archive of the python lib, we don't want the tests. Especially the test file badsyntax_future3.py does not compile. Use case: With this little addition, it becomes very easy to create a zip file of the whole python library. See the attached use case. -- components: Library (Lib) files: zipfile.diff keywords: patch messages: 200088 nosy: Christian.Tismer priority: normal severity: normal status: open title: make zipfile.PyZipFile more usable type: enhancement versions: Python 2.7 Added file: http://bugs.python.org/file32150/zipfile.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19274 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19274] make zipfile.PyZipFile more usable
Christian Tismer added the comment: Here is my use case as an example. With this patch above, I can easily create a .zip file of the standard lib. This was no longer possible at all, after revision 17558, from 2001-04-18: This is a test from __future__ import nested_scopes from __future__ import rested_snopes def f(x): def g(y): return x + y return g print f(2)(4) -- Added file: http://bugs.python.org/file32151/make_libzip.py ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19274 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9035] os.path.ismount on windows doesn't support windows mount points
Christian Tismer added the comment: Hi Tim, Yes, this would be great to get sorted out. Then we could make watchdog.py automatically configure itself for network mounts. Right now this makes no nense because of windows. cheers - chris -- nosy: +Christian.Tismer ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9035 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17941] namedtuple should support fully qualified name for more portable pickling
Christian Tismer added the comment: I would like to make an additional suggestion. (and I implemented this yesterday): Native namedtuple (not a derived class) can be made much simpler to handle when no module and class machinery is involved at all. The way I implemented it has no need for sys._getframes, and it does not store a reference to the class at all. The advantage of doing so is that this maximizes the compatibility with ordinary tuples. Ordinary tuples have no pickling issue at all, and this way the named tuple should behave as well. My implementation re-creates the namedtuple classes on the fly by a function in __reduce_ex__. There is no speed penalty for this because of caching the classes by their unique name and set of field names. This is IMHO the way things should work: A namedtuple replaces a builtin type, so it has the same pickling behavior: nothing needed. Rationale: tuples are used everywhere and dynamically. namedtuple should be as compatible to that as possible. By having to specify a module etc., this dynamic is partially lost. Limitation: When a class is derived from namedtuple, pickling support is no longer automated. This is compatible with ordinary tuples as well. Cheers - Chris -- hgrepos: +198 nosy: +Christian.Tismer ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17941 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
