I've updated the code. Flags are now set only when required, not on all types wholesale. It is possible to disable this with a new macro, STACKLESS_NO_TYPEINFO See how this affects your crash scenario
(still runs all the stackless unittests. with the STACKLESS_NO_TYPEINFO == 1 flag, it fails in the expected places when checking for type flags. K > -----Original Message----- > From: [email protected] [mailto:stackless- > [email protected]] On Behalf Of Kristján Valur Jónsson > Sent: 6. febrúar 2014 09:28 > To: The Stackless Python Mailing List > Subject: Re: [Stackless] Windows 2.7.6 stability > > I'm trying to familiarize myself with the code too :) > > There are two inheritance scenarios: > 1) when a non-heap class inherits from a base class. It then gets copies of > the slots from the base. > 2) When a heap class inherits from a base class. It then gets copies of the > slots from the base, but after any local slots have been filled with custom > __next__ etc. > > Having looked at the code some more and tried to understand all this > "stackless" flag passing, I've come to the conclusion that > 1) It is okay if we miss the 'can do stackless' flag on a slot. if a > stackles-aware > function is called with 'stackless==0' it just behaves as normal. So, passing > the correct stackless flags for the slots is good, since it enables the > stackless > performance enhancement, but it is not strictly necessary. In other words, > we would get away with never copying this property. On the other hand we > can also be more clever in copying the flags, particulary for external types > that inherit from one of the builtins.... > 3) there is only one flag per method so we might use a bitfield. > 4) Only a handful of types actually have the stackless property, that is, have > slots that can do stackless. It is possible to optimize stuff greatly by > adding > the stackless flag only to those particular ones during initialization. > > 5) This change causes a binary incompatibility with external modules that > were compiled using the stackless headers previously, since they would use > an extended type object. > > I think 5 is the only possible cause for concern here. For that reason, we > _may_ want to employ a new flag, one that means 'extensions are in the > Mapping member', and ignore the old one, but only if you think this is a > concern. > > I'M going to make a new version of this now, one that isn't as intrusive by > selectively patching only those types that we deem matter.... Using that > paradigm, we may, for example, disable the stackless flag feature altogether > at compile time, and then we can see if it has anything to do at all with the > PySide crashes. > > K > > > > > -----Original Message----- > > From: [email protected] [mailto:stackless- > > [email protected]] On Behalf Of Richard Tew > > Sent: 6. febrúar 2014 01:46 > > To: The Stackless Python Mailing List > > Subject: Re: [Stackless] Windows 2.7.6 stability > > > > Where do types inherit a bunch of slots from the builtin type? It's > > been a while since I had to be familiar with that part of the code. > > > > Should your changes have resulted in something that crashes harder and > > faster (and more consistently) than the old heap type version? > > > > Cheers, > > Richard. > > > > On 2/6/14, Kristján Valur Jónsson <[email protected]> wrote: > > > I think that the problem may be a bit more insidious than just the > > > size of the PyTypeObject. > > > Imagine this scenario: > > > An extension creates a C extension of a builtin type. This > > > extension is compiled against vanilla Python. So, its own default > > > flags don't have the STACKLESS flag. But it does inherit a bunch of > > > slots from the builtin stackless object, and those are defined to be > > > STACKLESS aware. > > > > > > so, stackless will call those inherited slots thinking they don't > > > adhere to the STACKLESS protocol. > > > > > > Is it safe to call "stackless" type calls using the old method? Or > > > is this purely an "added extra", so that the new, stackless aware, > > > slot methods _can_ work with the old calling mechanism? I don't > > > know. Hm, perhaps they do.... In which case, the stackless flag is > > > an opt-in thing. If it is missing, even for a stackless method, then it > > > would > be ok..... > > > > > > > > > I think that with the method here, of having the STACKLESS flag > > > indicate that there is extra data in the PyMappingMethods structure, > > > we can in fact apply the STACKLESS flag to any inherited type, which > > > calls PyType_Ready(). If the parent type has the STACKLESS flag, we > > > can modify its tp_as_mapping member to point to a dynamically > > > allocated, larger, structure, and then proceed to do inheritance > > > correctly. > > > > > > So, we can inherit the stackless flag for subtypes that were > > > compiled unaware of stackless. If PySide were for example to create > > > a subtype of PyFunctionType, then we could decorate it with the > > > STACKLESS flag and have it inherit the stackless slot flags, by > > > giving the subtype a new tp_as_mapping member. > > > > > > K > > > > > > > > > ________________________________________ > > > From: [email protected] > > > [[email protected]] on behalf of Richard Tew > > > [[email protected]] > > > Sent: Wednesday, February 05, 2014 19:58 > > > To: The Stackless Python Mailing List > > > Subject: Re: [Stackless] Windows 2.7.6 stability > > > > > > On one hand I wonder why we just didn't do this long ago. > > > > > > On the other, it crashes :-) I'll look into it later as I am > > > celebrating Waitangi Day in the traditional way of sitting around > > > doing nothing. > > > > > > When this is working, I'll rerelease 2.7.6 and try and get people to > > > rebuild all the binaries. > > > > > >> python27.dll!getset_get(PyGetSetDescrObject * > > >> descr=0x0056a1e8, _object * obj=0x03848b98, _object * > > >> type=0x00000000) Line 147 + 0xa > > bytes > > >> C > > > python27.dll!PyObject_Call(_object * func=0x0056a1e8, _object * > > > arg=0x03848b98, _object * kw=0x00000000) Line 2539 + 0x1c bytes C > > > python27.dll!PyObject_CallFunctionObjArgs(_object * > > > callable=0x0056a1e8, ...) Line 2786 + 0x13 bytes C > > > python27.dll!build_class(_object * methods=0x0384c390, _object * > > > bases=0x1e0efddf, _object * name=0x037ba270) Line 5117 + 0xf bytes > C > > > python27.dll!PyEval_EvalFrame_value(_frame * f=0x00502428, int > > > throwflag=58434160, _object * retval=0x0384c390) Line 3394 C > > > python27.dll!PyEval_EvalFrameEx_slp(_frame * f=0x00502428, int > > > throwflag=0, _object * retval=0x1e234ff4) Line 964 + 0x17 bytes C > > > python27.dll!slp_eval_frame_newstack(_frame * f=0x00502428, int > > > exc=0, _object * retval=0x1e234ff4) Line 521 + 0x1b bytes C > > > python27.dll!PyEval_EvalFrameEx_slp(_frame * f=0x00502428, int > > > throwflag=0, _object * retval=0x1e234ff4) Line 964 C > > > python27.dll!slp_frame_dispatch_top(_object * retval=0x1e234ff4) > > > Line 810 + 0x9 bytes C > > > python27.dll!slp_run_tasklet(_frame * f=0x00502428) Line 1504 C > > > python27.dll!slp_eval_frame(_frame * f=0x00502428) Line 317 + 0xa > > > bytes C > > > python27.dll!climb_stack_and_eval_frame(_frame * > > > f=0x00502428) Line > > > 274 + 0x9 bytes C > > > python27.dll!slp_eval_frame(_frame * f=0x00502428) Line 303 + 0x6 > > > bytes C > > > python27.dll!PyEval_EvalCodeEx(PyCodeObject * co=0x022faa40, > > > _object > > > * globals=0x00502428, _object * locals=0x004bc930, _object * * > > > args=0x00000000, int argcount=0, _object * * kws=0x00000000, int > > > kwcount=0, _object * * defs=0x00000000, int defcount=0, _object * > > > closure=0x00000000) Line 3697 + 0x6 bytes C > > > python27.dll!PyEval_EvalCode(PyCodeObject * co=0x022faa40, > > > _object > > > * > > > globals=0x004bc930, _object * locals=0x004bc930) Line 674 + 0x22 > > > bytes C > > > python27.dll!run_mod(_mod * mod=0x023613e8, const char * > > > filename=0x00000000, _object * globals=0x004bc930, _object * > > > locals=0x004bc930, PyCompilerFlags * flags=0x00000000, _arena * > > > arena=0x00000000) Line 1408 + 0x16 bytes C > > > python27.dll!PyRun_FileExFlags(_iobuf * fp=0x73247408, const > > > char * filename=0x00354be4, int start=257, _object * > > > globals=0x004bc930, _object * locals=0x004bc930, int closeit=1, > > PyCompilerFlags * > > > flags=0x0027ff00) Line 1393 C > > > python27.dll!PyRun_SimpleFileExFlags(_iobuf * fp=0x73247408, > > > const char * filename=0x00354be4, int closeit=1, PyCompilerFlags * > > > flags=0x0027ff00) Line 967 + 0x18 bytes C > > > python27.dll!PyRun_AnyFileExFlags(_iobuf * fp=0x73247408, > > > const char > > > * filename=0x00354be4, int closeit=1, PyCompilerFlags * > > > flags=0x0027ff00) Line 770 + 0x11 bytes C > > > python27.dll!Py_Main(int argc=2, char * * argv=0x00354ba0) > > > Line > > > 643 > > > + 0x27 bytes C > > > python.exe!__tmainCRTStartup() Line 582 + 0x17 bytes C > > > > > > From the output window... > > > > > > HEAP[python.exe]: HEAP: Free Heap block 3812418 modified at 3812438 > > > after it was freed Windows has triggered a breakpoint in python.exe. > > > > > > This may be due to a corruption of the heap, which indicates a bug > > > in python.exe or any of the DLLs it has loaded. > > > > > > This may also be due to the user pressing F12 while python.exe has focus. > > > > > > The output window may have more diagnostic information. > > > > > > Cheers, > > > Richard. > > > > > > On 2/6/14, Kristján Valur Jónsson <[email protected]> wrote: > > >> Ok, I have a proposed patch in a branch on > > >> https://bitbucket.org/krisvale/stackless-scratch > > >> > > >> The idea is to move the flags from the PyTypeObject into the > > >> PyMappingMethods structure, which is unlikely to be extended by > > >> third party apps. > > >> This runs the stackless testsuite. > > >> > > >> K > > >> > > >> From: [email protected] > > >> [mailto:[email protected]] On Behalf Of lars van > > >> Gemerden > > >> Sent: 5. febrúar 2014 12:17 > > >> To: The Stackless Python Mailing List > > >> Subject: Re: [Stackless] Windows 2.7.6 stability > > >> > > >> I would like to see better PySide compatibility, since my code > > >> relies on stackless and PySide. > > >> > > >> The crashes have become less though since maybe a year ago; maybe > > >> some improvements on the PySide? > > >> > > >> I am using stackless python 2.7.5. > > >> > > >> CL > > >> > > >> On Wed, Feb 5, 2014 at 9:55 AM, Kristján Valur Jónsson > > >> <[email protected]<mailto:[email protected]>> wrote: > > >> At one point I had a patch going in stackless, which changed the > > >> way we extended PyHeapType, IIRC. > > >> I think this is the way to go, make sure we stay compatible. > > >> > > >>> -----Original Message----- > > >>> From: > > >>> [email protected]<mailto:stackless- > > [email protected] > > >>> om> [mailto:stackless-<mailto:stackless-> > > >>> [email protected]<mailto:[email protected]>] On Behalf > Of > > >>> Richard Tew > > >>> Sent: 4. febrúar 2014 20:11 > > >>> To: The Stackless Python Mailing List > > >>> Subject: Re: [Stackless] Windows 2.7.6 stability > > >>> > > >>> Well, I think Christian was deep into it, and trying to get the > > >>> PySide people to do a compatibility patch or something. Last I > > >>> recall, from several months ago. > > >>> > > >>> It is indeed a pity we can't say we're compatible with all Python > > >>> extensions, and it would be good to get it fixed. > > >>> > > >>> Cheers, > > >>> Richard. > > >>> > > >>> On 2/4/14, Kristján Valur Jónsson > > >>> <[email protected]<mailto:[email protected]>> wrote: > > >>> > This is so annoying. Time for another stab at this problem, perhaps? > > >>> > > > >>> >> -----Original Message----- > > >>> >> From: > > >>> >> [email protected]<mailto:stackless- > > bounces@stackles > > >>> >> s.com> [mailto:stackless-<mailto:stackless-> > > >>> >> [email protected]<mailto:[email protected]>] On > Behalf > > Of > > >>> >> Richard Tew > > >>> >> Sent: 3. febrúar 2014 18:53 > > >>> >> To: The Stackless Python Mailing List > > >>> >> Subject: Re: [Stackless] Windows 2.7.6 stability > > >>> >> > > >>> >> No, thanks for the suggestion. But this is definitely PySide > > >>> >> and it's modification of the base object type that stackless > > >>> >> also modifies. > > >>> >> > > >>> >> Cheers, > > >>> >> Richard. > > >>> >> > > >>> >> On 2/4/14, Anselm Kruis > > >>> >> <[email protected]<mailto:a.kruis@science- > > computing.de > > >>> >> >> > > >>> >> wrote: > > >>> >> > Could it be related to the PGO optimised build? > > >>> >> > As far as I know, the mainline python installer is build > > >>> >> > without PGO. > > >>> >> > Our installer is build with PGO optimisation. > > >>> >> > > > >>> >> > Another question: can you try our 2.7.5 build? It is PGO > > >>> >> > optimized too. > > >>> >> > > > >>> >> > Cheers > > >>> >> > Anselm > > >>> >> > > > >>> >> > > > >>> >> > Am 03.02.2014 12<tel:03.02.2014%2012>:13, schrieb Kristján > > >>> >> > Valur > > >>> >> > Jónsson: > > >>> >> >> Hi. > > >>> >> >> Do you get this only with the installed version? > > >>> >> >> Could you try replacing it with your own build? If so, > > >>> >> >> could you go into the source code and disable stack spilling ? > > >>> >> >> You have to nerf the macro CSTACK_SAVE_NOW. > > >>> >> >> > > >>> >> >> I saw some mysterious crashes recently in a live build in > > >>> >> >> Shanghai that went away when I disabled this. > > >>> >> >> > > >>> >> >> K > > >>> >> >> > > >>> >> >>> -----Original Message----- > > >>> >> >>> From: > > >>> >> >>> [email protected]<mailto:stackless- > > bounces@stac > > >>> >> >>> kless.com> [mailto:stackless-<mailto:stackless-> > > >>> >> >>> [email protected]<mailto:[email protected]>] On > > >>> >> >>> Behalf Of Richard Tew > > >>> >> >>> Sent: 3. febrúar 2014 03:04 > > >>> >> >>> To: [email protected]<mailto:[email protected]> > > >>> >> >>> Subject: [Stackless] Windows 2.7.6 stability > > >>> >> >>> > > >>> >> >>> Hi, > > >>> >> >>> > > >>> >> >>> I've got the installer we provide for 2.7.6, on Windows. > > >>> >> >>> And I've been getting lots of non-crashing premature exits: > > >>> >> >>> > > >>> >> >>> SystemError: unknown opcode XXX lineno: 314, opcode: 0 > > >>> >> >>> > > >>> >> >>> It's not consistently reproducible using running the same > > >>> >> >>> code, but can be sometimes, and I'm not using any Stackless > > features. > > >>> >> >>> > > >>> >> >>> If I take the mainline python repo and sync to v2.7.6 and > > >>> >> >>> generate a dll, and put it in c:\python27, all the problems > > >>> >> >>> go away. > > >>> >> >>> > > >>> >> >>> I thought it might be pyside 1.1.2 which I was using, but > > >>> >> >>> upgraded that and the problem remained with pyside 1.2.1. > > >>> >> >>> That's the only external dependency my code uses. > > >>> >> >>> > > >>> >> >>> Anyone else using this installer? > > >>> >> >>> > > >>> >> >>> Cheers, > > >>> >> >>> Richard. > > >>> >> >>> > > >>> >> >>> _______________________________________________ > > >>> >> >>> Stackless mailing list > > >>> >> >>> [email protected]<mailto:[email protected]> > > >>> >> >>> http://www.stackless.com/mailman/listinfo/stackless > > >>> >> >> > > >>> >> >> > > >>> >> >> > > >>> >> >> _______________________________________________ > > >>> >> >> Stackless mailing list > > >>> >> >> [email protected]<mailto:[email protected]> > > >>> >> >> http://www.stackless.com/mailman/listinfo/stackless > > >>> >> >> > > >>> >> > > > >>> >> > -- > > >>> >> > Dipl. Phys. Anselm Kruis science + > > >>> >> > computing > > >>> >> > ag > > >>> >> > Senior Solution Architect Ingolstädter Str. > > >>> >> > 22 > > >>> >> > email > > >>> >> > [email protected]<mailto:A.Kruis@science- > > computing.de> > > >>> >> > 80807 München, Germany > > >>> >> > phone +49 89 356386 874<tel:%2B49%2089%20356386%20874> > fax > > >>> >> > 737 www.science-computing.de<http://www.science- > > computing.de> > > >>> >> > -- > > >>> >> > Vorstandsvorsitzender/Chairman of the board of management: > > >>> >> > Gerd-Lothar Leonhart > > >>> >> > Vorstand/Board of Management: > > >>> >> > Dr. Bernd Finkbeiner, Michael Heinrichs, Dr. Arno Steitz, Dr. > > >>> >> > Ingrid Zech Vorsitzender des Aufsichtsrats/ Chairman of the > > >>> >> > Supervisory > > >>> >> > Board: > > >>> >> > Philippe Miltin > > >>> >> > Sitz/Registered Office: Tuebingen > > >>> >> > Registergericht/Registration > > >>> >> > Court: Stuttgart Registernummer/Commercial Register No.: HRB > > >>> >> > 382196 > > >>> >> > > > >>> >> > > > >>> >> > _______________________________________________ > > >>> >> > Stackless mailing list > > >>> >> > [email protected]<mailto:[email protected]> > > >>> >> > http://www.stackless.com/mailman/listinfo/stackless > > >>> >> > > > >>> >> > > >>> >> _______________________________________________ > > >>> >> Stackless mailing list > > >>> >> [email protected]<mailto:[email protected]> > > >>> >> http://www.stackless.com/mailman/listinfo/stackless > > >>> > > > >>> > > > >>> > > > >>> > _______________________________________________ > > >>> > Stackless mailing list > > >>> > [email protected]<mailto:[email protected]> > > >>> > http://www.stackless.com/mailman/listinfo/stackless > > >>> > > > >>> > > >>> _______________________________________________ > > >>> Stackless mailing list > > >>> [email protected]<mailto:[email protected]> > > >>> http://www.stackless.com/mailman/listinfo/stackless > > >> > > >> > > >> > > >> _______________________________________________ > > >> Stackless mailing list > > >> [email protected]<mailto:[email protected]> > > >> http://www.stackless.com/mailman/listinfo/stackless > > >> > > >> > > >> > > >> -- > > >> ==================================== > > >> Lars van Gemerden > > >> [email protected]<mailto:[email protected]> > > >> +31 6 26 88 55 39 > > >> ==================================== > > >> > > > > > > _______________________________________________ > > > Stackless mailing list > > > [email protected] > > > http://www.stackless.com/mailman/listinfo/stackless > > > > > > _______________________________________________ > > > Stackless mailing list > > > [email protected] > > > http://www.stackless.com/mailman/listinfo/stackless > > > > > > > _______________________________________________ > > Stackless mailing list > > [email protected] > > http://www.stackless.com/mailman/listinfo/stackless > > > > _______________________________________________ > Stackless mailing list > [email protected] > http://www.stackless.com/mailman/listinfo/stackless _______________________________________________ Stackless mailing list [email protected] http://www.stackless.com/mailman/listinfo/stackless
