I can reproduce it with 2.7.4 and 2.7.5. --pawel
On Tue, Apr 14, 2015 at 2:13 AM, Andres Sommerhoff <sommerh...@gmail.com> wrote: > Hi Pawel, its actually very old to me (more than a year). I'm suffering > gradually as the code went more complex. At the beginning was not a big > issue. Now, it is not a chance to run some code under IronPython without the > weakref exception (so Im running it in Cpython or with the workarround of my > previous email). Please note that I have tested only with 2.7.4, not with > 2.7.5 yet (as 2.7.4 is the version that comes with SolverStudio software) > > Can you try the script yourself? > > Thank you for helping on this! > > Regards, > Andres > >> El 13-04-2015, a las 19:41, Pawel Jasinski <pawel.jasin...@gmail.com> >> escribió: >> >> did anybody try it with the 2.7.4? >> I wonder it is something we introduced in 2.7.5. >> --pawel >> >>> On Mon, Apr 13, 2015 at 6:29 PM, Andres Sommerhoff <sommerh...@gmail.com> >>> wrote: >>> Thank you Andrew, good to know you could replicate it. I went to very >>> similar workarround with __contains__(), but yours is more elegant with the >>> "while 1". However, I will recommend to repeat only a few times (to avoid a >>> possible endless loop) and to catch the two unexpected exception only >>> (SystemError, ValueError). So I suggest the following workaround based in >>> your proposal should be: >>> >>> def __contains__(self, item): >>> while 1: >>> try: >>> try: >>> wr = ref(item) >>> except TypeError: >>> return False >>> return wr in self.data >>> except (SystemError, ValueError): #i.e: Don't catch >>> KeyboardInterrupt, MemoryError, etc >>> try: i+=1 >>> except: i=0 # i declared here. Don't want waste >>> resource outside... (not Pythonic, I know) >>> if i==10: raise # Retry max 10 times and re-raise >>> >>> However, would like if someone can debug the error in the C# code of >>> weakref. I'm afraid that the workaround will leave some potential error open >>> beside __contains__(). I see than inside _weakrefset.py the functions add(), >>> remove(), discard(), __isub__(), etc are also using the problematic "ref()" >>> function. The error could eventually happen there or even outside >>> _weakrefset.py. >>> >>> Looking forward some IronPython Guru can check the c# code. I will >>> appreciate the help on that! In the meantime, I will use the __contains__() >>> as above, trying to tackle the most common case. >>> >>> Regards, >>> Andres Sommerhoff >>> >>> >>> >>> On Mon, Apr 13, 2015 at 11:55 AM, Andrew Graham <a...@agraham.demon.co.uk> >>> wrote: >>>> >>>> I confirm the same problem, except that it happens more often for me, >>>> about 50 to 60 times each time I run the test. >>>> It smells like some sort of “unsynchronised acess by different threads to >>>> an object problem ”. >>>> >>>> One (very) hacky way round it until the real cause is established is to >>>> modify __contains__() in _weakrefset.py line 68 to retry until it gets a >>>> proper result >>>> >>>> def __contains__(self, item): >>>> while 1: >>>> try: >>>> try: >>>> wr = ref(item) >>>> except TypeError: >>>> return False >>>> return wr in self.data >>>> except Exception: >>>> pass >>>> >>>> Regards >>>> >>>> Andy Graham >>>> >>>> From: Andres Sommerhoff >>>> Sent: Monday, April 13, 2015 6:39 AM >>>> To: ironpython-users@python.org >>>> Subject: [Ironpython-users] Fwd: weakref random "SystemError" and >>>> "ValueError" exception (affecting _weakrefset.py and abc.py) >>>> >>>> Dear IronPython gurus! >>>> >>>> Hopping you can help me with a kind of ramdom bug (unexpected SystemError >>>> and ValueError exception) in _weakrefset.py (it would be great if you can >>>> replicate it by running my script below. Please let me know). >>>> >>>> The error is random, but I managed to reproduce it by running the >>>> "ramdomly" ofending code 100.000 times inside a loop (Script attached). >>>> Note >>>> that the test takes only a few seconds, and in my PC it throws regularly >>>> between 5 to 30 exception for all those cycles). I see there are other >>>> people suffering for the same, but without solution or workaround yet >>>> (https://mail.python.org/pipermail/ironpython-users/2014-November/017332.html >>>> and https://github.com/IronLanguages/main/issues/1187) >>>> >>>> In my case, weakref error was related with an intensive use of >>>> isinstance() inside "Pulp" library (an optimization python library). Just >>>> for your reference: isintance() use internally a WeakSet as a cache for the >>>> class types, which in turn use weakref (see ABCMeta.__instancecheck__() in >>>> the standard file "abc.py"). >>>> >>>> In my test script I have isolated the problem to WeakSet only (I isolated >>>> it to clean the bug from the "abc.py" and "Pulp" library stuff). The >>>> exception happens inside WeakSet.__contains__() function (from >>>> _weakrefset.py file). As stated, it happens randomly, but apparently only >>>> related with a GC collect cycle into a memory "hungry" python script . I >>>> ran >>>> the test script in two PCs with similar results: Windows 7 64bits and >>>> Windows 7 32bits. Both using ipy.exe 32bit version 2.7.4 (2.7.0.40). The >>>> .NET version is 4.0.30319.18444 (32-bit). The test script does: >>>> >>>> It simulate a "memory intensive" python code (creating a weakref object of >>>> 2kb to 12kb in each loop. If smaller, like 0.1kb objects, then the bug >>>> don't >>>> show up) >>>> It manually runs GC.collect() every 1.000 cycles (should collect those >>>> weakref objects) >>>> ... and it repeat (100.000 times) the following "offending" boolean test: >>>> >>>> test = item in WeakSetObject #<- Repeated 100.000 times. >>>> #-> it fails between 10 to 20 times >>>> with an unexpected exception >>>> >>>> NOTE 1: The "item" is an object added to the WeakSet at the beginning of >>>> the script, but "item" should not GC collected as it also keeps a normal >>>> (not weak) reference alive. >>>> >>>> NOTE 2: The boolean test should give always True, which is the case 99.9% >>>> of the time. Less than 0.01%, the boolean test fails raising an exception >>>> of >>>> the type "ValueError" (Description:"Index was out of range") or a bit more >>>> frequent "SystemError" (Description:"Handle is not initialized"). This >>>> happens 5 to 30 times in 100.000 test cycle (Seems very small, but it is >>>> important enough to avoid a practical construction of a medium size >>>> optimization problem with "Pulp" library). >>>> >>>> Tracking down the error, the exception ValueError is raised in line 70 and >>>> the exception "SystemError" in line 73 of "_weakrefset.py" . >>>> >>>> On "Lib\_weakrefset.py" >>>> >>>> 35 :class WeakSet(object): >>>> .... >>>> 68 : def __contains__(self, item): >>>> 69 : try: >>>> *70: wr = ref(item) # <- here is raised "ValueError" >>>> Exception ("Index was out of range") >>>> 71 : except TypeError: >>>> 72 : return False >>>> *73: return wr in self.data # <- here is raised "SystemError" >>>> Exception ("Handle is not initialized") >>>> >>>> Continuing after the exception, when executing the same boolean test >>>> again, it works fine (same item, same WeakSetObject, same execution, local >>>> and global context. Script was not started again!). I.e. if you catch the >>>> exception and continue the execution, It is like as if the exception never >>>> happened before (it's like a runtime lapsus!). >>>> >>>> I believe to fix the source of the problem, Ironpython should trap or >>>> avoid the error in weakref module itself (C# code). I don't know how... can >>>> someone kindly help me to fix this? >>>> >>>> Cheers, >>>> Andres Sommerhoff >>>> >>>> >>>> >>>> >>>> >>>> ________________________________ >>>> _______________________________________________ >>>> Ironpython-users mailing list >>>> Ironpython-users@python.org >>>> https://mail.python.org/mailman/listinfo/ironpython-users >>> >>> >>> >>> _______________________________________________ >>> Ironpython-users mailing list >>> Ironpython-users@python.org >>> https://mail.python.org/mailman/listinfo/ironpython-users >>> _______________________________________________ Ironpython-users mailing list Ironpython-users@python.org https://mail.python.org/mailman/listinfo/ironpython-users
