Re: [pypy-dev] Fwd: Re: Adding init/variables to W_Root
Hi Armin, Thanks for the help! It seems like the problem is that somewhere in my code, I called .set_rbflags(x), and it's not checked that x is a dict. Frank On Fri, Jan 13, 2017 at 6:36 AM, Armin Rigo wrote: > Re-hi, > > On 13 January 2017 at 12:34, Armin Rigo wrote: > > says: you are storing in the rb_flags attribute something that is > > sometimes a dict, and sometimes an instance of W_Root. > > Another potential reason would be that there is one buggy call > somewhere that calls .set_rbflags(x) where x is an instance of W_Root > instead of a dictionary. > > > A bientôt, > > Armin. > ___ pypy-dev mailing list pypy-dev@python.org https://mail.python.org/mailman/listinfo/pypy-dev
Re: [pypy-dev] Fwd: Re: Adding init/variables to W_Root
Re-hi, On 13 January 2017 at 12:34, Armin Rigo wrote: > says: you are storing in the rb_flags attribute something that is > sometimes a dict, and sometimes an instance of W_Root. Another potential reason would be that there is one buggy call somewhere that calls .set_rbflags(x) where x is an instance of W_Root instead of a dictionary. A bientôt, Armin. ___ pypy-dev mailing list pypy-dev@python.org https://mail.python.org/mailman/listinfo/pypy-dev
Re: [pypy-dev] Fwd: Re: Adding init/variables to W_Root
Hi, On 12 January 2017 at 03:35, Frank Wang wrote: > Good catch! I was just trying to get something to work, but it seems like > you found another bug. I am now checking if it is None: No, I think these latest changes don't do anything. The error message says: you are storing in the rb_flags attribute something that is sometimes a dict, and sometimes an instance of W_Root. Check if there is somewhere else a subclass of W_Root with its own rb_flags attribute which would be set to a W_Root. Then you'd get a crash using that subclass, also untranslated, simply because 'rb_flags' is supposed to be two different things. To be on the safe side, you should change the name of the attribute in W_Root from rb_flags to something much more obscure like "_frank_rb_flags". A bientôt, Armin. ___ pypy-dev mailing list pypy-dev@python.org https://mail.python.org/mailman/listinfo/pypy-dev
Re: [pypy-dev] Fwd: Re: Adding init/variables to W_Root
Hi William,
Good catch! I was just trying to get something to work, but it seems like
you found another bug. I am now checking if it is None:
# union flags in w_1 and w_2 and propagate to result
if isinstance(w_result, W_Root) and w_result is not None:
rb_results = {}
if isinstance(w_1, W_Root) and w_1 is not None:
w_1_rbflags = w_1.get_rbflags()
rb_results.update(w_1_rbflags)
if isinstance(w_2, W_Root) and w_2 is not None:
w_2_rbflags = w_2.get_rbflags()
rb_results.update(w_2_rbflags)
w_result.set_rbflags(rb_results)
I now get this error:
[translation:ERROR] UnionError:
Offending annotations:
SomeInstance(can_be_None=False,
classdef=pypy.interpreter.baseobjspace.W_Root)
SomeOrderedDict(dictdef=<{SomeImpossibleValue(): SomeImpossibleValue()}>)
Occurred processing the following simple_call:
bound to {}> returning
v831 = simple_call(v830, rb_results_0)
In :
Happened at file /home/ubuntu/pypy2-v5.3.1-src/pypy/interpreter/pyopcode.py
line 55
==> w_result.set_rbflags(rb_results)
Known variable annotations:
v830 = SomePBC(can_be_None=False, descriptions={...1...},
knowntype=method, subset_of=None)
rb_results_0 = SomeOrderedDict(dictdef=<{SomeImpossibleValue():
SomeImpossibleValue()}>)
Processing block:
block@203 is a
in (pypy.interpreter.pyopcode:40)opcode_impl_for_div__star_2
containing the following operations:
v830 = getattr(w_result_0, ('set_rbflags'))
v831 = simple_call(v830, rb_results_0)
--end--
I assume this happens because w_result can be a "subset" of None. What is
the proper way to check if it's None?
Frank
On Wed, Jan 11, 2017 at 3:00 PM, William ML Leslie <
william.leslie@gmail.com> wrote:
> On 12 January 2017 at 09:08, Frank Wang wrote:
> > def binaryoperation(operationname):
> > """NOT_RPYTHON"""
> > def opimpl(self, *ignored):
> > operation = getattr(self.space, operationname)
> > w_2 = self.popvalue()
> > w_1 = self.popvalue()
> > w_result = operation(w_1, w_2)
> >
> > # union flags in w_1 and w_2 and propagate to result
> > w_1_rbflags = w_1.get_rbflags()
> > w_2_rbflags = w_2.get_rbflags()
> > w_1_rbflags.update(w_2_rbflags)
> > w_result.set_rbflags(w_1_rbflags)
> >
> > self.pushvalue(w_result)
>
> It looks like, if an app-level exception is in progress, w_result may
> be None. Try checking for that before setting the flags.
>
> Also: did you mean to alter w_1's rbflags? Seems strange that you'd
> alter w_1 but not w_2.
>
> --
> William Leslie
>
> Notice:
> Likely much of this email is, by the nature of copyright, covered
> under copyright law. You absolutely MAY reproduce any part of it in
> accordance with the copyright law of the nation you are reading this
> in. Any attempt to DENY YOU THOSE RIGHTS would be illegal without
> prior contractual agreement.
>
___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev
Re: [pypy-dev] Fwd: Re: Adding init/variables to W_Root
On 12 January 2017 at 09:08, Frank Wang wrote: > def binaryoperation(operationname): > """NOT_RPYTHON""" > def opimpl(self, *ignored): > operation = getattr(self.space, operationname) > w_2 = self.popvalue() > w_1 = self.popvalue() > w_result = operation(w_1, w_2) > > # union flags in w_1 and w_2 and propagate to result > w_1_rbflags = w_1.get_rbflags() > w_2_rbflags = w_2.get_rbflags() > w_1_rbflags.update(w_2_rbflags) > w_result.set_rbflags(w_1_rbflags) > > self.pushvalue(w_result) It looks like, if an app-level exception is in progress, w_result may be None. Try checking for that before setting the flags. Also: did you mean to alter w_1's rbflags? Seems strange that you'd alter w_1 but not w_2. -- William Leslie Notice: Likely much of this email is, by the nature of copyright, covered under copyright law. You absolutely MAY reproduce any part of it in accordance with the copyright law of the nation you are reading this in. Any attempt to DENY YOU THOSE RIGHTS would be illegal without prior contractual agreement. ___ pypy-dev mailing list pypy-dev@python.org https://mail.python.org/mailman/listinfo/pypy-dev
Re: [pypy-dev] Fwd: Re: Adding init/variables to W_Root
Hi Armin,
I've taken your advice, and my W_Root has these additions (in bold):
class W_Root(object):
_attrs_ = ['__weakref__', *'rb_flags'*]
_must_be_light_finalizer_ = True
user_overridden_class = False
# Riverbed flags
*rb_flags = None*
*def add_rbflags(self, new_flags):*
*if self.rb_flags is None:*
*self.rb_flags = {}*
*for flag in new_flags:*
*self.rb_flags[flag] = None*
*def set_rbflags(self, new_flags):*
*self.rb_flags = new_flags*
*def get_rbflags(self):*
*if self.rb_flags is None:*
*self.rb_flags = {}*
*return self.rb_flags*
In pyopcode.py, I've made the following modifications. I've shown them in
bold. The high level idea is that for a binary operation, I want to take
the union of the rb_flags in w_1 and w_2 and set the union to be the
rb_flags in w_result.
def binaryoperation(operationname):
"""NOT_RPYTHON"""
def opimpl(self, *ignored):
operation = getattr(self.space, operationname)
w_2 = self.popvalue()
w_1 = self.popvalue()
w_result = operation(w_1, w_2)
# union flags in w_1 and w_2 and propagate to result
*w_1_rbflags = w_1.get_rbflags()*
*w_2_rbflags = w_2.get_rbflags()*
*w_1_rbflags.update(w_2_rbflags)*
*w_result.set_rbflags(w_1_rbflags)*
self.pushvalue(w_result)
opimpl.binop = operationname
return func_with_new_name(opimpl, "opcode_impl_for_%s" % operationname)
However, when I do this, I get the following error:
[translation:ERROR] UnionError:
Offending annotations:
SomeOrderedDict(dictdef=<{SomeImpossibleValue(): SomeImpossibleValue()}>)
SomeInstance(can_be_None=True,
classdef=pypy.interpreter.baseobjspace.W_Root)
v945 = setattr(self_268, ('rb_flags'), v944)
In :
Happened at file
/home/ubuntu/pypy2-v5.3.1-src/pypy/interpreter/baseobjspace.py line 48
==> self.rb_flags = {}
Known variable annotations:
self_268 = SomeInstance(can_be_None=False,
classdef=pypy.interpreter.baseobjspace.W_Root)
v944 = SomeOrderedDict(dictdef=<{SomeImpossibleValue():
SomeImpossibleValue()}>)
Processing block:
block@15 is a
in (pypy.interpreter.baseobjspace:46)W_Root.get_rbflags
containing the following operations:
v944 = newdict()
v945 = setattr(self_268, ('rb_flags'), v944)
--end--
Any thoughts on this?
Frank
On Mon, Dec 19, 2016 at 10:32 PM, Frank Wang wrote:
> Hi Armin,
>
> It seems that with your suggestions, things have started to work!
>
> Thanks again for the help!
>
> On Mon, Dec 19, 2016 at 5:57 PM Armin Rigo wrote:
>
>> Hi Frank,
>>
>> On 20 December 2016 at 01:29, Frank Wang wrote:
>> > This also seems to happen in a lot of classes because W_Root is the
>> parent
>> > class for many classes. Is there a way to universally turn off this
>> > immutable flag (assuming it's okay) in all classes?
>>
>> You can hack at the place where the error is raised. Assuming you're
>> translating without the JIT, it should not cause too much problems.
>>
>> As an alternative to adding an attribute to every object, you could
>> also use a rpython.rlib.rweakref.RWeakKeyDictionary(W_Root, RbFlags),
>> for some class RbFlags which has got the rb_flags dictionary and/or
>> any other related data. The advantage is that it is a less intrusive
>> change, reduces memory usage if many objects don't need the extra
>> information at all, and avoids the immutable problem. The
>> inconvenient is that going from the object to the rb_flags dictionary
>> is slower.
>>
>> There are also alternatives that are better if you are ok with
>> supporting not *all* objects. For example, you can edit
>> objspace/std/mapdict.py to add a new dynamic attribute that is
>> internally called ``"rbflags", SPECIAL``; see ``"weakref", SPECIAL``
>> for an existing example. This would work for all objects that use
>> mapdicts, i.e. all instances of user-defined classes, and additionally
>> instances of many built-in types as well---but not instances of the
>> core types like "int" or "str" or "list" or "tuple" or "dict".
>>
>>
>> A bientôt,
>>
>> Armin.
>>
>
___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev
Re: [pypy-dev] Fwd: Re: Adding init/variables to W_Root
Hi Armin, It seems that with your suggestions, things have started to work! Thanks again for the help! On Mon, Dec 19, 2016 at 5:57 PM Armin Rigo wrote: > Hi Frank, > > On 20 December 2016 at 01:29, Frank Wang wrote: > > This also seems to happen in a lot of classes because W_Root is the > parent > > class for many classes. Is there a way to universally turn off this > > immutable flag (assuming it's okay) in all classes? > > You can hack at the place where the error is raised. Assuming you're > translating without the JIT, it should not cause too much problems. > > As an alternative to adding an attribute to every object, you could > also use a rpython.rlib.rweakref.RWeakKeyDictionary(W_Root, RbFlags), > for some class RbFlags which has got the rb_flags dictionary and/or > any other related data. The advantage is that it is a less intrusive > change, reduces memory usage if many objects don't need the extra > information at all, and avoids the immutable problem. The > inconvenient is that going from the object to the rb_flags dictionary > is slower. > > There are also alternatives that are better if you are ok with > supporting not *all* objects. For example, you can edit > objspace/std/mapdict.py to add a new dynamic attribute that is > internally called ``"rbflags", SPECIAL``; see ``"weakref", SPECIAL`` > for an existing example. This would work for all objects that use > mapdicts, i.e. all instances of user-defined classes, and additionally > instances of many built-in types as well---but not instances of the > core types like "int" or "str" or "list" or "tuple" or "dict". > > > A bientôt, > > Armin. > ___ pypy-dev mailing list pypy-dev@python.org https://mail.python.org/mailman/listinfo/pypy-dev
Re: [pypy-dev] Fwd: Re: Adding init/variables to W_Root
Hi Frank, On 20 December 2016 at 01:29, Frank Wang wrote: > This also seems to happen in a lot of classes because W_Root is the parent > class for many classes. Is there a way to universally turn off this > immutable flag (assuming it's okay) in all classes? You can hack at the place where the error is raised. Assuming you're translating without the JIT, it should not cause too much problems. As an alternative to adding an attribute to every object, you could also use a rpython.rlib.rweakref.RWeakKeyDictionary(W_Root, RbFlags), for some class RbFlags which has got the rb_flags dictionary and/or any other related data. The advantage is that it is a less intrusive change, reduces memory usage if many objects don't need the extra information at all, and avoids the immutable problem. The inconvenient is that going from the object to the rb_flags dictionary is slower. There are also alternatives that are better if you are ok with supporting not *all* objects. For example, you can edit objspace/std/mapdict.py to add a new dynamic attribute that is internally called ``"rbflags", SPECIAL``; see ``"weakref", SPECIAL`` for an existing example. This would work for all objects that use mapdicts, i.e. all instances of user-defined classes, and additionally instances of many built-in types as well---but not instances of the core types like "int" or "str" or "list" or "tuple" or "dict". A bientôt, Armin. ___ pypy-dev mailing list pypy-dev@python.org https://mail.python.org/mailman/listinfo/pypy-dev
Re: [pypy-dev] Fwd: Re: Adding init/variables to W_Root
Hi Carl,
Thanks for the tip! It's mostly working. I get this error:
[translation:ERROR] ImmutableConflictError: class has _immutable_=True, but
parent class
defines (at least) the mutable field 'rb_flags'
I'm tempted to set the flag to be False. Is there any reason that W_Root or
any of its subclasses cannot contain immutable variables?
Frank
On Mon, Dec 19, 2016 at 9:00 AM, Carl Friedrich Bolz wrote:
> Oops, forgot to cc pypy-dev...
>
>
> Carl Friedrich
>
> --
> *From:* Carl Friedrich Bolz
> *Sent:* December 19, 2016 5:59:01 PM GMT+01:00
> *To:* Frank Wang
> *Subject:* Re: [pypy-dev] Adding init/variables to W_Root
>
> Hi Frank,
>
> The solution is to add an _attrs_ = ('__weakref__', 'rb_flags',)
> declaration to the body of W_Root. It's like __slots__, but only for
> RPython.
>
> Cheers,
>
> Carl Friedrich
>
> On December 19, 2016 5:42:12 PM GMT+01:00, Frank Wang
> wrote:
>>
>> Hi Armin,
>>
>> If I modify the line to __slots__ = ('__weakref__', 'rb_flags',), I get
>> the following error:
>>
>> [translation:ERROR] AssertionError:
>> > object at 0x073e87c8>: to run register_finalizer() untranslated,
>> the object must not have __slots__
>>
>> If I remove that line completely, I get the following error,
>>
>> [translation:ERROR] Exception: > dictmultiobject.W_DictMultiObject'> has slots or _attrs_, but not its
>> base class
>> Processing block:
>> block@3 is a
>> in (pypy.interpreter.mixedmodule:83)MixedModule.getdictvalue
>> containing the following operations:
>>v0 = getattr(space_0, ('finditem_str'))
>>v1 = getattr(self_0, ('w_dict'))
>>w_value_0 = simple_call(v0, v1, name_0)
>>v2 = getattr(self_0, ('lazy'))
>>v3 = bool(v2)
>> --end--
>>
>> I wonder what error in your opinion is easier to get around.
>>
>> Thanks for the help!
>>
>> Frank
>>
>> On Mon, Dec 19, 2016 at 2:26 AM, Armin Rigo wrote:
>>
>>> Hi Frank,
>>>
>>> On 19 December 2016 at 05:29, Frank Wang wrote:
>>> > Attribute 'rb_flags' on >> .W_Root'>
>>> > should be read-only.
>>>
>>> Ah, did you adapt or remove this line in class W_Root?
>>>
>>> __slots__ = ('__weakref__',)
>>>
>>> The annotator complains, I think, because 'rb_flags' is forbidden by
>>> this line (which is useful to avoid having random attributes
>>> accidentally move up to W_Root).
>>>
>>>
>>> A bientôt,
>>>
>>> Armin.
>>>
>>
>> --
>>
>> pypy-dev mailing list
>> pypy-dev@python.org
>> https://mail.python.org/mailman/listinfo/pypy-dev
>>
>>
> ___
> pypy-dev mailing list
> pypy-dev@python.org
> https://mail.python.org/mailman/listinfo/pypy-dev
>
>
___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev
Re: [pypy-dev] Fwd: Re: Adding init/variables to W_Root
This also seems to happen in a lot of classes because W_Root is the parent
class for many classes. Is there a way to universally turn off this
immutable flag (assuming it's okay) in all classes?
Frank
On Mon, Dec 19, 2016 at 3:55 PM, Frank Wang wrote:
> Hi Carl,
>
> Thanks for the tip! It's mostly working. I get this error:
>
> [translation:ERROR] ImmutableConflictError: class pypy.module._cffi_backend.libraryobj.W_Library> has _immutable_=True, but
> parent class
> defines (at least) the mutable field 'rb_flags'
>
> I'm tempted to set the flag to be False. Is there any reason that W_Root
> or any of its subclasses cannot contain immutable variables?
>
> Frank
>
> On Mon, Dec 19, 2016 at 9:00 AM, Carl Friedrich Bolz
> wrote:
>
>> Oops, forgot to cc pypy-dev...
>>
>>
>> Carl Friedrich
>>
>> --
>> *From:* Carl Friedrich Bolz
>> *Sent:* December 19, 2016 5:59:01 PM GMT+01:00
>> *To:* Frank Wang
>> *Subject:* Re: [pypy-dev] Adding init/variables to W_Root
>>
>> Hi Frank,
>>
>> The solution is to add an _attrs_ = ('__weakref__', 'rb_flags',)
>> declaration to the body of W_Root. It's like __slots__, but only for
>> RPython.
>>
>> Cheers,
>>
>> Carl Friedrich
>>
>> On December 19, 2016 5:42:12 PM GMT+01:00, Frank Wang
>> wrote:
>>>
>>> Hi Armin,
>>>
>>> If I modify the line to __slots__ = ('__weakref__', 'rb_flags',), I get
>>> the following error:
>>>
>>> [translation:ERROR] AssertionError:
>>> >> object at 0x073e87c8>: to run register_finalizer() untranslated,
>>> the object must not have __slots__
>>>
>>> If I remove that line completely, I get the following error,
>>>
>>> [translation:ERROR] Exception: >> 'pypy.objspace.std.dictmultiobject.W_DictMultiObject'>
>>> has slots or _attrs_, but not its base class
>>> Processing block:
>>> block@3 is a
>>> in (pypy.interpreter.mixedmodule:83)MixedModule.getdictvalue
>>> containing the following operations:
>>>v0 = getattr(space_0, ('finditem_str'))
>>>v1 = getattr(self_0, ('w_dict'))
>>>w_value_0 = simple_call(v0, v1, name_0)
>>>v2 = getattr(self_0, ('lazy'))
>>>v3 = bool(v2)
>>> --end--
>>>
>>> I wonder what error in your opinion is easier to get around.
>>>
>>> Thanks for the help!
>>>
>>> Frank
>>>
>>> On Mon, Dec 19, 2016 at 2:26 AM, Armin Rigo
>>> wrote:
>>>
Hi Frank,
On 19 December 2016 at 05:29, Frank Wang wrote:
> Attribute 'rb_flags' on >>> .W_Root'>
> should be read-only.
Ah, did you adapt or remove this line in class W_Root?
__slots__ = ('__weakref__',)
The annotator complains, I think, because 'rb_flags' is forbidden by
this line (which is useful to avoid having random attributes
accidentally move up to W_Root).
A bientôt,
Armin.
>>>
>>> --
>>>
>>> pypy-dev mailing list
>>> pypy-dev@python.org
>>> https://mail.python.org/mailman/listinfo/pypy-dev
>>>
>>>
>> ___
>> pypy-dev mailing list
>> pypy-dev@python.org
>> https://mail.python.org/mailman/listinfo/pypy-dev
>>
>>
>
___
pypy-dev mailing list
pypy-dev@python.org
https://mail.python.org/mailman/listinfo/pypy-dev
