Re: [storm] Weird 'pickle' related error when adding many objects

[Andreas Kopecky]

Hi Justin,

I tried to reproduce what you describe here but failed at it.
I think you need to give us a little more information on what kind
of objects you try to create, what fields they got and (if possible)
an easy to reproduce testscript.

i tried something along the lines of:

>>> storage = []
>>> for i in xrange(1000):
>>>     obj = Test(u'testing name %i' % i)
>>>     storage.append(obj)
>>> print [store.add(obj).name for obj in storage]

whereas Test() is a very basic storm object - and it works like a charm

regards,
- Andreas Kopecky

Am 24.11.2008 um 01:32 schrieb Justin Warren:

Hi folks,

I've hit a weird problem that is beyond me at the moment. Hopefully
someone else can shed some light on the situation.

I create a bunch of objects, and attempt to add them all to the
database. If I only add about 100 objects, everything works fine. If I
attempt to add 150, I get a bunch of these errors:

Exception exceptions.AttributeError: AttributeError("'NoneType' object
has no attribute 'dumps'",) in <bound method
ObjectInfo._emit_object_deleted of {'store': <storm.store.Store object
at 0x139cc90>, 'primary_vars': (<storm.variables.IntVariable object at
0x1535910>, <storm.variables.DateTimeVariable object at 0x13bcc68>)}

'dumps' appears to only be used in variables.py as part of
pickle.dumps() calls. I added some print statements to get an idea of

seems that pickle == None within this method, even though the 'import
cPickle as pickle' statement at the start of the file works to begin
with.


import statement just before the problem line, thus:

   def get_state(self):
       try:
           import cPickle as pickle
           return (self._lazy_value, pickle.dumps(self._value, -1))
       except AttributeError, e:
           print "erk! found the problem here."
           print "pickle is:", pickle

And then I get this:
Exception exceptions.ImportError: ImportError('No module named

<storm.store.Store object at 0x139bc90>, 'primary_vars':
(<storm.variables.IntVariable object at 0x151c850>,
<storm.variables.DateTimeVariable object at 0x13a8fc8>)}>

It seems something is resetting pickle to None, as well as sys.path.
This seems really weird to me.

I'm using Thomas's twisted integration branch r214, and I've seen this

something, but I've searched for 'pickle' in all the code files and
found nothing. Nor can I find anything obviously updating sys.path.

Any clues, hints or help would be greatly appreciated.

--
Justin Warren <[EMAIL PROTECTED]>


--
storm mailing list
storm@lists.canonical.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/storm

what's going on, and narrowed the issue to ListVariable.get_state(). It I've checked my environment, and I can indeed import cPickle. I added an cPickle',) in <bound method ObjectInfo._emit_object_deleted of {'store': same issue in r207. It might be that the twisted integration is breaking

PGP.sig
Description: Signierter Teil der Nachricht

-- 
storm mailing list
storm@lists.canonical.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/storm