hi,

i'm trying to perform a ZCatalog.refreshCatalog() on a catalog with over 29,000 indexed objects. it churns for a good long time, and eventually fails with a long set of tracebacks, of which i've included a sample at the end of this message.


i think i understand the gist of the issue... it's trying to write an object (probably a CatalogBrain) to the database, but this object's __dict__ contains a value that is of type instancemethod, which isn't allowed for persistent objects.

the problem is that i can't figure out which specific objects are causing the problem. i've used pdb.post_mortem to get a debug prompt way down in the traceback, but the code goes in and out of C modules, so i'm missing a lot of what's happening. and when i interactively peek at the objects that are being indexed when the error happens, there doesn't seem to be anything wrong, and i can index the objects w/ no problem. i've even tried dropping the subtransaction threshold down to 1, so it will try to commit a savepoint after every object, but none of the objects being indexed seemed to have any problems.

i CAN verify that the instancemethod that is causing the problem renders like this:

<bound method SessionDataManager.getSessionData of <SessionDataManager at /session_data_manager>>

even that hasn't proven enough for me to concretely identify the source of the problem, though.

i've been working on this for a full day already, and am not sure how to proceed. does anyone have any debugging tips that might help me figure out what, exactly, is causing the reindex attempts to blow up?

thanks!

-r




Traceback (most recent call last):
File "/home/rob/topp/14000/builds/20080611/opencore/lib/zope/lib/python/Products/ZCatalog/ZCatalog.py", line 296, in refreshCatalog
    self.catalog_object(obj, p, pghandler=pghandler)
File "/home/rob/topp/14000/builds/20080611/opencore/zope/Products/CMFPlone/CatalogTool.py", line 367, in catalog_object
    self._increment_counter()
File "/home/rob/topp/14000/builds/20080611/opencore/zope/Products/CMFPlone/CatalogTool.py", line 395, in _increment_counter
    self._counter.change(1)
File "/home/rob/topp/14000/builds/20080611/opencore/lib/zope/lib/python/BTrees/Length.py", line 55, in change
    self.value += delta
File "/home/rob/topp/14000/builds/20080611/opencore/lib/zope/lib/python/ZODB/Connection.py", line 890, in register
    self._register(obj)
File "/home/rob/topp/14000/builds/20080611/opencore/lib/zope/lib/python/ZODB/Connection.py", line 900, in _register
    self.transaction_manager.get().join(self)
File "/home/rob/topp/14000/builds/20080611/opencore/lib/zope/lib/python/transaction/_transaction.py", line 273, in join
    self._prior_operation_failed() # doesn't return
File "/home/rob/topp/14000/builds/20080611/opencore/lib/zope/lib/python/transaction/_transaction.py", line 267, in _prior_operation_failed
    raise TransactionFailedError("An operation previously failed, "
TransactionFailedError: An operation previously failed, with traceback:

File "/home/rob/topp/14000/builds/20080611/opencore/lib/zope/lib/python/ZServer/PubCore/ZServerPublisher.py", line 25, in __init__
    response=b)
File "/home/rob/topp/14000/builds/20080611/opencore/lib/zope/lib/python/ZPublisher/Publish.py", line 401, in publish_module
    environ, debug, request, response)
File "/home/rob/topp/14000/builds/20080611/opencore/lib/zope/lib/python/ZPublisher/Publish.py", line 202, in publish_module_standard
    response = publish(request, module_name, after_list, debug=debug)
File "/home/rob/topp/14000/builds/20080611/opencore/lib/zope/lib/python/ZPublisher/Publish.py", line 119, in publish
    request, bind=1)
File "/home/rob/topp/14000/builds/20080611/opencore/lib/zope/lib/python/ZPublisher/mapply.py", line 88, in mapply
    if debug is not None: return debug(object,args,context)
File "/home/rob/topp/14000/builds/20080611/opencore/lib/zope/lib/python/ZPublisher/Publish.py", line 42, in call_object
    result=apply(object,args) # Type s<cr> to step into published object.
File "/home/rob/topp/14000/builds/20080611/opencore/lib/zope/lib/python/Products/ZCatalog/ZCatalog.py", line 260, in manage_catalogReindex
    self.refreshCatalog(clear=1, pghandler=handler)
File "/home/rob/topp/14000/builds/20080611/opencore/lib/zope/lib/python/Products/ZCatalog/ZCatalog.py", line 296, in refreshCatalog
    self.catalog_object(obj, p, pghandler=pghandler)
File "/home/rob/topp/14000/builds/20080611/opencore/zope/Products/CMFPlone/CatalogTool.py", line 385, in catalog_object
    update_metadata, pghandler=pghandler)
File "/home/rob/topp/14000/builds/20080611/opencore/zope/Products/CacheSetup/patch.py", line 96, in catalog_object
    uid, idxs, update_metadata, pghandler)
File "/home/rob/topp/14000/builds/20080611/opencore/zope/Products/CacheSetup/patch_utils.py", line 6, in call
    return getattr(self, PATTERN % __name__)(*args, **kw)
File "/home/rob/topp/14000/builds/20080611/opencore/lib/zope/lib/python/Products/ZCatalog/ZCatalog.py", line 560, in catalog_object
    transaction.savepoint(optimistic=True)
File "/home/rob/topp/14000/builds/20080611/opencore/lib/zope/lib/python/transaction/_manager.py", line 110, in savepoint
    return self.get().savepoint(optimistic)
File "/home/rob/topp/14000/builds/20080611/opencore/lib/zope/lib/python/transaction/_transaction.py", line 312, in savepoint
    self._saveAndRaiseCommitishError() # reraises!
File "/home/rob/topp/14000/builds/20080611/opencore/lib/zope/lib/python/transaction/_transaction.py", line 422, in _saveAndRaiseCommitishError
    t, v, tb = self._saveAndGetCommitishError()
File "/home/rob/topp/14000/builds/20080611/opencore/lib/zope/lib/python/transaction/_transaction.py", line 309, in savepoint
    savepoint = Savepoint(self, optimistic, *self._resources)
File "/home/rob/topp/14000/builds/20080611/opencore/lib/zope/lib/python/transaction/_transaction.py", line 737, in __init__
    savepoint = savepoint()
File "/home/rob/topp/14000/builds/20080611/opencore/lib/zope/lib/python/ZODB/Connection.py", line 1046, in savepoint
    self._commit(None)
File "/home/rob/topp/14000/builds/20080611/opencore/lib/zope/lib/python/ZODB/Connection.py", line 555, in _commit
    self._store_objects(ObjectWriter(obj), transaction)
File "/home/rob/topp/14000/builds/20080611/opencore/lib/zope/lib/python/ZODB/Connection.py", line 582, in _store_objects
    p = writer.serialize(obj)  # This calls __getstate__ of obj
File "/home/rob/topp/14000/builds/20080611/opencore/lib/zope/lib/python/ZODB/serialize.py", line 407, in serialize
    return self._dump(meta, obj.__getstate__())
File "/home/rob/topp/14000/builds/20080611/opencore/lib/zope/lib/python/ZODB/serialize.py", line 416, in _dump
    self._p.dump(state)
  File "copy_reg.py", line 69, in _reduce_ex
    raise TypeError, "can't pickle %s objects" % base.__name__
TypeError: can't pickle instancemethod objects

_______________________________________________
Zope maillist  -  Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **
(Related lists - http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope-dev )

Reply via email to