Nguyen Quan Son wrote:
Hi,
I have a problem with performance and memory consumption when trying to do some 
statistics, using following code:
...
docs = container.portal_catalog(meta_type='Document', ...)
for doc in docs:
    obj = doc.getObject()
    value = obj.attr
    ...

With about 10.000 documents this Python script takes 10 minutes and more than 500MB of 
memory, after that I had to restart Zope. I
am running Zope 2.6.1 + Plone 1.0 on Windows 2000, Xeon P4 with 1GB RAM.
What's wrong with this code? Any suggestion is appreciated.
Nguyen Quan Son.


it's not the catalog that's slow:


t1=time.time()
docs = container.portal_catalog(meta_type='Document', ...)
t2=time.time()
for doc in docs:
     obj = doc.getObject()
     value = obj.attr
     ...
t3=time.time()

print out the times and you'll see that the the finding is fast.
the problem is that you are inflating each and every document one after another, and that takes time.


Romain Slootmaekers.



_______________________________________________
Zope-Dev maillist - [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists - http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )






_______________________________________________
Zope-Dev maillist - [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists - http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )

Reply via email to