--On 8. Januar 2007 19:28:32 -0800 Allen Huang <[EMAIL PROTECTED]> wrote:

I have a data file that has over 110000 entry of 3 column data (string,
float, float)
currently I have written my program so it will do an entry by entry
processing with zope. This operation is like this
1. read data (the data file)
2. create product (a python product that store three field data: one
string and two float data) 3. update product (update the three field
entries)

Please name things the right way. A "Product" is basically a Zope/Python package that contains definitions of classes, scripts, templates etc.

You mean instances of a particular class?


when I first tried it out with the first 1000 entries it took about 30
seconds. That means its going to take 50 ~ 60 minutes for 110000 entries.

You're creating 110k instances for storing a string and two floats?
If yes, that's stupid idea.

You can persistent large amounts of data within a single instances
by using Zope BTrees.

It not every day that you have to process over 110000 data entries but
processing over 60 minutes is still kind of long.


What kind of processing?

-aj

Attachment: pgpcagwhkOkKm.pgp
Description: PGP signature

_______________________________________________
Zope maillist  -  Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope-dev )

Reply via email to