Allen Huang wrote at 2007-1-8 19:28 -0800:
>currently I have written my program so it will do an entry by entry processing 
>with zope. This operation is like this
>1. read data (the data file)
>2. create product (a python product that store three field data: one string 
>and two float data)
>3. update product (update the three field entries)
>when I first tried it out with the first 1000 entries it took about 30 
>seconds. That means its going to take 50 ~ 60 minutes for 110000 entries. 

You need to be especially careful in which container you dump
your entries. You *MUST NOT* use a standard folder for this.
It is not fit for larger amounts of entries. Use instead
a "BTreeFolder2".

Another way to speed up the creation process is to batch
the creation and commit a transaction only for every X entries.

Using these techniques, I expect that you will be able
to create about 100 entries per second.

Zope maillist  -
**   No cross posts or HTML encoding!  **
(Related lists - )

Reply via email to