"Morten W. Petersen" wrote:
> during testing of a mail product I've discovered that the Data.fs file may
> bloat considerably after storing 50 messages. Packing the database will
> reduce the Data.fs file to 20 MB (from 40 MB). Another thing is that
> storing 50 messages takes a *long time* on a 600Mhz 256 MB RAM system.
Did you catalog each message? What version of Zope?
> The thing that struck me was that transactions may cause a lot of
> overhead, both in database size and for execution speed, so, the thing I'm
> wondering about is if there exists a non-undoable storage, or something
> similar that doesn't support transactions..
Yep, see BerkeleyStorage.
> Another thing I noticed BTW, is that retrieving 500 messages and packing
> the database on every 50th message (all messages retrieved during one
> request) would lead to off the charts memory usage, the process ended up
> using 700 MB of RAM (had to add swap files as the footprint grew!)
You could either:
1) Import messages in a single transaction with subtransactions. This
would take less time and incur less memory consumption and database
bloat. We at DC have experimented with bulk-loading mail into Zope and
this is what works the best.
2) Change the database cache parameters to keep the RAM cache as low as
possible (set the timeout to 3 seconds and the target size to 10).
3) Manually zap the caches periodically, which is a capability of Zope
Zope-Dev maillist - [EMAIL PROTECTED]
** No cross posts or HTML encoding! **
(Related lists -