On Sep 30, 2008, at 1:38 PM, Dieter Maurer wrote:
> Jim Fulton wrote at 2008-9-19 13:45 -0400:
>> 2. We (ZC) are moving to 64-bit OSs. I've resisted this for a while
>> due to the extra memory overhead of 64-bit pointers in Python
>> programs, but I've finally (too late) come around to realizing that
>> the benefit far outweighs the cost. (In this case, the process was
>> around 900MB in size.
> That is very strange.
> On our Linux systems (Debian etch), the processes can use 2.7 to 2.9
> of memory before the os refuses to allocate more.
>> It was probably trying to malloc a few hundred
>> MB. The malloc failed despite the fact that there was more than 2GB
>> of available process address space and system memory.)
>> 3. I plan to add code to FileStorage's _finish that will, if there's
>> an error:
>> a. Log a critical message.
>> b. Try to roll back the disk commit.
I decided not to do this. Too complicated.
>> c. Close the file storage, causing subsequent reads and writes to
> Raise an easily recognizable exception.
I raise the original exception.
> In our error handling we look out for some nasty exceptions and
> a restart in such cases. The exception above might be such a nasty
The critical log entry should be easy enough to spot.
>> I considered some other ideas:
>> - Try to get FileStorage to repair it's meta data. This is certainly
>> theoretically doable. For example, it could re-build it's in-memory
>> index. At this point, that's the only thing in question. OTOH,
>> updating it is the only thing left to fail at this point. If
>> it fails, it seems likely that rebuilding it will fail as well.
>> - Have a storage server restart when a tpc_finish call fails. This
>> would work fine for FileStorage, but might be the wrong thing to do
>> for another storage. The server can't know.
> Why do you think that a failing "tpc_finish" is less critical
> for some other kind of storage?
It's not a question of criticality. It's a question of whether a
restart will fix the problem. I happen to know that a file storage
would be in a reasonable state after a restart. I don't know this to
be the case for some other storage.
For more information about ZODB, see the ZODB Wiki:
ZODB-Dev mailing list - ZODB-Dev@zope.org