Re: Re: [Zope-dev] ERROR(200) ZODB Couldn't load state for... Memory problem?
On Fri, 2004-07-23 at 16:21, Tim Peters wrote:
> [Chris McDonough]
> > ...
> > self._f[current] = open(self._p[current],'w+b')
> >
> > will be likely to fail at the last line if you're using
> > nonpersistent cache files, because self._p[current] is (bogus)
> > '1-None-0' (relative bogus filename).
>
> Is it really *likely* to fail?
I suppose it depends on the working directory of the shell/process used
to start Zope. Zope doesn't mess with the working directory on its own,
AFAIK.
If you follow Richard Stevens' ("UNIX Network Programming" guy,
apparently now dead) advice, he says that "well-behaved" daemon
processes should change their working directory to "/". So I suspect
there are daemonizers that do this.
Guido's zdrun daemon (which "zopectl" uses) gives you an option to set
the working directory of the daemonized process, but I don't use it
(neither zdrun nor the option, that is). It does nothing to the working
directory by default.
But I think the common case is that the program is run out of an
/etc/init.d "rc" script, and I suspect the working directory is "/" when
Zope gets started in that circumstance. Which I guess makes the error
understandable.
> It's just a name, and it's opened in
> 'w' ('+b') mode, not 'r' mode. That is, it creates the file -- no
> file of that name need already exist (and if one does, it tries to
> overrwrite it). Running on Windows most days, I'm not usually aware
> of all the permission bugs Linuxheads delight in torturing themselves
> with .
Yes. Gotta agree with you there. I don't think a day passes where I
don't want to rip the face off the guy who proclaimed that TCP ports
below 1024 couldn't be bound to by a user other than root. What a
disaster.
> > There should probably be a _using_persistent_cache flag attr rather than
> > trying to inspect self._p to find out if we're using persistent caches.
>
> +1. As you later discovered, this "hmm, let's try to guess what we're
> doing based on obscure droppings" business is a continuing bug
> factory.
Thankfully, Dieter fixed it so it doesn't (at least in this one case).
>
> > I may try to work up a patch + test for this later.
>
> I'm neutral on whether you try, but +1 on you actually doing it .
Too late! It's already fixed. I didn't know either. ;-) This thread
was full of sound and fury
- C
___
Zope-Dev maillist - [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )
Re: Re: [Zope-dev] ERROR(200) ZODB Couldn't load state for... Memory problem?
[Chris McDonough]
> ...
> self._f[current] = open(self._p[current],'w+b')
>
> will be likely to fail at the last line if you're using
> nonpersistent cache files, because self._p[current] is (bogus)
> '1-None-0' (relative bogus filename).
Is it really *likely* to fail? It's just a name, and it's opened in
'w' ('+b') mode, not 'r' mode. That is, it creates the file -- no
file of that name need already exist (and if one does, it tries to
overrwrite it). Running on Windows most days, I'm not usually aware
of all the permission bugs Linuxheads delight in torturing themselves
with .
> There should probably be a _using_persistent_cache flag attr rather than
> trying to inspect self._p to find out if we're using persistent caches.
+1. As you later discovered, this "hmm, let's try to guess what we're
doing based on obscure droppings" business is a continuing bug
factory.
> I may try to work up a patch + test for this later.
I'm neutral on whether you try, but +1 on you actually doing it .
___
Zope-Dev maillist - [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )
Re: Re: [Zope-dev] ERROR(200) ZODB Couldn't load state for... Memory problem?
[Chris McDonough] > ... > I think the problem is related to ZEO client storage "cache flips". Me too. The 3.2 ZEO cache alternates between two cache files, in the two-element list self._f. Both elements are initialized to None, and the index of the current file in use (0 or 1) is in self._current. There's an implicit assumption throughout the code that self._f[self._current] is always an actual file object, but the "flip logic" is excruciating and that global invariant certainly isn't self-evident. > I found an error in the log near the time of the "None has no attribute > seek" symptom indicating that the Zope process tried to "flip" a ZEO > cache file (by creating a new file) but UNIX file system permissions > apparently prevented it. This was a traceback ending somewhere in ClientCache.checkSize()? That's where a cache flip happens. It changes its idea of self._current (from 0 to 1 or from 1 to 0) *before* making sure there's an actual file object in "the other" self._f slot. So, e.g., if self._f started life as [good_file_object, None] and self._current started at 0, and it came time for a cache flip, and a new file object couldn't be created in self._f[1], self._current would end up as 1 anyway, pointing to None. But this code gives me a headache, and I'm not sure that can actually happen (despite that I hear you guys saying it is ). > But then I turned off persistent ZEO client cachefile storage (but omitting > the "zeo-client-name" parameter from zope.conf), believing this would be a > workaround, but it hasn't been. I gave up at that point and that's where I am now. Did you continue to get errors in the log near cache-flip times? I don't see a way for checkSize() to screw up unless an unexpected exception is raised. > ... > My theory is that it will happen as often as a Zope client's ZEO client storage > needs to flip its cache file. The cache file is only flipped when it exceeds a > certain size and it only exceeds a certain size after a certain pattern of usage > causes it to do so (lots of loads from the database of new items, typically). It appears that once self._f[self._current] is None, all future attempts by ZEO to store into its client cache will fail the same way. So I'd be even more surprised if you saw just one of these occur. > It would be nice if you could confirm this. Reading the Zope event log > file of the client that generated the error would be a good start. The log is everything here. The ZEO client cache logs most relevant messages at info level, producing msgs starting with "ZEC". ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
