Re: [Zope-dev] KeywordIndex errors: This should not happen.
Jean Jordaan wrote: > > Hmm! That's interesting. I was worried that we might be causing it > by tickling the Catalog the wrong way .. Did you ever get to the > bottom of the bugs? Do they have collector issues, perhaps? They were bugs in Zope's BTree implementation which were fixed for 2.6. cheers, Chris ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] KeywordIndex errors: This should not happen.
> It would be fine if we could get a reproducable testcase > for this problem. Absolutely. When the error we see occurs, the RID that is looked for has already gone missing. It looks like the real error happens silently sometime before, and not in all cases: the majority of the instances of the affected class are cataloged OK. -- Jean Jordaan Upfront Systems http://www.upfrontsystems.co.za ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] KeywordIndex errors: This should not happen.
On Tue, Oct 01, 2002 at 08:37:16AM +0200, Jean Jordaan wrote: > >>Have you tried it in 2.6? > > > >Yeah, this'd be my suggestion having suffered more terminally from > >exactly these bugs :-S > > Hmm! That's interesting. I was worried that we might be causing it > by tickling the Catalog the wrong way .. Did you ever get to the > bottom of the bugs? Do they have collector issues, perhaps? It would be fine if we could get a reproducable testcase for this problem. I have seen this problem multiple times in the past I have not been able to track it down to a simple testcase. Andreas ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] KeywordIndex errors: This should not happen.
>> Have you tried it in 2.6? > > Yeah, this'd be my suggestion having suffered more terminally from > exactly these bugs :-S Hmm! That's interesting. I was worried that we might be causing it by tickling the Catalog the wrong way .. Did you ever get to the bottom of the bugs? Do they have collector issues, perhaps? -- Jean Jordaan Upfront Systems http://www.upfrontsystems.co.za ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] KeywordIndex errors: This should not happen.
On Mon, 30 Sep 2002 09:50:56 -0400 Casey Duncan <[EMAIL PROTECTED]> wrote: > On Monday 30 September 2002 08:36 am, Roché Compaan wrote: > > Don't know if this will help but these errors specifically occur with > > 'lines' properties with Keyword Indexes on them. > > > > What is also significant is that there are legitimate duplicates in the > > lines properties and that they are duplicated on the index data as well. > > This doesn't feel right. I would have thought that only unique values > > should be stored in the index data. > > If you feel this is a bug, you should file a collector issue for it. I'm not sure yet ... -- Roché Compaan Upfront Systems http://www.upfrontsystems.co.za ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] KeywordIndex errors: This should not happen.
Toby Dickenson wrote: > >>Our app has been hitting these errors for months: > > Have you tried it in 2.6? A couple of bug in this area have been fixed since > 2.5.x Yeah, this'd be my suggestion having suffered more terminally from exactly these bugs :-S cheers, Chris ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] KeywordIndex errors: This should not happen.
> If you feel this is a bug, you should file a collector issue for it. We don't have a deep enough understanding of the Catalog to diagnose :/ We're not sure if it's a Catalog issue, or something we're doing wrong; i.e. we don't know how the Catalog gets into that state. -- Jean Jordaan Upfront Systems http://www.upfrontsystems.co.za ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] KeywordIndex errors: This should not happen.
On Monday 30 September 2002 08:36 am, Roché Compaan wrote: > Don't know if this will help but these errors specifically occur with > 'lines' properties with Keyword Indexes on them. > > What is also significant is that there are legitimate duplicates in the > lines properties and that they are duplicated on the index data as well. > This doesn't feel right. I would have thought that only unique values > should be stored in the index data. If you feel this is a bug, you should file a collector issue for it. -Casey ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] KeywordIndex errors: This should not happen.
On Monday 30 Sep 2002 1:17 pm, Jean Jordaan wrote: > Hi all > > Our app has been hitting these errors for months: Have you tried it in 2.6? A couple of bug in this area have been fixed since 2.5.x ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] KeywordIndex errors: This should not happen.
Don't know if this will help but these errors specifically occur with 'lines' properties with Keyword Indexes on them. What is also significant is that there are legitimate duplicates in the lines properties and that they are duplicated on the index data as well. This doesn't feel right. I would have thought that only unique values should be stored in the index data. -- Roché Compaan Upfront Systems http://www.upfrontsystems.co.za On Mon, 30 Sep 2002 14:17:17 +0200 Jean Jordaan <[EMAIL PROTECTED]> wrote: > Hi all > > Our app has been hitting these errors for months: > > 2002-09-30T11:06:27 ERROR(200) KeywordIndex unindex_object could not remove > documentId -444153566 from index ComponentVersions. This should not happen. > Traceback (innermost last): >File > /usr/local/zope/2-5-1/lib/python/Products/PluginIndexes/common/UnIndex.py, line > 168, in removeForwardIndexEntry > (Object: ComponentVersions) > KeyError: -444153566 ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
[Zope-dev] KeywordIndex errors: This should not happen.
Hi all
Our app has been hitting these errors for months:
2002-09-30T11:06:27 ERROR(200) KeywordIndex unindex_object could not remove
documentId -444153566 from index ComponentVersions. This should not happen.
Traceback (innermost last):
File
/usr/local/zope/2-5-1/lib/python/Products/PluginIndexes/common/UnIndex.py, line
168, in removeForwardIndexEntry
(Object: ComponentVersions)
KeyError: -444153566
Here is the relevant piece Zope code::
def removeForwardIndexEntry(self, entry, documentId):
"""Take the entry provided and remove any reference to documentId
in its entry in the index."""
global _marker
indexRow = self._index.get(entry, _marker)
if indexRow is not _marker:
try:
indexRow.remove(documentId)
if not indexRow:
del self._index[entry]
try: self.__len__.change(-1)
except AttributeError: pass # pre-BTrees-module instance
except AttributeError:
# index row is an int
del self._index[entry]
try: self.__len__.change(-1)
except AttributeError: pass # pre-BTrees-module instance
except:
import pdb; pdb.set_trace() #DBG
I inserted that 'set_trace' just before the error is logged. At this
point, the app state looks like this::
(Pdb) self
(Pdb) p self.id
'ComponentVersions'
(Pdb) entry
'44.131'
(Pdb) p indexRow
(Pdb) len(indexRow.keys())
566
(Pdb) indexRow.remove(documentId)
*** KeyError: -444153566
(Pdb) [self._index.keys()[n] for n in range(0,69)]
['', ' 2.13', '1', '1.14', '1.22', '1.25', '1.3', '1.6', '1.8.7',
'1.9.8', '17.9', '2.0.6', '2.1', '2.15', '2.2.1', '2.2.3m', '2.2.7',
'2.2.7m', '2.3.4m', '2.4', '2.4.2m', '2.4.3m', '2.4.4m', '2.4.5',
'2.5', '2.6.6m', '2.7.7m', '2.8.5m', '2.8.6m', '2.8.7m', '2.9',
'20.13-37', '22.13-42', '22.42-47', '22.5-42', '24.12-49',
'24.13-49', '24.19-49', '4.4', '4.9', '40.54e', '41.184', '42.95',
'43.11', '43.96', '43.96 ', '44.120', '44.131', '45.11', '7.3',
'7.7', '8.0', '8.1', '8.2', '8.4', '8.4.1', '8.5.9', '8.7.2m',
'8.7.6m', '8.8.9m', '8.9.6m', '8.9.7m', '9.0', '9.12', '9.2', '9.3',
'9.4 ', '9.8', 'x']
(Pdb) [indexRow.keys()[n] for n in range(0, len(indexRow.keys()))]
[-1948802264, -1018656130, -444153568, -111852827, -32258423,
-32258421, -32258398, -32258397, -32258396, 226891955, (and so on)
(Pdb) catalog = self.restrictedTraverse('Catalog')
(Pdb) catalog.getpath(-444153566)
'/Licenses/LicenseDocs/License_377'
(Pdb) catalog.getIndexDataForRID(-444153566)['ComponentVersions']
['2.4', '8.9.7m', '44.131', '44.131', '2.4.3m', '4.9', '9.12',
'24.13-49', '1.25', '2.15', '9.3', '2.8.6m']
Rephrasing that, I'd say: we're trying to remove the RID -444153566
from the list returned by: indexRow.keys()['44.131']
That RID is absent from that list, which raises a KeyError that
"should not happen". There *is* an entry in the Catalog for that RID,
which *has* '44.131' in the list under the 'ComponentVersions' key in
the dict returned by getIndexDataForRID.
Did the RID get removed from the KeywordIndex list of RIDs prematurely?
Or did it never get inserted? Can anyone shed any enlightenment? Where
else should I be looking?
--
Jean Jordaan
Upfront Systems http://www.upfrontsystems.co.za
___
Zope-Dev maillist - [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://lists.zope.org/mailman/listinfo/zope-announce
http://lists.zope.org/mailman/listinfo/zope )
