Your message dated Thu, 13 Oct 2016 15:24:35 +1300 with message-id <20161013022435.wgl64whzrsd2d...@survex.com> and subject line Re: Bug#724610: Acknowledgement (incrementally updated databases eventually become corrupt) has caused the Debian Bug report #724610, regarding incrementally updated databases eventually become corrupt to be marked as done.
This means that you claim that the problem has been dealt with. If this is not the case it is now your responsibility to reopen the Bug report if necessary, and/or fix the problem forthwith. (NB: If you are a system administrator and have no idea what this message is talking about, this may indicate a serious mail system misconfiguration somewhere. Please contact ow...@bugs.debian.org immediately.) -- 724610: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=724610 Debian Bug Tracking System Contact ow...@bugs.debian.org with problems
--- Begin Message ---Package: libsearch-xapian-perl Version: 1.2.10.0-1 Severity: normal ikiwiki's search plugin incrementally updates a xapian index as a wiki is edited. Certian of my larger wikis tend to corrupt themselves every month or two, preventing xapian-omega from finding anything. xapian-check says there is a btree error. I see this frequently on my main ikiwiki server. But I've heard from others who have also had this kind of corruption elsewhere so I don't think it's bad hardware. (I've also migrated my main ikiwiki server to new hardware at least twice in the years since I started experiencing this problem.) It used to be the case that this could cause Search::Xapian::WritableDatabase->new to throw an exception, as seen here: http://www.branchable.com/bugs/Exception:_Cannot_open_tables_at_consistent_revisions_at___47__usr__47__lib__47__perl5__47__Search__47__Xapian__47__WritableDatabase.pm_line_41./#comment-c159ea3f9be35fcd9ed0eeedb162e816 I have not seen that behavior since upgrading to wheezy. Now when the database is corrupt, further changes to it don't cause a crash (which is good); xapian-omega just never finds any matches. ikiwiki calls methods like add_term and replace_document_by_term and delete_document_by_term. It may be exercising the incremental update of xapian databases more frequently than is typical and exposing some bug. It's also the only thing in Debian that seems to use this perl library, so it could be that the bug in in this library and not in xapian itself. I suppose it's possible that the bug is in ikiwiki. For example, it could be that two ikiwiki processes end up manipulating the same xapian database concurrently. I don't know if that is allowed, or likely to corrupt it. But ikiwiki is supposed to use a lock to prevent two processes from both making stateful changes at the same time, and I have audited it and cannot find anywhere that it updates the xapian database without first taking that lock. I also wonder if this could be a problem with the flushing of the database. Ikiwiki relies on the automatic flushing behavior, and assumes that the database will be closed and flushed automatically when the xapian database object is destroyed at program termination. It also could be the case that ikiwiki sometimes crashes, and this could interfere with that. -- see shy jo
signature.asc
Description: Digital signature
--- End Message ---
--- Begin Message ---Control: fixed -1 1.2.21-1 Control: fixed -1 1.4.0-1 On Wed, Jul 01, 2015 at 02:21:23PM +1200, Olly Betts wrote: > It's possible it's a bug which has since been fixed - e.g. 1.2.21 > fixed a bug where the cursor could end up with bad data, which could > cause DatabaseCorruptError to be thrown when the database on disk was > fine. The bad data could theoretically end up written back, though I've > never actually seen that happen before something notices the data isn't > right and throws an exception. I'm increasingly believing this to be a manifestation of the bug referred to above (which was fixed in xapian-core 1.2.21-1, and that fix is also in 1.4.0-1). I'd like to have had a report from someone experiencing this before 1.2.21 and it going away upon upgrading. We don't have that, but there have been no further reports of this in over a year, and the newest version it has been reported in was 1.2.19-1 (or at least using libsearch-xapian-perl 1.2.19.0-1, and that's released upstream on the same schedule as xapian-core and typically updated in Debian at a similar time), and keeping this open indefinitely isn't useful, so I think it's time to close this bug. If anyone has seen this with a version >= 1.2.21 please follow-up with details and we can re-open. And if you were seeing this regularly but it's gone away, that would be useful to know too. Cheers, Olly
--- End Message ---
