Package: release.debian.org Severity: normal Tags: stretch User: release.debian....@packages.debian.org Usertags: pu
I'd like to fix https://bugs.debian.org/906007 in stretch. The glass backend (the default disk-based backend in Xapian 1.4.x) has a bug with long-lived cursors on a table in a WritableDatabase which can get into an invalid state, typically leading to a DatabaseCorruptError being thrown with the message: Db block overwritten - are there multiple writers? But in fact the on-disk database is not corrupted - it's just that the cursor in memory has got into an inconsistent state. It looks like we'll always detect the inconsistency before it can cause on-disk corruption but it's hard to be completely certain. The usage patterns of notmuch can trigger this bug (at least two different notmuch users have hit it, and both reported 1.4.7 fixed their problems). It's also been encountered by at least one other person in their own code (they provided a cut-down reproducer that helped pin it down). * This bug is already fixed in unstable (also in testing and stretch-backports) * The bug is of severity "important" * Bug meta-data is up to date * The fix is minimal and includes a detailed changelog entry * A source debdiff of the proposed change is attached * The proposed package is versioned correctly (+deb9u2) * The patch came from upstream release 1.4.7 and has been in unstable and testing for more than three weeks. The backported version 1.4.7-1~bpo9+1 has been confirmed as fixing the bug by affected users. The proposed update passes the extensive upstream testsuite which runs during the package build. * The update was built in a stretch chroot I'll upload the package shortly as per the updated workflow for stable updates. Cheers, Olly
diff -Nru xapian-core-1.4.3/debian/changelog xapian-core-1.4.3/debian/changelog --- xapian-core-1.4.3/debian/changelog 2018-07-06 09:52:48.000000000 +1200 +++ xapian-core-1.4.3/debian/changelog 2018-08-13 18:19:13.000000000 +1200 @@ -1,3 +1,12 @@ +xapian-core (1.4.3-2+deb9u2) stretch; urgency=medium + + * fix-glass-cursor-bug.patch: Fix glass backend bug with long-lived cursors + on a table in a WritableDatabase which could incorrectly lead to + DatabaseCorruptError being thrown when the database was actually OK. + (Closes: #906007) + + -- Olly Betts <o...@survex.com> Mon, 13 Aug 2018 18:19:13 +1200 + xapian-core (1.4.3-2+deb9u1) stretch; urgency=medium * Fix MSet::snippet() to escape HTML in all cases (CVE-2018-499). diff -Nru xapian-core-1.4.3/debian/patches/fix-glass-cursor-bug.patch xapian-core-1.4.3/debian/patches/fix-glass-cursor-bug.patch --- xapian-core-1.4.3/debian/patches/fix-glass-cursor-bug.patch 1970-01-01 12:00:00.000000000 +1200 +++ xapian-core-1.4.3/debian/patches/fix-glass-cursor-bug.patch 2018-08-13 18:17:34.000000000 +1200 @@ -0,0 +1,63 @@ +Description: Fix glass bug with WritableDatabase cursors + A long-lived cursor on a table in a WritableDatabase could get into + an invalid state, which typically resulted in a DatabaseCorruptError + being thrown with the message: + . + Db block overwritten - are there multiple writers? + . + But in fact the on-disk database is not corrupted - it's just that + the cursor in memory has got into an inconsistent state. It looks + like we'll always detect the inconsistency before it can cause on-disk + corruption but it's hard to be complete certain. + . + The bug is in code to rebuild the cursor when the underlying table + changes in ways which require that. That's a fairly rare occurrence + (with my C++ reproducer it happens 99 times out of 5000 commits). + . + In chert the equivalent code just marks the cursor's blocks as not yet + read, but in glass cursor blocks are reference counted and shared so we + can't simply do that as it could affect other cursors sharing the same + blocks. + . + So instead the glass code was leaving them with the contents they + previously had, except for copying the current root block from the + table's "built-in cursor". After the rebuild we seek the cursor to the + same key it was on before, and that mostly works because we follow down + each level in the Btree from the new root, except it can happen that the + old cursor contained a block number which has since been released and + reallocated, and in that case the block doesn't get reread and we try to + use its old contents, which violates the rule that a parent can't be + younger than its child and causes the exception. + . + This commit fixes this by simply resetting the rebuilt cursor to match + the current "built-in cursor" at all levels (not just the root), which + is cheap because of the reference counting. + . + Reported with a reproducer by Sylvain Taverne. Confirmed by David + Bremner as fixing a problem in notmuch without a reduced reproducer. +Author: Olly Betts <o...@survex.com> +Bug-Debian: https://bugs.debian.org/906007 +Origin: upstream +Forwarded: https://git.xapian.org/?p=xapian;a=commitdiff;h=70266397a517e213977f49c5b7c2c91389c88879 +Last-Update: 2018-08-13 + +--- a/backends/glass/glass_cursor.cc ++++ b/backends/glass/glass_cursor.cc +@@ -73,6 +73,9 @@ GlassCursor::rebuild() + { + int new_level = B->level; + if (new_level <= level) { ++ for (int j = 0; j < new_level; ++j) { ++ C[j].clone(B->C[j]); ++ } + for (int j = new_level; j <= level; ++j) { + C[j].destroy(); + } +@@ -81,6 +84,7 @@ GlassCursor::rebuild() + C = new Cursor[new_level + 1]; + for (int i = 0; i < level; ++i) { + C[i].swap(old_C[i]); ++ C[i].clone(B->C[i]); + } + delete [] old_C; + for (int j = level; j < new_level; ++j) { diff -Nru xapian-core-1.4.3/debian/patches/series xapian-core-1.4.3/debian/patches/series --- xapian-core-1.4.3/debian/patches/series 2018-07-06 09:47:57.000000000 +1200 +++ xapian-core-1.4.3/debian/patches/series 2018-08-13 18:14:06.000000000 +1200 @@ -1,2 +1,3 @@ fix-unweighted-and.patch cve-2018-0499-mset-snippet-escaping.patch +fix-glass-cursor-bug.patch
signature.asc
Description: PGP signature
