Re: Error while compacting: Bad position key

2018-07-12 Thread David Bremner
Mike Hommey  writes:

> Hi,
>
> When running `notmuch compact` today, it stopped with the following
> output:
>
> Compacting database...
> compacting table postlist
>  Reduced by 25% 648656K (2498904K -> 1850248K)
> compacting table docdata
>  Reduced by 15% 24K (152K -> 128K)
> compacting table termlist
>  Reduced by 1% 27008K (2211800K -> 2184792K)
> compacting table position
> Error while compacting: Bad position key

I had not seen anything like this before, but when I run xapian-check
from xapian 1.4.6-2, I see


termlist:
B-tree checked okay
doclen not within bounds
doclen not within bounds
doclen not within bounds
doclen not within bounds
doclen not within bounds
doclen not within bounds
doclen not within bounds
doclen not within bounds

[repeated 52498 times]

OTOH, notmuch compact completes for me, so that might be unrelated.
I'm copying the Xapian list for input.

> Compaction failed: A Xapian exception occurred
>
> Running xapian-check says:
>
> docdata:
> blocksize=8K items=2677 firstunused=19 revision=15425 levels=1 root=17
> B-tree checked okay
> docdata table structure checked OK
>
> termlist:
> blocksize=8K items=2986940 firstunused=276475 revision=15425 levels=2 
> root=271786
> B-tree checked okay
> doclen not within bounds
> (...)
> doclen not within bounds
> termlist table errors found: 107982
>
> postlist:
> blocksize=8K items=16090818 firstunused=312363 revision=15425 levels=3 
> root=249894
> B-tree checked okay
> postlist table structure checked OK
>
> position:
> blocksize=8K items=236476398 firstunused=653990 revision=15425 levels=3 
> root=598684
> xapian-check: DatabaseError: Block 459158 item 179: not in sorted order
>

What version of Xapian is this?  I guess an obvious question is if
there is some potential external cause of filesystem corruption
(e.g. smart errrors, power loss at an inconvenient time).

> Is there something I can do, or do I essentially need to completely
> rebuild the database (and if so, what's the best way to do it?)
>

xapian-check has an option F to attempt to fix things. It might not
help, but it shouldn't hurt. Otherwise something like

% notmuch dump > backup.txt
% mv Maildir/.notmuch $backup
% notmuch new
% notmuch restore < backup.txt

will rebuild the database.
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Error while compacting: Bad position key

2018-07-12 Thread Mike Hommey
Hi,

When running `notmuch compact` today, it stopped with the following
output:

Compacting database...
compacting table postlist
 Reduced by 25% 648656K (2498904K -> 1850248K)
compacting table docdata
 Reduced by 15% 24K (152K -> 128K)
compacting table termlist
 Reduced by 1% 27008K (2211800K -> 2184792K)
compacting table position
Error while compacting: Bad position key

Compaction failed: A Xapian exception occurred

Running xapian-check says:

docdata:
blocksize=8K items=2677 firstunused=19 revision=15425 levels=1 root=17
B-tree checked okay
docdata table structure checked OK

termlist:
blocksize=8K items=2986940 firstunused=276475 revision=15425 levels=2 
root=271786
B-tree checked okay
doclen not within bounds
(...)
doclen not within bounds
termlist table errors found: 107982

postlist:
blocksize=8K items=16090818 firstunused=312363 revision=15425 levels=3 
root=249894
B-tree checked okay
postlist table structure checked OK

position:
blocksize=8K items=236476398 firstunused=653990 revision=15425 levels=3 
root=598684
xapian-check: DatabaseError: Block 459158 item 179: not in sorted order


Is there something I can do, or do I essentially need to completely
rebuild the database (and if so, what's the best way to do it?)

Cheers,

Mike
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: Bug: fatal error with notmuch new, second run starts indexing all over again

2018-07-12 Thread Leonard Lausen
I run notmuch 0.27 and xapian 1.4.3 (from the Debian stretch repo).

Now that Xapian 1.4.6 is released (which should contain fixes to prevent
the database corruption that may occur due to this bug) I will upgrade
to it.

David Bremner  writes:
> While we're sorting out test data, what versions of notmuch and xapian
> are you running?
>
> d
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch