Re: [notmuch] Missing messages breaking threads

2009-12-22 Thread Carl Worth
On Tue, 22 Dec 2009 22:48:25 + (UTC), Olly Betts o...@survex.com wrote:
 This is just the sort of thing which Xapian's user metadata is there
 for.  It's essentially a key/value store which is versioned along with
 the rest of the Xapian database.  So to set it:
 
   database.set_metadata(version, 1);
 
 And to read (and default if not set):
 
   string version = database.get_metadata(version);
   if (version.empty()) version = 0;

Thanks, Olly!

That is exactly what we'll want here, and is much better than a magic
document.

-Carl (grateful to have a Xapian expert keeping watch on the list)


pgpXLbC5HmGJ2.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Re: [notmuch] Missing messages breaking threads

2009-12-18 Thread Carl Worth
On Fri, 18 Dec 2009 19:53:13 +, James Westby jw+deb...@jameswestby.net 
wrote:
 Or do we not index whatever dummy text we add? Or do we not
 even put it in? Or not even show it at all? I was just thinking
 of having Missing messages... showing up as the start of
 the thread, but maybe it's no needed.

Oh, I was assuming you wouldn't index any text. The UI can add missing
message for a document with no filename, for example.

 So, to summarise, I should first look at storing filesizes, then
 the collision code to make it index further when the filesize grows,
 and then finally the code to add documents for missing messages?

Some of the code areas to be touched will be changing soon, (at least as
far as when filenames appear and disappear). Hopefully I'll have
something posted for that sooner rather than later to avoid having to
redo too much work.

 The only thing I am unclear on is how to handle existing databases?
 Do we have any concept of versioning? Or should I just assume that
 filesize: may not be in the document and act appropriately?

My current, outstanding patch is going to be the first trigger for a
flag day where we'll all need to rewrite our databases.

We don't have any concept of versioning yet, but it would obviously be
easy to have a new version document with an increasing integer.

But even with my current patch I'm considering doing a graceful upgrade
of the database in-place rather than making the user do something like a
dump, delete, rebuild, restore. That would give a much better experience
than Your database is out-of-date, please rebuild it, so we'll see if
I pursue that in the end.

-Carl




pgpXvyQBVFou6.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Re: [notmuch] Missing messages breaking threads

2009-12-18 Thread James Westby
On Fri, 18 Dec 2009 12:52:58 -0800, Carl Worth cwo...@cworth.org wrote:
 On Fri, 18 Dec 2009 19:53:13 +, James Westby jw+deb...@jameswestby.net 
 wrote:
 Oh, I was assuming you wouldn't index any text. The UI can add missing
 message for a document with no filename, for example.

Works for me.

  So, to summarise, I should first look at storing filesizes, then
  the collision code to make it index further when the filesize grows,
  and then finally the code to add documents for missing messages?
 
 Some of the code areas to be touched will be changing soon, (at least as
 far as when filenames appear and disappear). Hopefully I'll have
 something posted for that sooner rather than later to avoid having to
 redo too much work.

That would be great. I'm learning all the code anyway, so there's not
a whole lot of knowledge being thrown away.

I've just sent an initial cut at the fist step.

  The only thing I am unclear on is how to handle existing databases?
  Do we have any concept of versioning? Or should I just assume that
  filesize: may not be in the document and act appropriately?
 
 My current, outstanding patch is going to be the first trigger for a
 flag day where we'll all need to rewrite our databases.
 
 We don't have any concept of versioning yet, but it would obviously be
 easy to have a new version document with an increasing integer.
 
 But even with my current patch I'm considering doing a graceful upgrade
 of the database in-place rather than making the user do something like a
 dump, delete, rebuild, restore. That would give a much better experience
 than Your database is out-of-date, please rebuild it, so we'll see if
 I pursue that in the end.

That sounds nice, I'd certainly prefer this sort of thing as it evolves.

Thanks,

James
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch