notmuch painfully slow

2011-05-06 Thread Xavier Maillard 
Hi,

On Tue, 26 Apr 2011 22:04:13 +0200, Florian Friesdorf  
wrote:
> 
> thank you all for your answers!

I second that and thanks to your performance page, I finally switched to
chert format database. That's day and night !

Thank you !

/Xavier

notmuch painfully slow

2011-04-26 Thread Florian Friesdorf 

thank you all for your answers!

I am now also on a compacted chert database and started a "performance"
section:

http://notmuchmail.org/performance/

-- 
Florian Friesdorf 
  GPG FPR: 7A13 5EEE 1421 9FC2 108D  BAAF 38F8 99A3 0C45 F083
Jabber/XMPP: flo at chaoflow.net
IRC: chaoflow on freenode,ircnet,blafasel,OFTC
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL:

notmuch painfully slow

2011-04-24 Thread Pieter Praet 
On Sat, 23 Apr 2011 07:18:52 +0200, Florian Friesdorf  
wrote:
> [...]
> Is this the switch from xapian 1.0.x to 1.2.x or are further steps
> necessary?
> 
> Are there instructions on how to do the switch? For the mailing list
> archive a link would be great - a quick search for 'xapian flint chert'
> was unsuccessful.
> 
> In case there are no instructions yet, I'm happy to conclude what is
> gathered here and add it to notmuch's webpage.
> 
> -- 
> Florian Friesdorf 
>   GPG FPR: 7A13 5EEE 1421 9FC2 108D  BAAF 38F8 99A3 0C45 F083
> Jabber/XMPP: flo at chaoflow.net
> IRC: chaoflow on freenode,ircnet,blafasel,OFTC
Non-text part: application/pgp-signature

On Sat, 23 Apr 2011 09:20:53 +0200, Xavier Maillard  wrote:
> [...]
> Obviously, no, I did not. I even did not know about that ;) What's the
> story with this and what's the way to do this exactly ?
> 
> Thank you.
> 
> /Xavier


Both Carl [1] and Sebastian [2] posted about this a little while ago.

Bottom line: Chert is the new db format (replacing Flint) as of Xapian
1.2.x [3], and should -in most cases- be alot faster, whilst consuming
less disk space.

If .notmuch/xapian contains a file called "iamflint", you're still on
the old format. Otherwise it's called (surprise) "iamchert". Perhaps
somewhat confusingly, "flintlock" remains though.

Switching your notmuch db is straightforward:
>> notmuch dump ${tagdump}
>> rm -rf .notmuch/xapian
>> notmuch new
>> notmuch restore ${tagdump}
>> mv .notmuch/xapian .notmuch/xapian-fat
>> xapian-compact --no-renumber .notmuch/xapian-fat .notmuch/xapian
>> rm -rf .notmuch/xapian-fat

The last 3 lines are entirely optional and will cause writes to be a
bit more expensive in the short term, but will significantly reduce
the db's size, which your SSD will surely appreciate.


Peace

-Pieter

[1] id:"87ocl1lut1.fsf at yoom.home.cworth.org"
[2] id:"87lj0iqa3c.fsf at SSpaeth.de"
[3] http://www.archlinux.org/packages/?q=xapian-core

Re: notmuch painfully slow

2011-04-24 Thread Pieter Praet 
On Sat, 23 Apr 2011 07:18:52 +0200, Florian Friesdorf f...@chaoflow.net wrote:
 [...]
 Is this the switch from xapian 1.0.x to 1.2.x or are further steps
 necessary?
 
 Are there instructions on how to do the switch? For the mailing list
 archive a link would be great - a quick search for 'xapian flint chert'
 was unsuccessful.
 
 In case there are no instructions yet, I'm happy to conclude what is
 gathered here and add it to notmuch's webpage.
 
 -- 
 Florian Friesdorf f...@chaoflow.net
   GPG FPR: 7A13 5EEE 1421 9FC2 108D  BAAF 38F8 99A3 0C45 F083
 Jabber/XMPP: f...@chaoflow.net
 IRC: chaoflow on freenode,ircnet,blafasel,OFTC
Non-text part: application/pgp-signature

On Sat, 23 Apr 2011 09:20:53 +0200, Xavier Maillard x...@gnu.org wrote:
 [...]
 Obviously, no, I did not. I even did not know about that ;) What's the
 story with this and what's the way to do this exactly ?
 
 Thank you.
 
 /Xavier


Both Carl [1] and Sebastian [2] posted about this a little while ago.

Bottom line: Chert is the new db format (replacing Flint) as of Xapian
1.2.x [3], and should -in most cases- be alot faster, whilst consuming
less disk space.

If .notmuch/xapian contains a file called iamflint, you're still on
the old format. Otherwise it's called (surprise) iamchert. Perhaps
somewhat confusingly, flintlock remains though.

Switching your notmuch db is straightforward:
 notmuch dump ${tagdump}
 rm -rf .notmuch/xapian
 notmuch new
 notmuch restore ${tagdump}
 mv .notmuch/xapian .notmuch/xapian-fat
 xapian-compact --no-renumber .notmuch/xapian-fat .notmuch/xapian
 rm -rf .notmuch/xapian-fat

The last 3 lines are entirely optional and will cause writes to be a
bit more expensive in the short term, but will significantly reduce
the db's size, which your SSD will surely appreciate.


Peace

-Pieter

[1] id:87ocl1lut1@yoom.home.cworth.org
[2] id:87lj0iqa3c@sspaeth.de
[3] http://www.archlinux.org/packages/?q=xapian-core
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

notmuch painfully slow

2011-04-23 Thread Xavier Maillard 
Hi Pieter,

On Fri, 22 Apr 2011 22:18:09 +0200, Pieter Praet  wrote:
> On Fri, 22 Apr 2011 16:14:52 +0200, Xavier Maillard  wrote:
> >
> > as time goes, notmuch is really becoming slower and slower. It may be
> > normal as it has more and more messages to "track" but I am not sure it
> > is that normal give these numbers:

[SNIP]

> > What is happening here ? Is there any maintenance task I should do onto
> > my mail collection ?
> > 
> > Thank you for your help.
> 
> Have you switched your Xapian database from flint to chert yet? This
> really makes a huge difference in responsiveness.

Obviously, no, I did not. I even did not know about that ;) What's the
story with this and what's the way to do this exactly ?

Thank you.

/Xavier

notmuch painfully slow

2011-04-23 Thread Florian Friesdorf 
On Fri, 22 Apr 2011 22:18:09 +0200, Pieter Praet  wrote:
> Have you switched your Xapian database from flint to chert yet? This
> really makes a huge difference in responsiveness.

Is this the switch from xapian 1.0.x to 1.2.x or are further steps
necessary?

Are there instructions on how to do the switch? For the mailing list
archive a link would be great - a quick search for 'xapian flint chert'
was unsuccessful.

In case there are no instructions yet, I'm happy to conclude what is
gathered here and add it to notmuch's webpage.

-- 
Florian Friesdorf 
  GPG FPR: 7A13 5EEE 1421 9FC2 108D  BAAF 38F8 99A3 0C45 F083
Jabber/XMPP: flo at chaoflow.net
IRC: chaoflow on freenode,ircnet,blafasel,OFTC
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL:

notmuch painfully slow

2011-04-23 Thread Simon Fondrie-Teitler 
On Sat, 23 Apr 2011 07:18:52 +0200, Florian Friesdorf  
wrote:
> On Fri, 22 Apr 2011 22:18:09 +0200, Pieter Praet  wrote:
> Is this the switch from xapian 1.0.x to 1.2.x or are further steps
> necessary?
> 
> Are there instructions on how to do the switch? For the mailing list
> archive a link would be great - a quick search for 'xapian flint chert'
> was unsuccessful.
Notmuch has been very slow for me, and I decided to give cert a try. A
search turned up this message:
http://notmuchmail.org/pipermail/notmuch/2010/001105.html 
as well as: http://notmuchmail.org/pipermail/notmuch/2011/004048.html

It seems you have to rebuild the database with "XAPIAN_PREFER_CHERT=1
notmuch new"

I have done this, and it does seemed to have sped up notmuch. 

  --Simon
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL:

Re: notmuch painfully slow

2011-04-23 Thread Simon Fondrie-Teitler 
On Sat, 23 Apr 2011 07:18:52 +0200, Florian Friesdorf f...@chaoflow.net wrote:
 On Fri, 22 Apr 2011 22:18:09 +0200, Pieter Praet pie...@praet.org wrote:
 Is this the switch from xapian 1.0.x to 1.2.x or are further steps
 necessary?
 
 Are there instructions on how to do the switch? For the mailing list
 archive a link would be great - a quick search for 'xapian flint chert'
 was unsuccessful.
Notmuch has been very slow for me, and I decided to give cert a try. A
search turned up this message:
http://notmuchmail.org/pipermail/notmuch/2010/001105.html 
as well as: http://notmuchmail.org/pipermail/notmuch/2011/004048.html

It seems you have to rebuild the database with XAPIAN_PREFER_CHERT=1
notmuch new

I have done this, and it does seemed to have sped up notmuch. 

  --Simon


pgplZXFb40o8G.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

notmuch painfully slow

2011-04-22 Thread Pieter Praet 
On Fri, 22 Apr 2011 16:14:52 +0200, Xavier Maillard  wrote:
> Hi,
> 
> as time goes, notmuch is really becoming slower and slower. It may be
> normal as it has more and more messages to "track" but I am not sure it
> is that normal give these numbers:
> 
> - notmuch version: notmuch 0.5-133-gc509598
> - notmuch count: 18475
> - uname: Linux kcals 2.6.33.4-smp #2 SMP Wed May 12 22:47:36 CDT 2010
>  i686 Intel(R) Atom(TM) CPU N270   @ 1.60GHz GenuineIntel GNU/Linux
> - Load average: 0.00 0.00 0.00 1/144 28377
> - Memory:
>  total   used   free sharedbuffers cached
> Mem:   992881111  0253500
> -/+ buffers/cache:127865
> Swap:  506  0505
> 
> This is on a Slackware GNU/linux system on a Eee PC 901 (thus SSD
> storage). I am using my computer *exclusively* in the virtual linux
> console (tty) and 99% of my tasks are done through GNU Emacs (dunno if
> is of any interest for this problem though).
> 
> My sync script is taking really long time to achieve, updating one mail
> at a time is also really slow (several seconds). In the worst cases,
> updating my collection can take upto 15 minutes and updating just one
> message can take up to the minute too !
> 
> What is happening here ? Is there any maintenance task I should do onto
> my mail collection ?
> 
> Thank you for your help.

Have you switched your Xapian database from flint to chert yet? This
really makes a huge difference in responsiveness.

> /Xavier
> ___
> notmuch mailing list
> notmuch at notmuchmail.org
> http://notmuchmail.org/mailman/listinfo/notmuch


Peace

-Pieter

notmuch painfully slow

2011-04-22 Thread Xavier Maillard 
Hi,

as time goes, notmuch is really becoming slower and slower. It may be
normal as it has more and more messages to "track" but I am not sure it
is that normal give these numbers:

- notmuch version: notmuch 0.5-133-gc509598
- notmuch count: 18475
- uname: Linux kcals 2.6.33.4-smp #2 SMP Wed May 12 22:47:36 CDT 2010
 i686 Intel(R) Atom(TM) CPU N270   @ 1.60GHz GenuineIntel GNU/Linux
- Load average: 0.00 0.00 0.00 1/144 28377
- Memory:
 total   used   free sharedbuffers cached
Mem:   992881111  0253500
-/+ buffers/cache:127865
Swap:  506  0505

This is on a Slackware GNU/linux system on a Eee PC 901 (thus SSD
storage). I am using my computer *exclusively* in the virtual linux
console (tty) and 99% of my tasks are done through GNU Emacs (dunno if
is of any interest for this problem though).

My sync script is taking really long time to achieve, updating one mail
at a time is also really slow (several seconds). In the worst cases,
updating my collection can take upto 15 minutes and updating just one
message can take up to the minute too !

What is happening here ? Is there any maintenance task I should do onto
my mail collection ?

Thank you for your help.

/Xavier

notmuch painfully slow

2011-04-22 Thread Xavier Maillard 
Hi,

as time goes, notmuch is really becoming slower and slower. It may be
normal as it has more and more messages to track but I am not sure it
is that normal give these numbers:

- notmuch version: notmuch 0.5-133-gc509598
- notmuch count: 18475
- uname: Linux kcals 2.6.33.4-smp #2 SMP Wed May 12 22:47:36 CDT 2010
 i686 Intel(R) Atom(TM) CPU N270   @ 1.60GHz GenuineIntel GNU/Linux
- Load average: 0.00 0.00 0.00 1/144 28377
- Memory:
 total   used   free sharedbuffers cached
Mem:   992881111  0253500
-/+ buffers/cache:127865
Swap:  506  0505

This is on a Slackware GNU/linux system on a Eee PC 901 (thus SSD
storage). I am using my computer *exclusively* in the virtual linux
console (tty) and 99% of my tasks are done through GNU Emacs (dunno if
is of any interest for this problem though).

My sync script is taking really long time to achieve, updating one mail
at a time is also really slow (several seconds). In the worst cases,
updating my collection can take upto 15 minutes and updating just one
message can take up to the minute too !

What is happening here ? Is there any maintenance task I should do onto
my mail collection ?

Thank you for your help.

/Xavier
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: notmuch painfully slow

2011-04-22 Thread Pieter Praet 
On Fri, 22 Apr 2011 16:14:52 +0200, Xavier Maillard x...@gnu.org wrote:
 Hi,
 
 as time goes, notmuch is really becoming slower and slower. It may be
 normal as it has more and more messages to track but I am not sure it
 is that normal give these numbers:
 
 - notmuch version: notmuch 0.5-133-gc509598
 - notmuch count: 18475
 - uname: Linux kcals 2.6.33.4-smp #2 SMP Wed May 12 22:47:36 CDT 2010
  i686 Intel(R) Atom(TM) CPU N270   @ 1.60GHz GenuineIntel GNU/Linux
 - Load average: 0.00 0.00 0.00 1/144 28377
 - Memory:
  total   used   free sharedbuffers cached
 Mem:   992881111  0253500
 -/+ buffers/cache:127865
 Swap:  506  0505
 
 This is on a Slackware GNU/linux system on a Eee PC 901 (thus SSD
 storage). I am using my computer *exclusively* in the virtual linux
 console (tty) and 99% of my tasks are done through GNU Emacs (dunno if
 is of any interest for this problem though).
 
 My sync script is taking really long time to achieve, updating one mail
 at a time is also really slow (several seconds). In the worst cases,
 updating my collection can take upto 15 minutes and updating just one
 message can take up to the minute too !
 
 What is happening here ? Is there any maintenance task I should do onto
 my mail collection ?
 
 Thank you for your help.

Have you switched your Xapian database from flint to chert yet? This
really makes a huge difference in responsiveness.

 /Xavier
 ___
 notmuch mailing list
 notmuch@notmuchmail.org
 http://notmuchmail.org/mailman/listinfo/notmuch


Peace

-Pieter
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: notmuch painfully slow

2011-04-22 Thread Florian Friesdorf 
On Fri, 22 Apr 2011 22:18:09 +0200, Pieter Praet pie...@praet.org wrote:
 Have you switched your Xapian database from flint to chert yet? This
 really makes a huge difference in responsiveness.

Is this the switch from xapian 1.0.x to 1.2.x or are further steps
necessary?

Are there instructions on how to do the switch? For the mailing list
archive a link would be great - a quick search for 'xapian flint chert'
was unsuccessful.

In case there are no instructions yet, I'm happy to conclude what is
gathered here and add it to notmuch's webpage.

-- 
Florian Friesdorf f...@chaoflow.net
  GPG FPR: 7A13 5EEE 1421 9FC2 108D  BAAF 38F8 99A3 0C45 F083
Jabber/XMPP: f...@chaoflow.net
IRC: chaoflow on freenode,ircnet,blafasel,OFTC


pgp63dieUMoFQ.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch