[notmuch] emacs mode performance issue

[Carl Worth]

On Fri, 11 Dec 2009 09:40:23 -0800, Carl Worth  wrote:
> Which should let you tar these up and you can send them to me privately
> and I'll try to replicate and fix the bug.

Thanks for passing on the messages, Dirk. If anyone's curious the
message that triggered the bug is a public mail on the Linux Kernel
Mailing List:

id:alpine.LFD.2.00.0912081304070.3560 at localhost.localdomain

(It does have to be viewed in thread context---indented at least 13
columns to trigger the bug.)

The bug was an infinite loop add the button for a hidden citation,
(where our emacs lisp code got a bit confused about where the next line
was and infinitely looped on the same citation adding a never ending
sequence of "[1-line citation]" buttons).

I fixed the infinite loop and while in the area made a few other minor
improvements to the citation-hiding code:

  * No longer consider '>' as introducing a citation if not in first
column.

  * No longer insert an the extra blank line after the citation button.

  * Add text to the citation button to tell new users that they can
click or press Enter on the button to show the citation.

I'd still like to add one more feature which is a keybinding to make all
the citations for the current message visible. (It could even do the
whole thread like the button we used to have, but *only* if it ensured
that the current position of point within the current buffer and the
current window remained unchanged.)

I hope that's helpful,

-Carl
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
URL: 

[notmuch] emacs mode performance issue

[Carl Worth]

On Fri, 11 Dec 2009 08:24:02 -0800, Carl Worth  wrote:
> In the meantime, if I could get my hands on a message that triggers this
> bug, that would be useful. (Here's a case where it would be nice to have
> the "notmuch search --output=maildir" command I had talked about so that
> we could export the results of a particular search.)

Without a nice command to export a search as a maildir, here are some
things you can do in the meantime, Dirk. Consider this a free tutorial
in using the notmuch command line.

First, run a notmuch search to isolate the thread. This might be exactly
the search you were using within your notmuch emacs buffer, plus maybe a
keyword or two from the subject of the particular thread. So perhaps
something like:

notmuch search tag:lkml subject:"some phrase here"

[notmuch] emacs mode performance issue

[Carl Worth]

On Thu, 10 Dec 2009 22:10:13 -0800, Dirk Hohndel  
wrote:
> I'm in the search results window in Emacs, on an LKML thread with 140+
> messages. I hit return to view this thread - Emacs consumes 100% CPU
> but even after waiting 3 minutes it doesn't display the result (this is
> on a fast system Lenovo x200s). 
> 
> C-g stops the process and gets me dumped into a clearly partially
> processed buffer.
> 
> Is there a good way to collect more profiling information to figure out
> why this is so slow?

My guess is that it's not a problem of being slow.

It sounds much more likely that some of our lame emacs lisp code has
gotten itself into an infinite loop.

We've got a bunch of silly, ad-hoc code for jumping around the emacs
buffer looking for various things. And then we have invisible characters
in the emacs buffer which make some of the movement commands behave
differently, and unreliably. So there's some ugliness here.

The right fix is to move more of the parsing logic into the C code where
I can actually comprehend things, and then emit some fully-quoted
structure that we can walk with simple, reliable emacs code.

In the meantime, if I could get my hands on a message that triggers this
bug, that would be useful. (Here's a case where it would be nice to have
the "notmuch search --output=maildir" command I had talked about so that
we could export the results of a particular search.)

-Carl
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
URL: 

Re: [notmuch] emacs mode performance issue

[Carl Worth]

On Thu, 10 Dec 2009 22:10:13 -0800, Dirk Hohndel hohn...@infradead.org wrote:
 I'm in the search results window in Emacs, on an LKML thread with 140+
 messages. I hit return to view this thread - Emacs consumes 100% CPU
 but even after waiting 3 minutes it doesn't display the result (this is
 on a fast system Lenovo x200s). 
 
 C-g stops the process and gets me dumped into a clearly partially
 processed buffer.
 
 Is there a good way to collect more profiling information to figure out
 why this is so slow?

My guess is that it's not a problem of being slow.

It sounds much more likely that some of our lame emacs lisp code has
gotten itself into an infinite loop.

We've got a bunch of silly, ad-hoc code for jumping around the emacs
buffer looking for various things. And then we have invisible characters
in the emacs buffer which make some of the movement commands behave
differently, and unreliably. So there's some ugliness here.

The right fix is to move more of the parsing logic into the C code where
I can actually comprehend things, and then emit some fully-quoted
structure that we can walk with simple, reliable emacs code.

In the meantime, if I could get my hands on a message that triggers this
bug, that would be useful. (Here's a case where it would be nice to have
the notmuch search --output=maildir command I had talked about so that
we could export the results of a particular search.)

-Carl


pgp7WIj5GfkCp.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [notmuch] emacs mode performance issue

[Carl Worth]

On Fri, 11 Dec 2009 08:24:02 -0800, Carl Worth cwo...@cworth.org wrote:
 In the meantime, if I could get my hands on a message that triggers this
 bug, that would be useful. (Here's a case where it would be nice to have
 the notmuch search --output=maildir command I had talked about so that
 we could export the results of a particular search.)

Without a nice command to export a search as a maildir, here are some
things you can do in the meantime, Dirk. Consider this a free tutorial
in using the notmuch command line.

First, run a notmuch search to isolate the thread. This might be exactly
the search you were using within your notmuch emacs buffer, plus maybe a
keyword or two from the subject of the particular thread. So perhaps
something like:

notmuch search tag:lkml subject:some phrase here

From that you should be able to see the thread ID of interest, so you
can display the whole thread, (for reading it without hitting the emacs
infinite-loop bug), by just copy-and-paste of the thread ID to notmuch
show:

notmuch show thread:thread-id-here

And at that point you could manually archive the thread with:

notmuch tag -inbox thread:thread-id-here

So that should let you workaround the bug to at least read the thread.

As for debugging, we don't yet have any mechanism in the emacs code to
view a single message without viewing the entire thread, so there's not
an easy way for you to isolate which message is triggering the bug. But
you can at least get the list of filenames from the notmuch show output:

notmuch show thread:thread-id-here | grep message{ | sed 
's/.*filename:\(.*\)/\1/'

Which should let you tar these up and you can send them to me privately
and I'll try to replicate and fix the bug.

I hope that's helpful,

-Carl



pgpoFwcRaOwxX.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [notmuch] emacs mode performance issue

[Carl Worth]

On Fri, 11 Dec 2009 09:40:23 -0800, Carl Worth cwo...@cworth.org wrote:
 Which should let you tar these up and you can send them to me privately
 and I'll try to replicate and fix the bug.

Thanks for passing on the messages, Dirk. If anyone's curious the
message that triggered the bug is a public mail on the Linux Kernel
Mailing List:

id:alpine.lfd.2.00.0912081304070.3...@localhost.localdomain

(It does have to be viewed in thread context---indented at least 13
columns to trigger the bug.)

The bug was an infinite loop add the button for a hidden citation,
(where our emacs lisp code got a bit confused about where the next line
was and infinitely looped on the same citation adding a never ending
sequence of [1-line citation] buttons).

I fixed the infinite loop and while in the area made a few other minor
improvements to the citation-hiding code:

  * No longer consider '' as introducing a citation if not in first
column.

  * No longer insert an the extra blank line after the citation button.

  * Add text to the citation button to tell new users that they can
click or press Enter on the button to show the citation.

I'd still like to add one more feature which is a keybinding to make all
the citations for the current message visible. (It could even do the
whole thread like the button we used to have, but *only* if it ensured
that the current position of point within the current buffer and the
current window remained unchanged.)

I hope that's helpful,

-Carl


pgpV7hIROycq8.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[notmuch] emacs mode performance issue

[Dirk Hohndel]

I'm in the search results window in Emacs, on an LKML thread with 140+
messages. I hit return to view this thread - Emacs consumes 100% CPU
but even after waiting 3 minutes it doesn't display the result (this is
on a fast system Lenovo x200s). 

C-g stops the process and gets me dumped into a clearly partially
processed buffer.

Is there a good way to collect more profiling information to figure out
why this is so slow?

/D


-- 
Dirk Hohndel
Intel Open Source Technology Center