[sup-talk] [Crash] "thread_by_subject: true" leads to "wrong id called on nil"

Alexandra Walford Mon, 10 Mar 2008 21:46:47 -0700

Hi all,

I am getting frequent sup crashes with "wrong id called on nil" (using the
latest git master), when thread_by_subject is true.


The crash generally happens when the thread index is being built as a result of
a search, or when opening the inbox (which is really just another sort of
search).

I have managed to narrow down one particular crash to two messages sent to the
sup-talk list.  If I set up a new maildir containing only those two messages
and a new sup config, accepting all sup-config defaults and changing
thread_by_subject to true, then I can reproduce the crash reliably.  The second
message References: the first message, although the subjects are dissimilar.

I have attached the two messages, a transcript of my test case session
(including backtrace), and sup-exception-log.txt.  I am not yet familiar enough
with sup or ruby to offer a decent patch; my current fix just hides any thread
which will cause a crash, on line 208 of lib/sup/modes/thread-index-mode.rb in
git master, which is a hopeless workaround really.

Thanks,

Alexandra

-- 
[EMAIL PROTECTED]

From: Christopher Warrington <[EMAIL PROTECTED]>
To: sup-talk <[email protected]>
Date: Mon, 03 Mar 2008 01:07:55 -0600
Message-Id: <[EMAIL PROTECTED]>
User-Agent: Sup/git
Subject: [sup-talk] Maildir source with one message
Precedence: list
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Sender: [EMAIL PROTECTED]
Errors-To: [EMAIL PROTECTED]

In playing with offline IMAP and Maildir sources (much faster than a native 
IMAP source and it can be used offline!), I noticed an odd bug.

If there is only one message in a Maildir source, neither sup not
sup-sync can see this message. Adding another message makes both
visible messages visible.

--- here begins speculation ---

I *THINK* that this has to do with the definitions Source.start_offset
and Source.end_offset. As I understand it, they should define a range
of message ids [start_offset, end_offset).

Now, when start_offset = end_offset--as is the case when there is only
one message, bad things happen. As I understand the math:
(a,a) = {}
[a,a) = {} *our case
(a,a] = {}
[a,a] = {a.

Changing maildir.rb:126 from:
    @ids.last
to
    @ids.last + 1
appears to have fixed the problem without ill-effect.

Simple inspection of imap.rp leads be to believe that it will have the
same problem.

A mbox source will have the same problem iff the mbox is empty.

--- here ends speculation ---

-- 
Christopher Warrington <[EMAIL PROTECTED]>
_______________________________________________
sup-talk mailing list
[email protected]
http://rubyforge.org/mailman/listinfo/sup-talk

From: Christopher Warrington <[EMAIL PROTECTED]>
To: [email protected]
Date: Mon,  3 Mar 2008 02:00:33 -0600
Message-Id: <[EMAIL PROTECTED]>
In-Reply-To: <[EMAIL PROTECTED]>
References: <[EMAIL PROTECTED]>
Cc: Christopher Warrington <[EMAIL PROTECTED]>
Subject: [sup-talk] [PATCH] fixed off-by-one error in imap.rb and maildir.rb
Precedence: list
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Sender: [EMAIL PROTECTED]
Errors-To: [EMAIL PROTECTED]

The end_offset reported by imap and maildir sources was incorrect if there was
only one message in the source. Since end_offset is exclusive, we must add one 
to
the last known id to get include all valid message ids in the range.
---
 lib/sup/imap.rb    |    2 +-
 lib/sup/maildir.rb |    2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/lib/sup/imap.rb b/lib/sup/imap.rb
index e785597..1d36976 100644
--- a/lib/sup/imap.rb
+++ b/lib/sup/imap.rb
@@ -176,7 +176,7 @@ class IMAP < Source
 
   def end_offset
     unsynchronized_scan_mailbox
-    @ids.last
+    @ids.last + 1
   end
   synchronized :end_offset
 
diff --git a/lib/sup/maildir.rb b/lib/sup/maildir.rb
index 5c9600d..584e657 100644
--- a/lib/sup/maildir.rb
+++ b/lib/sup/maildir.rb
@@ -123,7 +123,7 @@ class Maildir < Source
 
   def end_offset
     scan_mailbox :rescan => true
-    @ids.last
+    @ids.last + 1
   end
 
   def pct_done; 100.0 * (@ids.index(cur_offset) || 0).to_f / (@ids.length - 
1).to_f; end
-- 
1.5.3.7

_______________________________________________
sup-talk mailing list
[email protected]
http://rubyforge.org/mailman/listinfo/sup-talk

--- RuntimeError from thread: load threads for thread-index-mode
wrong id called on nil
/usr/local/lib/ruby/gems/1.8/gems/sup-999/lib/sup.rb:13:in `id'
/usr/local/lib/ruby/gems/1.8/gems/sup-999/lib/sup/modes/thread-index-mode.rb:208:in
 `update'
/usr/local/lib/ruby/gems/1.8/gems/sup-999/lib/sup/hook.rb:123:in `sort_by'
/usr/local/lib/ruby/gems/1.8/gems/sup-999/lib/sup/modes/thread-index-mode.rb:208:in
 `each'
/usr/local/lib/ruby/gems/1.8/gems/sup-999/lib/sup/modes/thread-index-mode.rb:208:in
 `sort_by'
/usr/local/lib/ruby/gems/1.8/gems/sup-999/lib/sup/modes/thread-index-mode.rb:208:in
 `update'
/usr/local/lib/ruby/gems/1.8/gems/sup-999/lib/sup/modes/thread-index-mode.rb:206:in
 `synchronize'
/usr/local/lib/ruby/gems/1.8/gems/sup-999/lib/sup/modes/thread-index-mode.rb:206:in
 `update'
/usr/local/lib/ruby/gems/1.8/gems/sup-999/lib/sup/modes/thread-index-mode.rb:505:in
 `__unprotected_load_n_threads'
(eval):12:in `load_n_threads'
/usr/local/lib/ruby/gems/1.8/gems/sup-999/lib/sup/modes/thread-index-mode.rb:478:in
 `load_n_threads_background'
/usr/local/lib/ruby/gems/1.8/gems/sup-999/lib/sup.rb:60:in `reporting_thread'
/usr/local/lib/ruby/gems/1.8/gems/sup-999/lib/sup.rb:58:in `initialize'
/usr/local/lib/ruby/gems/1.8/gems/sup-999/lib/sup.rb:58:in `new'
/usr/local/lib/ruby/gems/1.8/gems/sup-999/lib/sup.rb:58:in `reporting_thread'
/usr/local/lib/ruby/gems/1.8/gems/sup-999/lib/sup/modes/thread-index-mode.rb:477:in
 `load_n_threads_background'
/usr/local/lib/ruby/gems/1.8/gems/sup-999/lib/sup/modes/thread-index-mode.rb:547:in
 `__unprotected_load_threads'
(eval):12:in `load_threads'
/usr/local/lib/ruby/gems/1.8/gems/sup-999/bin/sup:200
/usr/local/bin/sup:19:in `load'
/usr/local/bin/sup:19
--- SystemExit from thread: main
wrong id called on nil
/usr/local/lib/ruby/gems/1.8/gems/sup-999/lib/sup.rb:64:in `select'
/usr/local/lib/ruby/gems/1.8/gems/sup-999/lib/sup/buffer.rb:31:in 
`nonblocking_getch'
/usr/local/lib/ruby/gems/1.8/gems/sup-999/bin/sup:217
/usr/local/bin/sup:19:in `load'
/usr/local/bin/sup:19

[EMAIL PROTECTED]:~ $> sup-config
[Tue Mar 11 08:48:12 +1000 2008] using character set encoding "UTF-8"
[Tue Mar 11 08:48:12 +1000 2008] crypto: detected gpg binary in /usr/bin/gpg
Howdy neighbor! This here's sup-config, ready to help you jack in to
the next generation of digital cyberspace: the text-based email
program. Get ready to be the envy of everyone in your internets
with your amazing keyboarding skills! Jump from email to email with
nary a click of the mouse!

Just answer these simple questions and you'll be on your way.

What's your name? (enter for "ajbw mail"):
What's your (primary) email address? (enter for "[EMAIL PROTECTED]"):
Ok, your header will look like this:
  From: ajbw mail <[EMAIL PROTECTED]>

Do you have any alternate email addresses that also receive email?
If so, enter them now, separated by spaces.
Alternate email addresses:
What file contains your signature? (enter for "/home/ajbwmail/.signature"):
What editor would you like to use? (enter for "/usr/bin/vim -f -c 'setlocal 
spell spelllang=en_us' -c
'set filetype=mail'"):
Ok, I've saved you up a nice lil' /home/ajbwmail/.sup/config.yaml.

Now, we'll tell Sup where to find all your email.
Current sources:
  No sources!

1. Add a new source.
2. Done adding sources!
Your wish? 1
Ok, adding a new source.
1. mbox file
2. maildir directory
3. remote mbox file (accessible via ssh)
4. IMAP server (secure)
5. IMAP server (unsecure)
6. Get me out of here!
What type of mail source is it? 2
Ok, now for the details.
What's the full path to the maildir directory? (enter for 
"/var/spool/mail/ajbwmail"): /home/ajbwmail/test/sup-talk
I'm going to add this source: maildir:/home/ajbwmail/test/sup-talk
Does that look right? (enter for "y"):
Does this source ever receive new messages? (enter for "y"):
Should new messages be automatically archived? (I.e. not appear in your inbox, 
though still be
accessible via search.) (enter for "n"):
Enter any labels to be automatically added to all messages from this source, 
separated by spaces (or
'none'):
Ok, trying to run "/usr/local/bin/sup-add 
maildir:/home/ajbwmail/test/sup-talk"...
[Tue Mar 11 08:48:31 +1000 2008] using character set encoding "UTF-8"
[Tue Mar 11 08:48:32 +1000 2008] crypto: detected gpg binary in /usr/bin/gpg
[Tue Mar 11 08:48:32 +1000 2008] locking /home/ajbwmail/.sup/lock...
Adding maildir:/home/ajbwmail/test/sup-talk...
[Tue Mar 11 08:48:32 +1000 2008] saving index and sources...
[Tue Mar 11 08:48:32 +1000 2008] unlocking /home/ajbwmail/.sup/lock...
Great! Added!

Now, we'll tell Sup where to find all your email.
Current sources:
* maildir:/home/ajbwmail/test/sup-talk

1. Add a new source.
2. Done adding sources!
Your wish? 2

Ok. The final step is to import all your messages into the Sup index.
Depending on how many messages are in the sources, this could take
quite a while.

Run sup-sync to import all messages now? (enter for "n"): y
Ok, trying to run "/usr/local/bin/sup-sync --all-sources"...
[Tue Mar 11 08:48:37 +1000 2008] using character set encoding "UTF-8"
[Tue Mar 11 08:48:38 +1000 2008] crypto: detected gpg binary in /usr/bin/gpg
[Tue Mar 11 08:48:38 +1000 2008] locking /home/ajbwmail/.sup/lock...
[Tue Mar 11 08:48:38 +1000 2008] creating index...
Scanning maildir:/home/ajbwmail/test/sup-talk...
[Tue Mar 11 08:48:38 +1000 2008] scanning maildir...
[Tue Mar 11 08:48:38 +1000 2008] done scanning maildir
Scanned 2, added 2, updated 0 messages from 
maildir:/home/ajbwmail/test/sup-talk.
[Tue Mar 11 08:48:38 +1000 2008] saving index and sources...
[Tue Mar 11 08:48:38 +1000 2008] unlocking /home/ajbwmail/.sup/lock...
Great! It worked!
[Tue Mar 11 08:48:38 +1000 2008] loading index...
[Tue Mar 11 08:48:38 +1000 2008] loaded index of 2 messages

Okee doke, you've got yourself an index of 2 messages. Looks
like you're ready to jack in to cyberspace there, cowboy.

Just one last command:

  /usr/local/bin/sup

Have fun!
[EMAIL PROTECTED]:~ $> vim .sup/config.yaml
[EMAIL PROTECTED]:~ $> grep thread_by !$
grep thread_by .sup/config.yaml
:thread_by_subject: false
[EMAIL PROTECTED]:~ $> sup
[Tue Mar 11 08:49:08 +1000 2008] using character set encoding "UTF-8"
[Tue Mar 11 08:49:08 +1000 2008] locking /home/ajbwmail/.sup/lock...
[Tue Mar 11 08:49:08 +1000 2008] crypto: detected gpg binary in /usr/bin/gpg
[Tue Mar 11 08:49:08 +1000 2008] loading index...
[Tue Mar 11 08:49:08 +1000 2008] loaded index of 2 messages
[Tue Mar 11 08:49:08 +1000 2008] no draft source, auto-adding...
[Tue Mar 11 08:49:08 +1000 2008] no sent mail source, auto-adding...
[Tue Mar 11 08:49:08 +1000 2008] starting curses
[Tue Mar 11 08:49:10 +1000 2008] stopped cursing
[Tue Mar 11 08:49:10 +1000 2008] no fatal errors. good job, william.
[Tue Mar 11 08:49:10 +1000 2008] saving index and sources...
[Tue Mar 11 08:49:10 +1000 2008] unlocking /home/ajbwmail/.sup/lock...
[EMAIL PROTECTED]:~ $> vim .sup/config.yaml
[EMAIL PROTECTED]:~ $> grep thread_by .sup/config.yaml
:thread_by_subject: true
[EMAIL PROTECTED]:~ $> sup
[Tue Mar 11 08:49:26 +1000 2008] using character set encoding "UTF-8"
[Tue Mar 11 08:49:26 +1000 2008] locking /home/ajbwmail/.sup/lock...
[Tue Mar 11 08:49:26 +1000 2008] crypto: detected gpg binary in /usr/bin/gpg
[Tue Mar 11 08:49:26 +1000 2008] loading index...
[Tue Mar 11 08:49:26 +1000 2008] loaded index of 2 messages
[Tue Mar 11 08:49:26 +1000 2008] starting curses
[Tue Mar 11 08:49:28 +1000 2008] stopped cursing
[Tue Mar 11 08:49:28 +1000 2008] oh crap, an exception
[Tue Mar 11 08:49:28 +1000 2008] unlocking /home/ajbwmail/.sup/lock...
----------------------------------------------------------------
I'm very sorry. It seems that an error occurred in Sup. Please
accept my sincere apologies. If you don't mind, please send the
contents of sup-exception-log.txt and a brief report of the
circumstances to sup-talk at rubyforge dot orgs so that I might
address this problem. Thank you!

Sincerely,
William
----------------------------------------------------------------
--- RuntimeError from thread: load threads for thread-index-mode
wrong id called on nil
/usr/local/lib/ruby/gems/1.8/gems/sup-999/lib/sup.rb:13:in `id'
/usr/local/lib/ruby/gems/1.8/gems/sup-999/lib/sup/modes/thread-index-mode.rb:208:in
 `update'
/usr/local/lib/ruby/gems/1.8/gems/sup-999/lib/sup/hook.rb:123:in `sort_by'
/usr/local/lib/ruby/gems/1.8/gems/sup-999/lib/sup/modes/thread-index-mode.rb:208:in
 `each'
/usr/local/lib/ruby/gems/1.8/gems/sup-999/lib/sup/modes/thread-index-mode.rb:208:in
 `sort_by'
/usr/local/lib/ruby/gems/1.8/gems/sup-999/lib/sup/modes/thread-index-mode.rb:208:in
 `update'
/usr/local/lib/ruby/gems/1.8/gems/sup-999/lib/sup/modes/thread-index-mode.rb:206:in
 `synchronize'
/usr/local/lib/ruby/gems/1.8/gems/sup-999/lib/sup/modes/thread-index-mode.rb:206:in
 `update'
/usr/local/lib/ruby/gems/1.8/gems/sup-999/lib/sup/modes/thread-index-mode.rb:505:in
 `__unprotected_load_n_threads'
(eval):12:in `load_n_threads'
/usr/local/lib/ruby/gems/1.8/gems/sup-999/lib/sup/modes/thread-index-mode.rb:478:in
 `load_n_threads_background'
/usr/local/lib/ruby/gems/1.8/gems/sup-999/lib/sup.rb:60:in `reporting_thread'
/usr/local/lib/ruby/gems/1.8/gems/sup-999/lib/sup.rb:58:in `initialize'
/usr/local/lib/ruby/gems/1.8/gems/sup-999/lib/sup.rb:58:in `new'
/usr/local/lib/ruby/gems/1.8/gems/sup-999/lib/sup.rb:58:in `reporting_thread'
/usr/local/lib/ruby/gems/1.8/gems/sup-999/lib/sup/modes/thread-index-mode.rb:477:in
 `load_n_threads_background'
/usr/local/lib/ruby/gems/1.8/gems/sup-999/lib/sup/modes/thread-index-mode.rb:547:in
 `__unprotected_load_threads'
(eval):12:in `load_threads'
/usr/local/lib/ruby/gems/1.8/gems/sup-999/bin/sup:200
/usr/local/bin/sup:19:in `load'
/usr/local/bin/sup:19
--- SystemExit from thread: main
wrong id called on nil
/usr/local/lib/ruby/gems/1.8/gems/sup-999/lib/sup.rb:64:in `select'
/usr/local/lib/ruby/gems/1.8/gems/sup-999/lib/sup/buffer.rb:31:in 
`nonblocking_getch'
/usr/local/lib/ruby/gems/1.8/gems/sup-999/bin/sup:217
/usr/local/bin/sup:19:in `load'
/usr/local/bin/sup:19
[EMAIL PROTECTED]:~ $>

_______________________________________________
sup-talk mailing list
[email protected]
http://rubyforge.org/mailman/listinfo/sup-talk

[sup-talk] [Crash] "thread_by_subject: true" leads to "wrong id called on nil"

Reply via email to