Re: [PATCH 14/25] Fix old style notmuch-fcc-dirs configuration check.

2011-06-02 Thread Dmitry Kurochkin
Hi Carl.

On Wed, 01 Jun 2011 22:10:07 -0700, Carl Worth cwo...@cworth.org wrote:
 On Sat, 28 May 2011 14:51:49 -0700, Jameson Graef Rollins 
 jroll...@finestructure.net wrote:
 Hi Jamie,
 
 I've pushed the next few patches up to this point, (with only one
 functional change---I fixed a new test case to correctly use
 notmuch_search_sanitize to avoid spurious failures unmatching thread ID
 values).
 
 This patch, however, isn't ready. The big problem is in this commit
 message:
 
  From: Dmitry Kurochkin dmitry.kuroch...@gmail.com
  
  In notmuch 0.5 notmuch-fcc-dirs style changed.  The previous code
  did not correctly identify an old configuration and, as a
  consequence, broke new configurations.
 
 There are several things there that are too vague, (previous code,
 old configuration, new configurations).

Well, it says that changes are in notmuch 0.5.  So old and previous
refer to pre-0.5 (i.e. 0.4) and new refers to 0.5.

 What kind of configuration
 is broken?

Any configuration when `notmuch-fcc-dirs' is a list.  That variable has
a nice documentation.

 How does the change here help?
 

It fixes detection of old-style pre-0.5 setting for `notmuch-fcc-dirs'.

 It would be easier to understand the code if there were a corresponding
 test case for it. I'd even be willing to help write a test case, but
 there are not enough specifics in that commit message to even tell me
 what to test.
 

I do not think we need a test for this fix.  What we need are tests for
FCC functionality when notmuch-fcc-dirs is a list.

((and (listp notmuch-fcc-dirs)
  -   (= 1 (length (car notmuch-fcc-dirs
  +   (stringp (car notmuch-fcc-dirs)))
 
 So the old configuration was a single string?

No, it is a list with a string as the first element.  May I refer you to
git show 0.4:emacs/notmuch-maildir-fcc.el and current
emacs/notmuch-maildir-fcc.el?  It has good documentation with examples
for notmuch-fcc-dirs.  Also NEWS for 0.5 describe this change.

 And this has been
 inadvertently broken since 0.5?
 

Old configuration format was changed in 0.5 in an incompatible way.
There is a check for the unsupported old-style configuration.  But the
check is broken and results in an error when running with a valid
new-style configuration.

 ;; Old style - no longer works.
 (error Invalid `notmuch-fcc-dirs' setting (old style)))
 
 Yikes. That vague phrasing (old style) is already in the code in error
 messages as well.
 

Yes.

 Dmitry, can you help me know what's going on here?

I hope above helps.

 (Preferably by
 sending a newer commit with a more thorough commit message.)
 

I am not sure what you expect from the commit message here.  IMO it is
enough for this small bugfix and those who interested can always refer
to documentation for details.

Regards,
  Dmitry

 Thanks,
 
 -Carl
Non-text part: application/pgp-signature
 ___
 notmuch mailing list
 notmuch@notmuchmail.org
 http://notmuchmail.org/mailman/listinfo/notmuch
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Re: [python] get all messages of a thread

2011-06-02 Thread Sebastian Spaeth
On Wed, 1 Jun 2011 15:35:35 +1000, Brian May wrote:
 Oh, I see, for your code, there is a implied call to __len__, and the
 __len__ function is completely broken for the reasons described in the
 documentation:

It seems to have been a bad idea to implement __len__ at all for the
Messsages() construct in the python bindings, and I wonder if I should
remove it.

On the other hand, it seems that list(Messages()) implicitely calls
len(), or so it seems from the error that we get when trying to list() a
messages object.

An alternative is to implement len() as a call to count_messages() which
has for me so far always returned the correct number of messages without
using up the iterator. However, the xapian docs explicitely state that
it does not guarantee that the count will be correct, so len() might
return a wrong message size (potentially).

What would be the best way to solve this (besides fixing the C api to
allow to reset the iterator ;-) ?)

I could implement a custom .as_list() function that returns the
Messages() object as a list that is guaranteed to be stable, by copying
out the Message() objects into a list.

Sebastian


pgpw0PXWEka8y.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


python: get_filenames()

2011-06-02 Thread Sebastian Spaeth
A head up for the distributed gits and python users:
I just pushed a commit to cworth/master implementing
Message().get_filenames(). This will return a generator of absolute
filenames for a certain Message-ID.

For further information I refer to
http://git.notmuchmail.org/git/notmuch/commit/b31247c354b54a3cbeb1c7f9df830e16f7c921d9
and the code (and it's documentation).

Let me know if it works for you.

Sebastian


pgpHuBF5jCwjv.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


notmuch crashing on misformated emails

2011-06-02 Thread Sebastian Spaeth
in python:
  list(Threads())

leads to a program abortion (bombing out of python):
Internal error: Failed to read timestamp value from document. 
(lib/message.cc:731).

Can we not crash the app on misformated emails?

Sebastian


pgpj9XsvsvjQZ.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Re: [python] get all messages of a thread

2011-06-02 Thread Brian May
On 2 June 2011 17:05, Sebastian Spaeth sebast...@sspaeth.de wrote:

 What would be the best way to solve this (besides fixing the C api to
 allow to reset the iterator ;-) ?)


 I am not really familiar with the code. So am I correct in making the
following assumptions?

* It is not easy to fix the C api to reset the iterator (what about
repeating the search?)

* The only accurate way to get the number of messages is to iterate through
every search result and count them?

If so, then len(...) I think might be very slow if there are a large number
of elements.

Maybe it might be easier/better to implement object.__nonzero__(self)
 instead of the object.__len__(self) method?

http://docs.python.org/reference/datamodel.html

object.__nonzero__(self)
Called to implement truth value testing and the built-in operation bool();
should return False or True, or their integer equivalents 0 or 1. When this
method is not defined, __len__() is called, if it is defined, and the object
is considered true if its result is nonzero. If a class defines neither
__len__() nor __nonzero__(), all its instances are considered true.

object.__len__(self)
Called to implement the built-in function len(). Should return the length of
the object, an integer = 0. Also, an object that doesn’t define a
__nonzero__() method and whose __len__() method returns zero is considered
to be false in a Boolean context.

-- 
Brian May br...@microcomaustralia.com.au
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


[PATCH 0/4] set test prereqs (Emacs, GDB, GPG) v3

2011-06-02 Thread Pieter Praet
Rebased the set test prereqs (Emacs, GDB, GPG) [1] patch series
to Jameson's release-candidate/0.6 branch (0c0b4172).

The GDB dependency for the atomicity tests was dropped since
Austin has already included this in his amdragon/atomic-new-v4
branch [2], which has been included in release-candidate/0.6.

Also, some crypto tests which depend not only on GPG but Emacs as
well have been updated accordingly.

Jameson, could this be merged in? It helps prevent false
negatives in the absence of Emacs/GnuPG/GDB.


[1] id:1305274636-19975-1-git-send-email-pie...@praet.org
[2] id:banlktikehvhroyjnrn9rlwbnz_dbolf...@mail.gmail.com

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


[PATCH 1/4] test: add 'GnuPG' prereq to dependent 'crypto' tests

2011-06-02 Thread Pieter Praet
Adds a new test that checks for the presence of 'gpg',
and adds that test as a prereq to all subsequent tests
that rely on GnuPG.

This causes tests with unmet dependencies to be skipped.

Signed-off-by: Pieter Praet pie...@praet.org
---
 test/crypto |   33 +++--
 1 files changed, 19 insertions(+), 14 deletions(-)

diff --git a/test/crypto b/test/crypto
index 8e92016..59f7989 100755
--- a/test/crypto
+++ b/test/crypto
@@ -7,6 +7,11 @@
 test_description='PGP/MIME signature verification and decryption'
 . ./test-lib.sh
 
+# GnuPG is a prereq.
+test_expect_success prereq: GnuPG is present which gpg \
+ test_set_prereq GPG
+
+
 add_gnupg_home ()
 {
 local output
@@ -31,7 +36,7 @@ FINGERPRINT=$(gpg --no-tty --list-secret-keys --with-colons 
--fingerprint | grep
 # although I can't figure out why
 add_email_corpus
 
-test_expect_success 'emacs delivery of signed message' \
+test_expect_success GPG 'emacs delivery of signed message' \
 'emacs_deliver_message \
 test signed message 001 \
 This is a test signed message. \
@@ -64,7 +69,7 @@ expected='[[[{id: X,
  {id: 3,
  content-type: application/pgp-signature}]}]},
  ['
-test_expect_equal \
+test_expect_equal GPG \
 $output \
 $expected
 
@@ -99,7 +104,7 @@ expected='[[[{id: X,
  {id: 3,
  content-type: application/pgp-signature}]}]},
  ['
-test_expect_equal \
+test_expect_equal GPG \
 $output \
 $expected
 
@@ -132,7 +137,7 @@ expected='[[[{id: X,
  {id: 3,
  content-type: application/pgp-signature}]}]},
  ['
-test_expect_equal \
+test_expect_equal GPG \
 $output \
 $expected
 mv ${GNUPGHOME}{.bak,}
@@ -141,7 +146,7 @@ mv ${GNUPGHOME}{.bak,}
 cat EOF TESTATTACHMENT
 This is a test file.
 EOF
-test_expect_success 'emacs delivery of encrypted message with attachment' \
+test_expect_success GPG 'emacs delivery of encrypted message with attachment' \
 'emacs_deliver_message \
 test encrypted message 001 \
 This is a test encrypted message.\n \
@@ -176,7 +181,7 @@ Non-text part: application/octet-stream
 part}
 body}
 message}'
-test_expect_equal \
+test_expect_equal GPG \
 $output \
 $expected
 
@@ -211,7 +216,7 @@ expected='[[[{id: X,
  content-type: application/octet-stream,
  filename: TESTATTACHMENT}]}]}]},
  ['
-test_expect_equal \
+test_expect_equal GPG \
 $output \
 $expected
 
@@ -222,7 +227,7 @@ output=$(notmuch show --format=json --part=4 --decrypt 
subject:test encrypted m
 expected='{id: 4,
  content-type: text/plain,
  content: This is a test encrypted message.\n}'
-test_expect_equal \
+test_expect_equal GPG \
 $output \
 $expected
 
@@ -232,7 +237,7 @@ notmuch show \
 --part=5 \
 --decrypt \
 subject:test encrypted message 001 OUTPUT
-test_expect_equal_file OUTPUT TESTATTACHMENT
+test_expect_equal_file GPG OUTPUT TESTATTACHMENT
 
 test_begin_subtest decryption failure with missing key
 mv ${GNUPGHOME}{,.bak}
@@ -259,12 +264,12 @@ expected='[[[{id: X,
  {id: 3,
  content-type: application/octet-stream}]}]},
  ['
-test_expect_equal \
+test_expect_equal GPG \
 $output \
 $expected
 mv ${GNUPGHOME}{.bak,}
 
-test_expect_success 'emacs delivery of encrypted + signed message' \
+test_expect_success GPG 'emacs delivery of encrypted + signed message' \
 'emacs_deliver_message \
 test encrypted message 002 \
 This is another test encrypted message.\n \
@@ -299,7 +304,7 @@ expected='[[[{id: X,
  content-type: text/plain,
  content: This is another test encrypted message.\n}]}]},
  ['
-test_expect_equal \
+test_expect_equal GPG \
 $output \
 $expected
 
@@ -313,7 +318,7 @@ On 01 Jan 2000 12:00:00 -, Notmuch Test Suite 
test_su...@notmuchmail.org w
 Non-text part: multipart/encrypted
 Non-text part: application/pgp-encrypted
  This is another test encrypted message.'
-test_expect_equal \
+test_expect_equal GPG \
 $output \
 $expected
 
@@ -354,7 +359,7 @@ expected='[[[{id: X,
  {id: 3,
  content-type: application/pgp-signature}]}]},
  ['
-test_expect_equal \
+test_expect_equal GPG \
 $output \
 $expected
 
-- 
1.7.4.1

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


[PATCH 2/4] test: add 'Emacs' prereq to dependent 'crypto' tests

2011-06-02 Thread Pieter Praet
Adds a new test that checks for the presence of 'emacs',
and adds that test as a prereq to all subsequent tests
that rely on Emacs.

This causes tests with unmet dependencies to be skipped.

Signed-off-by: Pieter Praet pie...@praet.org
---
 test/crypto |   17 ++---
 1 files changed, 14 insertions(+), 3 deletions(-)

diff --git a/test/crypto b/test/crypto
index 59f7989..c69307d 100755
--- a/test/crypto
+++ b/test/crypto
@@ -7,10 +7,21 @@
 test_description='PGP/MIME signature verification and decryption'
 . ./test-lib.sh
 
+# Emacs is a prereq.
+test_expect_success prereq: Emacs is present which emacs \
+ test_set_prereq EMACS
+
 # GnuPG is a prereq.
 test_expect_success prereq: GnuPG is present which gpg \
  test_set_prereq GPG
 
+# Some tests have multiple prereqs, but the test_expect_* functions
+# accept only a single argument as prereq tag, and using test_have_prereq
+# in and around tests causes various errors for me, so a dirty workaround
+# will have to do for the time being.
+test_have_prereq EMACS  test_have_prereq GPG \
+ test_set_prereq EMACS+GPG
+
 
 add_gnupg_home ()
 {
@@ -36,7 +47,7 @@ FINGERPRINT=$(gpg --no-tty --list-secret-keys --with-colons 
--fingerprint | grep
 # although I can't figure out why
 add_email_corpus
 
-test_expect_success GPG 'emacs delivery of signed message' \
+test_expect_success EMACS+GPG 'emacs delivery of signed message' \
 'emacs_deliver_message \
 test signed message 001 \
 This is a test signed message. \
@@ -146,7 +157,7 @@ mv ${GNUPGHOME}{.bak,}
 cat EOF TESTATTACHMENT
 This is a test file.
 EOF
-test_expect_success GPG 'emacs delivery of encrypted message with attachment' \
+test_expect_success EMACS+GPG 'emacs delivery of encrypted message with 
attachment' \
 'emacs_deliver_message \
 test encrypted message 001 \
 This is a test encrypted message.\n \
@@ -269,7 +280,7 @@ test_expect_equal GPG \
 $expected
 mv ${GNUPGHOME}{.bak,}
 
-test_expect_success GPG 'emacs delivery of encrypted + signed message' \
+test_expect_success EMACS+GPG 'emacs delivery of encrypted + signed message' \
 'emacs_deliver_message \
 test encrypted message 002 \
 This is another test encrypted message.\n \
-- 
1.7.4.1

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


[PATCH 3/4] test: add 'Emacs' prereq to dependent 'emacs' tests

2011-06-02 Thread Pieter Praet
Adds a new test that checks for the presence of 'emacs',
and adds that test as a prereq to all subsequent tests
that rely on Emacs.

This causes tests with unmet dependencies to be skipped.

Signed-off-by: Pieter Praet pie...@praet.org
---
 test/emacs |   43 ---
 1 files changed, 24 insertions(+), 19 deletions(-)

diff --git a/test/emacs b/test/emacs
index da8ba68..10138a6 100755
--- a/test/emacs
+++ b/test/emacs
@@ -2,65 +2,70 @@
 test_description=emacs interface
 . test-lib.sh
 
+# Emacs is a prereq.
+test_expect_success prereq: Emacs is present which emacs \
+ test_set_prereq EMACS
+
+
 EXPECTED=../emacs.expected-output
 
 add_email_corpus
 
 test_begin_subtest Basic notmuch-hello view in emacs
 test_emacs '(notmuch-hello) (princ (buffer-string))' OUTPUT
-test_expect_equal_file OUTPUT $EXPECTED/notmuch-hello
+test_expect_equal_file EMACS OUTPUT $EXPECTED/notmuch-hello
 
 test_begin_subtest Saved search with 0 results
 test_emacs '(setq notmuch-show-empty-saved-searches t) (setq 
notmuch-saved-searches '\''((inbox . tag:inbox) (unread . tag:unread) 
(empty . tag:doesnotexist))) (notmuch-hello) (princ (buffer-string))' 
OUTPUT
-test_expect_equal_file OUTPUT $EXPECTED/notmuch-hello-with-empty
+test_expect_equal_file EMACS OUTPUT $EXPECTED/notmuch-hello-with-empty
 
 test_begin_subtest No saved searches displayed (all with 0 results)
 test_emacs '(setq notmuch-saved-searches '\''((empty . tag:doesnotexist))) 
(notmuch-hello) (princ (buffer-string))' OUTPUT
-test_expect_equal_file OUTPUT $EXPECTED/notmuch-hello-no-saved-searches
+test_expect_equal_file EMACS OUTPUT $EXPECTED/notmuch-hello-no-saved-searches
 
 test_begin_subtest Basic notmuch-search view in emacs
 test_emacs '(notmuch-search tag:inbox) (notmuch-test-wait) (princ 
(buffer-string))' OUTPUT
-test_expect_equal_file OUTPUT $EXPECTED/notmuch-search-tag-inbox
+test_expect_equal_file EMACS OUTPUT $EXPECTED/notmuch-search-tag-inbox
 
 test_begin_subtest Navigation of notmuch-hello to search results
 test_emacs '(notmuch-hello) (goto-char (point-min)) (re-search-forward 
inbox) (widget-button-press (point)) (notmuch-test-wait) (princ 
(buffer-string))' OUTPUT
-test_expect_equal_file OUTPUT $EXPECTED/notmuch-hello-view-inbox
+test_expect_equal_file EMACS OUTPUT $EXPECTED/notmuch-hello-view-inbox
 
 test_begin_subtest Basic notmuch-show view in emacs
 maildir_storage_thread=$(notmuch search --output=threads 
id:20091117190054.gu3...@dottiness.seas.harvard.edu)
 test_emacs (notmuch-show \$maildir_storage_thread\) (princ 
(buffer-string)) OUTPUT
-test_expect_equal_file OUTPUT $EXPECTED/notmuch-show-thread-maildir-storage
+test_expect_equal_file EMACS OUTPUT 
$EXPECTED/notmuch-show-thread-maildir-storage
 
 test_begin_subtest Navigation of notmuch-search to thread view
 test_emacs '(notmuch-search tag:inbox) (notmuch-test-wait) (goto-char 
(point-min)) (re-search-forward Working with Maildir) 
(notmuch-search-show-thread) (notmuch-test-wait) (princ (buffer-string))' 
OUTPUT
-test_expect_equal_file OUTPUT $EXPECTED/notmuch-show-thread-maildir-storage
+test_expect_equal_file EMACS OUTPUT 
$EXPECTED/notmuch-show-thread-maildir-storage
 
 test_begin_subtest Add tag from search view
 os_x_darwin_thread=$(notmuch search --output=threads 
id:ddd65cda0911171950o4eea4389v86de9525e4605...@mail.gmail.com)
 test_emacs (notmuch-search \$os_x_darwin_thread\) (notmuch-test-wait) 
(notmuch-search-add-tag \tag-from-search-view\)
 output=$(notmuch search $os_x_darwin_thread | notmuch_search_sanitize)
-test_expect_equal $output thread:XXX   2009-11-18 [4/4] Jjgod Jiang, 
Alexander Botero-Lowry; [notmuch] Mac OS X/Darwin compatibility issues (inbox 
tag-from-search-view unread)
+test_expect_equal EMACS $output thread:XXX   2009-11-18 [4/4] Jjgod Jiang, 
Alexander Botero-Lowry; [notmuch] Mac OS X/Darwin compatibility issues (inbox 
tag-from-search-view unread)
 
 test_begin_subtest Remove tag from search view
 test_emacs (notmuch-search \$os_x_darwin_thread\) (notmuch-test-wait) 
(notmuch-search-remove-tag \tag-from-search-view\)
 output=$(notmuch search $os_x_darwin_thread | notmuch_search_sanitize)
-test_expect_equal $output thread:XXX   2009-11-18 [4/4] Jjgod Jiang, 
Alexander Botero-Lowry; [notmuch] Mac OS X/Darwin compatibility issues (inbox 
unread)
+test_expect_equal EMACS $output thread:XXX   2009-11-18 [4/4] Jjgod Jiang, 
Alexander Botero-Lowry; [notmuch] Mac OS X/Darwin compatibility issues (inbox 
unread)
 
 test_begin_subtest Add tag from notmuch-show view
 test_emacs (notmuch-show \$os_x_darwin_thread\) (notmuch-show-add-tag 
\tag-from-show-view\)
 output=$(notmuch search $os_x_darwin_thread | notmuch_search_sanitize)
-test_expect_equal $output thread:XXX   2009-11-18 [4/4] Jjgod Jiang, 
Alexander Botero-Lowry; [notmuch] Mac OS X/Darwin compatibility issues (inbox 
tag-from-show-view unread)
+test_expect_equal EMACS $output thread:XXX   2009-11-18 [4/4] Jjgod Jiang, 
Alexander Botero-Lowry; [notmuch] Mac OS X/Darwin 

Re: [PATCH 0/4] set test prereqs (Emacs, GDB, GPG) v3

2011-06-02 Thread Austin Clements
This looks good to me, and should be merged sooner rather than later
because it touches a lot of lines.

If atomic-new-v4 doesn't happen to get merged before this, then
id:1305206080-17461-1-git-send-email-amdra...@mit.edu and
id:1305206110-17511-1-git-send-email-amdra...@mit.edu should get
merged, since they fix test-lib problems with dependencies (they're
included in the atomic-new-v4 branch because that uses dependencies,
but aren't related to atomicity).

Quoth Pieter Praet on Jun 02 at  2:03 pm:
 Rebased the set test prereqs (Emacs, GDB, GPG) [1] patch series
 to Jameson's release-candidate/0.6 branch (0c0b4172).
 
 The GDB dependency for the atomicity tests was dropped since
 Austin has already included this in his amdragon/atomic-new-v4
 branch [2], which has been included in release-candidate/0.6.
 
 Also, some crypto tests which depend not only on GPG but Emacs as
 well have been updated accordingly.
 
 Jameson, could this be merged in? It helps prevent false
 negatives in the absence of Emacs/GnuPG/GDB.
 
 
 [1] id:1305274636-19975-1-git-send-email-pie...@praet.org
 [2] id:banlktikehvhroyjnrn9rlwbnz_dbolf...@mail.gmail.com
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Re: [python] get all messages of a thread

2011-06-02 Thread Sebastian Spaeth
On Thu, 2 Jun 2011 19:43:29 +1000, Brian May wrote:
 On 2 June 2011 17:05, Sebastian Spaeth sebast...@sspaeth.de wrote:
 
  What would be the best way to solve this (besides fixing the C api to
  allow to reset the iterator ;-) ?)

 * It is not easy to fix the C api to reset the iterator (what about
 repeating the search?)

I am not sure about the difficulty of that, I am not a C-kind of
guy. Repeating the search would be easy but potentially gives you
different results since the db could have changed since then.
 
 * The only accurate way to get the number of messages is to iterate through
 every search result and count them?

There is count_messages() which wraps notmuch_query_count_messages which
invokes some very quick xapian function. But the xapian docs explicitely
state that it's xapians best guess and not guaranteed to be the real
number of messages (although it always was in my attempts). And you
don't want len() to return an approximation of the iterator length

 If so, then len(...) I think might be very slow if there are a large number
 of elements.
 
 Maybe it might be easier/better to implement object.__nonzero__(self)
  instead of the object.__len__(self) method?
 
 http://docs.python.org/reference/datamodel.html
 
 object.__nonzero__(self)
 Called to implement truth value testing and the built-in operation bool();
 should return False or True, or their integer equivalents 0 or 1. When this
 method is not defined, __len__() is called, if it is defined, and the object
 is considered true if its result is nonzero. If a class defines neither
 __len__() nor __nonzero__(), all its instances are considered true.

Interesting, did not know about this one. I guess that would solve the
example with the:

t = query.search_threads()

if t:
  for thread in t:
print thread

Actually the if t: is no longer needed, I just tried a query returning
no Threads and 

for thread in threads:
  pass

works just fine with an empty Threads() object.

I made the change, and implemented __nonzero__ and removed the len()
method. It just doesn't make sense on 1-time iterators. (I documented
the change in the API docs). Sorry if this breaks existing code.

list(Threads()) works just fine too, it just took a while to create a
list of 13k Thread() objects on this laptop. (and list() will of course
not return until it is finished).


Sebastian


pgp2HrsCIo0F1.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Re: [python] get all messages of a thread

2011-06-02 Thread Sebastian Spaeth
On Thu, 02 Jun 2011 16:20:14 +0200, Sebastian Spaeth wrote:

 I made the change, and implemented __nonzero__ and removed the len()
 method. It just doesn't make sense on 1-time iterators. (I documented
 the change in the API docs). Sorry if this breaks existing code.

FYI

OK, I just pushed a change that adds the __nonzero__ function thus
making:

t=Threads()
if t:
  len(t)

work, but I did not pull the len() function out. The explicit if t
test is however not needed for

t = q.search_threads()
for thread in t:
  print thread

works just fine with empty results.

Sebastian


pgpr2siKpn6fF.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Re: [python] get all messages of a thread

2011-06-02 Thread Austin Clements
On Thu, Jun 2, 2011 at 10:20 AM, Sebastian Spaeth sebast...@sspaeth.de wrote:
 On Thu, 2 Jun 2011 19:43:29 +1000, Brian May wrote:
 On 2 June 2011 17:05, Sebastian Spaeth sebast...@sspaeth.de wrote:

  What would be the best way to solve this (besides fixing the C api to
  allow to reset the iterator ;-) ?)

 * It is not easy to fix the C api to reset the iterator (what about
 repeating the search?)

 I am not sure about the difficulty of that, I am not a C-kind of
 guy. Repeating the search would be easy but potentially gives you
 different results since the db could have changed since then.

Not too hard.  Here's an utterly untested patch that implements
iterator resetting for notmuch_messages_t iterators.  It *should* be
much more efficient than performing the query again, but if you use
it, I'd love to know if that's actually true.

This may not be useful if __len__ is gone, unless you really want to
turn Messages/Threads into iterators rather than generators (as I've
pointed out before, there is absolutely nothing unusual or un-Pythonic
about how Messages/Threads works right now [well, except for the
presence of __len__ in a generator, I suppose]).

diff --git a/lib/messages.c b/lib/messages.c
index 7bcd1ab..085691c 100644
--- a/lib/messages.c
+++ b/lib/messages.c
@@ -80,7 +80,8 @@ _notmuch_messages_create (notmuch_message_list_t *list)
return NULL;

 messages-is_of_list_type = TRUE;
-messages-iterator = list-head;
+messages-head = list-head;
+notmuch_messages_reset (messages);

 return messages;
 }
@@ -137,6 +138,15 @@ notmuch_messages_move_to_next (notmuch_messages_t
*messages)
 }

 void
+notmuch_messages_reset (notmuch_messages_t *messages)
+{
+if (! messages-is_of_list_type)
+   return _notmuch_mset_messages_reset (messages);
+
+messages-iterator = messages-head;
+}
+
+void
 notmuch_messages_destroy (notmuch_messages_t *messages)
 {
 talloc_free (messages);
diff --git a/lib/notmuch-private.h b/lib/notmuch-private.h
index 02e24ee..805d60c 100644
--- a/lib/notmuch-private.h
+++ b/lib/notmuch-private.h
@@ -413,6 +413,7 @@ typedef struct _notmuch_message_list {
  */
 struct visible _notmuch_messages {
 notmuch_bool_t is_of_list_type;
+notmuch_message_node_t *head;
 notmuch_message_node_t *iterator;
 };

@@ -441,6 +442,9 @@ _notmuch_mset_messages_get (notmuch_messages_t *messages);
 void
 _notmuch_mset_messages_move_to_next (notmuch_messages_t *messages);

+void
+_notmuch_mset_messages_reset (notmuch_messages_t *messages);
+
 notmuch_bool_t
 _notmuch_doc_id_set_contains (notmuch_doc_id_set_t *doc_ids,
   unsigned int doc_id);
diff --git a/lib/notmuch.h b/lib/notmuch.h
index 9cdcec0..044cfaa 100644
--- a/lib/notmuch.h
+++ b/lib/notmuch.h
@@ -734,6 +734,15 @@ notmuch_messages_get (notmuch_messages_t *messages);
 void
 notmuch_messages_move_to_next (notmuch_messages_t *messages);

+/* Reset the 'messages' iterator back to the first message.
+ *
+ * For iterators returned from notmuch_query_search_messages, this is
+ * both more efficient than performing the query a second time and
+ * guaranteed to result in the same messages as the first iteration.
+ */
+void
+notmuch_messages_reset (notmuch_messages_t *messages);
+
 /* Destroy a notmuch_messages_t object.
  *
  * It's not strictly necessary to call this function. All memory from
diff --git a/lib/query.cc b/lib/query.cc
index 6f02b04..1e75be0 100644
--- a/lib/query.cc
+++ b/lib/query.cc
@@ -32,6 +32,7 @@ struct _notmuch_query {
 typedef struct _notmuch_mset_messages {
 notmuch_messages_t base;
 notmuch_database_t *notmuch;
+Xapian::MSet mset;
 Xapian::MSetIterator iterator;
 Xapian::MSetIterator iterator_end;
 } notmuch_mset_messages_t;
@@ -128,6 +129,7 @@ notmuch_query_search_messages (notmuch_query_t *query)
messages-base.is_of_list_type = FALSE;
messages-base.iterator = NULL;
messages-notmuch = notmuch;
+   new (messages-mset) Xapian::MSet ();
new (messages-iterator) Xapian::MSetIterator ();
new (messages-iterator_end) Xapian::MSetIterator ();

@@ -181,8 +183,8 @@ notmuch_query_search_messages (notmuch_query_t *query)

mset = enquire.get_mset (0, notmuch-xapian_db-get_doccount ());

-   messages-iterator = mset.begin ();
-   messages-iterator_end = mset.end ();
+   messages-mset = mset;
+   _notmuch_mset_messages_reset (messages-base);

return messages-base;

@@ -257,6 +259,17 @@ _notmuch_mset_messages_move_to_next
(notmuch_messages_t *messages)
 mset_messages-iterator++;
 }

+void
+_notmuch_mset_messages_reset (notmuch_messages_t *messages)
+{
+notmuch_mset_messages_t *mset_messages;
+
+mset_messages = (notmuch_mset_messages_t *) messages;
+
+mset_messages-iterator = mset_messages-mset.begin ();
+mset_messages-iterator_end = mset_messages-mset.end ();
+}
+
 static notmuch_bool_t
 _notmuch_doc_id_set_init (void *ctx,
  

Re: notmuch crashing on misformated emails

2011-06-02 Thread Sebastian Spaeth
On Thu, 02 Jun 2011 09:16:33 +0200, Sebastian Spaeth sebast...@sspaeth.de 
wrote:
 in python:
   list(Threads())
 
 leads to a program abortion (bombing out of python):
 Internal error: Failed to read timestamp value from document. 
 (lib/message.cc:731).
 
 Can we not crash the app on misformated emails?

Just to follow up, I cannot reproduce this anymore. It works fine
now. Sorry for the noise.

Sebastian


pgpci9JcT3iXp.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Re: Decryption fails

2011-06-02 Thread Jameson Graef Rollins
On Tue, 31 May 2011 19:33:29 +0200, Felix Geller fgel...@gmail.com wrote:
 I get the following trace when using show --decrypt to decrypt a
 specific message (have to kill the process to actually get the trace):
 
 #0  0x0001006121a6 in poll ()
 #1  0x00010006d3d2 in gpg_ctx_op_step ()
 #2  0x00010006e5c7 in gpg_decrypt ()
 #3  0x0001000566cf in g_mime_multipart_encrypted_decrypt ()
 #4  0x0001a413 in show_message_part (part=0x10606fc20,
 #state=0x7fff5fbfd1c0, format=0x10002ef80, params=0x7fff5fbfd2c0,
 #first=1) at show-message.c:71
 
 So I guess it ends up looping or waiting in poll(), but I can't tell why
 it would do that. I guess the next step is to post to the gmime mailing
 list, or?

Hey, Felix.  Yeah, I unfortunately don't have any other suggestions
other than asking the gmime folks.

Interestingly, I see the following message in your crypto test output:

  Error: search term did not match precisely one message.

which seems to indicate that the desired message wasn't actually
delivered properly, contrary to what the emacs delivery tests are
stating.  Not sure how that could be related, though, since it looks
like the trace that you show above definitely looks like gmime caught in
a poll loop.

jamie.


pgp4EpZBs1AvD.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


[PATCH] Do not attept to output part raw if part is not GMimePart.

2011-06-02 Thread Jameson Graef Rollins
This was a minor oversite in checking of part type when outputing
content raw.  This was causing gmime was to throw an exception to
stderr.

Unfortunately the gmime exception was not being caught by notmuch, or
the test suite.  I'm not sure if notmuch should have done anything in
this case, but certainly the test suite should be capable of detecting
that something unexpected was output to stderr.
---
 notmuch-show.c |5 +++--
 1 files changed, 3 insertions(+), 2 deletions(-)

diff --git a/notmuch-show.c b/notmuch-show.c
index 9267d02..3b4f775 100644
--- a/notmuch-show.c
+++ b/notmuch-show.c
@@ -646,14 +646,15 @@ format_part_content_raw (GMimeObject *part)
 {
 GMimeStream *stream_stdout;
 GMimeStream *stream_filter = NULL;
-GMimeDataWrapper *wrapper;
+GMimeDataWrapper *wrapper = NULL;
 
 stream_stdout = g_mime_stream_file_new (stdout);
 g_mime_stream_file_set_owner (GMIME_STREAM_FILE (stream_stdout), FALSE);
 
 stream_filter = g_mime_stream_filter_new (stream_stdout);
 
-wrapper = g_mime_part_get_content_object (GMIME_PART (part));
+if (GMIME_IS_PART (part))
+   wrapper = g_mime_part_get_content_object (GMIME_PART (part));
 
 if (wrapper  stream_filter)
g_mime_data_wrapper_write_to_stream (wrapper, stream_filter);
-- 
1.7.4.4

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Re: Decryption fails

2011-06-02 Thread Felix Geller
On Thu, 02 Jun 2011 08:35:49 -0700, Jameson Graef Rollins 
jroll...@finestructure.net wrote:
Non-text part: multipart/signed

 Hey, Felix.  Yeah, I unfortunately don't have any other suggestions
 other than asking the gmime folks.

Jeff replied and sent me a working patch :) Not sure yet how he prefers
to publish the patch, but the problem is fixed.
 
 Interestingly, I see the following message in your crypto test output:
 
   Error: search term did not match precisely one message.
 
 which seems to indicate that the desired message wasn't actually
 delivered properly, contrary to what the emacs delivery tests are
 stating.  Not sure how that could be related, though, since it looks
 like the trace that you show above definitely looks like gmime caught in
 a poll loop.

Just ran the tests with a fixed poll function and it seems that many
problems remain. Don't have the time at the moment to take a closer
look, but I attached the output.

Anyway, I'm a glad decrypting user now ;)


Cheers,
Felix

 jamie.
Non-text part: application/pgp-signature
crypto: Testing PGP/MIME signature verification and decryption
 PASS   emacs delivery of signed message
 FAIL   signature verification
--- crypto.2.expected   2011-06-02 16:46:27.0 +
+++ crypto.2.output 2011-06-02 16:46:27.0 +
@@ -1,23 +1 @@
-[[[{id: X,
- match: true,
- filename: Y,
- timestamp: 946728000,
- date_relative: 2000-01-01,
- tags: [inbox,signed],
- headers: {Subject: test signed message 001,
- From: Notmuch Test Suite test_su...@notmuchmail.org,
- To: test_su...@notmuchmail.org,
- Cc: ,
- Bcc: ,
- Date: 01 Jan 2000 12:00:00 -},
- body: [{id: 1,
- sigstatus: [{status: good,
- fingerprint: 5AEAB11F5E33DCE875DDB75B6D92612D94E46381,
- created: 946728000}],
- content-type: multipart/signed,
- content: [{id: 2,
- content-type: text/plain,
- content: This is a test signed message.\n},
- {id: 3,
- content-type: application/pgp-signature}]}]},
- [
+[[[{id: X,n match: true,n filename: Y,n timestamp: 
946728000,n date_relative: 2000-01-01,n tags: [inbox,signed],n 
headers: {Subject: test signed message 001,n From: Notmuch Test Suite 
test_su...@notmuchmail.org,n To: test_su...@notmuchmail.org,n Cc: ,n 
Bcc: ,n Date: 01 Jan 2000 12:00:00 -},n body: [{id: 1,n 
sigstatus: [{status: good,n fingerprint: 
5AEAB11F5E33DCE875DDB75B6D92612D94E46381,n created: 946728000}],n 
content-type: multipart/signed,n content: [{id: 2,n content-type: 
text/plain,n content: This is a test signed message.\n},n {id: 3,n 
content-type: application/pgp-signature}]}]},n [
 FAIL   signature verification with full owner trust
--- crypto.3.expected   2011-06-02 16:46:27.0 +
+++ crypto.3.output 2011-06-02 16:46:27.0 +
@@ -1,24 +1 @@
-[[[{id: X,
- match: true,
- filename: Y,
- timestamp: 946728000,
- date_relative: 2000-01-01,
- tags: [inbox,signed],
- headers: {Subject: test signed message 001,
- From: Notmuch Test Suite test_su...@notmuchmail.org,
- To: test_su...@notmuchmail.org,
- Cc: ,
- Bcc: ,
- Date: 01 Jan 2000 12:00:00 -},
- body: [{id: 1,
- sigstatus: [{status: good,
- fingerprint: 5AEAB11F5E33DCE875DDB75B6D92612D94E46381,
- created: 946728000,
- userid:  Notmuch Test Suite test_su...@notmuchmail.org 
(INSECURE!)}],
- content-type: multipart/signed,
- content: [{id: 2,
- content-type: text/plain,
- content: This is a test signed message.\n},
- {id: 3,
- content-type: application/pgp-signature}]}]},
- [
+[[[{id: X,n match: true,n filename: Y,n timestamp: 
946728000,n date_relative: 2000-01-01,n tags: [inbox,signed],n 
headers: {Subject: test signed message 001,n From: Notmuch Test Suite 
test_su...@notmuchmail.org,n To: test_su...@notmuchmail.org,n Cc: ,n 
Bcc: ,n Date: 01 Jan 2000 12:00:00 -},n body: [{id: 1,n 
sigstatus: [{status: good,n fingerprint: 
5AEAB11F5E33DCE875DDB75B6D92612D94E46381,n created: 946728000,n userid:  
Notmuch Test Suite test_su...@notmuchmail.org (INSECURE!)}],n 
content-type: multipart/signed,n content: [{id: 2,n content-type: 
text/plain,n content: This is a test signed message.\n},n {id: 3,n 
content-type: application/pgp-signature}]}]},n [
 FAIL   signature verification with signer key unavailable
--- crypto.4.expected   2011-06-02 16:46:28.0 +
+++ crypto.4.output 2011-06-02 16:46:28.0 +
@@ -1,23 +1 @@
-[[[{id: X,
- match: true,
- filename: Y,
- timestamp: 946728000,
- date_relative: 2000-01-01,
- tags: 

[PATCH] emacs: Fix handling of message/rfc822 parts.

2011-06-02 Thread Jameson Graef Rollins
After the recent part-handling overhaul, emacs handling of
message/rfc822 parts broke, not outputting the message contents.  This
fixes then handling as is, but there are still problems with how
notmuch is outputting message parts that needs to be addressed (for
instance, message headers are not being output at the moment).

This also indicates that we need a test for output of message parts in
emacs.
---
 emacs/notmuch-show.el |   27 ---
 1 files changed, 16 insertions(+), 11 deletions(-)

diff --git a/emacs/notmuch-show.el b/emacs/notmuch-show.el
index 9837c7a..714200a 100644
--- a/emacs/notmuch-show.el
+++ b/emacs/notmuch-show.el
@@ -516,21 +516,26 @@ current buffer, if possible.
   t)
 
 (defun notmuch-show-insert-part-message/rfc822 (msg part content-type nth 
depth declared-type)
-  (let* ((message-part (plist-get part :content))
-(inner-parts (plist-get message-part :content)))
-(notmuch-show-insert-part-header nth declared-type content-type nil)
-;; Override `notmuch-message-headers' to force `From' to be
-;; displayed.
-(let ((notmuch-message-headers '(From Subject To Cc Date)))
-  (notmuch-show-insert-headers (plist-get part :headers)))
-;; Blank line after headers to be compatible with the normal
-;; message display.
-(insert \n)
+  (notmuch-show-insert-part-header nth declared-type content-type nil)
+  (let ((inner-parts (plist-get part :content))
+   (start (point)))
+(if (plist-member part :headers)
+   (progn
+ ;; Override `notmuch-message-headers' to force `From' to be
+ ;; displayed.
+ (let ((notmuch-message-headers '(From Subject To Cc Date)))
+   (notmuch-show-insert-headers (plist-get part :headers)))
+ ;; Blank line after headers to be compatible with the normal
+ ;; message display.
+ (insert foo\n)))
 
 ;; Show all of the parts.
 (mapc (lambda (inner-part)
(notmuch-show-insert-bodypart msg inner-part depth))
- inner-parts))
+ inner-parts)
+
+(when notmuch-show-indent-multipart
+  (indent-rigidly start (point) 1)))
   t)
 
 (defun notmuch-show-insert-part-text/plain (msg part content-type nth depth 
declared-type)
-- 
1.7.4.4

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Re: Fwd: Re: compile error of current git on F15

2011-06-02 Thread Daniel Kahn Gillmor
I got the following response off-list from the gmime lead, which he's ok
with my re-posting here:

On 06/02/2011 08:53 AM, Jeffrey Stedfast wrote:
 I don't know why Fedora 15 ships 2.5.x, that seems like a really silly
 thing to do. I think I recall the Balsa guys bringing this up in the
 past and I thought I explained to the Fedora guys that 2.5.x was API
 unstable. Perhaps there was some misunderstanding or something.
 
 That said, as far as timeline goes, I think 2.6 is basically ready for a
 final release, I've just been busy with other stuff lately. Assuming you
 don't have any major outstanding issues with the crypto stuff, I can
 just roll out a 2.6.0 release (which hopefully F15 will pick up, or, if
 not, just check gmime-2.6 = 2.5.7) at any time.
 
 I think notmuch is the only thing making heavy use of the 2.6 crypto
 stuff right now, so you guys have priority.

So this actually makes me re-think my earlier objections to patches
against an unstable API, on two grounds:

 0) if we're the driving force for 2.6 crypto, we should exercise it so
we can give reasonable feedback before stabilization happens.

 1) it sounds like the API is near stabilization anyway, so it won't be
too much hair-pulling, other than any changes we recommend.

So if we can just get 0.6 released, i'll commit to making patches to
support gmime 2.6 for the next version.

(i'll probably start by urging debian's gmime folks to put a version of
gmime2.6 into the experimental repo for us to play with)

--dkg

PS i still recommend that F15 should not ship gmime2.6 by default.  that
seems like a recipe for this kind of trouble.



signature.asc
Description: OpenPGP digital signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


problems with message/rfc822 parts

2011-06-02 Thread Jameson Graef Rollins
Hey, folks.  I've been noticing a couple of issues with message/rfc822
part handling in recent builds of notmuch/master.  They are rooted in
the new part handling rework that was done recently.  I just want to
mention them here, to make people aware of them, until we get a chance
to address them.

The first issue is that show --format=raw --part on a message/rfc822
part is throwing a gmime exception, even though notmuch is still
producing output and returning successfully.  I just sent a patch to the
list addressing the issues.  However, the test suite was apparently
unable to catch that something unexpected was being output to stderr, so
we still need to address that.

Another issue is that we're no longer outputting any of the headers of
message/rfc822 parts.  This shouldn't be too hard to hack back in, so
hopefully I can get to that this weekend.

There was also a problem with the emacs display of message/rfc822 parts,
and I just sent a patch to the list about that as well.

However, there is a larger issue related to the ones above which is that
message parts in general are not being handled consistently, including
the top level message.  amdragon and I were actually discussing this on
irc the other day.  A possible solution is to make a general
message-part-handling function, that handles output of headers and the
message body, and call that function both on the top level message and
on any message sub parts.  I think that would make things a lot cleaner,
and would make the output more consistent and intuitive.

A somewhat related issue is that the raw format for parts is not
really doing the expected thing in certain cases, particularly with
regards to message parts.  dkg and I sort of realized this when we were
reworking the part handling a couple weeks ago.  One expects, rightfully
so, that show --format=raw --part on a message part would output the
full, raw, well-formatted rfc822 message enclosed in that part.
Currently that is not the case, since, for instance, the headers are not
output and the part handling is recursing over all the message
sub-parts.

I don't know necessarily know the right solution for either of these
issues, although I have some ideas (assuming I don't run into gmime
limitations).  Feedback and suggestions would be wonderful.
Unfortunately I don't have time to work on any of these issues at the
moment (hopefully this weekend), but I just wanted to throw them out
there, in case anyone wants to tackle them before I get to them.

jamie.


pgpOalvQowegn.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Re: Decryption fails

2011-06-02 Thread Jameson Graef Rollins
On Thu, 02 Jun 2011 18:49:22 +0200, Felix Geller fgel...@gmail.com wrote:
 Jeff replied and sent me a working patch :) Not sure yet how he prefers
 to publish the patch, but the problem is fixed.

That's great!  What did Jeff say exactly?  Is the patch to gmime 2.4?
Did he mention that he was including them in upstream, hopefully in the
next release?

  Interestingly, I see the following message in your crypto test output:
  
Error: search term did not match precisely one message.
  
  which seems to indicate that the desired message wasn't actually
  delivered properly, contrary to what the emacs delivery tests are
  stating.  Not sure how that could be related, though, since it looks
  like the trace that you show above definitely looks like gmime caught in
  a poll loop.
 
 Just ran the tests with a fixed poll function and it seems that many
 problems remain. Don't have the time at the moment to take a closer
 look, but I attached the output.

Those test failures look very strange to me.  There are a bunch of 'n's
being output after the commas in the json output.  Felix, are you sure
you haven't modified your source at all?

jamie.


pgp4mvsSQW5VC.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Multiple sender identities (composing)

2011-06-02 Thread Brian May
On 1 June 2011 16:42, Thomas Jost  wrote:

> There's a function that changes the signature according to the From
> header in the message I sent on this list yesterday
> (id:"87pqmznczk.fsf at thor.loria.fr", near the end of the message).
>
>
Thanks for the pointer, it seems to work fine.
-- 
Brian May 
-- next part --
An HTML attachment was scrubbed...
URL: 
<http://notmuchmail.org/pipermail/notmuch/attachments/20110602/c7fceb25/attachment.html>


[PATCH 14/25] Fix old style notmuch-fcc-dirs configuration check.

2011-06-02 Thread Dmitry Kurochkin
Hi Carl.

On Wed, 01 Jun 2011 22:10:07 -0700, Carl Worth  wrote:
> On Sat, 28 May 2011 14:51:49 -0700, Jameson Graef Rollins  finestructure.net> wrote:
> Hi Jamie,
> 
> I've pushed the next few patches up to this point, (with only one
> functional change---I fixed a new test case to correctly use
> notmuch_search_sanitize to avoid spurious failures unmatching thread ID
> values).
> 
> This patch, however, isn't ready. The big problem is in this commit
> message:
> 
> > From: Dmitry Kurochkin 
> > 
> > In notmuch 0.5 notmuch-fcc-dirs style changed.  The previous code
> > did not correctly identify an old configuration and, as a
> > consequence, broke new configurations.
> 
> There are several things there that are too vague, ("previous code",
> "old configuration", "new configurations").

Well, it says that changes are in notmuch 0.5.  So "old" and "previous"
refer to pre-0.5 (i.e. 0.4) and "new" refers to 0.5.

> What kind of configuration
> is broken?

Any configuration when `notmuch-fcc-dirs' is a list.  That variable has
a nice documentation.

> How does the change here help?
> 

It fixes detection of old-style pre-0.5 setting for `notmuch-fcc-dirs'.

> It would be easier to understand the code if there were a corresponding
> test case for it. I'd even be willing to help write a test case, but
> there are not enough specifics in that commit message to even tell me
> what to test.
> 

I do not think we need a test for this fix.  What we need are tests for
FCC functionality when notmuch-fcc-dirs is a list.

> >   ((and (listp notmuch-fcc-dirs)
> > -   (= 1 (length (car notmuch-fcc-dirs
> > +   (stringp (car notmuch-fcc-dirs)))
> 
> So the "old configuration" was a single string?

No, it is a list with a string as the first element.  May I refer you to
"git show 0.4:emacs/notmuch-maildir-fcc.el" and current
emacs/notmuch-maildir-fcc.el?  It has good documentation with examples
for notmuch-fcc-dirs.  Also NEWS for 0.5 describe this change.

> And this has been
> inadvertently broken since 0.5?
> 

Old configuration format was changed in 0.5 in an incompatible way.
There is a check for the unsupported old-style configuration.  But the
check is broken and results in an error when running with a valid
new-style configuration.

> >;; Old style - no longer works.
> >(error "Invalid `notmuch-fcc-dirs' setting (old style)"))
> 
> Yikes. That vague phrasing ("old style") is already in the code in error
> messages as well.
> 

Yes.

> Dmitry, can you help me know what's going on here?

I hope above helps.

> (Preferably by
> sending a newer commit with a more thorough commit message.)
> 

I am not sure what you expect from the commit message here.  IMO it is
enough for this small bugfix and those who interested can always refer
to documentation for details.

Regards,
  Dmitry

> Thanks,
> 
> -Carl
Non-text part: application/pgp-signature
> ___
> notmuch mailing list
> notmuch at notmuchmail.org
> http://notmuchmail.org/mailman/listinfo/notmuch


[python] get all messages of a thread

2011-06-02 Thread Sebastian Spaeth
On Wed, 1 Jun 2011 15:35:35 +1000, Brian May wrote:
> Oh, I see, for your code, there is a implied call to __len__, and the
> __len__ function is completely broken for the reasons described in the
> documentation:

It seems to have been a bad idea to implement __len__ at all for the
Messsages() construct in the python bindings, and I wonder if I should
remove it.

On the other hand, it seems that list(Messages()) implicitely calls
len(), or so it seems from the error that we get when trying to list() a
messages object.

An alternative is to implement len() as a call to count_messages() which
has for me so far always returned the correct number of messages without
using up the iterator. However, the xapian docs explicitely state that
it does not guarantee that the count will be correct, so len() might
return a wrong message size (potentially).

What would be the best way to solve this (besides fixing the C api to
allow to reset the iterator ;-) ?)

I could implement a custom .as_list() function that returns the
Messages() object as a list that is guaranteed to be stable, by copying
out the Message() objects into a list.

Sebastian
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
URL: 
<http://notmuchmail.org/pipermail/notmuch/attachments/20110602/7c0dd31e/attachment.pgp>


python: get_filenames()

2011-06-02 Thread Sebastian Spaeth
A head up for the distributed gits and python users:
I just pushed a commit to cworth/master implementing
Message().get_filenames(). This will return a generator of absolute
filenames for a certain Message-ID.

For further information I refer to
http://git.notmuchmail.org/git/notmuch/commit/b31247c354b54a3cbeb1c7f9df830e16f7c921d9
and the code (and it's documentation).

Let me know if it works for you.

Sebastian
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
URL: 
<http://notmuchmail.org/pipermail/notmuch/attachments/20110602/481666e2/attachment.pgp>


notmuch crashing on misformated emails

2011-06-02 Thread Sebastian Spaeth
in python:
  list(Threads())

leads to a program abortion (bombing out of python):
Internal error: Failed to read timestamp value from document. 
(lib/message.cc:731).

Can we not crash the app on misformated emails?

Sebastian
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
URL: 
<http://notmuchmail.org/pipermail/notmuch/attachments/20110602/ca89b851/attachment.pgp>


[python] get all messages of a thread

2011-06-02 Thread Brian May
On 2 June 2011 17:05, Sebastian Spaeth  wrote:

> What would be the best way to solve this (besides fixing the C api to
> allow to reset the iterator ;-) ?)
>
>
 I am not really familiar with the code. So am I correct in making the
following assumptions?

* It is not easy to fix the C api to reset the iterator (what about
repeating the search?)

* The only accurate way to get the number of messages is to iterate through
every search result and count them?

If so, then len(...) I think might be very slow if there are a large number
of elements.

Maybe it might be easier/better to implement object.__nonzero__(self)
 instead of the object.__len__(self) method?

http://docs.python.org/reference/datamodel.html

object.__nonzero__(self)
Called to implement truth value testing and the built-in operation bool();
should return False or True, or their integer equivalents 0 or 1. When this
method is not defined, __len__() is called, if it is defined, and the object
is considered true if its result is nonzero. If a class defines neither
__len__() nor __nonzero__(), all its instances are considered true.

object.__len__(self)
Called to implement the built-in function len(). Should return the length of
the object, an integer >= 0. Also, an object that doesn?t define a
__nonzero__() method and whose __len__() method returns zero is considered
to be false in a Boolean context.

-- 
Brian May 
-- next part --
An HTML attachment was scrubbed...
URL: 
<http://notmuchmail.org/pipermail/notmuch/attachments/20110602/9b854d5c/attachment.html>


[PATCH 0/4] set test prereqs (Emacs, GDB, GPG) v3

2011-06-02 Thread Pieter Praet
Rebased the "set test prereqs (Emacs, GDB, GPG)" [1] patch series
to Jameson's release-candidate/0.6 branch (0c0b4172).

The GDB dependency for the atomicity tests was dropped since
Austin has already included this in his amdragon/atomic-new-v4
branch [2], which has been included in release-candidate/0.6.

Also, some crypto tests which depend not only on GPG but Emacs as
well have been updated accordingly.

Jameson, could this be merged in? It helps prevent false
negatives in the absence of Emacs/GnuPG/GDB.


[1] id:"1305274636-19975-1-git-send-email-pieter at praet.org"
[2] id:"BANLkTikEhvhroyJNRN9RLwBNz_DbOLfzBQ at mail.gmail.com"



[PATCH 1/4] test: add 'GnuPG' prereq to dependent 'crypto' tests

2011-06-02 Thread Pieter Praet
Adds a new test that checks for the presence of 'gpg',
and adds that test as a prereq to all subsequent tests
that rely on GnuPG.

This causes tests with unmet dependencies to be skipped.

Signed-off-by: Pieter Praet 
---
 test/crypto |   33 +++--
 1 files changed, 19 insertions(+), 14 deletions(-)

diff --git a/test/crypto b/test/crypto
index 8e92016..59f7989 100755
--- a/test/crypto
+++ b/test/crypto
@@ -7,6 +7,11 @@
 test_description='PGP/MIME signature verification and decryption'
 . ./test-lib.sh

+# GnuPG is a prereq.
+test_expect_success "prereq: GnuPG is present" "which gpg" \
+&& test_set_prereq GPG
+
+
 add_gnupg_home ()
 {
 local output
@@ -31,7 +36,7 @@ FINGERPRINT=$(gpg --no-tty --list-secret-keys --with-colons 
--fingerprint | grep
 # although I can't figure out why
 add_email_corpus

-test_expect_success 'emacs delivery of signed message' \
+test_expect_success GPG 'emacs delivery of signed message' \
 'emacs_deliver_message \
 "test signed message 001" \
 "This is a test signed message." \
@@ -64,7 +69,7 @@ expected='[[[{"id": "X",
  {"id": 3,
  "content-type": "application/pgp-signature"}]}]},
  ['
-test_expect_equal \
+test_expect_equal GPG \
 "$output" \
 "$expected"

@@ -99,7 +104,7 @@ expected='[[[{"id": "X",
  {"id": 3,
  "content-type": "application/pgp-signature"}]}]},
  ['
-test_expect_equal \
+test_expect_equal GPG \
 "$output" \
 "$expected"

@@ -132,7 +137,7 @@ expected='[[[{"id": "X",
  {"id": 3,
  "content-type": "application/pgp-signature"}]}]},
  ['
-test_expect_equal \
+test_expect_equal GPG \
 "$output" \
 "$expected"
 mv "${GNUPGHOME}"{.bak,}
@@ -141,7 +146,7 @@ mv "${GNUPGHOME}"{.bak,}
 cat OUTPUT
-test_expect_equal_file OUTPUT TESTATTACHMENT
+test_expect_equal_file GPG OUTPUT TESTATTACHMENT

 test_begin_subtest "decryption failure with missing key"
 mv "${GNUPGHOME}"{,.bak}
@@ -259,12 +264,12 @@ expected='[[[{"id": "X",
  {"id": 3,
  "content-type": "application/octet-stream"}]}]},
  ['
-test_expect_equal \
+test_expect_equal GPG \
 "$output" \
 "$expected"
 mv "${GNUPGHOME}"{.bak,}

-test_expect_success 'emacs delivery of encrypted + signed message' \
+test_expect_success GPG 'emacs delivery of encrypted + signed message' \
 'emacs_deliver_message \
 "test encrypted message 002" \
 "This is another test encrypted message.\n" \
@@ -299,7 +304,7 @@ expected='[[[{"id": "X",
  "content-type": "text/plain",
  "content": "This is another test encrypted message.\n"}]}]},
  ['
-test_expect_equal \
+test_expect_equal GPG \
 "$output" \
 "$expected"

@@ -313,7 +318,7 @@ On 01 Jan 2000 12:00:00 -, Notmuch Test Suite 
 w
 Non-text part: multipart/encrypted
 Non-text part: application/pgp-encrypted
 > This is another test encrypted message.'
-test_expect_equal \
+test_expect_equal GPG \
 "$output" \
 "$expected"

@@ -354,7 +359,7 @@ expected='[[[{"id": "X",
  {"id": 3,
  "content-type": "application/pgp-signature"}]}]},
  ['
-test_expect_equal \
+test_expect_equal GPG \
 "$output" \
 "$expected"

-- 
1.7.4.1



[PATCH 2/4] test: add 'Emacs' prereq to dependent 'crypto' tests

2011-06-02 Thread Pieter Praet
Adds a new test that checks for the presence of 'emacs',
and adds that test as a prereq to all subsequent tests
that rely on Emacs.

This causes tests with unmet dependencies to be skipped.

Signed-off-by: Pieter Praet 
---
 test/crypto |   17 ++---
 1 files changed, 14 insertions(+), 3 deletions(-)

diff --git a/test/crypto b/test/crypto
index 59f7989..c69307d 100755
--- a/test/crypto
+++ b/test/crypto
@@ -7,10 +7,21 @@
 test_description='PGP/MIME signature verification and decryption'
 . ./test-lib.sh

+# Emacs is a prereq.
+test_expect_success "prereq: Emacs is present" "which emacs" \
+&& test_set_prereq EMACS
+
 # GnuPG is a prereq.
 test_expect_success "prereq: GnuPG is present" "which gpg" \
 && test_set_prereq GPG

+# Some tests have multiple prereqs, but the test_expect_* functions
+# accept only a single argument as prereq tag, and using test_have_prereq
+# in and around tests causes various errors for me, so a dirty workaround
+# will have to do for the time being.
+test_have_prereq EMACS && test_have_prereq GPG \
+&& test_set_prereq EMACS+GPG
+

 add_gnupg_home ()
 {
@@ -36,7 +47,7 @@ FINGERPRINT=$(gpg --no-tty --list-secret-keys --with-colons 
--fingerprint | grep
 # although I can't figure out why
 add_email_corpus

-test_expect_success GPG 'emacs delivery of signed message' \
+test_expect_success EMACS+GPG 'emacs delivery of signed message' \
 'emacs_deliver_message \
 "test signed message 001" \
 "This is a test signed message." \
@@ -146,7 +157,7 @@ mv "${GNUPGHOME}"{.bak,}
 cat 

[PATCH 3/4] test: add 'Emacs' prereq to dependent 'emacs' tests

2011-06-02 Thread Pieter Praet
Adds a new test that checks for the presence of 'emacs',
and adds that test as a prereq to all subsequent tests
that rely on Emacs.

This causes tests with unmet dependencies to be skipped.

Signed-off-by: Pieter Praet 
---
 test/emacs |   43 ---
 1 files changed, 24 insertions(+), 19 deletions(-)

diff --git a/test/emacs b/test/emacs
index da8ba68..10138a6 100755
--- a/test/emacs
+++ b/test/emacs
@@ -2,65 +2,70 @@
 test_description="emacs interface"
 . test-lib.sh

+# Emacs is a prereq.
+test_expect_success "prereq: Emacs is present" "which emacs" \
+&& test_set_prereq EMACS
+
+
 EXPECTED=../emacs.expected-output

 add_email_corpus

 test_begin_subtest "Basic notmuch-hello view in emacs"
 test_emacs '(notmuch-hello) (princ (buffer-string))' >OUTPUT
-test_expect_equal_file OUTPUT $EXPECTED/notmuch-hello
+test_expect_equal_file EMACS OUTPUT $EXPECTED/notmuch-hello

 test_begin_subtest "Saved search with 0 results"
 test_emacs '(setq notmuch-show-empty-saved-searches t) (setq 
notmuch-saved-searches '\''(("inbox" . "tag:inbox") ("unread" . "tag:unread") 
("empty" . "tag:doesnotexist"))) (notmuch-hello) (princ (buffer-string))' 
>OUTPUT
-test_expect_equal_file OUTPUT $EXPECTED/notmuch-hello-with-empty
+test_expect_equal_file EMACS OUTPUT $EXPECTED/notmuch-hello-with-empty

 test_begin_subtest "No saved searches displayed (all with 0 results)"
 test_emacs '(setq notmuch-saved-searches '\''(("empty" . "tag:doesnotexist"))) 
(notmuch-hello) (princ (buffer-string))' >OUTPUT
-test_expect_equal_file OUTPUT $EXPECTED/notmuch-hello-no-saved-searches
+test_expect_equal_file EMACS OUTPUT $EXPECTED/notmuch-hello-no-saved-searches

 test_begin_subtest "Basic notmuch-search view in emacs"
 test_emacs '(notmuch-search "tag:inbox") (notmuch-test-wait) (princ 
(buffer-string))' >OUTPUT
-test_expect_equal_file OUTPUT $EXPECTED/notmuch-search-tag-inbox
+test_expect_equal_file EMACS OUTPUT $EXPECTED/notmuch-search-tag-inbox

 test_begin_subtest "Navigation of notmuch-hello to search results"
 test_emacs '(notmuch-hello) (goto-char (point-min)) (re-search-forward 
"inbox") (widget-button-press (point)) (notmuch-test-wait) (princ 
(buffer-string))' >OUTPUT
-test_expect_equal_file OUTPUT $EXPECTED/notmuch-hello-view-inbox
+test_expect_equal_file EMACS OUTPUT $EXPECTED/notmuch-hello-view-inbox

 test_begin_subtest "Basic notmuch-show view in emacs"
 maildir_storage_thread=$(notmuch search --output=threads 
id:20091117190054.GU3165 at dottiness.seas.harvard.edu)
 test_emacs "(notmuch-show \"$maildir_storage_thread\") (princ 
(buffer-string))" >OUTPUT
-test_expect_equal_file OUTPUT $EXPECTED/notmuch-show-thread-maildir-storage
+test_expect_equal_file EMACS OUTPUT 
$EXPECTED/notmuch-show-thread-maildir-storage

 test_begin_subtest "Navigation of notmuch-search to thread view"
 test_emacs '(notmuch-search "tag:inbox") (notmuch-test-wait) (goto-char 
(point-min)) (re-search-forward "Working with Maildir") 
(notmuch-search-show-thread) (notmuch-test-wait) (princ (buffer-string))' 
>OUTPUT
-test_expect_equal_file OUTPUT $EXPECTED/notmuch-show-thread-maildir-storage
+test_expect_equal_file EMACS OUTPUT 
$EXPECTED/notmuch-show-thread-maildir-storage

 test_begin_subtest "Add tag from search view"
 os_x_darwin_thread=$(notmuch search --output=threads 
id:ddd65cda0911171950o4eea4389v86de9525e46052d3 at mail.gmail.com)
 test_emacs "(notmuch-search \"$os_x_darwin_thread\") (notmuch-test-wait) 
(notmuch-search-add-tag \"tag-from-search-view\")"
 output=$(notmuch search $os_x_darwin_thread | notmuch_search_sanitize)
-test_expect_equal "$output" "thread:XXX   2009-11-18 [4/4] Jjgod Jiang, 
Alexander Botero-Lowry; [notmuch] Mac OS X/Darwin compatibility issues (inbox 
tag-from-search-view unread)"
+test_expect_equal EMACS "$output" "thread:XXX   2009-11-18 [4/4] Jjgod Jiang, 
Alexander Botero-Lowry; [notmuch] Mac OS X/Darwin compatibility issues (inbox 
tag-from-search-view unread)"

 test_begin_subtest "Remove tag from search view"
 test_emacs "(notmuch-search \"$os_x_darwin_thread\") (notmuch-test-wait) 
(notmuch-search-remove-tag \"tag-from-search-view\")"
 output=$(notmuch search $os_x_darwin_thread | notmuch_search_sanitize)
-test_expect_equal "$output" "thread:XXX   2009-11-18 [4/4] Jjgod Jiang, 
Alexander Botero-Lowry; [notmuch] Mac OS X/Darwin compatibility issues (inbox 
unread)"
+test_expect_equal EMACS "$output" "thread:XXX   2009-11-18 [4/4] Jjgod Jiang, 
Alexander Botero-Lowry; [notmuch] Mac OS X/Darwin compatibility issues (inbox 
unread)"

 test_begin_subtest "Add tag from notmuch-show view"
 test_emacs "(notmuch-show \"$os_x_darwin_thread\") (notmuch-show-add-tag 
\"tag-from-show-view\")"
 output=$(notmuch search $os_x_darwin_thread | notmuch_search_sanitize)
-test_expect_equal "$output" "thread:XXX   2009-11-18 [4/4] Jjgod Jiang, 
Alexander Botero-Lowry; [notmuch] Mac OS X/Darwin compatibility issues (inbox 
tag-from-show-view unread)"
+test_expect_equal EMACS "$output" "thread:XXX   

[PATCH 4/4] test: add 'Emacs' prereq to dependent 'emacs-large-search-buffer' tests

2011-06-02 Thread Pieter Praet
Adds a new test that checks for the presence of 'emacs',
and adds that test as a prereq to all subsequent tests
that rely on Emacs.

This causes tests with unmet dependencies to be skipped.

Signed-off-by: Pieter Praet 
---
 test/emacs-large-search-buffer |9 +++--
 1 files changed, 7 insertions(+), 2 deletions(-)

diff --git a/test/emacs-large-search-buffer b/test/emacs-large-search-buffer
index c78ce33..063f121 100755
--- a/test/emacs-large-search-buffer
+++ b/test/emacs-large-search-buffer
@@ -2,6 +2,11 @@
 test_description="Emacs with large search results buffer"
 . test-lib.sh

+# Emacs is a prereq.
+test_expect_success "prereq: Emacs is present" "which emacs" \
+&& test_set_prereq EMACS
+
+
 x=xx # 10
 x=$x$x$x$x$x$x$x$x$x$x # 100
 x=$x$x$x$x$x$x$x$x$x # 900
@@ -23,11 +28,11 @@ expected="$(notmuch search '*' | sed -e 
's/^thread:[0-9a-f]*  //' -e 's/;//' -e
 End of search results."

 output=$(test_emacs '(notmuch-search "*") (notmuch-test-wait) (princ 
(buffer-string))' | sed -e s',  *, ,g' -e 's/xxx*/[BLOB]/g')
-test_expect_equal "$output" "$expected"
+test_expect_equal EMACS "$output" "$expected"

 test_begin_subtest "Ensure that emacs doesn't drop error messages"
 output=$(test_emacs '(notmuch-search "--this-option-does-not-exist") 
(notmuch-test-wait) (princ (buffer-string))')
-test_expect_equal "$output" "Error: Unexpected output from notmuch search:
+test_expect_equal EMACS "$output" "Error: Unexpected output from notmuch 
search:
 Unrecognized option: --this-option-does-not-exist
 End of search results. (process returned 1)"

-- 
1.7.4.1



[PATCH 0/4] set test prereqs (Emacs, GDB, GPG) v3

2011-06-02 Thread Austin Clements
This looks good to me, and should be merged sooner rather than later
because it touches a lot of lines.

If atomic-new-v4 doesn't happen to get merged before this, then
id:"1305206080-17461-1-git-send-email-amdragon at mit.edu" and
id:"1305206110-17511-1-git-send-email-amdragon at mit.edu" should get
merged, since they fix test-lib problems with dependencies (they're
included in the atomic-new-v4 branch because that uses dependencies,
but aren't related to atomicity).

Quoth Pieter Praet on Jun 02 at  2:03 pm:
> Rebased the "set test prereqs (Emacs, GDB, GPG)" [1] patch series
> to Jameson's release-candidate/0.6 branch (0c0b4172).
> 
> The GDB dependency for the atomicity tests was dropped since
> Austin has already included this in his amdragon/atomic-new-v4
> branch [2], which has been included in release-candidate/0.6.
> 
> Also, some crypto tests which depend not only on GPG but Emacs as
> well have been updated accordingly.
> 
> Jameson, could this be merged in? It helps prevent false
> negatives in the absence of Emacs/GnuPG/GDB.
> 
> 
> [1] id:"1305274636-19975-1-git-send-email-pieter at praet.org"
> [2] id:"BANLkTikEhvhroyJNRN9RLwBNz_DbOLfzBQ at mail.gmail.com"


[python] get all messages of a thread

2011-06-02 Thread Sebastian Spaeth
On Thu, 2 Jun 2011 19:43:29 +1000, Brian May wrote:
> On 2 June 2011 17:05, Sebastian Spaeth  wrote:
> 
> > What would be the best way to solve this (besides fixing the C api to
> > allow to reset the iterator ;-) ?)

> * It is not easy to fix the C api to reset the iterator (what about
> repeating the search?)

I am not sure about the difficulty of that, I am not a C-kind of
guy. Repeating the search would be easy but potentially gives you
different results since the db could have changed since then.

> * The only accurate way to get the number of messages is to iterate through
> every search result and count them?

There is count_messages() which wraps notmuch_query_count_messages which
invokes some very quick xapian function. But the xapian docs explicitely
state that it's xapians best guess and not guaranteed to be the real
number of messages (although it always was in my attempts). And you
don't want len() to return an approximation of the iterator length

> If so, then len(...) I think might be very slow if there are a large number
> of elements.

> Maybe it might be easier/better to implement object.__nonzero__(self)
>  instead of the object.__len__(self) method?
> 
> http://docs.python.org/reference/datamodel.html
> 
> object.__nonzero__(self)
> Called to implement truth value testing and the built-in operation bool();
> should return False or True, or their integer equivalents 0 or 1. When this
> method is not defined, __len__() is called, if it is defined, and the object
> is considered true if its result is nonzero. If a class defines neither
> __len__() nor __nonzero__(), all its instances are considered true.

Interesting, did not know about this one. I guess that would solve the
example with the:

t = query.search_threads()

if t:
  for thread in t:
print thread

Actually the "if t:" is no longer needed, I just tried a query returning
no Threads and 

for thread in threads:
  pass

works just fine with an empty Threads() object.

I made the change, and implemented __nonzero__ and removed the len()
method. It just doesn't make sense on 1-time iterators. (I documented
the change in the API docs). Sorry if this breaks existing code.

list(Threads()) works just fine too, it just took a while to create a
list of 13k Thread() objects on this laptop. (and list() will of course
not return until it is finished).


Sebastian
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
URL: 
<http://notmuchmail.org/pipermail/notmuch/attachments/20110602/924e2df9/attachment.pgp>


[python] get all messages of a thread

2011-06-02 Thread Sebastian Spaeth
On Thu, 02 Jun 2011 16:20:14 +0200, Sebastian Spaeth wrote:

> I made the change, and implemented __nonzero__ and removed the len()
> method. It just doesn't make sense on 1-time iterators. (I documented
> the change in the API docs). Sorry if this breaks existing code.

FYI

OK, I just pushed a change that adds the __nonzero__ function thus
making:

t=Threads()
if t:
  len(t)

work, but I did not pull the len() function out. The explicit "if t"
test is however not needed for

t = q.search_threads()
for thread in t:
  print thread

works just fine with empty results.

Sebastian
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
URL: 
<http://notmuchmail.org/pipermail/notmuch/attachments/20110602/055da5b8/attachment-0001.pgp>


[python] get all messages of a thread

2011-06-02 Thread Austin Clements
On Thu, Jun 2, 2011 at 10:20 AM, Sebastian Spaeth  
wrote:
> On Thu, 2 Jun 2011 19:43:29 +1000, Brian May wrote:
>> On 2 June 2011 17:05, Sebastian Spaeth  wrote:
>>
>> > What would be the best way to solve this (besides fixing the C api to
>> > allow to reset the iterator ;-) ?)
>
>> * It is not easy to fix the C api to reset the iterator (what about
>> repeating the search?)
>
> I am not sure about the difficulty of that, I am not a C-kind of
> guy. Repeating the search would be easy but potentially gives you
> different results since the db could have changed since then.

Not too hard.  Here's an utterly untested patch that implements
iterator resetting for notmuch_messages_t iterators.  It *should* be
much more efficient than performing the query again, but if you use
it, I'd love to know if that's actually true.

This may not be useful if __len__ is gone, unless you really want to
turn Messages/Threads into iterators rather than generators (as I've
pointed out before, there is absolutely nothing unusual or un-Pythonic
about how Messages/Threads works right now [well, except for the
presence of __len__ in a generator, I suppose]).

diff --git a/lib/messages.c b/lib/messages.c
index 7bcd1ab..085691c 100644
--- a/lib/messages.c
+++ b/lib/messages.c
@@ -80,7 +80,8 @@ _notmuch_messages_create (notmuch_message_list_t *list)
return NULL;

 messages->is_of_list_type = TRUE;
-messages->iterator = list->head;
+messages->head = list->head;
+notmuch_messages_reset (messages);

 return messages;
 }
@@ -137,6 +138,15 @@ notmuch_messages_move_to_next (notmuch_messages_t
*messages)
 }

 void
+notmuch_messages_reset (notmuch_messages_t *messages)
+{
+if (! messages->is_of_list_type)
+   return _notmuch_mset_messages_reset (messages);
+
+messages->iterator = messages->head;
+}
+
+void
 notmuch_messages_destroy (notmuch_messages_t *messages)
 {
 talloc_free (messages);
diff --git a/lib/notmuch-private.h b/lib/notmuch-private.h
index 02e24ee..805d60c 100644
--- a/lib/notmuch-private.h
+++ b/lib/notmuch-private.h
@@ -413,6 +413,7 @@ typedef struct _notmuch_message_list {
  */
 struct visible _notmuch_messages {
 notmuch_bool_t is_of_list_type;
+notmuch_message_node_t *head;
 notmuch_message_node_t *iterator;
 };

@@ -441,6 +442,9 @@ _notmuch_mset_messages_get (notmuch_messages_t *messages);
 void
 _notmuch_mset_messages_move_to_next (notmuch_messages_t *messages);

+void
+_notmuch_mset_messages_reset (notmuch_messages_t *messages);
+
 notmuch_bool_t
 _notmuch_doc_id_set_contains (notmuch_doc_id_set_t *doc_ids,
   unsigned int doc_id);
diff --git a/lib/notmuch.h b/lib/notmuch.h
index 9cdcec0..044cfaa 100644
--- a/lib/notmuch.h
+++ b/lib/notmuch.h
@@ -734,6 +734,15 @@ notmuch_messages_get (notmuch_messages_t *messages);
 void
 notmuch_messages_move_to_next (notmuch_messages_t *messages);

+/* Reset the 'messages' iterator back to the first message.
+ *
+ * For iterators returned from notmuch_query_search_messages, this is
+ * both more efficient than performing the query a second time and
+ * guaranteed to result in the same messages as the first iteration.
+ */
+void
+notmuch_messages_reset (notmuch_messages_t *messages);
+
 /* Destroy a notmuch_messages_t object.
  *
  * It's not strictly necessary to call this function. All memory from
diff --git a/lib/query.cc b/lib/query.cc
index 6f02b04..1e75be0 100644
--- a/lib/query.cc
+++ b/lib/query.cc
@@ -32,6 +32,7 @@ struct _notmuch_query {
 typedef struct _notmuch_mset_messages {
 notmuch_messages_t base;
 notmuch_database_t *notmuch;
+Xapian::MSet mset;
 Xapian::MSetIterator iterator;
 Xapian::MSetIterator iterator_end;
 } notmuch_mset_messages_t;
@@ -128,6 +129,7 @@ notmuch_query_search_messages (notmuch_query_t *query)
messages->base.is_of_list_type = FALSE;
messages->base.iterator = NULL;
messages->notmuch = notmuch;
+   new (>mset) Xapian::MSet ();
new (>iterator) Xapian::MSetIterator ();
new (>iterator_end) Xapian::MSetIterator ();

@@ -181,8 +183,8 @@ notmuch_query_search_messages (notmuch_query_t *query)

mset = enquire.get_mset (0, notmuch->xapian_db->get_doccount ());

-   messages->iterator = mset.begin ();
-   messages->iterator_end = mset.end ();
+   messages->mset = mset;
+   _notmuch_mset_messages_reset (>base);

return >base;

@@ -257,6 +259,17 @@ _notmuch_mset_messages_move_to_next
(notmuch_messages_t *messages)
 mset_messages->iterator++;
 }

+void
+_notmuch_mset_messages_reset (notmuch_messages_t *messages)
+{
+notmuch_mset_messages_t *mset_messages;
+
+mset_messages = (notmuch_mset_messages_t *) messages;
+
+mset_messages->iterator = mset_messages->mset.begin ();
+mset_messages->iterator_end = mset_messages->mset.end ();
+}
+
 static notmuch_bool_t
 _notmuch_doc_id_set_init (void *ctx,
  notmuch_doc_id_set_t *doc_ids,


notmuch crashing on misformated emails

2011-06-02 Thread Sebastian Spaeth
On Thu, 02 Jun 2011 09:16:33 +0200, Sebastian Spaeth  
wrote:
> in python:
>   list(Threads())
> 
> leads to a program abortion (bombing out of python):
> Internal error: Failed to read timestamp value from document. 
> (lib/message.cc:731).
> 
> Can we not crash the app on misformated emails?

Just to follow up, I cannot reproduce this anymore. It works fine
now. Sorry for the noise.

Sebastian
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
URL: 
<http://notmuchmail.org/pipermail/notmuch/attachments/20110602/b1447f38/attachment.pgp>


Decryption fails

2011-06-02 Thread Jameson Graef Rollins
On Tue, 31 May 2011 19:33:29 +0200, Felix Geller  wrote:
> I get the following trace when using show --decrypt to decrypt a
> specific message (have to kill the process to actually get the trace):
> 
> #0  0x0001006121a6 in poll ()
> #1  0x00010006d3d2 in gpg_ctx_op_step ()
> #2  0x00010006e5c7 in gpg_decrypt ()
> #3  0x0001000566cf in g_mime_multipart_encrypted_decrypt ()
> #4  0x0001a413 in show_message_part (part=0x10606fc20,
> #state=0x7fff5fbfd1c0, format=0x10002ef80, params=0x7fff5fbfd2c0,
> #first=1) at show-message.c:71
> 
> So I guess it ends up looping or waiting in poll(), but I can't tell why
> it would do that. I guess the next step is to post to the gmime mailing
> list, or?

Hey, Felix.  Yeah, I unfortunately don't have any other suggestions
other than asking the gmime folks.

Interestingly, I see the following message in your crypto test output:

  Error: search term did not match precisely one message.

which seems to indicate that the desired message wasn't actually
delivered properly, contrary to what the emacs delivery tests are
stating.  Not sure how that could be related, though, since it looks
like the trace that you show above definitely looks like gmime caught in
a poll loop.

jamie.
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL: 
<http://notmuchmail.org/pipermail/notmuch/attachments/20110602/e9eb228c/attachment.pgp>


[PATCH] Do not attept to output part raw if part is not GMimePart.

2011-06-02 Thread Jameson Graef Rollins
This was a minor oversite in checking of part type when outputing
content raw.  This was causing gmime was to throw an exception to
stderr.

Unfortunately the gmime exception was not being caught by notmuch, or
the test suite.  I'm not sure if notmuch should have done anything in
this case, but certainly the test suite should be capable of detecting
that something unexpected was output to stderr.
---
 notmuch-show.c |5 +++--
 1 files changed, 3 insertions(+), 2 deletions(-)

diff --git a/notmuch-show.c b/notmuch-show.c
index 9267d02..3b4f775 100644
--- a/notmuch-show.c
+++ b/notmuch-show.c
@@ -646,14 +646,15 @@ format_part_content_raw (GMimeObject *part)
 {
 GMimeStream *stream_stdout;
 GMimeStream *stream_filter = NULL;
-GMimeDataWrapper *wrapper;
+GMimeDataWrapper *wrapper = NULL;

 stream_stdout = g_mime_stream_file_new (stdout);
 g_mime_stream_file_set_owner (GMIME_STREAM_FILE (stream_stdout), FALSE);

 stream_filter = g_mime_stream_filter_new (stream_stdout);

-wrapper = g_mime_part_get_content_object (GMIME_PART (part));
+if (GMIME_IS_PART (part))
+   wrapper = g_mime_part_get_content_object (GMIME_PART (part));

 if (wrapper && stream_filter)
g_mime_data_wrapper_write_to_stream (wrapper, stream_filter);
-- 
1.7.4.4



Decryption fails

2011-06-02 Thread Felix Geller
On Thu, 02 Jun 2011 08:35:49 -0700, Jameson Graef Rollins  wrote:
Non-text part: multipart/signed

> Hey, Felix.  Yeah, I unfortunately don't have any other suggestions
> other than asking the gmime folks.

Jeff replied and sent me a working patch :) Not sure yet how he prefers
to publish the patch, but the problem is fixed.

> Interestingly, I see the following message in your crypto test output:
> 
>   Error: search term did not match precisely one message.
> 
> which seems to indicate that the desired message wasn't actually
> delivered properly, contrary to what the emacs delivery tests are
> stating.  Not sure how that could be related, though, since it looks
> like the trace that you show above definitely looks like gmime caught in
> a poll loop.

Just ran the tests with a "fixed" poll function and it seems that many
problems remain. Don't have the time at the moment to take a closer
look, but I attached the output.

Anyway, I'm a glad decrypting user now ;)


Cheers,
Felix

> jamie.
Non-text part: application/pgp-signature
-- next part --
An embedded and charset-unspecified text was scrubbed...
Name: crypto.txt
URL: 
<http://notmuchmail.org/pipermail/notmuch/attachments/20110602/c80ad35b/attachment-0001.txt>
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 202 bytes
Desc: not available
URL: 
<http://notmuchmail.org/pipermail/notmuch/attachments/20110602/c80ad35b/attachment-0001.pgp>


[PATCH] emacs: Fix handling of message/rfc822 parts.

2011-06-02 Thread Jameson Graef Rollins
After the recent part-handling overhaul, emacs handling of
message/rfc822 parts broke, not outputting the message contents.  This
"fixes" then handling as is, but there are still problems with how
notmuch is outputting message parts that needs to be addressed (for
instance, message headers are not being output at the moment).

This also indicates that we need a test for output of message parts in
emacs.
---
 emacs/notmuch-show.el |   27 ---
 1 files changed, 16 insertions(+), 11 deletions(-)

diff --git a/emacs/notmuch-show.el b/emacs/notmuch-show.el
index 9837c7a..714200a 100644
--- a/emacs/notmuch-show.el
+++ b/emacs/notmuch-show.el
@@ -516,21 +516,26 @@ current buffer, if possible."
   t)

 (defun notmuch-show-insert-part-message/rfc822 (msg part content-type nth 
depth declared-type)
-  (let* ((message-part (plist-get part :content))
-(inner-parts (plist-get message-part :content)))
-(notmuch-show-insert-part-header nth declared-type content-type nil)
-;; Override `notmuch-message-headers' to force `From' to be
-;; displayed.
-(let ((notmuch-message-headers '("From" "Subject" "To" "Cc" "Date")))
-  (notmuch-show-insert-headers (plist-get part :headers)))
-;; Blank line after headers to be compatible with the normal
-;; message display.
-(insert "\n")
+  (notmuch-show-insert-part-header nth declared-type content-type nil)
+  (let ((inner-parts (plist-get part :content))
+   (start (point)))
+(if (plist-member part :headers)
+   (progn
+ ;; Override `notmuch-message-headers' to force `From' to be
+ ;; displayed.
+ (let ((notmuch-message-headers '("From" "Subject" "To" "Cc" "Date")))
+   (notmuch-show-insert-headers (plist-get part :headers)))
+ ;; Blank line after headers to be compatible with the normal
+ ;; message display.
+ (insert "foo\n")))

 ;; Show all of the parts.
 (mapc (lambda (inner-part)
(notmuch-show-insert-bodypart msg inner-part depth))
- inner-parts))
+ inner-parts)
+
+(when notmuch-show-indent-multipart
+  (indent-rigidly start (point) 1)))
   t)

 (defun notmuch-show-insert-part-text/plain (msg part content-type nth depth 
declared-type)
-- 
1.7.4.4



Fwd: Re: compile error of current git on F15

2011-06-02 Thread Daniel Kahn Gillmor
I got the following response off-list from the gmime lead, which he's ok
with my re-posting here:

On 06/02/2011 08:53 AM, Jeffrey Stedfast wrote:
> I don't know why Fedora 15 ships 2.5.x, that seems like a really silly
> thing to do. I think I recall the Balsa guys bringing this up in the
> past and I thought I explained to the Fedora guys that 2.5.x was API
> unstable. Perhaps there was some misunderstanding or something.
> 
> That said, as far as timeline goes, I think 2.6 is basically ready for a
> final release, I've just been busy with other stuff lately. Assuming you
> don't have any major outstanding issues with the crypto stuff, I can
> just roll out a 2.6.0 release (which hopefully F15 will pick up, or, if
> not, just check gmime-2.6 >= 2.5.7) at any time.
> 
> I think notmuch is the only thing making heavy use of the 2.6 crypto
> stuff right now, so you guys have priority.

So this actually makes me re-think my earlier objections to patches
against an unstable API, on two grounds:

 0) if we're the driving force for 2.6 crypto, we should exercise it so
we can give reasonable feedback before stabilization happens.

 1) it sounds like the API is near stabilization anyway, so it won't be
too much hair-pulling, other than any changes we recommend.

So if we can just get 0.6 released, i'll commit to making patches to
support gmime 2.6 for the next version.

(i'll probably start by urging debian's gmime folks to put a version of
gmime2.6 into the experimental repo for us to play with)

--dkg

PS i still recommend that F15 should not ship gmime2.6 by default.  that
seems like a recipe for this kind of trouble.

-- next part --
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 1030 bytes
Desc: OpenPGP digital signature
URL: 
<http://notmuchmail.org/pipermail/notmuch/attachments/20110602/1c576422/attachment.pgp>


problems with message/rfc822 parts

2011-06-02 Thread Jameson Graef Rollins
Hey, folks.  I've been noticing a couple of issues with message/rfc822
part handling in recent builds of notmuch/master.  They are rooted in
the new part handling rework that was done recently.  I just want to
mention them here, to make people aware of them, until we get a chance
to address them.

The first issue is that show --format=raw --part on a message/rfc822
part is throwing a gmime exception, even though notmuch is still
producing output and returning successfully.  I just sent a patch to the
list addressing the issues.  However, the test suite was apparently
unable to catch that something unexpected was being output to stderr, so
we still need to address that.

Another issue is that we're no longer outputting any of the headers of
message/rfc822 parts.  This shouldn't be too hard to hack back in, so
hopefully I can get to that this weekend.

There was also a problem with the emacs display of message/rfc822 parts,
and I just sent a patch to the list about that as well.

However, there is a larger issue related to the ones above which is that
message parts in general are not being handled consistently, including
the top level message.  amdragon and I were actually discussing this on
irc the other day.  A possible solution is to make a general
message-part-handling function, that handles output of headers and the
message body, and call that function both on the top level message and
on any message sub parts.  I think that would make things a lot cleaner,
and would make the output more consistent and intuitive.

A somewhat related issue is that the "raw" format for parts is not
really doing the expected thing in certain cases, particularly with
regards to message parts.  dkg and I sort of realized this when we were
reworking the part handling a couple weeks ago.  One expects, rightfully
so, that show --format=raw --part on a message part would output the
full, raw, well-formatted rfc822 message enclosed in that part.
Currently that is not the case, since, for instance, the headers are not
output and the part handling is recursing over all the message
sub-parts.

I don't know necessarily know the right solution for either of these
issues, although I have some ideas (assuming I don't run into gmime
limitations).  Feedback and suggestions would be wonderful.
Unfortunately I don't have time to work on any of these issues at the
moment (hopefully this weekend), but I just wanted to throw them out
there, in case anyone wants to tackle them before I get to them.

jamie.
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL: 
<http://notmuchmail.org/pipermail/notmuch/attachments/20110602/f0f869ca/attachment.pgp>


Decryption fails

2011-06-02 Thread Jameson Graef Rollins
On Thu, 02 Jun 2011 18:49:22 +0200, Felix Geller  wrote:
> Jeff replied and sent me a working patch :) Not sure yet how he prefers
> to publish the patch, but the problem is fixed.

That's great!  What did Jeff say exactly?  Is the patch to gmime 2.4?
Did he mention that he was including them in upstream, hopefully in the
next release?

> > Interestingly, I see the following message in your crypto test output:
> > 
> >   Error: search term did not match precisely one message.
> > 
> > which seems to indicate that the desired message wasn't actually
> > delivered properly, contrary to what the emacs delivery tests are
> > stating.  Not sure how that could be related, though, since it looks
> > like the trace that you show above definitely looks like gmime caught in
> > a poll loop.
> 
> Just ran the tests with a "fixed" poll function and it seems that many
> problems remain. Don't have the time at the moment to take a closer
> look, but I attached the output.

Those test failures look very strange to me.  There are a bunch of 'n's
being output after the commas in the json output.  Felix, are you sure
you haven't modified your source at all?

jamie.
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL: 
<http://notmuchmail.org/pipermail/notmuch/attachments/20110602/dc1c3afe/attachment.pgp>


notmuch crashing on misformated emails

2011-06-02 Thread Patrick Totzke
This actually sounds very familiar. I had this error before 
but had different problems at the time. Will try to isolate 
this when it happens again.
/p

Excerpts from Sebastian Spaeth's message of Thu Jun 02 15:40:43 +0100 2011:
> On Thu, 02 Jun 2011 09:16:33 +0200, Sebastian Spaeth  SSpaeth.de> wrote:
> > in python:
> >   list(Threads())
> > 
> > leads to a program abortion (bombing out of python):
> > Internal error: Failed to read timestamp value from document. 
> > (lib/message.cc:731).
> > 
> > Can we not crash the app on misformated emails?
> 
> Just to follow up, I cannot reproduce this anymore. It works fine
> now. Sorry for the noise.
> 
> Sebastian
-- next part --
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: not available
URL: 
<http://notmuchmail.org/pipermail/notmuch/attachments/20110602/94db5326/attachment.pgp>
-- next part --
An embedded and charset-unspecified text was scrubbed...
Name: not available
URL: 
<http://notmuchmail.org/pipermail/notmuch/attachments/20110602/94db5326/attachment.ksh>