Fix for emacs 24

2011-09-15 Thread E.J. Neafsey
Dan,

What a flower!

Speaking of plants, John Banaszak and I worked together to identify a
rare prairie plant that he had found.  It was an Agalinis auriculata
(aka Tomanthera auriculata, or earleaf false foxglove).  It was a lot of
fun. I have attached a picture from the web.

Dad

-- next part --
A non-text attachment was scrubbed...
Name: agalinis_aur.jpg
Type: image/jpeg
Size: 99667 bytes
Desc: agalinis auriculata 
URL: 
<http://notmuchmail.org/pipermail/notmuch/attachments/20110915/229a34e8/attachment-0001.jpg>
-- next part --


On Mon, 01 Aug 2011 11:12:36 -0600, richardmurri at gmail.com wrote:
> 
> From c034e4567dc08f81708f6ab8ac974c9b8b83a508 Mon Sep 17 00:00:00 2001
> Date: Mon, 1 Aug 2011 10:48:31 -0600
> Subject: [PATCH] Update for emacs 24.
> 
> Emacs added another parameter to the send mail functions.  This caused
> emacs to error out.
> ---
>  emacs/notmuch-mua.el |3 ++-
>  1 files changed, 2 insertions(+), 1 deletions(-)
> 
> diff --git a/emacs/notmuch-mua.el b/emacs/notmuch-mua.el
> index 274c5da..808e184 100644
> --- a/emacs/notmuch-mua.el
> +++ b/emacs/notmuch-mua.el
> @@ -125,7 +125,8 @@ list."
>(message-goto-to))
>  
>  (defun notmuch-mua-mail (&optional to subject other-headers continue
> -switch-function yank-action send-actions)
> +switch-function yank-action send-actions
> +return-action)
>"Invoke the notmuch mail composition window."
>(interactive)
>  
> -- 
> 1.7.6
> 
> 
> ___
> notmuch mailing list
> notmuch at notmuchmail.org
> http://notmuchmail.org/mailman/listinfo/notmuch

-- 
E.J. Neafsey


Python Unicode Was: Not much database creation error

2011-09-15 Thread Austin Clements
This seems like a symptom of a much bigger problem.  Shouldn't the
bindings be checking or coercing the type of *every* argument that
gets passed from the caller through to a ctypes-wrapped libnotmuch
function?  Otherwise, a simple type error in a caller, like passing a
number instead of a string to one of the Python methods, would likely
result in a segfault.

On Thu, Sep 15, 2011 at 1:41 PM, Martin Owens  wrote:
> Hello Sebastian,
>
> I've attached a diff for some proposed changes to help make dealing with
> unicode and strings in the bindings more regular. I noticed some of the
> methods were protected and others were not.
>
> Let me know.
>
> Best Regards, Martin Owens
>
> On Thu, 2011-09-08 at 15:45 +0200, Sebastian Spaeth wrote:
>> I have to admit, I am not deep into the libnotmuch/xapian internals,
>> so
>> I might be the wrong person to ask (I just do some python bindings to
>> whatever libnotmuch provides).


Unicode Paths

2011-09-15 Thread Austin Clements
On Tue, Sep 13, 2011 at 11:55 PM, Martin Owens  wrote:
> Hello Again,
>
> I notice in the lib code notmuch_database_open(),
> notmuch_database_create() these functions use const char *path for the
> directory path input. Is this unicode safe?
>
> The python bindings (and ctype docs) seem to suggest using something
> called 'wchar_t *' for accepting unicode but that's for C not C++.
>
> Is this something that should be patched?

char* is the correct type for paths on POSIX systems.  The *meaning*
of those bytes is a more complicated matter and depends on your locale
settings.  On old systems it was generally ASCII, on modern systems
it's generally UTF-8, and it can be many other things.  However, as a
consequence of UNIX's C heritage, it is *always* terminated with a
NULL byte and cannot contain embedded NULL's.  Any encoding that
doesn't satisfy this would not be a valid encoding for file names (you
couldn't even pass such a file name to the open() system call, because
it expects a NULL-terminated byte string).

wchar_t is another matter entirely.  wchar_t is the type used by C to
represent wide strings internally, which generally (but not
necessarily!) means it stores a Unicode code point.  However, this
isn't an encoding, and different compilers can give wchar_t different
meanings, so wchar_t strings aren't generally appropriate for storing
or sharing between processes or with the kernel.


Python Unicode Was: Not much database creation error

2011-09-15 Thread Martin Owens
Hello Sebastian,

I've attached a diff for some proposed changes to help make dealing with
unicode and strings in the bindings more regular. I noticed some of the
methods were protected and others were not.

Let me know.

Best Regards, Martin Owens

On Thu, 2011-09-08 at 15:45 +0200, Sebastian Spaeth wrote:
> I have to admit, I am not deep into the libnotmuch/xapian internals,
> so
> I might be the wrong person to ask (I just do some python bindings to
> whatever libnotmuch provides). 
-- next part --
A non-text attachment was scrubbed...
Name: unicode.patch
Type: text/x-patch
Size: 6195 bytes
Desc: not available
URL: 
<http://notmuchmail.org/pipermail/notmuch/attachments/20110915/cd193020/attachment-0001.bin>


Unicode Paths

2011-09-15 Thread Martin Owens
It looks like the python variables do include null, my investigations
show that the problem also effects tag names.

The symptoms can be seen when trying to use the python interface and
using unicode tag names or paths. Instead of seeing 'mytag1' we see 'm'
and instead of '/my/path/to/mail' we see '/' thus causing issues were
the db amusingly was trying to write to root.

I'll see if there is a way to remove the nulls from the strings in the
python bindings.

Martin,

On Wed, 2011-09-14 at 12:38 +0800, Kan-Ru Chen wrote:
> I think as long as the path does not contain embedded null character
> then it is safe. Most posix filesystem does not allow null character
> in
> the filename so you cannot use UTF-16 or UTF-32 to encode the unicode
> path. 



Re: Python Unicode Was: Not much database creation error

2011-09-15 Thread Austin Clements
This seems like a symptom of a much bigger problem.  Shouldn't the
bindings be checking or coercing the type of *every* argument that
gets passed from the caller through to a ctypes-wrapped libnotmuch
function?  Otherwise, a simple type error in a caller, like passing a
number instead of a string to one of the Python methods, would likely
result in a segfault.

On Thu, Sep 15, 2011 at 1:41 PM, Martin Owens  wrote:
> Hello Sebastian,
>
> I've attached a diff for some proposed changes to help make dealing with
> unicode and strings in the bindings more regular. I noticed some of the
> methods were protected and others were not.
>
> Let me know.
>
> Best Regards, Martin Owens
>
> On Thu, 2011-09-08 at 15:45 +0200, Sebastian Spaeth wrote:
>> I have to admit, I am not deep into the libnotmuch/xapian internals,
>> so
>> I might be the wrong person to ask (I just do some python bindings to
>> whatever libnotmuch provides).
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Re: Unicode Paths

2011-09-15 Thread Austin Clements
On Tue, Sep 13, 2011 at 11:55 PM, Martin Owens  wrote:
> Hello Again,
>
> I notice in the lib code notmuch_database_open(),
> notmuch_database_create() these functions use const char *path for the
> directory path input. Is this unicode safe?
>
> The python bindings (and ctype docs) seem to suggest using something
> called 'wchar_t *' for accepting unicode but that's for C not C++.
>
> Is this something that should be patched?

char* is the correct type for paths on POSIX systems.  The *meaning*
of those bytes is a more complicated matter and depends on your locale
settings.  On old systems it was generally ASCII, on modern systems
it's generally UTF-8, and it can be many other things.  However, as a
consequence of UNIX's C heritage, it is *always* terminated with a
NULL byte and cannot contain embedded NULL's.  Any encoding that
doesn't satisfy this would not be a valid encoding for file names (you
couldn't even pass such a file name to the open() system call, because
it expects a NULL-terminated byte string).

wchar_t is another matter entirely.  wchar_t is the type used by C to
represent wide strings internally, which generally (but not
necessarily!) means it stores a Unicode code point.  However, this
isn't an encoding, and different compilers can give wchar_t different
meanings, so wchar_t strings aren't generally appropriate for storing
or sharing between processes or with the kernel.
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Python Unicode Was: Not much database creation error

2011-09-15 Thread Martin Owens
Hello Sebastian,

I've attached a diff for some proposed changes to help make dealing with
unicode and strings in the bindings more regular. I noticed some of the
methods were protected and others were not.

Let me know.

Best Regards, Martin Owens

On Thu, 2011-09-08 at 15:45 +0200, Sebastian Spaeth wrote:
> I have to admit, I am not deep into the libnotmuch/xapian internals,
> so
> I might be the wrong person to ask (I just do some python bindings to
> whatever libnotmuch provides). 
diff --git a/bindings/python/notmuch/database.py b/bindings/python/notmuch/database.py
index b070019..4f44de0 100644
--- a/bindings/python/notmuch/database.py
+++ b/bindings/python/notmuch/database.py
@@ -19,12 +19,11 @@ Copyright 2010 Sebastian Spaeth '
 
 import os
 from ctypes import c_int, c_char_p, c_void_p, c_uint, c_long, byref
-from notmuch.globals import nmlib, STATUS, NotmuchError, Enum
+from notmuch.globals import nmlib, STATUS, NotmuchError, Enum, _str
 from notmuch.thread import Threads
 from notmuch.message import Messages, Message
 from notmuch.tag import Tags
 
-
 class Database(object):
 """Represents a notmuch database (wraps notmuch_database_t)
 
@@ -105,7 +104,6 @@ class Database(object):
 Database._std_db_path = self._get_user_default_db()
 path = Database._std_db_path
 
-assert isinstance(path, basestring), 'Path must be a string or None.'
 if create == False:
 self.open(path, mode)
 else:
@@ -136,7 +134,7 @@ class Database(object):
 raise NotmuchError(message="Cannot create db, this Database() "
"already has an open one.")
 
-res = Database._create(path, Database.MODE.READ_WRITE)
+res = Database._create(_str(path), Database.MODE.READ_WRITE)
 
 if res is None:
 raise NotmuchError(
@@ -156,9 +154,7 @@ class Database(object):
 :exception: Raises :exc:`NotmuchError` in case
 of any failure (after printing an error message on stderr).
 """
-if isinstance(path, unicode):
-path = path.encode('utf-8') 
-res = Database._open(path, mode)
+res = Database._open(_str(path), mode)
 
 if res is None:
 raise NotmuchError(
@@ -263,12 +259,10 @@ class Database(object):
 #we got a relative path, make it absolute
 abs_dirpath = os.path.abspath(os.path.join(self.get_path(), path))
 
-if isinstance(path, unicode):
-path = path.encode('UTF-8')
-dir_p = Database._get_directory(self._db, path)
+dir_p = Database._get_directory(self._db, _str(path))
 
 # return the Directory, init it with the absolute path
-return Directory(abs_dirpath, dir_p, self)
+return Directory(_str(abs_dirpath), dir_p, self)
 
 def add_message(self, filename, sync_maildir_flags=False):
 """Adds a new message to the database
@@ -325,7 +319,7 @@ class Database(object):
 
 msg_p = c_void_p()
 status = nmlib.notmuch_database_add_message(self._db,
-  filename,
+  _str(filename),
   byref(msg_p))
 
 if not status in [STATUS.SUCCESS, STATUS.DUPLICATE_MESSAGE_ID]:
@@ -394,7 +388,7 @@ class Database(object):
 # Raise a NotmuchError if not initialized
 self._verify_initialized_db()
 
-msg_p = Database._find_message(self._db, msgid)
+msg_p = Database._find_message(self._db, _str(msgid))
 if msg_p is None:
 return None
 return Message(msg_p, self)
@@ -404,8 +398,7 @@ class Database(object):
with this filename.
 """
 self._verify_initialized_db()
-
-msg_p = Database._find_message_by_filename(self._db, filename)
+msg_p = Database._find_message_by_filename(self._db, _str(filename))
 return msg_p and Message(msg_p, self) or None
 
 def get_all_tags(self):
@@ -558,11 +551,8 @@ class Query(object):
 raise NotmuchError(STATUS.NOT_INITIALIZED)
 # create reference to parent db to keep it alive
 self._db = db
-if isinstance(querystr, unicode):
-# xapian takes utf-8 encoded byte arrays
-querystr = querystr.encode('utf-8')
 # create query, return None if too little mem available
-query_p = Query._create(db.db_p, querystr)
+query_p = Query._create(db.db_p, _str(querystr))
 if query_p is None:
 NotmuchError(STATUS.NULL_POINTER)
 self._query = query_p
diff --git a/bindings/python/notmuch/globals.py b/bindings/python/notmuch/globals.py
index 77f2905..5c81076 100644
--- a/bindings/python/notmuch/globals.py
+++ b/bindings/python/notmuch/globals.py
@@ -98,3 +98,13 @@ class NotmuchError(Exception):
 return self.args[0]
 else:
 re

Re: Unicode Paths

2011-09-15 Thread Martin Owens
It looks like the python variables do include null, my investigations
show that the problem also effects tag names.

The symptoms can be seen when trying to use the python interface and
using unicode tag names or paths. Instead of seeing 'mytag1' we see 'm'
and instead of '/my/path/to/mail' we see '/' thus causing issues were
the db amusingly was trying to write to root.

I'll see if there is a way to remove the nulls from the strings in the
python bindings.

Martin,

On Wed, 2011-09-14 at 12:38 +0800, Kan-Ru Chen wrote:
> I think as long as the path does not contain embedded null character
> then it is safe. Most posix filesystem does not allow null character
> in
> the filename so you cannot use UTF-16 or UTF-32 to encode the unicode
> path. 

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


[PATCH 4/4] tags_to_maildir_flags: Don't rename if no flags change

2011-09-15 Thread Louis Rilling
notmuch_message_tags_to_maildir_flags() unconditionally moves messages from
maildir directory "new/" to maildir directory "cur/", which makes messages lose
their "new" status in the MUA. However some users want to keep this "new"
status after, for instance, an auto-tagging of new messages.

However, as Austin mentioned and according to the maildir specification,
messages living in "new/" are not allowed to have flags, even if mutt allows it
to happen. For this reason, this patch prevents moving messages from "new/" to
"cur/", only if no flags have to be changed. It's hopefully enough to satisfy
mutt (and maybe other MUAs showing the "new" status) users checking the "new"
status.

Changelog:
* v2: Fix bool type as well as NULL returned despite having no errors (Austin
  Clements)
* v4: Tag the related test (contributed by Michal Sojka) as working

Signed-off-by: Louis Rilling 
---
 lib/message.cc|   12 +++-
 test/maildir-sync |2 +-
 2 files changed, 12 insertions(+), 2 deletions(-)

diff --git a/lib/message.cc b/lib/message.cc
index b1b2942..c003729 100644
--- a/lib/message.cc
+++ b/lib/message.cc
@@ -1139,7 +1139,7 @@ _get_maildir_flag_actions (notmuch_message_t *message,
  * compute the new maildir filename.
  *
  * If the existing filename is in the directory "new", the new
- * filename will be in the directory "cur".
+ * filename will be in the directory "cur", unless no flags are changed.
  *
  * After a sequence of ":2," in the filename, any subsequent
  * single-character flags will be added or removed according to the
@@ -1162,6 +1162,7 @@ _new_maildir_filename (void *ctx,
 char *filename_new, *dir;
 char flag_map[128];
 int flags_in_map = 0;
+notmuch_bool_t flags_changed = FALSE;
 unsigned int i;
 char *s;

@@ -1202,6 +1203,7 @@ _new_maildir_filename (void *ctx,
if (flag_map[flag] == 0) {
flag_map[flag] = 1;
flags_in_map++;
+   flags_changed = TRUE;
}
 }

@@ -1210,9 +1212,17 @@ _new_maildir_filename (void *ctx,
if (flag_map[flag]) {
flag_map[flag] = 0;
flags_in_map--;
+   flags_changed = TRUE;
}
 }

+/* No need to rename. Messages in new/ can be kept in new/.
+ * Note: We don't even try to fix buggy messages having flags and living in
+ * new/. It's not our business.
+ */
+if (!flags_changed)
+   return talloc_strdup (ctx, filename);
+
 filename_new = (char *) talloc_size (ctx,
 info - filename +
 strlen (":2,") + flags_in_map + 1);
diff --git a/test/maildir-sync b/test/maildir-sync
index b3e90ae..e1ad81c 100755
--- a/test/maildir-sync
+++ b/test/maildir-sync
@@ -92,7 +92,7 @@ test_begin_subtest "Adding non-maildir tags does not move 
message from new to cu
 add_message [subject]='"Message to stay in new"' [date]='"Sat, 01 Jan 2000 
12:00:00 -"' [filename]='message-to-stay-in-new' [dir]=new
 notmuch tag +donotmove subject:"Message to stay in new"
 output=$(cd "$MAIL_DIR"; ls */message-to-stay-in-new*)
-test_expect_equal_failure "$output" "new/message-to-stay-in-new"
+test_expect_equal "$output" "new/message-to-stay-in-new"

 test_begin_subtest "Removing 'S' flag from existing filename adds 'unread' tag"
 add_message [subject]='"Removing S flag"' [filename]='removing-s-flag:2,S' 
[dir]=cur
-- 
1.7.2.5



[PATCH 3/4] test: Adding non-maildir tags does not move message from new to cur

2011-09-15 Thread Louis Rilling
From: Michal Sojka 

Some MUA's like mutt show the difference between "new" emails living in maildir
directory new/, and "old" emails living in maildir directory cur/. However
notmuch tag unconditionally moves selected messages from new/ to cur/, even if
no maildir synchronized tag is changed.

While maildir specification forbids messages with tags living in new/, there is
no need to move messages to cur/ when no maildir synchronized tag is changed.
Thus notmuch can remain transparent with respect to other MUA's.

[ Edited commit log to better describe the intended changes, and tag the
  test as broken until the actual changes are implemented -- Louis Rilling ]

Signed-off-by: Louis Rilling 
---
 test/maildir-sync |6 ++
 1 files changed, 6 insertions(+), 0 deletions(-)

diff --git a/test/maildir-sync b/test/maildir-sync
index a60854f..b3e90ae 100755
--- a/test/maildir-sync
+++ b/test/maildir-sync
@@ -88,6 +88,12 @@ test_expect_equal "$output" "No new mail."
 # creating new directories in the mail store, then it should be
 # creating all necessary database state for those directories.

+test_begin_subtest "Adding non-maildir tags does not move message from new to 
cur"
+add_message [subject]='"Message to stay in new"' [date]='"Sat, 01 Jan 2000 
12:00:00 -"' [filename]='message-to-stay-in-new' [dir]=new
+notmuch tag +donotmove subject:"Message to stay in new"
+output=$(cd "$MAIL_DIR"; ls */message-to-stay-in-new*)
+test_expect_equal_failure "$output" "new/message-to-stay-in-new"
+
 test_begin_subtest "Removing 'S' flag from existing filename adds 'unread' tag"
 add_message [subject]='"Removing S flag"' [filename]='removing-s-flag:2,S' 
[dir]=cur
 output=$(notmuch search subject:"Removing S flag" | notmuch_search_sanitize)
-- 
1.7.2.5



[PATCH 2/4] tags_to_maildir_flags: Cleanup double assignement

2011-09-15 Thread Louis Rilling
The for loop right after already does the job.

Signed-off-by: Louis Rilling 
---
 lib/message.cc |2 --
 1 files changed, 0 insertions(+), 2 deletions(-)

diff --git a/lib/message.cc b/lib/message.cc
index cf651e5..b1b2942 100644
--- a/lib/message.cc
+++ b/lib/message.cc
@@ -1172,8 +1172,6 @@ _new_maildir_filename (void *ctx,
 if (info == NULL) {
info = filename + strlen(filename);
 } else {
-   flags = info + 3;
-
/* Loop through existing flags in filename. */
for (flags = info + 3, last_flag = 0;
 *flags;
-- 
1.7.2.5



[PATCH 1/4] lib: Kill last usage of C++ type bool

2011-09-15 Thread Louis Rilling
Signed-off-by: Louis Rilling 
---
 lib/message.cc |   12 ++--
 1 files changed, 6 insertions(+), 6 deletions(-)

diff --git a/lib/message.cc b/lib/message.cc
index d993cde..cf651e5 100644
--- a/lib/message.cc
+++ b/lib/message.cc
@@ -49,16 +49,16 @@ struct visible _notmuch_message {
 struct maildir_flag_tag {
 char flag;
 const char *tag;
-bool inverse;
+notmuch_bool_t inverse;
 };

 /* ASCII ordered table of Maildir flags and associated tags */
 static struct maildir_flag_tag flag2tag[] = {
-{ 'D', "draft",   false},
-{ 'F', "flagged", false},
-{ 'P', "passed",  false},
-{ 'R', "replied", false},
-{ 'S', "unread",  true }
+{ 'D', "draft",   FALSE},
+{ 'F', "flagged", FALSE},
+{ 'P', "passed",  FALSE},
+{ 'R', "replied", FALSE},
+{ 'S', "unread",  TRUE }
 };

 /* We end up having to call the destructor explicitly because we had
-- 
1.7.2.5



[PATCH 0/4 v4] lib: Maildir flags synchronization fixes

2011-09-15 Thread Louis Rilling
Hello,

Here is the updated series of fixes I have around maildir flags
synchronization. The first two patches are just cleanups that can be applied
independently.

The intent for the fourth patch (detailed in the commit log) is to allow mutt
users to keep using the "new" status, as long as notmuch can respect the
maildir specification.

The third patch implements a test for the new desired behavior. From recent
discussions I decided to put it before the actual changes, but it certainly can
move after if this is preferred.

Changelog:
* v4: 
 - rebased on top of release 0.8
 - included the test contributed by Michal Sojka
* v3: Added patch to kill the last usage of C++ type bool
* v2: Fix bool type as well as NULL returned despite having no errors (Austin
  Clements)

Thanks,

Louis


Louis Rilling (3):
  lib: Kill last usage of C++ type bool
  tags_to_maildir_flags: Cleanup double assignement
  tags_to_maildir_flags: Don't rename if no flags change

Michal Sojka (1):
  test: Adding non-maildir tags does not move message from new to cur

 lib/message.cc|   26 +-
 test/maildir-sync |6 ++
 2 files changed, 23 insertions(+), 9 deletions(-)