Fix for emacs 24
Dan, What a flower! Speaking of plants, John Banaszak and I worked together to identify a rare prairie plant that he had found. It was an Agalinis auriculata (aka Tomanthera auriculata, or earleaf false foxglove). It was a lot of fun. I have attached a picture from the web. Dad -- next part -- A non-text attachment was scrubbed... Name: agalinis_aur.jpg Type: image/jpeg Size: 99667 bytes Desc: agalinis auriculata URL: <http://notmuchmail.org/pipermail/notmuch/attachments/20110915/229a34e8/attachment-0001.jpg> -- next part -- On Mon, 01 Aug 2011 11:12:36 -0600, richardmurri at gmail.com wrote: > > From c034e4567dc08f81708f6ab8ac974c9b8b83a508 Mon Sep 17 00:00:00 2001 > Date: Mon, 1 Aug 2011 10:48:31 -0600 > Subject: [PATCH] Update for emacs 24. > > Emacs added another parameter to the send mail functions. This caused > emacs to error out. > --- > emacs/notmuch-mua.el |3 ++- > 1 files changed, 2 insertions(+), 1 deletions(-) > > diff --git a/emacs/notmuch-mua.el b/emacs/notmuch-mua.el > index 274c5da..808e184 100644 > --- a/emacs/notmuch-mua.el > +++ b/emacs/notmuch-mua.el > @@ -125,7 +125,8 @@ list." >(message-goto-to)) > > (defun notmuch-mua-mail (&optional to subject other-headers continue > -switch-function yank-action send-actions) > +switch-function yank-action send-actions > +return-action) >"Invoke the notmuch mail composition window." >(interactive) > > -- > 1.7.6 > > > ___ > notmuch mailing list > notmuch at notmuchmail.org > http://notmuchmail.org/mailman/listinfo/notmuch -- E.J. Neafsey
Python Unicode Was: Not much database creation error
This seems like a symptom of a much bigger problem. Shouldn't the bindings be checking or coercing the type of *every* argument that gets passed from the caller through to a ctypes-wrapped libnotmuch function? Otherwise, a simple type error in a caller, like passing a number instead of a string to one of the Python methods, would likely result in a segfault. On Thu, Sep 15, 2011 at 1:41 PM, Martin Owens wrote: > Hello Sebastian, > > I've attached a diff for some proposed changes to help make dealing with > unicode and strings in the bindings more regular. I noticed some of the > methods were protected and others were not. > > Let me know. > > Best Regards, Martin Owens > > On Thu, 2011-09-08 at 15:45 +0200, Sebastian Spaeth wrote: >> I have to admit, I am not deep into the libnotmuch/xapian internals, >> so >> I might be the wrong person to ask (I just do some python bindings to >> whatever libnotmuch provides).
Unicode Paths
On Tue, Sep 13, 2011 at 11:55 PM, Martin Owens wrote: > Hello Again, > > I notice in the lib code notmuch_database_open(), > notmuch_database_create() these functions use const char *path for the > directory path input. Is this unicode safe? > > The python bindings (and ctype docs) seem to suggest using something > called 'wchar_t *' for accepting unicode but that's for C not C++. > > Is this something that should be patched? char* is the correct type for paths on POSIX systems. The *meaning* of those bytes is a more complicated matter and depends on your locale settings. On old systems it was generally ASCII, on modern systems it's generally UTF-8, and it can be many other things. However, as a consequence of UNIX's C heritage, it is *always* terminated with a NULL byte and cannot contain embedded NULL's. Any encoding that doesn't satisfy this would not be a valid encoding for file names (you couldn't even pass such a file name to the open() system call, because it expects a NULL-terminated byte string). wchar_t is another matter entirely. wchar_t is the type used by C to represent wide strings internally, which generally (but not necessarily!) means it stores a Unicode code point. However, this isn't an encoding, and different compilers can give wchar_t different meanings, so wchar_t strings aren't generally appropriate for storing or sharing between processes or with the kernel.
Python Unicode Was: Not much database creation error
Hello Sebastian, I've attached a diff for some proposed changes to help make dealing with unicode and strings in the bindings more regular. I noticed some of the methods were protected and others were not. Let me know. Best Regards, Martin Owens On Thu, 2011-09-08 at 15:45 +0200, Sebastian Spaeth wrote: > I have to admit, I am not deep into the libnotmuch/xapian internals, > so > I might be the wrong person to ask (I just do some python bindings to > whatever libnotmuch provides). -- next part -- A non-text attachment was scrubbed... Name: unicode.patch Type: text/x-patch Size: 6195 bytes Desc: not available URL: <http://notmuchmail.org/pipermail/notmuch/attachments/20110915/cd193020/attachment-0001.bin>
Unicode Paths
It looks like the python variables do include null, my investigations show that the problem also effects tag names. The symptoms can be seen when trying to use the python interface and using unicode tag names or paths. Instead of seeing 'mytag1' we see 'm' and instead of '/my/path/to/mail' we see '/' thus causing issues were the db amusingly was trying to write to root. I'll see if there is a way to remove the nulls from the strings in the python bindings. Martin, On Wed, 2011-09-14 at 12:38 +0800, Kan-Ru Chen wrote: > I think as long as the path does not contain embedded null character > then it is safe. Most posix filesystem does not allow null character > in > the filename so you cannot use UTF-16 or UTF-32 to encode the unicode > path.
Re: Python Unicode Was: Not much database creation error
This seems like a symptom of a much bigger problem. Shouldn't the bindings be checking or coercing the type of *every* argument that gets passed from the caller through to a ctypes-wrapped libnotmuch function? Otherwise, a simple type error in a caller, like passing a number instead of a string to one of the Python methods, would likely result in a segfault. On Thu, Sep 15, 2011 at 1:41 PM, Martin Owens wrote: > Hello Sebastian, > > I've attached a diff for some proposed changes to help make dealing with > unicode and strings in the bindings more regular. I noticed some of the > methods were protected and others were not. > > Let me know. > > Best Regards, Martin Owens > > On Thu, 2011-09-08 at 15:45 +0200, Sebastian Spaeth wrote: >> I have to admit, I am not deep into the libnotmuch/xapian internals, >> so >> I might be the wrong person to ask (I just do some python bindings to >> whatever libnotmuch provides). ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
Re: Unicode Paths
On Tue, Sep 13, 2011 at 11:55 PM, Martin Owens wrote: > Hello Again, > > I notice in the lib code notmuch_database_open(), > notmuch_database_create() these functions use const char *path for the > directory path input. Is this unicode safe? > > The python bindings (and ctype docs) seem to suggest using something > called 'wchar_t *' for accepting unicode but that's for C not C++. > > Is this something that should be patched? char* is the correct type for paths on POSIX systems. The *meaning* of those bytes is a more complicated matter and depends on your locale settings. On old systems it was generally ASCII, on modern systems it's generally UTF-8, and it can be many other things. However, as a consequence of UNIX's C heritage, it is *always* terminated with a NULL byte and cannot contain embedded NULL's. Any encoding that doesn't satisfy this would not be a valid encoding for file names (you couldn't even pass such a file name to the open() system call, because it expects a NULL-terminated byte string). wchar_t is another matter entirely. wchar_t is the type used by C to represent wide strings internally, which generally (but not necessarily!) means it stores a Unicode code point. However, this isn't an encoding, and different compilers can give wchar_t different meanings, so wchar_t strings aren't generally appropriate for storing or sharing between processes or with the kernel. ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
Python Unicode Was: Not much database creation error
Hello Sebastian,
I've attached a diff for some proposed changes to help make dealing with
unicode and strings in the bindings more regular. I noticed some of the
methods were protected and others were not.
Let me know.
Best Regards, Martin Owens
On Thu, 2011-09-08 at 15:45 +0200, Sebastian Spaeth wrote:
> I have to admit, I am not deep into the libnotmuch/xapian internals,
> so
> I might be the wrong person to ask (I just do some python bindings to
> whatever libnotmuch provides).
diff --git a/bindings/python/notmuch/database.py b/bindings/python/notmuch/database.py
index b070019..4f44de0 100644
--- a/bindings/python/notmuch/database.py
+++ b/bindings/python/notmuch/database.py
@@ -19,12 +19,11 @@ Copyright 2010 Sebastian Spaeth '
import os
from ctypes import c_int, c_char_p, c_void_p, c_uint, c_long, byref
-from notmuch.globals import nmlib, STATUS, NotmuchError, Enum
+from notmuch.globals import nmlib, STATUS, NotmuchError, Enum, _str
from notmuch.thread import Threads
from notmuch.message import Messages, Message
from notmuch.tag import Tags
-
class Database(object):
"""Represents a notmuch database (wraps notmuch_database_t)
@@ -105,7 +104,6 @@ class Database(object):
Database._std_db_path = self._get_user_default_db()
path = Database._std_db_path
-assert isinstance(path, basestring), 'Path must be a string or None.'
if create == False:
self.open(path, mode)
else:
@@ -136,7 +134,7 @@ class Database(object):
raise NotmuchError(message="Cannot create db, this Database() "
"already has an open one.")
-res = Database._create(path, Database.MODE.READ_WRITE)
+res = Database._create(_str(path), Database.MODE.READ_WRITE)
if res is None:
raise NotmuchError(
@@ -156,9 +154,7 @@ class Database(object):
:exception: Raises :exc:`NotmuchError` in case
of any failure (after printing an error message on stderr).
"""
-if isinstance(path, unicode):
-path = path.encode('utf-8')
-res = Database._open(path, mode)
+res = Database._open(_str(path), mode)
if res is None:
raise NotmuchError(
@@ -263,12 +259,10 @@ class Database(object):
#we got a relative path, make it absolute
abs_dirpath = os.path.abspath(os.path.join(self.get_path(), path))
-if isinstance(path, unicode):
-path = path.encode('UTF-8')
-dir_p = Database._get_directory(self._db, path)
+dir_p = Database._get_directory(self._db, _str(path))
# return the Directory, init it with the absolute path
-return Directory(abs_dirpath, dir_p, self)
+return Directory(_str(abs_dirpath), dir_p, self)
def add_message(self, filename, sync_maildir_flags=False):
"""Adds a new message to the database
@@ -325,7 +319,7 @@ class Database(object):
msg_p = c_void_p()
status = nmlib.notmuch_database_add_message(self._db,
- filename,
+ _str(filename),
byref(msg_p))
if not status in [STATUS.SUCCESS, STATUS.DUPLICATE_MESSAGE_ID]:
@@ -394,7 +388,7 @@ class Database(object):
# Raise a NotmuchError if not initialized
self._verify_initialized_db()
-msg_p = Database._find_message(self._db, msgid)
+msg_p = Database._find_message(self._db, _str(msgid))
if msg_p is None:
return None
return Message(msg_p, self)
@@ -404,8 +398,7 @@ class Database(object):
with this filename.
"""
self._verify_initialized_db()
-
-msg_p = Database._find_message_by_filename(self._db, filename)
+msg_p = Database._find_message_by_filename(self._db, _str(filename))
return msg_p and Message(msg_p, self) or None
def get_all_tags(self):
@@ -558,11 +551,8 @@ class Query(object):
raise NotmuchError(STATUS.NOT_INITIALIZED)
# create reference to parent db to keep it alive
self._db = db
-if isinstance(querystr, unicode):
-# xapian takes utf-8 encoded byte arrays
-querystr = querystr.encode('utf-8')
# create query, return None if too little mem available
-query_p = Query._create(db.db_p, querystr)
+query_p = Query._create(db.db_p, _str(querystr))
if query_p is None:
NotmuchError(STATUS.NULL_POINTER)
self._query = query_p
diff --git a/bindings/python/notmuch/globals.py b/bindings/python/notmuch/globals.py
index 77f2905..5c81076 100644
--- a/bindings/python/notmuch/globals.py
+++ b/bindings/python/notmuch/globals.py
@@ -98,3 +98,13 @@ class NotmuchError(Exception):
return self.args[0]
else:
re
Re: Unicode Paths
It looks like the python variables do include null, my investigations show that the problem also effects tag names. The symptoms can be seen when trying to use the python interface and using unicode tag names or paths. Instead of seeing 'mytag1' we see 'm' and instead of '/my/path/to/mail' we see '/' thus causing issues were the db amusingly was trying to write to root. I'll see if there is a way to remove the nulls from the strings in the python bindings. Martin, On Wed, 2011-09-14 at 12:38 +0800, Kan-Ru Chen wrote: > I think as long as the path does not contain embedded null character > then it is safe. Most posix filesystem does not allow null character > in > the filename so you cannot use UTF-16 or UTF-32 to encode the unicode > path. ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
[PATCH 4/4] tags_to_maildir_flags: Don't rename if no flags change
notmuch_message_tags_to_maildir_flags() unconditionally moves messages from
maildir directory "new/" to maildir directory "cur/", which makes messages lose
their "new" status in the MUA. However some users want to keep this "new"
status after, for instance, an auto-tagging of new messages.
However, as Austin mentioned and according to the maildir specification,
messages living in "new/" are not allowed to have flags, even if mutt allows it
to happen. For this reason, this patch prevents moving messages from "new/" to
"cur/", only if no flags have to be changed. It's hopefully enough to satisfy
mutt (and maybe other MUAs showing the "new" status) users checking the "new"
status.
Changelog:
* v2: Fix bool type as well as NULL returned despite having no errors (Austin
Clements)
* v4: Tag the related test (contributed by Michal Sojka) as working
Signed-off-by: Louis Rilling
---
lib/message.cc| 12 +++-
test/maildir-sync |2 +-
2 files changed, 12 insertions(+), 2 deletions(-)
diff --git a/lib/message.cc b/lib/message.cc
index b1b2942..c003729 100644
--- a/lib/message.cc
+++ b/lib/message.cc
@@ -1139,7 +1139,7 @@ _get_maildir_flag_actions (notmuch_message_t *message,
* compute the new maildir filename.
*
* If the existing filename is in the directory "new", the new
- * filename will be in the directory "cur".
+ * filename will be in the directory "cur", unless no flags are changed.
*
* After a sequence of ":2," in the filename, any subsequent
* single-character flags will be added or removed according to the
@@ -1162,6 +1162,7 @@ _new_maildir_filename (void *ctx,
char *filename_new, *dir;
char flag_map[128];
int flags_in_map = 0;
+notmuch_bool_t flags_changed = FALSE;
unsigned int i;
char *s;
@@ -1202,6 +1203,7 @@ _new_maildir_filename (void *ctx,
if (flag_map[flag] == 0) {
flag_map[flag] = 1;
flags_in_map++;
+ flags_changed = TRUE;
}
}
@@ -1210,9 +1212,17 @@ _new_maildir_filename (void *ctx,
if (flag_map[flag]) {
flag_map[flag] = 0;
flags_in_map--;
+ flags_changed = TRUE;
}
}
+/* No need to rename. Messages in new/ can be kept in new/.
+ * Note: We don't even try to fix buggy messages having flags and living in
+ * new/. It's not our business.
+ */
+if (!flags_changed)
+ return talloc_strdup (ctx, filename);
+
filename_new = (char *) talloc_size (ctx,
info - filename +
strlen (":2,") + flags_in_map + 1);
diff --git a/test/maildir-sync b/test/maildir-sync
index b3e90ae..e1ad81c 100755
--- a/test/maildir-sync
+++ b/test/maildir-sync
@@ -92,7 +92,7 @@ test_begin_subtest "Adding non-maildir tags does not move
message from new to cu
add_message [subject]='"Message to stay in new"' [date]='"Sat, 01 Jan 2000
12:00:00 -"' [filename]='message-to-stay-in-new' [dir]=new
notmuch tag +donotmove subject:"Message to stay in new"
output=$(cd "$MAIL_DIR"; ls */message-to-stay-in-new*)
-test_expect_equal_failure "$output" "new/message-to-stay-in-new"
+test_expect_equal "$output" "new/message-to-stay-in-new"
test_begin_subtest "Removing 'S' flag from existing filename adds 'unread' tag"
add_message [subject]='"Removing S flag"' [filename]='removing-s-flag:2,S'
[dir]=cur
--
1.7.2.5
[PATCH 3/4] test: Adding non-maildir tags does not move message from new to cur
From: Michal Sojka Some MUA's like mutt show the difference between "new" emails living in maildir directory new/, and "old" emails living in maildir directory cur/. However notmuch tag unconditionally moves selected messages from new/ to cur/, even if no maildir synchronized tag is changed. While maildir specification forbids messages with tags living in new/, there is no need to move messages to cur/ when no maildir synchronized tag is changed. Thus notmuch can remain transparent with respect to other MUA's. [ Edited commit log to better describe the intended changes, and tag the test as broken until the actual changes are implemented -- Louis Rilling ] Signed-off-by: Louis Rilling --- test/maildir-sync |6 ++ 1 files changed, 6 insertions(+), 0 deletions(-) diff --git a/test/maildir-sync b/test/maildir-sync index a60854f..b3e90ae 100755 --- a/test/maildir-sync +++ b/test/maildir-sync @@ -88,6 +88,12 @@ test_expect_equal "$output" "No new mail." # creating new directories in the mail store, then it should be # creating all necessary database state for those directories. +test_begin_subtest "Adding non-maildir tags does not move message from new to cur" +add_message [subject]='"Message to stay in new"' [date]='"Sat, 01 Jan 2000 12:00:00 -"' [filename]='message-to-stay-in-new' [dir]=new +notmuch tag +donotmove subject:"Message to stay in new" +output=$(cd "$MAIL_DIR"; ls */message-to-stay-in-new*) +test_expect_equal_failure "$output" "new/message-to-stay-in-new" + test_begin_subtest "Removing 'S' flag from existing filename adds 'unread' tag" add_message [subject]='"Removing S flag"' [filename]='removing-s-flag:2,S' [dir]=cur output=$(notmuch search subject:"Removing S flag" | notmuch_search_sanitize) -- 1.7.2.5
[PATCH 2/4] tags_to_maildir_flags: Cleanup double assignement
The for loop right after already does the job.
Signed-off-by: Louis Rilling
---
lib/message.cc |2 --
1 files changed, 0 insertions(+), 2 deletions(-)
diff --git a/lib/message.cc b/lib/message.cc
index cf651e5..b1b2942 100644
--- a/lib/message.cc
+++ b/lib/message.cc
@@ -1172,8 +1172,6 @@ _new_maildir_filename (void *ctx,
if (info == NULL) {
info = filename + strlen(filename);
} else {
- flags = info + 3;
-
/* Loop through existing flags in filename. */
for (flags = info + 3, last_flag = 0;
*flags;
--
1.7.2.5
[PATCH 1/4] lib: Kill last usage of C++ type bool
Signed-off-by: Louis Rilling
---
lib/message.cc | 12 ++--
1 files changed, 6 insertions(+), 6 deletions(-)
diff --git a/lib/message.cc b/lib/message.cc
index d993cde..cf651e5 100644
--- a/lib/message.cc
+++ b/lib/message.cc
@@ -49,16 +49,16 @@ struct visible _notmuch_message {
struct maildir_flag_tag {
char flag;
const char *tag;
-bool inverse;
+notmuch_bool_t inverse;
};
/* ASCII ordered table of Maildir flags and associated tags */
static struct maildir_flag_tag flag2tag[] = {
-{ 'D', "draft", false},
-{ 'F', "flagged", false},
-{ 'P', "passed", false},
-{ 'R', "replied", false},
-{ 'S', "unread", true }
+{ 'D', "draft", FALSE},
+{ 'F', "flagged", FALSE},
+{ 'P', "passed", FALSE},
+{ 'R', "replied", FALSE},
+{ 'S', "unread", TRUE }
};
/* We end up having to call the destructor explicitly because we had
--
1.7.2.5
[PATCH 0/4 v4] lib: Maildir flags synchronization fixes
Hello, Here is the updated series of fixes I have around maildir flags synchronization. The first two patches are just cleanups that can be applied independently. The intent for the fourth patch (detailed in the commit log) is to allow mutt users to keep using the "new" status, as long as notmuch can respect the maildir specification. The third patch implements a test for the new desired behavior. From recent discussions I decided to put it before the actual changes, but it certainly can move after if this is preferred. Changelog: * v4: - rebased on top of release 0.8 - included the test contributed by Michal Sojka * v3: Added patch to kill the last usage of C++ type bool * v2: Fix bool type as well as NULL returned despite having no errors (Austin Clements) Thanks, Louis Louis Rilling (3): lib: Kill last usage of C++ type bool tags_to_maildir_flags: Cleanup double assignement tags_to_maildir_flags: Don't rename if no flags change Michal Sojka (1): test: Adding non-maildir tags does not move message from new to cur lib/message.cc| 26 +- test/maildir-sync |6 ++ 2 files changed, 23 insertions(+), 9 deletions(-)
