[PATCH] test: optionally print subtest number
On Wed, 14 Dec 2011 00:58:35 +0400, Dmitry Kurochkin wrote: > On Tue, 13 Dec 2011 22:55:18 +0200, Tomi Ollila wrote: > > On Wed, 14 Dec 2011 00:15:43 +0400, Dmitry Kurochkin > gmail.com> wrote: > > > > > > Looks good to me. Except for tabs taking too much space. Perhaps the > > > following would be better? > > > > > > printf "%-4s" "[$(($test_count - 1))]" > > > > I attempted the same size reduction. Therefore I know that > > this should to be either > > > > printf " %-4s" "[$(($test_count - 1))]" > > or > > printf "\t%-4s" "[$(($test_count - 1))]" > > > > (or something similar) so that there is space betweem BROKEN and [num] > > > > Oh, apparently, not enough testing on my side. > > I vote for the first version with a space. Have to agree, Now that I checked tabs were not used in output lines before this addition. > > Regards, > Dmitry Tomi > > > This takes 4 bytes out from width (and drops tab as field separator) > > (and only few lines goes over 80 char width (some goes even with this > > reduction). So ... > > > > > > > Regards, > > > Dmitry > > > > Tomi
[PATCH] test: optionally print subtest number
On Wed, 14 Dec 2011 00:15:43 +0400, Dmitry Kurochkin wrote: > On Tue, 13 Dec 2011 15:56:47 -0400, David Bremner > wrote: > > From: David Bremner > > > > The idea is that $test_count could be used in tests to label > > intermediate files. The output enabled by this patch (and --debug) > > helps figure out which OUTPUT.nn file belongs to which test in case > > several subtests write to OUTPUT.$test_count > > --- > > > > Is there something that depends on the test format? I find it pretty > > handy to have the subtest numbers, but I don't want to break some > > other tools. I followed the existing style of conditionally defining > > functions, but maybe someone with more bash-fu can improve that. > > > > Looks good to me. Except for tabs taking too much space. Perhaps the > following would be better? > > printf "%-4s" "[$(($test_count - 1))]" I attempted the same size reduction. Therefore I know that this should to be either printf " %-4s" "[$(($test_count - 1))]" or printf "\t%-4s" "[$(($test_count - 1))]" (or something similar) so that there is space betweem BROKEN and [num] This takes 4 bytes out from width (and drops tab as field separator) (and only few lines goes over 80 char width (some goes even with this reduction). So ... > Regards, > Dmitry Tomi
[PATCH] test: optionally print subtest number
On Tue, 13 Dec 2011 15:56:47 -0400, David Bremner wrote:
> From: David Bremner
>
> The idea is that $test_count could be used in tests to label
> intermediate files. The output enabled by this patch (and --debug)
> helps figure out which OUTPUT.nn file belongs to which test in case
> several subtests write to OUTPUT.$test_count
> ---
>
> Is there something that depends on the test format? I find it pretty
> handy to have the subtest numbers, but I don't want to break some
> other tools. I followed the existing style of conditionally defining
> functions, but maybe someone with more bash-fu can improve that.
+1. I would also say '+1' even non-debug format (and update dependend code,
if there is any).
Tomi
>
> test/test-lib.sh | 12
> 1 files changed, 12 insertions(+), 0 deletions(-)
>
> diff --git a/test/test-lib.sh b/test/test-lib.sh
> index 6be93fe..f5c 100644
> --- a/test/test-lib.sh
> +++ b/test/test-lib.sh
> @@ -116,6 +116,16 @@ do
> esac
> done
>
> +if test -n "$debug"; then
> +print_subtest () {
> + printf "\t[%d]\t" $(($test_count - 1))
> +}
> +else
> +print_subtest () {
> + true
> +}
> +fi
> +
> if test -n "$color"; then
> say_color () {
> (
> @@ -132,6 +142,7 @@ if test -n "$color"; then
> printf " "
> printf "$@"
> tput sgr0
> + print_subtest
> )
> }
> else
> @@ -140,6 +151,7 @@ else
> shift
> printf " "
> printf "$@"
> + print_subtest
> }
> fi
>
> --
> 1.7.5.4
>
> ___
> notmuch mailing list
> notmuch at notmuchmail.org
> http://notmuchmail.org/mailman/listinfo/notmuch
>
--
Tomi Ollila !! tomi.ollila at nixu.fi !! +358 400 888 366
Nixu Oy !! http://www.nixu.fi/ !! Keilaranta 15, FI-02150 Espoo, Finland
[PATCH] notmuch: unref charset_filter (once more) to fix one memory leak
On Tue, 13 Dec 2011 20:18:48 +0200, tomi.ollila at iki.fi wrote:
> From: Tomi Ollila
>
> In my test case added g_object_unref(charset_filter) reduces memory
> consumption over 90% when 'notmuch show --format=text "*"' is
> executed (~11000 messages, RES ~330M -> ~25M).
> ---
> Thanks Dmitry. I did not realize unref unreferences, does not deallocate
> memory/data structures. g_mime_stream_filter_add() takes an additional
> reference to the charset_filter object.
Looks good to me.
Regards,
Dmitry
> notmuch-show.c |5 -
> 1 files changed, 4 insertions(+), 1 deletions(-)
>
> diff --git a/notmuch-show.c b/notmuch-show.c
> index 873a7c4..8da3295 100644
> --- a/notmuch-show.c
> +++ b/notmuch-show.c
> @@ -471,9 +471,12 @@ show_text_part_content (GMimeObject *part, GMimeStream
> *stream_out)
> /* This result can be NULL for things like "unknown-8bit".
>* Don't set a NULL filter as that makes GMime print
>* annoying assertion-failure messages on stderr. */
> - if (charset_filter)
> + if (charset_filter) {
> g_mime_stream_filter_add (GMIME_STREAM_FILTER (stream_filter),
> charset_filter);
> + g_object_unref (charset_filter);
> + }
> +
> }
>
> wrapper = g_mime_part_get_content_object (GMIME_PART (part));
> --
> 1.7.6.1
>
> ___
> notmuch mailing list
> notmuch at notmuchmail.org
> http://notmuchmail.org/mailman/listinfo/notmuch
[PATCH v3 4/4] emacs: add notmuch-hello-hook
On Tue, 13 Dec 2011 18:32:12 +0100, Thomas Jost wrote: > This hook is called every time the notmuch-hello buffer is updated. > --- > emacs/notmuch-hello.el |9 - > 1 files changed, 8 insertions(+), 1 deletions(-) > > diff --git a/emacs/notmuch-hello.el b/emacs/notmuch-hello.el > index 0fe9c1d..112b40b 100644 > --- a/emacs/notmuch-hello.el > +++ b/emacs/notmuch-hello.el > @@ -131,6 +131,11 @@ So: > (integer :tag "Number of characters") > (float :tag "Fraction of window"))) > > +(defcustom notmuch-hello-hook nil > + "Functions called after populating a `notmuch-hello' buffer." > + :type 'hook > + :group 'notmuch) > + > (defcustom notmuch-thousands-separator "," >"The string used as a thousands separator. > > @@ -579,7 +584,9 @@ Complete list of currently available key bindings: > (widget-forward 1))) > >(unless (widget-at) > - (notmuch-hello-goto-search) > + (notmuch-hello-goto-search)) > + > + (run-hooks 'notmuch-hello-hook > I spent some time finding out why run-hooks are not on the top level. Turns out it is inside two let statements. Can we move to the top level of `notmuch-hello' to make it clear that it is always run (i.e. there are no ifs or whens)? BTW this would replace one of the oldest custom patches which I have in my master branch :) Regards, Dmitry > (defun notmuch-folder () >"Deprecated function for invoking notmuch---calling `notmuch' is preferred > now." > -- > 1.7.8 > > ___ > notmuch mailing list > notmuch at notmuchmail.org > http://notmuchmail.org/mailman/listinfo/notmuch
[PATCH] notmuch: unref charset_filter (once more) to fix one memory leak
From: Tomi Ollila
In my test case added g_object_unref(charset_filter) reduces memory
consumption over 90% when 'notmuch show --format=text "*"' is
executed (~11000 messages, RES ~330M -> ~25M).
---
Thanks Dmitry. I did not realize unref unreferences, does not deallocate
memory/data structures. g_mime_stream_filter_add() takes an additional
reference to the charset_filter object.
notmuch-show.c |5 -
1 files changed, 4 insertions(+), 1 deletions(-)
diff --git a/notmuch-show.c b/notmuch-show.c
index 873a7c4..8da3295 100644
--- a/notmuch-show.c
+++ b/notmuch-show.c
@@ -471,9 +471,12 @@ show_text_part_content (GMimeObject *part, GMimeStream
*stream_out)
/* This result can be NULL for things like "unknown-8bit".
* Don't set a NULL filter as that makes GMime print
* annoying assertion-failure messages on stderr. */
- if (charset_filter)
+ if (charset_filter) {
g_mime_stream_filter_add (GMIME_STREAM_FILTER (stream_filter),
charset_filter);
+ g_object_unref (charset_filter);
+ }
+
}
wrapper = g_mime_part_get_content_object (GMIME_PART (part));
--
1.7.6.1
[PATCH] test: optionally print subtest number
On Tue, 13 Dec 2011 15:25:14 -0800, Jameson Graef Rollins wrote: > On Tue, 13 Dec 2011 19:18:16 -0400, David Bremner > wrote: > > We don't actually have test names, at least not ones directly suitable > > for file names. I guess we could encode them or something, is that what > > you mean? > > Don't we? There is at least a word/phrase that's consistent between the > actual test script names, what is listed in the TESTS variable (below), > and what is output when the tests are run (ie. "basic:" or > "search-folder-coherence:"). Are those not test names? OK, we can call those test names. This thread/patch is about "subtest numbers", so finer granularity. There are an average of 10 subtests per test, those are are the things that don't have names. d
[PATCH] test: optionally print subtest number
On Tue, 13 Dec 2011 14:22:21 -0800, Jameson Graef Rollins wrote: > I've only been vaguely following this "test count" stuff, but I'm not > sure I understand what's the point of giving tests a number that is > ultimately mutable. Why not just label things by the test name, instead > of the count? That wouldn't require keeping track of number/name > mapping, which will change over time. We don't actually have test names, at least not ones directly suitable for file names. I guess we could encode them or something, is that what you mean? d
Re: [PATCH v6 1/2] emacs: User-defined sections in notmuch-hello
Hi Daniel.
I have finished reviewing this patch at last. Sorry, it is a bit messy.
Overall, I like the patch. It is a very nice improvement.
I am sure I have missed some important points, but I guess this is the
best I can do right now. Perhaps I will find more comments for the next
version of the patch :)
As we already discussed on IRC, there are some trailing whitespaces to
cleanup.
Here is the review:
+(defvar notmuch-custom-section-options
s/notmuch-custom-section-options/notmuch-hello-custom-section-options/ for
consistency?
+(:filter-count (string :tag "Different filter message counts"))
It was not clear to me what this option is for from the docstring.
Perhaps something like: "Count query filter, if different from :filter"?
+(:initially-hidden (const :tag "Hide this on startup?" t))
"This" refers to section, right? If yes, let's state it explicitly:
"Hide this section on startup". Also, we should probably remove the
question mark, or add it to other options for consistency.
Should the default be to show all sections?
+(:hide-if-empty (const :tag "Hide if empty" t)))
As I understand, this controls whether the whole sections is visible.
It is not clear what "if empty" means. Does it mean that all queries
are empty? Or all queries are empty and :show-empty-sections is
false? Consider changing to something like: "Hide this section if all
queries are empty [and hidden]".
+ `(list :tag ""
+(const :tag "" notmuch-hello-insert-query-list)
Do we need to explicitly specify empty tags? Aren't they empty by
default?
+ :tag "Customized tag-list (see docstring for details)"
+ :tag "Customized queries section (see docstring for details)"
Perhaps it would be more useful to add reference to
`notmuch-hello-sections'? I.e. "see `notmuch-hello-sections' for
details.
Please s/Customized tag-list/Customized tag-list section/ everywhere for
consistency (or remove section from "Customized queries section").
+Each entry of this list should be a function of no arguments that
+should return if `notmuch-hello-target' is produced as part of its
+output and nil otherwise.
Something is missing between "return if". IMO it is really hard to
understand what the function should actually do and what it should
return. Are this functions expected to add section content to current
position? As I understand, the return value indicates whether cursor
should be positioned somewhere inside this section. It is a minor
detail, but it is described in the first (and complex sentence) as if
it was the most important part. Consider moving the return and "no
arguments" to the 3rd paragraph which describes details about the
functions. I would also swap 2nd and 3rd paragraph. Smth like:
The list contains functions which are used to construct sections in
notmuch-hello buffer. When notmuch-hello buffer is constructed,
these functions are run in the order they appear in this list. Each
function produces a section simply by adding content to the current
buffer. A section should not end with an empty line, because a
newline will be inserted after each section by `notmuch-hello'.
Each function should take no arguments. If the produced section
includes `notmuch-hello-target' (i.e. cursor should be positioned
inside this section), the function should return [something].
Otherwise, it should return nil.
For convenience an element can also be a list of the form (FUNC ARG1
ARG2 .. ARGN) in which case FUNC will be applied to the rest of the
list.
[ details about customized tag-list and queries sections ]
This is just a draft. Feel free to use it or ignore it.
+ For convenience an element can also be
Remove space the leading space and do `fill-paragraph'.
+ (function :tag "Custom function"
Perhaps "Custom section" would be more accurate?
+ "Button at position of point before rebuilding the notmuch-buffer
Missing dot at the end.
s/Button/Button text/?
+This variable contains the string of the button, if any, the
s/the string/text/ or label?
+rebuilt. This is never actually set globally and defined as a
s/is never actually set/should never be set/?
+(defvar notmuch-hello-hidden-sections nil
+ "List of query section titles whose contents are hidden")
Is this really for query sections only?
Does this duplicate :initially-hidden option from
notmuch-custom-section-options?
How about adding a global alist variable notmuch-hello-state to store
the state needed for section functions? Currently, it would contain
two values: :first-run and :target. This would allow us to add more
state variables in the future without polluting the global namespace.
Also, it would make clear what variables are section function are
supposed to use and perhaps even change (docstring should explain that
of course).
`notmuch-hello-filtered-query':
+ (and result (concat query " and (" result ")"
How about using the result as query instead of filter? I.e.
[PATCH v3 4/4] emacs: add notmuch-hello-hook
This hook is called every time the notmuch-hello buffer is updated. --- emacs/notmuch-hello.el |9 - 1 files changed, 8 insertions(+), 1 deletions(-) diff --git a/emacs/notmuch-hello.el b/emacs/notmuch-hello.el index 0fe9c1d..112b40b 100644 --- a/emacs/notmuch-hello.el +++ b/emacs/notmuch-hello.el @@ -131,6 +131,11 @@ So: (integer :tag "Number of characters") (float :tag "Fraction of window"))) +(defcustom notmuch-hello-hook nil + "Functions called after populating a `notmuch-hello' buffer." + :type 'hook + :group 'notmuch) + (defcustom notmuch-thousands-separator "," "The string used as a thousands separator. @@ -579,7 +584,9 @@ Complete list of currently available key bindings: (widget-forward 1))) (unless (widget-at) - (notmuch-hello-goto-search) + (notmuch-hello-goto-search)) + + (run-hooks 'notmuch-hello-hook (defun notmuch-folder () "Deprecated function for invoking notmuch---calling `notmuch' is preferred now." -- 1.7.8
[PATCH v3 3/4] emacs: rename notmuch-decimal-separator to notmuch-thousands-separator
In 123,456.78, "." is the decimal separator, but "," is the thousands separator. This commit also mentions the space being used as thousands separator in several European countries. --- emacs/notmuch-hello.el |8 1 files changed, 4 insertions(+), 4 deletions(-) diff --git a/emacs/notmuch-hello.el b/emacs/notmuch-hello.el index 0582cae..0fe9c1d 100644 --- a/emacs/notmuch-hello.el +++ b/emacs/notmuch-hello.el @@ -131,10 +131,10 @@ So: (integer :tag "Number of characters") (float :tag "Fraction of window"))) -(defcustom notmuch-decimal-separator "," - "The string used as a decimal separator. +(defcustom notmuch-thousands-separator "," + "The string used as a thousands separator. -Typically \",\" in the US and UK and \".\" in Europe." +Typically \",\" in the US and UK and \".\" or \" \" in Europe." :group 'notmuch :type 'string) @@ -159,7 +159,7 @@ Typically \",\" in the US and UK and \".\" in Europe." (apply #'concat (number-to-string (car result)) (mapcar (lambda (elem) - (format "%s%03d" notmuch-decimal-separator elem)) + (format "%s%03d" notmuch-thousands-separator elem)) (cdr result) (defun notmuch-hello-trim (search) -- 1.7.8
[PATCH v3 2/4] emacs: Add a face for crypto parts headers
Commit cb841878 introduced new parts handlers for crypto parts, but also hardcoded values for their headers face. This replaces these hardcoded values with a customizable face. --- emacs/notmuch-crypto.el |5 + emacs/notmuch-show.el |4 ++-- 2 files changed, 7 insertions(+), 2 deletions(-) diff --git a/emacs/notmuch-crypto.el b/emacs/notmuch-crypto.el index 44fccae..67c26af 100644 --- a/emacs/notmuch-crypto.el +++ b/emacs/notmuch-crypto.el @@ -37,6 +37,11 @@ mode." :group 'notmuch :type 'boolean) +(defface notmuch-crypto-part-header + '((t (:foreground "blue"))) + "Face used for crypto parts headers." + :group 'notmuch) + (defface notmuch-crypto-signature-good '((t (:background "green" :foreground "black"))) "Face used for good signatures." diff --git a/emacs/notmuch-show.el b/emacs/notmuch-show.el index 33ee3d8..ec9c52c 100644 --- a/emacs/notmuch-show.el +++ b/emacs/notmuch-show.el @@ -457,7 +457,7 @@ current buffer, if possible." (defun notmuch-show-insert-part-multipart/signed (msg part content-type nth depth declared-type) (let ((button (notmuch-show-insert-part-header nth declared-type content-type nil))) -(button-put button 'face '(:foreground "blue")) +(button-put button 'face 'notmuch-crypto-part-header) ;; add signature status button if sigstatus provided (if (plist-member part :sigstatus) (let* ((from (notmuch-show-get-header :From msg)) @@ -479,7 +479,7 @@ current buffer, if possible." (defun notmuch-show-insert-part-multipart/encrypted (msg part content-type nth depth declared-type) (let ((button (notmuch-show-insert-part-header nth declared-type content-type nil))) -(button-put button 'face '(:foreground "blue")) +(button-put button 'face 'notmuch-crypto-part-header) ;; add encryption status button if encstatus specified (if (plist-member part :encstatus) (let ((encstatus (car (plist-get part :encstatus -- 1.7.8
[PATCH v3 1/4] emacs: Let the user choose where to compose new mails
Reusing the current window to compose a new mail may be troublesome for the
user. This patch introduces a new customizable variable, notmuch-mua-compose-in,
which lets the user choose where to create the mail buffer: in the current
window (current and default behaviour), in a new window, or in a new frame.
---
emacs/notmuch-mua.el | 40 ++--
1 files changed, 38 insertions(+), 2 deletions(-)
diff --git a/emacs/notmuch-mua.el b/emacs/notmuch-mua.el
index 8824b08..90834d6 100644
--- a/emacs/notmuch-mua.el
+++ b/emacs/notmuch-mua.el
@@ -31,6 +31,21 @@
:group 'notmuch
:type 'hook)
+(defcustom notmuch-mua-compose-in 'current-window
+ "Where to create the mail buffer used to compose a new message.
+ Possible values are `current-window' (default), `new-window'
+ and `new-frame'. If set to `current-window', the mail buffer
+ will be displayed in the current window, so the old buffer will
+ be restored when the mail buffer is killed. If set to
+ `new-window' or `new-frame', the mail buffer will be displayed
+ in a new window/frame that will be destroyed when the buffer is
+ killed. You may want to customize `message-kill-buffer-on-exit'
+ accordingly."
+ :group 'notmuch
+ :type '(choice (const :tag "Compose in the current window" current-window)
+(const :tag "Compose mail in a new window" new-window)
+(const :tag "Compose mail in a new frame" new-frame)))
+
(defcustom notmuch-mua-user-agent-function 'notmuch-mua-user-agent-full
"Function used to generate a `User-Agent:' string. If this is
`nil' then no `User-Agent:' will be generated."
@@ -48,6 +63,23 @@ list."
;;
+(defun notmuch-mua-get-switch-function ()
+ "Get a switch function according to `notmuch-mua-compose-in'."
+ (cond ((eq notmuch-mua-compose-in 'current-window)
+'switch-to-buffer)
+ ((eq notmuch-mua-compose-in 'new-window)
+'switch-to-buffer-other-window)
+ ((eq notmuch-mua-compose-in 'new-frame)
+'switch-to-buffer-other-frame)
+ (t (error "Invalid value for `notmuch-mua-compose-in'"
+
+(defun notmuch-mua-maybe-set-window-dedicated ()
+ "Set the selected window as dedicated according to
+`notmuch-mua-compose-in'."
+ (when (or (eq notmuch-mua-compose-in 'new-frame)
+ (eq notmuch-mua-compose-in 'new-window))
+(set-window-dedicated-p (selected-window) t)))
+
(defun notmuch-mua-user-agent-full ()
"Generate a `User-Agent:' string suitable for notmuch."
(concat (notmuch-mua-user-agent-notmuch)
@@ -99,7 +131,8 @@ list."
((same-window-regexps '("\\*mail .*")))
(notmuch-mua-mail (mail-header 'to headers)
(mail-header 'subject headers)
- (message-headers-to-generate headers t '(to subject
+ (message-headers-to-generate headers t '(to subject))
+ nil (notmuch-mua-get-switch-function)))
;; insert the message body - but put it in front of the signature
;; if one is present
(goto-char (point-max))
@@ -112,6 +145,7 @@ list."
(message-goto-body))
(defun notmuch-mua-forward-message ()
+ (funcall (notmuch-mua-get-switch-function) (current-buffer))
(message-forward)
(when notmuch-mua-user-agent-function
@@ -121,6 +155,7 @@ list."
(message-sort-headers)
(message-hide-headers)
(set-buffer-modified-p nil)
+ (notmuch-mua-maybe-set-window-dedicated)
(message-goto-to))
@@ -143,6 +178,7 @@ list."
(message-sort-headers)
(message-hide-headers)
(set-buffer-modified-p nil)
+ (notmuch-mua-maybe-set-window-dedicated)
(message-goto-to))
@@ -199,7 +235,7 @@ the From: address first."
(let ((other-headers
(when (or prompt-for-sender notmuch-always-prompt-for-sender)
(list (cons 'from (notmuch-mua-prompt-for-sender))
-(notmuch-mua-mail nil nil other-headers)))
+(notmuch-mua-mail nil nil other-headers nil
(notmuch-mua-get-switch-function
(defun notmuch-mua-new-forward-message (&optional prompt-for-sender)
"Invoke the notmuch message forwarding window.
--
1.7.8
[PATCH v3 0/4] Several minor Emacs enhancements
Hi David et al., Here is a rebased version of this patch series, with more descriptive commit messages. Please tell me if there are still ways to enhance them. English is not my native language and it's sometimes hard to figure out what's relevant and how to word it :) Best regards, Thomas Thomas Jost (4): emacs: Let the user choose where to compose new mails emacs: Add a face for crypto parts headers emacs: rename notmuch-decimal-separator to notmuch-thousands-separator emacs: add notmuch-hello-hook emacs/notmuch-crypto.el |5 + emacs/notmuch-hello.el | 17 - emacs/notmuch-mua.el| 40 ++-- emacs/notmuch-show.el |4 ++-- 4 files changed, 57 insertions(+), 9 deletions(-) -- 1.7.8
[PATCH 1/2] test: add a function to run Python tests
On Sun, 11 Dec 2011 18:58:18 +0400, Dmitry Kurochkin wrote: > Perhaps we should have a test-lib.py for test-specific stuff like this > (similar to test-lib.el)? I think it would be cleaner and makes it easy > to add more Python test auxiliary functions later. Well, right now that would probably be overkill: the stdout-to-file redirection takes 3 lines at most. Besides we would still need to set some environment variables to run python with the correct directories. But of course if we need to add more helper functions for running python tests, then a test-lib.py would be nice. Maybe we could even think about using one of Python's unit test libs for that: doctest, unittest (both in the standard library) and nose seem to be the most popular ones. Regards, -- Thomas/Schnouki -- next part -- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 489 bytes Desc: not available URL: <http://notmuchmail.org/pipermail/notmuch/attachments/20111213/d3209294/attachment.pgp>
[PATCH 5/5] python: add get_mtime() to the Message class
--- bindings/python/notmuch/message.py | 20 1 files changed, 20 insertions(+), 0 deletions(-) diff --git a/bindings/python/notmuch/message.py b/bindings/python/notmuch/message.py index ce8e718..56f56c2 100644 --- a/bindings/python/notmuch/message.py +++ b/bindings/python/notmuch/message.py @@ -293,6 +293,10 @@ class Message(object): _get_date.argtypes = [NotmuchMessageP] _get_date.restype = c_long +_get_mtime = nmlib.notmuch_message_get_mtime +_get_mtime.argtypes = [NotmuchMessageP] +_get_mtime.restype = c_long + _get_header = nmlib.notmuch_message_get_header _get_header.argtypes = [NotmuchMessageP, c_char_p] _get_header.restype = c_char_p @@ -401,6 +405,22 @@ class Message(object): raise NotmuchError(STATUS.NOT_INITIALIZED) return Message._get_date(self._msg) +def get_mtime(self): +"""Returns time_t of the message mtime + +The mtime is the timestamp of the last time the message was modified, +e.g. the time it was added to the database or the last time a tag was +added or removed. + +:returns: A time_t timestamp. +:rtype: c_unit64 +:exception: :exc:`NotmuchError` STATUS.NOT_INITIALIZED if the message +is not initialized. +""" +if self._msg is None: +raise NotmuchError(STATUS.NOT_INITIALIZED) +return Message._get_mtime(self._msg) + def get_header(self, header): """Get the value of the specified header. -- 1.7.8
[PATCH 4/5] show: include mtime in JSON output
This could be used by a UI implementation somehow.
---
notmuch-show.c |7 ---
1 files changed, 4 insertions(+), 3 deletions(-)
diff --git a/notmuch-show.c b/notmuch-show.c
index 873a7c4..7279601 100644
--- a/notmuch-show.c
+++ b/notmuch-show.c
@@ -202,17 +202,18 @@ format_message_json (const void *ctx, notmuch_message_t
*message, unused (int in
notmuch_tags_t *tags;
int first = 1;
void *ctx_quote = talloc_new (ctx);
-time_t date;
+time_t date, mtime;
const char *relative_date;
date = notmuch_message_get_date (message);
relative_date = notmuch_time_relative_date (ctx, date);
+mtime = notmuch_message_get_mtime (message);
-printf ("\"id\": %s, \"match\": %s, \"filename\": %s, \"timestamp\": %ld,
\"date_relative\": \"%s\", \"tags\": [",
+printf ("\"id\": %s, \"match\": %s, \"filename\": %s, \"timestamp\": %ld,
\"date_relative\": \"%s\", \"mtime\": %ld, \"tags\": [",
json_quote_str (ctx_quote, notmuch_message_get_message_id
(message)),
notmuch_message_get_flag (message, NOTMUCH_MESSAGE_FLAG_MATCH) ?
"true" : "false",
json_quote_str (ctx_quote, notmuch_message_get_filename (message)),
- date, relative_date);
+ date, relative_date, mtime);
for (tags = notmuch_message_get_tags (message);
notmuch_tags_valid (tags);
--
1.7.8
[PATCH 3/5] lib: Make MTIME values searchable
Tag modification times are now searchable as ranges (just like regular message
dates) with the "mtime:" prefix.
---
lib/database-private.h |1 +
lib/database.cc|3 +++
notmuch.1 | 14 --
notmuch.c | 13 ++---
4 files changed, 26 insertions(+), 5 deletions(-)
diff --git a/lib/database-private.h b/lib/database-private.h
index 88532d5..e71c8e4 100644
--- a/lib/database-private.h
+++ b/lib/database-private.h
@@ -52,6 +52,7 @@ struct _notmuch_database {
Xapian::QueryParser *query_parser;
Xapian::TermGenerator *term_gen;
Xapian::ValueRangeProcessor *value_range_processor;
+Xapian::ValueRangeProcessor *mtime_value_range_processor;
};
/* Return the list of terms from the given iterator matching a prefix.
diff --git a/lib/database.cc b/lib/database.cc
index 6dc6f73..cc970c1 100644
--- a/lib/database.cc
+++ b/lib/database.cc
@@ -677,12 +677,14 @@ notmuch_database_open (const char *path,
notmuch->term_gen = new Xapian::TermGenerator;
notmuch->term_gen->set_stemmer (Xapian::Stem ("english"));
notmuch->value_range_processor = new Xapian::NumberValueRangeProcessor
(NOTMUCH_VALUE_TIMESTAMP);
+ notmuch->mtime_value_range_processor = new
Xapian::NumberValueRangeProcessor (NOTMUCH_VALUE_MTIME, "mtime:");
notmuch->query_parser->set_default_op (Xapian::Query::OP_AND);
notmuch->query_parser->set_database (*notmuch->xapian_db);
notmuch->query_parser->set_stemmer (Xapian::Stem ("english"));
notmuch->query_parser->set_stemming_strategy
(Xapian::QueryParser::STEM_SOME);
notmuch->query_parser->add_valuerangeprocessor
(notmuch->value_range_processor);
+ notmuch->query_parser->add_valuerangeprocessor
(notmuch->mtime_value_range_processor);
for (i = 0; i < ARRAY_SIZE (BOOLEAN_PREFIX_EXTERNAL); i++) {
prefix_t *prefix = &BOOLEAN_PREFIX_EXTERNAL[i];
@@ -726,6 +728,7 @@ notmuch_database_close (notmuch_database_t *notmuch)
delete notmuch->query_parser;
delete notmuch->xapian_db;
delete notmuch->value_range_processor;
+delete notmuch->mtime_value_range_processor;
talloc_free (notmuch);
}
diff --git a/notmuch.1 b/notmuch.1
index 3dbd67e..2235096 100644
--- a/notmuch.1
+++ b/notmuch.1
@@ -644,6 +644,8 @@ terms to match against specific portions of an email, (where
folder:
+ mtime:
+
The
.B from:
prefix is used to match the name or address of the sender of an email
@@ -707,8 +709,8 @@ operators, but will have to be protected from
interpretation by the
shell, (such as by putting quotation marks around any parenthesized
expression).
-Finally, results can be restricted to only messages within a
-particular time range, (based on the Date: header) with a syntax of:
+Results can be restricted to only messages within a particular time range,
+(based on the Date: header) with a syntax of:
..
@@ -721,6 +723,14 @@ specify a date range to return messages from 2009\-10\-01
until the
current time:
$(date +%s \-d 2009\-10\-01)..$(date +%s)
+
+Finally, the
+.B mtime:
+prefix can be used to search for messages which were modified (e.g. tags were
+added or removed) within a particular time range, with the same syntax as
+before:
+
+ mtime:..
.SH HOOKS
Hooks are scripts (or arbitrary executables or symlinks to such) that notmuch
invokes before and after certain actions. These scripts reside in
diff --git a/notmuch.c b/notmuch.c
index c0ce026..443cf59 100644
--- a/notmuch.c
+++ b/notmuch.c
@@ -71,6 +71,7 @@ static const char search_terms_help[] =
"\t\tid:\n"
"\t\tthread:\n"
"\t\tfolder:\n"
+"\t\tmtime:\n"
"\n"
"\tThe from: prefix is used to match the name or address of\n"
"\tthe sender of an email message.\n"
@@ -112,8 +113,8 @@ static const char search_terms_help[] =
"\tinterpretation by the shell, (such as by putting quotation\n"
"\tmarks around any parenthesized expression).\n"
"\n"
-"\tFinally, results can be restricted to only messages within a\n"
-"\tparticular time range, (based on the Date: header) with:\n"
+"\tResults can be restricted to only messages within a particular\n"
+"\ttime range, (based on the Date: header) with:\n"
"\n"
"\t\t..\n"
"\n"
@@ -125,7 +126,13 @@ static const char search_terms_help[] =
"\tfollowing syntax would specify a date range to return messages\n"
"\tfrom 2009-10-01 until the current time:\n"
"\n"
-"\t\t$(date +%%s -d 2009-10-01)..$(date +%%s)\n\n";
+"\t\t$(date +%%s -d 2009-10-01)..$(date +%%s)\n\n"
+"\n"
+"\tFinally, the mtime: prefix can be used to search for messages\n"
+"\twhich were modified (e.g. tags were added or removed) within a\n"
+"\tparticular time range, with the same syntax as before:\n"
+"\n"
+"\t\tmtime:..\n";
static const char hooks_help[] =
"\tHooks are scripts (or arbitrary executables or symlinks to such)
[PATCH 2/5] lib: Add a MTIME value to every mail document
This is a time_t value, similar to the message date (TIMESTAMP). It is first set
when the message is added to the database, and is then updated every time a tag
is added or removed. It can thus be used for doing incremental dumps of the
database or for synchronizing it between several computers.
This value can be read freely (with notmuch_message_get_mtime()) but for now it
can't be set to an arbitrary value: it can only be set to "now" when updated.
There's no specific reason for this except that I don't really see a real use
case for setting it to an arbitrary value.
---
lib/database.cc |7 ++-
lib/message.cc| 32
lib/notmuch-private.h |6 +-
lib/notmuch.h |4
4 files changed, 47 insertions(+), 2 deletions(-)
diff --git a/lib/database.cc b/lib/database.cc
index 2025189..6dc6f73 100644
--- a/lib/database.cc
+++ b/lib/database.cc
@@ -81,7 +81,7 @@ typedef struct {
* STRING is the name of a file within that
* directory for this mail message.
*
- *A mail document also has four values:
+ *A mail document also has five values:
*
* TIMESTAMP: The time_t value corresponding to the message's
* Date header.
@@ -92,6 +92,9 @@ typedef struct {
*
* SUBJECT:The value of the "Subject" header
*
+ * MTIME: The time_t value corresponding to the last time
+ * a tag was added or removed on the message.
+ *
* In addition, terms from the content of the message are added with
* "from", "to", "attachment", and "subject" prefixes for use by the
* user in searching. Similarly, terms from the path of the mail
@@ -1735,6 +1738,8 @@ notmuch_database_add_message (notmuch_database_t *notmuch,
date = notmuch_message_file_get_header (message_file, "date");
_notmuch_message_set_header_values (message, date, from, subject);
+_notmuch_message_update_mtime (message);
+
_notmuch_message_index_file (message, filename);
} else {
ret = NOTMUCH_STATUS_DUPLICATE_MESSAGE_ID;
diff --git a/lib/message.cc b/lib/message.cc
index 0075425..0c98589 100644
--- a/lib/message.cc
+++ b/lib/message.cc
@@ -830,6 +830,34 @@ _notmuch_message_set_header_values (notmuch_message_t
*message,
message->doc.add_value (NOTMUCH_VALUE_SUBJECT, subject);
}
+/* Get the message mtime, i.e. when it was added or the last time a tag was
+ * added/removed. */
+time_t
+notmuch_message_get_mtime (notmuch_message_t *message)
+{
+std::string value;
+
+try {
+ value = message->doc.get_value (NOTMUCH_VALUE_MTIME);
+} catch (Xapian::Error &error) {
+ INTERNAL_ERROR ("Failed to read mtime value from document.");
+ return 0;
+}
+
+return Xapian::sortable_unserialise (value);
+}
+
+/* Set the message mtime to "now". */
+void
+_notmuch_message_update_mtime (notmuch_message_t *message)
+{
+time_t time_value;
+
+time_value = time (NULL);
+message->doc.add_value (NOTMUCH_VALUE_MTIME,
+Xapian::sortable_serialise (time_value));
+}
+
/* Synchronize changes made to message->doc out into the database. */
void
_notmuch_message_sync (notmuch_message_t *message)
@@ -994,6 +1022,8 @@ notmuch_message_add_tag (notmuch_message_t *message, const
char *tag)
private_status);
}
+_notmuch_message_update_mtime (message);
+
if (! message->frozen)
_notmuch_message_sync (message);
@@ -1022,6 +1052,8 @@ notmuch_message_remove_tag (notmuch_message_t *message,
const char *tag)
private_status);
}
+_notmuch_message_update_mtime (message);
+
if (! message->frozen)
_notmuch_message_sync (message);
diff --git a/lib/notmuch-private.h b/lib/notmuch-private.h
index 60a932f..9859872 100644
--- a/lib/notmuch-private.h
+++ b/lib/notmuch-private.h
@@ -95,7 +95,8 @@ typedef enum {
NOTMUCH_VALUE_TIMESTAMP = 0,
NOTMUCH_VALUE_MESSAGE_ID,
NOTMUCH_VALUE_FROM,
-NOTMUCH_VALUE_SUBJECT
+NOTMUCH_VALUE_SUBJECT,
+NOTMUCH_VALUE_MTIME
} notmuch_value_t;
/* Xapian (with flint backend) complains if we provide a term longer
@@ -276,6 +277,9 @@ _notmuch_message_set_header_values (notmuch_message_t
*message,
const char *from,
const char *subject);
void
+_notmuch_message_update_mtime (notmuch_message_t *message);
+
+void
_notmuch_message_sync (notmuch_message_t *message);
notmuch_status_t
diff --git a/lib/notmuch.h b/lib/notmuch.h
index 9f23a10..643ebce 100644
--- a/lib/notmuch.h
+++ b/lib/notmuch.h
@@ -910,6 +910,10 @@ notmuch_message_set_flag (notmuch_message_t *message,
time_t
notmuch_message_get_date (notmuch_message_t *message);
+/* Get the mtime of 'message' as a time_t value. */
+time_t
+notmuch_message_get_mtime (notmuch_message_t *message);
+
/* Get th
[PATCH 1/5] Fix comments about what is stored in the database
Commit 567bcbc2 introduced two new values for each message (content of the
"From" and "Subject" headers), but the comments about the database schema had
not been updated accordingly.
---
lib/database.cc |6 +-
1 files changed, 5 insertions(+), 1 deletions(-)
diff --git a/lib/database.cc b/lib/database.cc
index 98f101e..2025189 100644
--- a/lib/database.cc
+++ b/lib/database.cc
@@ -81,13 +81,17 @@ typedef struct {
* STRING is the name of a file within that
* directory for this mail message.
*
- *A mail document also has two values:
+ *A mail document also has four values:
*
* TIMESTAMP: The time_t value corresponding to the message's
* Date header.
*
* MESSAGE_ID: The unique ID of the mail mess (see "id" above)
*
+ * FROM: The value of the "From" header
+ *
+ * SUBJECT:The value of the "Subject" header
+ *
* In addition, terms from the content of the message are added with
* "from", "to", "attachment", and "subject" prefixes for use by the
* user in searching. Similarly, terms from the path of the mail
--
1.7.8
[PATCH 0/5] Store message modification times in the DB
Hello world, This is a patch series I've been working on for some time in order to be able to sync my tags on several computers. I'm posting it now, but please consider it as a RFC rather than something that is ready to be pushed. The basic idea is to the last time each message was modified, i.e. "the message was added to the DB", "a tag was added" or "a tag was removed". This mtime is accessible through a library function and in the JSON output of "notmuch show". It is also searchable with the "mtime:" prefix and with timestamp ranges, like for searching messages by date: notmuch search mtime:$(date +%s 2011-12-01)..$(date +%s) This can then be used in scripts or helper programs to do incremental dumps or tags synchronization. (I already have a script to do incremental backups, but it needs some cleaning, and I'm still working on something for sync'ing tags, but it's starting to work really well; I'll post them later). This can be seen as an alternative to David Bremner's jlog branch, but with several differences: + no external dependency + everything is stored in the notmuch DB: atomicity for free! - when a message is removed we lose everything about it, which makes the sync process more complicated - for a human, it's harder to manipulate timestamps than log messages - this can store much less data than a proper log system On IRC amdragon suggested using a simple sequence number instead of a timestamp. This would indeed eliminate the need for proper time synchronization between computers one would want to keep in sync, and it would reduce the risk of time-going-backward problems, but IMHO it would cause more problems: no global clock --> no simple way to tell if DB #A is more recent than DB #B. So, here are the patches: - first a little fix to the comments describing the DB schema (not specific to this patch series at all, I just noticed it when rebasing this series) - the second commit adds the MTIME value to the database schema, and creates the functions used to update and access this value. - the third commit makes the MTIME value searchable with a range syntax. - the fourth commit adds the MTIME to the JSON output of "notmuch show". - the fifth and last commit adds Message.get_mtime() to the Python bindings. Please tell me what you think of this. Best regards, Thomas Thomas Jost (5): Fix comments about what is stored in the database lib: Add a MTIME value to every mail document lib: Make MTIME values searchable show: include mtime in JSON output python: add get_mtime() to the Message class bindings/python/notmuch/message.py | 20 lib/database-private.h |1 + lib/database.cc| 14 +- lib/message.cc | 32 lib/notmuch-private.h |6 +- lib/notmuch.h |4 notmuch-show.c |7 --- notmuch.1 | 14 -- notmuch.c | 13 ++--- 9 files changed, 101 insertions(+), 10 deletions(-) -- 1.7.8
[PATCH] notmuch: unref charset_filter to fix one memory leak
On Tue, 13 Dec 2011 14:03:33 +0200, Tomi Ollila wrote:
> In my use case g_object_unref(charset_filter) reduces memory
> consumption over 90% when 'notmuch show --format=text "*"' is
> executed (~11000 messages, RES ~330M -> ~25M).
> ---
> notmuch-show.c |4 +++-
> 1 files changed, 3 insertions(+), 1 deletions(-)
>
> diff --git a/notmuch-show.c b/notmuch-show.c
> index 873a7c4..23d7368 100644
> --- a/notmuch-show.c
> +++ b/notmuch-show.c
> @@ -450,6 +450,7 @@ show_text_part_content (GMimeObject *part, GMimeStream
> *stream_out)
> {
> GMimeContentType *content_type = g_mime_object_get_content_type
> (GMIME_OBJECT (part));
> GMimeStream *stream_filter = NULL;
> +GMimeFilter *charset_filter = NULL;
> GMimeDataWrapper *wrapper;
> const char *charset;
>
> @@ -466,7 +467,6 @@ show_text_part_content (GMimeObject *part, GMimeStream
> *stream_out)
>
> charset = g_mime_object_get_content_type_parameter (part, "charset");
> if (charset) {
> - GMimeFilter *charset_filter;
> charset_filter = g_mime_filter_charset_new (charset, "UTF-8");
> /* This result can be NULL for things like "unknown-8bit".
>* Don't set a NULL filter as that makes GMime print
> @@ -479,6 +479,8 @@ show_text_part_content (GMimeObject *part, GMimeStream
> *stream_out)
> wrapper = g_mime_part_get_content_object (GMIME_PART (part));
> if (wrapper && stream_filter)
> g_mime_data_wrapper_write_to_stream (wrapper, stream_filter);
> +if (charset_filter)
> + g_object_unref (charset_filter);
Why can't we do this inside the if (charset) block?
Regards,
Dmitry
> if (stream_filter)
> g_object_unref(stream_filter);
> }
> --
> 1.7.6.1
> ___
> notmuch mailing list
> notmuch at notmuchmail.org
> http://notmuchmail.org/mailman/listinfo/notmuch
[Alpha PATCH 6/6] test: second set of dump/restore --format=notmuch tests
From: David Bremner
These one need the completed functionality in notmuch-restore. Fairly
exotic tags are tested, but no weird message id's.
---
test/dump-restore | 33 +
1 files changed, 33 insertions(+), 0 deletions(-)
diff --git a/test/dump-restore b/test/dump-restore
index 48caf4e..122de5c 100755
--- a/test/dump-restore
+++ b/test/dump-restore
@@ -94,4 +94,37 @@ notmuch dump --format=sup -- from:cworth | tr -d \(\) >
EXPECTED.$test_count
notmuch dump --format=notmuch -- from:cworth > OUTPUT.$test_count
test_expect_equal_file OUTPUT.$test_count EXPECTED.$test_count
+test_begin_subtest "format=notmuch, # round-trip"
+notmuch dump --format=sup | sort > EXPECTED.$test_count
+notmuch dump --format=notmuch | notmuch restore --format=notmuch
+notmuch dump --format=sup | sort > OUTPUT.$test_count
+test_expect_equal_file EXPECTED.$test_count OUTPUT.$test_count
+
+tag1='comic_swear=$&^%$^%\\//-+$^%$'
+enc1=$($TEST_DIRECTORY/hex-xcode e "$tag1")
+
+tag2=$(printf 'this\n tag\t has\n spaces')
+enc2=$($TEST_DIRECTORY/hex-xcode e "$tag2")
+
+notmuch dump --format=notmuch > BACKUP
+
+notmuch tag +"$tag1" +"$tag2" -inbox -unread "*"
+
+test_begin_subtest 'format=notmuch, round trip with strange tags'
+ notmuch dump --format=notmuch > EXPECTED.$test_count
+ notmuch dump --format=notmuch | notmuch restore --format=notmuch
+ notmuch dump --format=notmuch > OUTPUT.$test_count
+test_expect_equal_file EXPECTED.$test_count OUTPUT.$test_count
+
+
+test_begin_subtest 'format=notmuch, checking encoded output'
+cp /dev/null EXPECTED.$test_count
+notmuch dump --format=notmuch -- from:cworth |\
+awk "{ print \$1 \" $enc1 $enc2\" }" > EXPECTED.$test_count
+
+notmuch dump --format=notmuch -- from:cworth > OUTPUT.$test_count
+test_expect_equal_file EXPECTED.$test_count OUTPUT.$test_count
+
+notmuch restore --format=notmuch < BACKUP
+
test_done
--
1.7.5.4
[Alpha PATCH 5/6] notmuch-restore: add --format=notmuch support
From: David Bremner
This is format is whitespace separated tokens, encoded by
util/hex-escape.c
---
notmuch-restore.c | 83
1 files changed, 64 insertions(+), 19 deletions(-)
diff --git a/notmuch-restore.c b/notmuch-restore.c
index 87d9772..f392590 100644
--- a/notmuch-restore.c
+++ b/notmuch-restore.c
@@ -19,6 +19,7 @@
*/
#include "notmuch-client.h"
+#include "dump-restore-private.h"
int
notmuch_restore_command (unused (void *ctx), int argc, char *argv[])
@@ -35,6 +36,7 @@ notmuch_restore_command (unused (void *ctx), int argc, char
*argv[])
regex_t regex;
int rerr;
int opt_index;
+int input_format = DUMP_FORMAT_SUP;
config = notmuch_config_open (ctx, NULL, NULL);
if (config == NULL)
@@ -48,6 +50,7 @@ notmuch_restore_command (unused (void *ctx), int argc, char
*argv[])
synchronize_flags = notmuch_config_get_maildir_synchronize_flags (config);
notmuch_opt_desc_t options[] = {
+ FORMAT_DESC (input_format),
{ NOTMUCH_OPT_POSITION, &input_file_name, 0, 0, 0 },
{ NOTMUCH_OPT_BOOLEAN, &accumulate, "accumulate", 'a', 0 },
{ 0, 0, 0, 0, 0 }
@@ -81,33 +84,63 @@ notmuch_restore_command (unused (void *ctx), int argc, char
*argv[])
* non-space characters for the message-id, then one or more
* spaces, then a list of space-separated tags as a sequence of
* characters within literal '(' and ')'. */
-if ( xregcomp (®ex,
- "^([^ ]+) \\(([^)]*)\\)$",
- REG_EXTENDED) )
- INTERNAL_ERROR("compile time constant regex failed.");
+if (input_format == DUMP_FORMAT_SUP) {
+ if ( xregcomp (®ex,
+ "^([^ ]+) \\(([^)]*)\\)$",
+ REG_EXTENDED) )
+ INTERNAL_ERROR("compile time constant regex failed.");
+}
+
+
+/* These are out here to re-use the buffers with hex_decode */
+
+char *message_id = NULL;
+size_t message_id_size=0;
+char *tag = NULL;
+size_t tag_size=0;
while ((line_len = getline (&line, &line_size, input)) != -1) {
regmatch_t match[3];
- char *message_id, *file_tags, *tag, *next;
+ char *file_tags, *next;
notmuch_message_t *message = NULL;
+
notmuch_status_t status;
notmuch_tags_t *db_tags;
char *db_tags_str;
chomp_newline (line);
- rerr = xregexec (®ex, line, 3, match, 0);
- if (rerr == REG_NOMATCH)
- {
- fprintf (stderr, "Warning: Ignoring invalid input line: %s\n",
-line);
- continue;
+ /* Silently ignore blank lines */
+
+ if (line[0] == '\0') {
+ continue;
+ }
+
+ if (input_format == DUMP_FORMAT_SUP) {
+ rerr = xregexec (®ex, line, 3, match, 0);
+ if (rerr == REG_NOMATCH)
+ {
+ fprintf (stderr, "Warning: Ignoring invalid input line: %s\n",
+line);
+ continue;
+ }
+ message_id = talloc_strndup (notmuch, line + match[1].rm_so,
+match[1].rm_eo - match[1].rm_so);
+ file_tags = talloc_strndup (notmuch, line + match[2].rm_so,
+ match[2].rm_eo - match[2].rm_so);
+ } else {
+ char *p = line;
+ char *raw_mid;
+
+ raw_mid = strsep (&p, " \t");
+
+ if (hex_decode (notmuch, raw_mid,
+ &message_id, &message_id_size) != HEX_SUCCESS)
+ return 1;
+
+ file_tags = xstrdup (p);
}
- message_id = xstrndup (line + match[1].rm_so,
- match[1].rm_eo - match[1].rm_so);
- file_tags = xstrndup (line + match[2].rm_so,
- match[2].rm_eo - match[2].rm_so);
status = notmuch_database_find_message (notmuch, message_id, &message);
if (status || message == NULL) {
@@ -153,7 +186,16 @@ notmuch_restore_command (unused (void *ctx), int argc,
char *argv[])
next = file_tags;
while (next) {
- tag = strsep (&next, " ");
+ char *raw_tag = strsep (&next, " ");
+
+ if (input_format == DUMP_FORMAT_NOTMUCH) {
+ if (hex_decode (notmuch, raw_tag,
+ &tag, &tag_size) != HEX_SUCCESS)
+ return 1;
+ } else {
+ tag = talloc_strdup(notmuch, raw_tag);
+ }
+
if (*tag == '\0')
continue;
status = notmuch_message_add_tag (message, tag);
@@ -175,11 +217,14 @@ notmuch_restore_command (unused (void *ctx), int argc,
char *argv[])
if (message)
notmuch_message_destroy (message);
message = NULL;
- free (message_id);
- free (file_tags);
+ if (input_format == DUMP_FORMAT_SUP) {
+ talloc_free (message_id);
+ talloc_free (file_tags);
[Alpha PATCH 4/6] test: add test for dump --format=notmuch
From: David Bremner The first test is really to test our assumptions about the corpus, namely that a certain set of message-id's is safe (i.e. doesn't change under hex-escaping). We then check dump output as best we can without functionality-to-come in notmuch-restore. --- test/dump-restore | 12 1 files changed, 12 insertions(+), 0 deletions(-) diff --git a/test/dump-restore b/test/dump-restore index 439e998..48caf4e 100755 --- a/test/dump-restore +++ b/test/dump-restore @@ -82,4 +82,16 @@ test_begin_subtest "dump outfile -- from:cworth" notmuch dump dump-outfile-dash-inbox.actual -- from:cworth test_expect_equal_file dump-cworth.expected dump-outfile-dash-inbox.actual +test_begin_subtest "Check for a safe set of message-ids" +notmuch search --output=messages from:cworth > EXPECTED.$test_count +notmuch search --output=messages from:cworth |\ + $TEST_DIRECTORY/hex-xcode e > OUTPUT.$test_count +test_expect_equal_file OUTPUT.$test_count EXPECTED.$test_count + +# we have observed that cworth has sane message-ids, and hopefully sane tags. +test_begin_subtest "dump --format=notmuch -- from:cworth" +notmuch dump --format=sup -- from:cworth | tr -d \(\) > EXPECTED.$test_count +notmuch dump --format=notmuch -- from:cworth > OUTPUT.$test_count +test_expect_equal_file OUTPUT.$test_count EXPECTED.$test_count + test_done -- 1.7.5.4
[Alpha PATCH 3/6] notmuch-dump: add --format=(notmuch|sup)
From: David Bremner
sup is the old format, and remains the default.
Each line of the notmuch format is "msg_id tag tag...tag" where each
space seperated token is 'hex-encoded' to remove troubling characters.
In particular this format won't have the same problem with e.g. spaces
in message-ids or tags; they will be round-trip-able.
---
dump-restore-private.h | 20
notmuch-dump.c | 40 ++--
2 files changed, 54 insertions(+), 6 deletions(-)
create mode 100644 dump-restore-private.h
diff --git a/dump-restore-private.h b/dump-restore-private.h
new file mode 100644
index 000..eda3219
--- /dev/null
+++ b/dump-restore-private.h
@@ -0,0 +1,20 @@
+#ifndef DUMP_RESTORE_PRIVATE_H
+#define DUMP_RESTORE_PRIVATE_H
+
+#include "hex-escape.h"
+#include "command-line-arguments.h"
+
+typedef enum dump_formats {
+DUMP_FORMAT_SUP,
+DUMP_FORMAT_NOTMUCH
+} dump_format_t;
+
+#define FORMAT_DESC(out) \
+ { \
+ NOTMUCH_OPT_KEYWORD, &out, "format", 'f', \
+ (notmuch_keyword_t []){ { "sup", DUMP_FORMAT_SUP }, \
+ { "notmuch", DUMP_FORMAT_NOTMUCH }, \
+ {0, 0} }\
+ }
+
+#endif
diff --git a/notmuch-dump.c b/notmuch-dump.c
index a735875..5f788f9 100644
--- a/notmuch-dump.c
+++ b/notmuch-dump.c
@@ -19,6 +19,7 @@
*/
#include "notmuch-client.h"
+#include "dump-restore-private.h"
int
notmuch_dump_command (unused (void *ctx), int argc, char *argv[])
@@ -44,7 +45,10 @@ notmuch_dump_command (unused (void *ctx), int argc, char
*argv[])
char *output_file_name = NULL;
int opt_index;
+int output_format = DUMP_FORMAT_SUP;
+
notmuch_opt_desc_t options[] = {
+ FORMAT_DESC (output_format),
{ NOTMUCH_OPT_POSITION, &output_file_name, 0, 0, 0 },
{ 0, 0, 0, 0, 0 }
};
@@ -85,29 +89,53 @@ notmuch_dump_command (unused (void *ctx), int argc, char
*argv[])
*/
notmuch_query_set_sort (query, NOTMUCH_SORT_UNSORTED);
+char *buffer=NULL;
+size_t buffer_size=0;
+
for (messages = notmuch_query_search_messages (query);
notmuch_messages_valid (messages);
notmuch_messages_move_to_next (messages))
{
int first = 1;
- message = notmuch_messages_get (messages);
+ const char *message_id;
- fprintf (output,
-"%s (", notmuch_message_get_message_id (message));
+ message = notmuch_messages_get (messages);
+ message_id = notmuch_message_get_message_id (message);
+
+ if (output_format == DUMP_FORMAT_SUP) {
+ fprintf (output, "%s (", message_id);
+ } else {
+ if (hex_encode (notmuch, message_id,
+ &buffer, &buffer_size) != HEX_SUCCESS)
+ return 1;
+ fprintf (output, "%s ", buffer);
+ }
for (tags = notmuch_message_get_tags (message);
notmuch_tags_valid (tags);
notmuch_tags_move_to_next (tags))
{
+ const char *tag_str = notmuch_tags_get (tags);
+
if (! first)
- fprintf (output, " ");
+ fputs (" ", output);
- fprintf (output, "%s", notmuch_tags_get (tags));
+ if (output_format == DUMP_FORMAT_SUP) {
+ fputs (tag_str, output);
+ } else {
+ if (hex_encode (notmuch, tag_str,
+ &buffer, &buffer_size) != HEX_SUCCESS)
+ return 1;
+ fputs (buffer, output);
+ }
first = 0;
}
- fprintf (output, ")\n");
+ if (output_format == DUMP_FORMAT_SUP)
+ fputs (")\n",output);
+ else
+ fputs ("\n", output);
notmuch_message_destroy (message);
}
--
1.7.5.4
[Alpha PATCH 2/6] test: add test for hex_(encode|decode)
From: David Bremner
The test binary hex-xcode can also be used to check for memory leaks.
---
test/.gitignore |1 +
test/Makefile.local |6 +++-
test/basic |2 +-
test/hex-escaping | 20 +
test/hex-xcode.c| 76 +++
test/notmuch-test |1 +
6 files changed, 104 insertions(+), 2 deletions(-)
create mode 100755 test/hex-escaping
create mode 100644 test/hex-xcode.c
diff --git a/test/.gitignore b/test/.gitignore
index e63c689..be7ab5e 100644
--- a/test/.gitignore
+++ b/test/.gitignore
@@ -3,4 +3,5 @@ corpus.mail
smtp-dummy
symbol-test
arg-test
+hex-xcode
tmp.*
diff --git a/test/Makefile.local b/test/Makefile.local
index 6cb6c82..d27826a 100644
--- a/test/Makefile.local
+++ b/test/Makefile.local
@@ -13,6 +13,9 @@ smtp_dummy_modules = $(smtp_dummy_srcs:.c=.o)
$(dir)/arg-test: $(dir)/arg-test.o command-line-arguments.o util/libutil.a
$(call quiet,CC) -I. $^ -o $@
+$(dir)/hex-xcode: $(dir)/hex-xcode.o util/libutil.a
+ $(call quiet,CC) -I. $^ -o $@ -ltalloc
+
$(dir)/smtp-dummy: $(smtp_dummy_modules)
$(call quiet,CC) $^ -o $@
@@ -21,7 +24,8 @@ $(dir)/symbol-test: $(dir)/symbol-test.o
.PHONY: test check
-test-binaries: $(dir)/arg-test $(dir)/smtp-dummy $(dir)/symbol-test
+test-binaries: $(dir)/arg-test $(dir)/hex-xcode \
+$(dir)/smtp-dummy $(dir)/symbol-test
test: all test-binaries
@${dir}/notmuch-test $(OPTIONS)
diff --git a/test/basic b/test/basic
index d6aed24..af57026 100755
--- a/test/basic
+++ b/test/basic
@@ -54,7 +54,7 @@ test_begin_subtest 'Ensure that all available tests will be
run by notmuch-test'
eval $(sed -n -e '/^TESTS="$/,/^"$/p' $TEST_DIRECTORY/notmuch-test)
tests_in_suite=$(for i in $TESTS; do echo $i; done | sort)
available=$(find "$TEST_DIRECTORY" -maxdepth 1 -type f -executable -printf
'%f\n' | \
-sed -r -e
"/^(aggregate-results.sh|notmuch-test|smtp-dummy|test-verbose|symbol-test|arg-test)$/d"
| \
+sed -r -e
"/^(aggregate-results.sh|notmuch-test|smtp-dummy|test-verbose|symbol-test|arg-test|hex-xcode)$/d"
| \
sort)
test_expect_equal "$tests_in_suite" "$available"
diff --git a/test/hex-escaping b/test/hex-escaping
new file mode 100755
index 000..d0a993e
--- /dev/null
+++ b/test/hex-escaping
@@ -0,0 +1,20 @@
+#!/usr/bin/env bash
+test_description="hex encoding and decoding"
+. ./test-lib.sh
+
+test_begin_subtest "round trip"
+find $TEST_DIRECTORY/corpus -type f -print | sort | xargs cat > EXPECTED
+$TEST_DIRECTORY/hex-xcode e < EXPECTED | $TEST_DIRECTORY/hex-xcode d > OUTPUT
+test_expect_equal_file OUTPUT EXPECTED
+
+test_begin_subtest "punctuation"
+tag1='comic_swear=$&^%$^%\\//-+$^%$'
+tag_enc1=$($TEST_DIRECTORY/hex-xcode e "$tag1")
+test_expect_equal "$tag_enc1"
"comic_swear=%24%26%5e%25%24%5e%25%5c%5c%2f%2f-+%24%5e%25%24"
+
+test_begin_subtest "round trip newlines"
+printf 'this\n tag\t has\n spaces\n' > EXPECTED.$test_count
+$TEST_DIRECTORY/hex-xcode e < EXPECTED.$test_count |\
+ $TEST_DIRECTORY/hex-xcode d > OUTPUT.$test_count
+test_expect_equal_file EXPECTED.$test_count OUTPUT.$test_count
+test_done
diff --git a/test/hex-xcode.c b/test/hex-xcode.c
new file mode 100644
index 000..fff3e01
--- /dev/null
+++ b/test/hex-xcode.c
@@ -0,0 +1,76 @@
+/* No, nothing to to with IDE from Apple Inc.
+ testbed for ../util/hex-escape.c.
+
+ usage: hex-xcode (e|d) < foo
+
+ e is for encode
+ d is for decode
+ */
+
+#include "notmuch-client.h"
+#include "hex-escape.h"
+#include
+
+static int
+xcode(void *ctx, char dir, char *in, char **buf_p, size_t *size_p)
+{
+hex_status_t status;
+
+if (dir == 'e')
+ status = hex_encode (ctx, in, buf_p, size_p);
+else
+ status = hex_decode (ctx, in, buf_p, size_p);
+
+if (status == HEX_SUCCESS)
+ puts(*buf_p);
+
+return status;
+}
+
+
+int main (int argc, char **argv){
+
+assert(argc > 1 && argv[1]);
+
+char dir=argv[1][0];
+
+void *ctx = talloc_new(NULL);
+
+char *line = NULL;
+size_t line_size;
+ssize_t line_len;
+
+char *buffer=NULL;
+size_t buf_size=0;
+
+int arg_index=2;
+notmuch_bool_t read_stdin=TRUE;
+
+for (arg_index=2; arg_index < argc; arg_index++) {
+
+ if (xcode (ctx, dir, argv[arg_index],
+ &buffer, &buf_size) != HEX_SUCCESS)
+ return 1;
+
+ read_stdin=FALSE;
+}
+
+if (!read_stdin)
+ return 0;
+
+while ((line_len = getline (&line, &line_size, stdin)) != -1) {
+
+ chomp_newline (line);
+
+ if (xcode (ctx, dir, line, &buffer, &buf_size) != HEX_SUCCESS)
+ return 1;
+
+}
+
+if (line)
+ free(line);
+
+talloc_free(ctx);
+
+return 0;
+}
diff --git a/test/notmuch-test b/test/notmuch-test
index ded79e8..af72c73 100755
--- a/test/notmuch-test
+++ b/test/notmuch-test
@@ -50,6 +50,7 @@ TESTS="
python
hooks
argument-parsing
+ hex-escaping
"
TESTS=${NOTMUCH_TESTS:=$TESTS}
--
1.
[Alpha PATCH 1/6] util/hex-escape.[ch]: encoding/decoding strings into restricted character set
From: David Bremner
The character set is chosen to be suitable for pathnames, and the same as that
used by contrib/nmbug
---
util/Makefile.local |2 +-
util/hex-escape.c | 150 +++
util/hex-escape.h | 15 +
3 files changed, 166 insertions(+), 1 deletions(-)
create mode 100644 util/hex-escape.c
create mode 100644 util/hex-escape.h
diff --git a/util/Makefile.local b/util/Makefile.local
index 26e4c3f..2e63932 100644
--- a/util/Makefile.local
+++ b/util/Makefile.local
@@ -3,7 +3,7 @@
dir := util
extra_cflags += -I$(srcdir)/$(dir)
-libutil_c_srcs := $(dir)/xutil.c $(dir)/error_util.c
+libutil_c_srcs := $(dir)/xutil.c $(dir)/error_util.c $(dir)/hex-escape.c
libutil_modules := $(libutil_c_srcs:.c=.o)
diff --git a/util/hex-escape.c b/util/hex-escape.c
new file mode 100644
index 000..3e08465
--- /dev/null
+++ b/util/hex-escape.c
@@ -0,0 +1,150 @@
+/* pathname.c - Manage encoding and decoding of byte strings into path names
+ *
+ * Copyright (c) 2011 David Bremner
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 3 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see http://www.gnu.org/licenses/ .
+ *
+ * Author: David Bremner
+ */
+
+#include
+#include
+#include "error_util.h"
+#include "hex-escape.h"
+
+static const size_t default_buf_size=1024;
+
+static const char* output_charset=
+"ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+-_@=.:,";
+
+static const char escape_char='%';
+
+static int
+is_output (char c) {
+return (strchr (output_charset, c) != NULL);
+}
+
+
+static int
+maybe_realloc(void *ctx, size_t needed, char **out, size_t *out_size)
+{
+if (*out_size < needed) {
+
+ if (*out == NULL)
+ *out = talloc_size(ctx,needed);
+ else
+ *out = talloc_realloc(ctx,*out,char,needed);
+
+ if (*out == NULL)
+ return 0;
+
+ *out_size = needed;
+}
+return 1;
+}
+
+
+hex_status_t
+hex_encode (void *ctx, const char *in, char **out, size_t *out_size)
+{
+
+const char *p;
+char *q;
+
+int escape_count=0;
+size_t needed;
+
+for (p = in; *p; p++)
+ escape_count += (! is_output (*p));
+
+needed = strlen (in) + 2*escape_count + 1;
+
+if (*out == NULL)
+ *out_size=0;
+
+if (!maybe_realloc (ctx, needed, out, out_size))
+ return HEX_OUT_OF_MEMORY;
+
+q = *out;
+p = in;
+
+while (*p) {
+ if (is_output (*p)) {
+ *q++ = *p++;
+ } else {
+ sprintf (q, "%%%02x", *p++);
+ q += 3;
+ }
+}
+
+*q = '\0';
+return HEX_SUCCESS;
+}
+
+
+hex_status_t
+hex_decode (void *ctx, const char *in, char **out, size_t *out_size) {
+
+char buf[3];
+
+const char *p;
+char *q;
+
+size_t escape_count = 0;
+size_t needed = 0;
+
+size_t len = strlen (in);
+
+for (p = in; *p; p++)
+ escape_count += (*p == escape_char);
+
+needed = len - escape_count*2 +1;
+
+if (!maybe_realloc(ctx, needed, out, out_size))
+ return HEX_OUT_OF_MEMORY;
+
+p = in;
+q = *out;
+buf[2]=0;
+
+
+while (*p) {
+
+ if (*p == escape_char) {
+
+ char *endp;
+
+ if (len < 3)
+ return HEX_SYNTAX_ERROR;
+
+ buf[0]=p[1];
+ buf[1]=p[2];
+
+ *q = strtol(buf, &endp, 16);
+
+ if (endp != buf+2)
+ return HEX_SYNTAX_ERROR;
+
+ len -= 3;
+ p += 3;
+ q++;
+ } else {
+ *q++ = *p++;
+ }
+}
+
+*q = '\0';
+
+return HEX_SUCCESS;
+}
diff --git a/util/hex-escape.h b/util/hex-escape.h
new file mode 100644
index 000..98ecbe0
--- /dev/null
+++ b/util/hex-escape.h
@@ -0,0 +1,15 @@
+#ifndef _HEX_ESCAPE_H
+#define _HEX_ESCAPE_H
+
+typedef enum hex_status {
+HEX_SUCCESS = 0,
+HEX_SYNTAX_ERROR,
+HEX_OUT_OF_MEMORY
+} hex_status_t;
+
+hex_status_t
+hex_encode (void *talloc_ctx, const char *in, char **out, size_t *out_size);
+
+hex_status_t
+hex_decode (void *talloc_ctx, const char *in, char **out, size_t *out_size);
+#endif
--
1.7.5.4
New Dump/Restore Format
Hi All; There are some style/doc issues remaining, but because bugs in dump and restore really suck, I thought I would ask for early feedback on functionality. I'm particularly interested in how the new dump format works for weird message-ids (spaces and so on). If you have public messages with tricky message-id's, I'd appreciate adding those messages to the test suite. Things to bikeshed now: name(s) of the formats; sup and notmuch are maybe not ideal. The format itself? The encoding format? The latter is chosen for compatibility with nmbug, but we could discussing using a bigger character set. Things I know about - not enough tests - no man page, online docs. - no API docs for hex_encode/blah. I think the code in hex-escape.[ch] is otherwise ready for (second) review; I'll probably do another review of the code in notmuch-(dump|restore).c myself for clarity, so you might want to wait for the next round before diving in. If you prefer pull from git, you can get these patches on branch "new-dump" at git://pivot.cs.unb.ca/notmuch.git
Re: [PATCH] test: optionally print subtest number
On Tue, 13 Dec 2011 15:25:14 -0800, Jameson Graef Rollins wrote: > On Tue, 13 Dec 2011 19:18:16 -0400, David Bremner wrote: > > We don't actually have test names, at least not ones directly suitable > > for file names. I guess we could encode them or something, is that what > > you mean? > > Don't we? There is at least a word/phrase that's consistent between the > actual test script names, what is listed in the TESTS variable (below), > and what is output when the tests are run (ie. "basic:" or > "search-folder-coherence:"). Are those not test names? OK, we can call those test names. This thread/patch is about "subtest numbers", so finer granularity. There are an average of 10 subtests per test, those are are the things that don't have names. d ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
[PATCH] test: optionally print subtest number
From: David Bremner
The idea is that $test_count could be used in tests to label
intermediate files. The output enabled by this patch (and --debug)
helps figure out which OUTPUT.nn file belongs to which test in case
several subtests write to OUTPUT.$test_count
---
Is there something that depends on the test format? I find it pretty
handy to have the subtest numbers, but I don't want to break some
other tools. I followed the existing style of conditionally defining
functions, but maybe someone with more bash-fu can improve that.
test/test-lib.sh | 12
1 files changed, 12 insertions(+), 0 deletions(-)
diff --git a/test/test-lib.sh b/test/test-lib.sh
index 6be93fe..f5c 100644
--- a/test/test-lib.sh
+++ b/test/test-lib.sh
@@ -116,6 +116,16 @@ do
esac
done
+if test -n "$debug"; then
+print_subtest () {
+ printf "\t[%d]\t" $(($test_count - 1))
+}
+else
+print_subtest () {
+ true
+}
+fi
+
if test -n "$color"; then
say_color () {
(
@@ -132,6 +142,7 @@ if test -n "$color"; then
printf " "
printf "$@"
tput sgr0
+ print_subtest
)
}
else
@@ -140,6 +151,7 @@ else
shift
printf " "
printf "$@"
+ print_subtest
}
fi
--
1.7.5.4
Re: [PATCH] test: optionally print subtest number
On Tue, 13 Dec 2011 15:35:53 -0800, Jameson Graef Rollins wrote: > On Wed, 14 Dec 2011 03:24:23 +0400, Dmitry Kurochkin > wrote: > > BTW I have some plans to introduce optional explicit test ids that can > > be used for inter-test dependencies. E.g.: > > > > test_begin_subtest test-id-1 "A subtest" > > > > ;; in another test requre that test-id-1 passed > > test_require_subtest test-id-1 > > Would the required test need to be listed twice, both on the > begin_subtest line *and* in the require_subtest line? > > And again, why would the test id have to be any different that the > existing test names? The tests already have names, so I don't > understand why we would want to introduce some other kind of > identification. Seems like it's just going to add extra confusion. > What you listed in the other email are test scripts, each with many subtests. I was talking about dependencies between subtests, not test scripts. > And speaking of which, I sometimes worry that the test infrastructure > itself is getting too complicated. Pretty soon we're going to need > tests for the tests. We already have them :) Though, pretty limited. > I don't necessarily see the need to all of these > extra features in the test suite, so I worry that it's just making > everything harder to debug. > I hope we can keep balance here. Without inter-subtest dependencies, we have unhealthy situation where some tests may be skipped because of missing prerequisites, but test that depend on them are failing. The only alternative I see is to rewrite these tests to remove the dependencies. But that would complicate test cases itself, so I believe inter-subtest dependencies is a better option. Regards, Dmitry > jamie. ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
Re: [PATCH] test: optionally print subtest number
On Wed, 14 Dec 2011 03:24:23 +0400, Dmitry Kurochkin wrote: > BTW I have some plans to introduce optional explicit test ids that can > be used for inter-test dependencies. E.g.: > > test_begin_subtest test-id-1 "A subtest" > > ;; in another test requre that test-id-1 passed > test_require_subtest test-id-1 Would the required test need to be listed twice, both on the begin_subtest line *and* in the require_subtest line? And again, why would the test id have to be any different that the existing test names? The tests already have names, so I don't understand why we would want to introduce some other kind of identification. Seems like it's just going to add extra confusion. And speaking of which, I sometimes worry that the test infrastructure itself is getting too complicated. Pretty soon we're going to need tests for the tests. I don't necessarily see the need to all of these extra features in the test suite, so I worry that it's just making everything harder to debug. jamie. pgp9TD61sKYHo.pgp Description: PGP signature ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
[PATCH] test: optionally print subtest number
On Wed, 14 Dec 2011 03:24:23 +0400, Dmitry Kurochkin wrote: > BTW I have some plans to introduce optional explicit test ids that can > be used for inter-test dependencies. E.g.: > > test_begin_subtest test-id-1 "A subtest" > > ;; in another test requre that test-id-1 passed > test_require_subtest test-id-1 Would the required test need to be listed twice, both on the begin_subtest line *and* in the require_subtest line? And again, why would the test id have to be any different that the existing test names? The tests already have names, so I don't understand why we would want to introduce some other kind of identification. Seems like it's just going to add extra confusion. And speaking of which, I sometimes worry that the test infrastructure itself is getting too complicated. Pretty soon we're going to need tests for the tests. I don't necessarily see the need to all of these extra features in the test suite, so I worry that it's just making everything harder to debug. jamie. -- next part -- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: not available URL: <http://notmuchmail.org/pipermail/notmuch/attachments/20111213/f59534cd/attachment.pgp>
Re: [PATCH] test: optionally print subtest number
On Tue, 13 Dec 2011 19:18:16 -0400, David Bremner wrote: > We don't actually have test names, at least not ones directly suitable > for file names. I guess we could encode them or something, is that what > you mean? Don't we? There is at least a word/phrase that's consistent between the actual test script names, what is listed in the TESTS variable (below), and what is output when the tests are run (ie. "basic:" or "search-folder-coherence:"). Are those not test names? jamie. TESTS=" basic help-test new count search search-output search-by-folder search-position-overlap-bug search-insufficient-from-quoting search-limiting tagging json multipart thread-naming raw reply dump-restore uuencode thread-order author-order from-guessing long-id encoding emacs emacs-large-search-buffer maildir-sync crypto symbol-hiding search-folder-coherence atomicity python hooks argument-parsing " pgpU47OLxshrk.pgp Description: PGP signature ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
[PATCH] test: optionally print subtest number
On Tue, 13 Dec 2011 19:18:16 -0400, David Bremner wrote: > We don't actually have test names, at least not ones directly suitable > for file names. I guess we could encode them or something, is that what > you mean? Don't we? There is at least a word/phrase that's consistent between the actual test script names, what is listed in the TESTS variable (below), and what is output when the tests are run (ie. "basic:" or "search-folder-coherence:"). Are those not test names? jamie. TESTS=" basic help-test new count search search-output search-by-folder search-position-overlap-bug search-insufficient-from-quoting search-limiting tagging json multipart thread-naming raw reply dump-restore uuencode thread-order author-order from-guessing long-id encoding emacs emacs-large-search-buffer maildir-sync crypto symbol-hiding search-folder-coherence atomicity python hooks argument-parsing " -- next part -- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: not available URL: <http://notmuchmail.org/pipermail/notmuch/attachments/20111213/83d30334/attachment.pgp>
Re: [PATCH] test: optionally print subtest number
On Tue, 13 Dec 2011 19:18:16 -0400, David Bremner wrote: > On Tue, 13 Dec 2011 14:22:21 -0800, Jameson Graef Rollins > wrote: > > > I've only been vaguely following this "test count" stuff, but I'm not > > sure I understand what's the point of giving tests a number that is > > ultimately mutable. Why not just label things by the test name, instead > > of the count? That wouldn't require keeping track of number/name > > mapping, which will change over time. > > We don't actually have test names, at least not ones directly suitable > for file names. I guess we could encode them or something, is that what > you mean? > BTW I have some plans to introduce optional explicit test ids that can be used for inter-test dependencies. E.g.: test_begin_subtest test-id-1 "A subtest" ;; in another test requre that test-id-1 passed test_require_subtest test-id-1 Regards, Dmitry > d > ___ > notmuch mailing list > notmuch@notmuchmail.org > http://notmuchmail.org/mailman/listinfo/notmuch ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
Re: [PATCH] test: optionally print subtest number
On Tue, 13 Dec 2011 14:22:21 -0800, Jameson Graef Rollins wrote: > I've only been vaguely following this "test count" stuff, but I'm not > sure I understand what's the point of giving tests a number that is > ultimately mutable. Why not just label things by the test name, instead > of the count? That wouldn't require keeping track of number/name > mapping, which will change over time. We don't actually have test names, at least not ones directly suitable for file names. I guess we could encode them or something, is that what you mean? d ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
Re: [PATCH] test: optionally print subtest number
On Tue, 13 Dec 2011 15:56:47 -0400, David Bremner wrote: > The idea is that $test_count could be used in tests to label > intermediate files. The output enabled by this patch (and --debug) > helps figure out which OUTPUT.nn file belongs to which test in case > several subtests write to OUTPUT.$test_count I've only been vaguely following this "test count" stuff, but I'm not sure I understand what's the point of giving tests a number that is ultimately mutable. Why not just label things by the test name, instead of the count? That wouldn't require keeping track of number/name mapping, which will change over time. jamie. pgpnNvruW2Eyx.pgp Description: PGP signature ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
[PATCH] test: optionally print subtest number
On Tue, 13 Dec 2011 15:56:47 -0400, David Bremner wrote: > The idea is that $test_count could be used in tests to label > intermediate files. The output enabled by this patch (and --debug) > helps figure out which OUTPUT.nn file belongs to which test in case > several subtests write to OUTPUT.$test_count I've only been vaguely following this "test count" stuff, but I'm not sure I understand what's the point of giving tests a number that is ultimately mutable. Why not just label things by the test name, instead of the count? That wouldn't require keeping track of number/name mapping, which will change over time. jamie. -- next part -- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: not available URL: <http://notmuchmail.org/pipermail/notmuch/attachments/20111213/b1964bd7/attachment.pgp>
[PATCH] notmuch: unref charset_filter to fix one memory leak
In my use case g_object_unref(charset_filter) reduces memory
consumption over 90% when 'notmuch show --format=text "*"' is
executed (~11000 messages, RES ~330M -> ~25M).
---
notmuch-show.c |4 +++-
1 files changed, 3 insertions(+), 1 deletions(-)
diff --git a/notmuch-show.c b/notmuch-show.c
index 873a7c4..23d7368 100644
--- a/notmuch-show.c
+++ b/notmuch-show.c
@@ -450,6 +450,7 @@ show_text_part_content (GMimeObject *part, GMimeStream
*stream_out)
{
GMimeContentType *content_type = g_mime_object_get_content_type
(GMIME_OBJECT (part));
GMimeStream *stream_filter = NULL;
+GMimeFilter *charset_filter = NULL;
GMimeDataWrapper *wrapper;
const char *charset;
@@ -466,7 +467,6 @@ show_text_part_content (GMimeObject *part, GMimeStream
*stream_out)
charset = g_mime_object_get_content_type_parameter (part, "charset");
if (charset) {
- GMimeFilter *charset_filter;
charset_filter = g_mime_filter_charset_new (charset, "UTF-8");
/* This result can be NULL for things like "unknown-8bit".
* Don't set a NULL filter as that makes GMime print
@@ -479,6 +479,8 @@ show_text_part_content (GMimeObject *part, GMimeStream
*stream_out)
wrapper = g_mime_part_get_content_object (GMIME_PART (part));
if (wrapper && stream_filter)
g_mime_data_wrapper_write_to_stream (wrapper, stream_filter);
+if (charset_filter)
+ g_object_unref (charset_filter);
if (stream_filter)
g_object_unref(stream_filter);
}
--
1.7.6.1
Re: [PATCH] test: optionally print subtest number
On Wed, 14 Dec 2011 00:58:35 +0400, Dmitry Kurochkin wrote: > On Tue, 13 Dec 2011 22:55:18 +0200, Tomi Ollila wrote: > > On Wed, 14 Dec 2011 00:15:43 +0400, Dmitry Kurochkin > > wrote: > > > > > > Looks good to me. Except for tabs taking too much space. Perhaps the > > > following would be better? > > > > > > printf "%-4s" "[$(($test_count - 1))]" > > > > I attempted the same size reduction. Therefore I know that > > this should to be either > > > > printf " %-4s" "[$(($test_count - 1))]" > > or > > printf "\t%-4s" "[$(($test_count - 1))]" > > > > (or something similar) so that there is space betweem BROKEN and [num] > > > > Oh, apparently, not enough testing on my side. > > I vote for the first version with a space. Have to agree, Now that I checked tabs were not used in output lines before this addition. > > Regards, > Dmitry Tomi > > > This takes 4 bytes out from width (and drops tab as field separator) > > (and only few lines goes over 80 char width (some goes even with this > > reduction). So ... > > > > > > > Regards, > > > Dmitry > > > > Tomi ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
Re: [PATCH] test: optionally print subtest number
On Tue, 13 Dec 2011 22:55:18 +0200, Tomi Ollila wrote: > On Wed, 14 Dec 2011 00:15:43 +0400, Dmitry Kurochkin > wrote: > > On Tue, 13 Dec 2011 15:56:47 -0400, David Bremner wrote: > > > From: David Bremner > > > > > > The idea is that $test_count could be used in tests to label > > > intermediate files. The output enabled by this patch (and --debug) > > > helps figure out which OUTPUT.nn file belongs to which test in case > > > several subtests write to OUTPUT.$test_count > > > --- > > > > > > Is there something that depends on the test format? I find it pretty > > > handy to have the subtest numbers, but I don't want to break some > > > other tools. I followed the existing style of conditionally defining > > > functions, but maybe someone with more bash-fu can improve that. > > > > > > > Looks good to me. Except for tabs taking too much space. Perhaps the > > following would be better? > > > > printf "%-4s" "[$(($test_count - 1))]" > > I attempted the same size reduction. Therefore I know that > this should to be either > > printf " %-4s" "[$(($test_count - 1))]" > or > printf "\t%-4s" "[$(($test_count - 1))]" > > (or something similar) so that there is space betweem BROKEN and [num] > Oh, apparently, not enough testing on my side. I vote for the first version with a space. Regards, Dmitry > This takes 4 bytes out from width (and drops tab as field separator) > (and only few lines goes over 80 char width (some goes even with this > reduction). So ... > > > > Regards, > > Dmitry > > Tomi > ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
Re: [PATCH] test: optionally print subtest number
On Wed, 14 Dec 2011 00:15:43 +0400, Dmitry Kurochkin wrote: > On Tue, 13 Dec 2011 15:56:47 -0400, David Bremner wrote: > > From: David Bremner > > > > The idea is that $test_count could be used in tests to label > > intermediate files. The output enabled by this patch (and --debug) > > helps figure out which OUTPUT.nn file belongs to which test in case > > several subtests write to OUTPUT.$test_count > > --- > > > > Is there something that depends on the test format? I find it pretty > > handy to have the subtest numbers, but I don't want to break some > > other tools. I followed the existing style of conditionally defining > > functions, but maybe someone with more bash-fu can improve that. > > > > Looks good to me. Except for tabs taking too much space. Perhaps the > following would be better? > > printf "%-4s" "[$(($test_count - 1))]" I attempted the same size reduction. Therefore I know that this should to be either printf " %-4s" "[$(($test_count - 1))]" or printf "\t%-4s" "[$(($test_count - 1))]" (or something similar) so that there is space betweem BROKEN and [num] This takes 4 bytes out from width (and drops tab as field separator) (and only few lines goes over 80 char width (some goes even with this reduction). So ... > Regards, > Dmitry Tomi ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
Re: [PATCH] test: optionally print subtest number
On Tue, 13 Dec 2011 15:56:47 -0400, David Bremner wrote:
> From: David Bremner
>
> The idea is that $test_count could be used in tests to label
> intermediate files. The output enabled by this patch (and --debug)
> helps figure out which OUTPUT.nn file belongs to which test in case
> several subtests write to OUTPUT.$test_count
> ---
>
> Is there something that depends on the test format? I find it pretty
> handy to have the subtest numbers, but I don't want to break some
> other tools. I followed the existing style of conditionally defining
> functions, but maybe someone with more bash-fu can improve that.
+1. I would also say '+1' even non-debug format (and update dependend code,
if there is any).
Tomi
>
> test/test-lib.sh | 12
> 1 files changed, 12 insertions(+), 0 deletions(-)
>
> diff --git a/test/test-lib.sh b/test/test-lib.sh
> index 6be93fe..f5c 100644
> --- a/test/test-lib.sh
> +++ b/test/test-lib.sh
> @@ -116,6 +116,16 @@ do
> esac
> done
>
> +if test -n "$debug"; then
> +print_subtest () {
> + printf "\t[%d]\t" $(($test_count - 1))
> +}
> +else
> +print_subtest () {
> + true
> +}
> +fi
> +
> if test -n "$color"; then
> say_color () {
> (
> @@ -132,6 +142,7 @@ if test -n "$color"; then
> printf " "
> printf "$@"
> tput sgr0
> + print_subtest
> )
> }
> else
> @@ -140,6 +151,7 @@ else
> shift
> printf " "
> printf "$@"
> + print_subtest
> }
> fi
>
> --
> 1.7.5.4
>
> ___
> notmuch mailing list
> notmuch@notmuchmail.org
> http://notmuchmail.org/mailman/listinfo/notmuch
>
--
Tomi Ollila !! tomi.oll...@nixu.fi !! +358 400 888 366
Nixu Oy !! http://www.nixu.fi/ !! Keilaranta 15, FI-02150 Espoo, Finland
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch
[Alpha PATCH 5/6] notmuch-restore: add --format=notmuch support
From: David Bremner
This is format is whitespace separated tokens, encoded by
util/hex-escape.c
---
notmuch-restore.c | 83
1 files changed, 64 insertions(+), 19 deletions(-)
diff --git a/notmuch-restore.c b/notmuch-restore.c
index 87d9772..f392590 100644
--- a/notmuch-restore.c
+++ b/notmuch-restore.c
@@ -19,6 +19,7 @@
*/
#include "notmuch-client.h"
+#include "dump-restore-private.h"
int
notmuch_restore_command (unused (void *ctx), int argc, char *argv[])
@@ -35,6 +36,7 @@ notmuch_restore_command (unused (void *ctx), int argc, char
*argv[])
regex_t regex;
int rerr;
int opt_index;
+int input_format = DUMP_FORMAT_SUP;
config = notmuch_config_open (ctx, NULL, NULL);
if (config == NULL)
@@ -48,6 +50,7 @@ notmuch_restore_command (unused (void *ctx), int argc, char
*argv[])
synchronize_flags = notmuch_config_get_maildir_synchronize_flags (config);
notmuch_opt_desc_t options[] = {
+ FORMAT_DESC (input_format),
{ NOTMUCH_OPT_POSITION, &input_file_name, 0, 0, 0 },
{ NOTMUCH_OPT_BOOLEAN, &accumulate, "accumulate", 'a', 0 },
{ 0, 0, 0, 0, 0 }
@@ -81,33 +84,63 @@ notmuch_restore_command (unused (void *ctx), int argc, char
*argv[])
* non-space characters for the message-id, then one or more
* spaces, then a list of space-separated tags as a sequence of
* characters within literal '(' and ')'. */
-if ( xregcomp (®ex,
- "^([^ ]+) \\(([^)]*)\\)$",
- REG_EXTENDED) )
- INTERNAL_ERROR("compile time constant regex failed.");
+if (input_format == DUMP_FORMAT_SUP) {
+ if ( xregcomp (®ex,
+ "^([^ ]+) \\(([^)]*)\\)$",
+ REG_EXTENDED) )
+ INTERNAL_ERROR("compile time constant regex failed.");
+}
+
+
+/* These are out here to re-use the buffers with hex_decode */
+
+char *message_id = NULL;
+size_t message_id_size=0;
+char *tag = NULL;
+size_t tag_size=0;
while ((line_len = getline (&line, &line_size, input)) != -1) {
regmatch_t match[3];
- char *message_id, *file_tags, *tag, *next;
+ char *file_tags, *next;
notmuch_message_t *message = NULL;
+
notmuch_status_t status;
notmuch_tags_t *db_tags;
char *db_tags_str;
chomp_newline (line);
- rerr = xregexec (®ex, line, 3, match, 0);
- if (rerr == REG_NOMATCH)
- {
- fprintf (stderr, "Warning: Ignoring invalid input line: %s\n",
-line);
- continue;
+ /* Silently ignore blank lines */
+
+ if (line[0] == '\0') {
+ continue;
+ }
+
+ if (input_format == DUMP_FORMAT_SUP) {
+ rerr = xregexec (®ex, line, 3, match, 0);
+ if (rerr == REG_NOMATCH)
+ {
+ fprintf (stderr, "Warning: Ignoring invalid input line: %s\n",
+line);
+ continue;
+ }
+ message_id = talloc_strndup (notmuch, line + match[1].rm_so,
+match[1].rm_eo - match[1].rm_so);
+ file_tags = talloc_strndup (notmuch, line + match[2].rm_so,
+ match[2].rm_eo - match[2].rm_so);
+ } else {
+ char *p = line;
+ char *raw_mid;
+
+ raw_mid = strsep (&p, " \t");
+
+ if (hex_decode (notmuch, raw_mid,
+ &message_id, &message_id_size) != HEX_SUCCESS)
+ return 1;
+
+ file_tags = xstrdup (p);
}
- message_id = xstrndup (line + match[1].rm_so,
- match[1].rm_eo - match[1].rm_so);
- file_tags = xstrndup (line + match[2].rm_so,
- match[2].rm_eo - match[2].rm_so);
status = notmuch_database_find_message (notmuch, message_id, &message);
if (status || message == NULL) {
@@ -153,7 +186,16 @@ notmuch_restore_command (unused (void *ctx), int argc,
char *argv[])
next = file_tags;
while (next) {
- tag = strsep (&next, " ");
+ char *raw_tag = strsep (&next, " ");
+
+ if (input_format == DUMP_FORMAT_NOTMUCH) {
+ if (hex_decode (notmuch, raw_tag,
+ &tag, &tag_size) != HEX_SUCCESS)
+ return 1;
+ } else {
+ tag = talloc_strdup(notmuch, raw_tag);
+ }
+
if (*tag == '\0')
continue;
status = notmuch_message_add_tag (message, tag);
@@ -175,11 +217,14 @@ notmuch_restore_command (unused (void *ctx), int argc,
char *argv[])
if (message)
notmuch_message_destroy (message);
message = NULL;
- free (message_id);
- free (file_tags);
+ if (input_format == DUMP_FORMAT_SUP) {
+ talloc_free (message_id);
+ talloc_free (fi
[Alpha PATCH 3/6] notmuch-dump: add --format=(notmuch|sup)
From: David Bremner
sup is the old format, and remains the default.
Each line of the notmuch format is "msg_id tag tag...tag" where each
space seperated token is 'hex-encoded' to remove troubling characters.
In particular this format won't have the same problem with e.g. spaces
in message-ids or tags; they will be round-trip-able.
---
dump-restore-private.h | 20
notmuch-dump.c | 40 ++--
2 files changed, 54 insertions(+), 6 deletions(-)
create mode 100644 dump-restore-private.h
diff --git a/dump-restore-private.h b/dump-restore-private.h
new file mode 100644
index 000..eda3219
--- /dev/null
+++ b/dump-restore-private.h
@@ -0,0 +1,20 @@
+#ifndef DUMP_RESTORE_PRIVATE_H
+#define DUMP_RESTORE_PRIVATE_H
+
+#include "hex-escape.h"
+#include "command-line-arguments.h"
+
+typedef enum dump_formats {
+DUMP_FORMAT_SUP,
+DUMP_FORMAT_NOTMUCH
+} dump_format_t;
+
+#define FORMAT_DESC(out) \
+ { \
+ NOTMUCH_OPT_KEYWORD, &out, "format", 'f', \
+ (notmuch_keyword_t []){ { "sup", DUMP_FORMAT_SUP }, \
+ { "notmuch", DUMP_FORMAT_NOTMUCH }, \
+ {0, 0} }\
+ }
+
+#endif
diff --git a/notmuch-dump.c b/notmuch-dump.c
index a735875..5f788f9 100644
--- a/notmuch-dump.c
+++ b/notmuch-dump.c
@@ -19,6 +19,7 @@
*/
#include "notmuch-client.h"
+#include "dump-restore-private.h"
int
notmuch_dump_command (unused (void *ctx), int argc, char *argv[])
@@ -44,7 +45,10 @@ notmuch_dump_command (unused (void *ctx), int argc, char
*argv[])
char *output_file_name = NULL;
int opt_index;
+int output_format = DUMP_FORMAT_SUP;
+
notmuch_opt_desc_t options[] = {
+ FORMAT_DESC (output_format),
{ NOTMUCH_OPT_POSITION, &output_file_name, 0, 0, 0 },
{ 0, 0, 0, 0, 0 }
};
@@ -85,29 +89,53 @@ notmuch_dump_command (unused (void *ctx), int argc, char
*argv[])
*/
notmuch_query_set_sort (query, NOTMUCH_SORT_UNSORTED);
+char *buffer=NULL;
+size_t buffer_size=0;
+
for (messages = notmuch_query_search_messages (query);
notmuch_messages_valid (messages);
notmuch_messages_move_to_next (messages))
{
int first = 1;
- message = notmuch_messages_get (messages);
+ const char *message_id;
- fprintf (output,
-"%s (", notmuch_message_get_message_id (message));
+ message = notmuch_messages_get (messages);
+ message_id = notmuch_message_get_message_id (message);
+
+ if (output_format == DUMP_FORMAT_SUP) {
+ fprintf (output, "%s (", message_id);
+ } else {
+ if (hex_encode (notmuch, message_id,
+ &buffer, &buffer_size) != HEX_SUCCESS)
+ return 1;
+ fprintf (output, "%s ", buffer);
+ }
for (tags = notmuch_message_get_tags (message);
notmuch_tags_valid (tags);
notmuch_tags_move_to_next (tags))
{
+ const char *tag_str = notmuch_tags_get (tags);
+
if (! first)
- fprintf (output, " ");
+ fputs (" ", output);
- fprintf (output, "%s", notmuch_tags_get (tags));
+ if (output_format == DUMP_FORMAT_SUP) {
+ fputs (tag_str, output);
+ } else {
+ if (hex_encode (notmuch, tag_str,
+ &buffer, &buffer_size) != HEX_SUCCESS)
+ return 1;
+ fputs (buffer, output);
+ }
first = 0;
}
- fprintf (output, ")\n");
+ if (output_format == DUMP_FORMAT_SUP)
+ fputs (")\n",output);
+ else
+ fputs ("\n", output);
notmuch_message_destroy (message);
}
--
1.7.5.4
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch
[Alpha PATCH 6/6] test: second set of dump/restore --format=notmuch tests
From: David Bremner
These one need the completed functionality in notmuch-restore. Fairly
exotic tags are tested, but no weird message id's.
---
test/dump-restore | 33 +
1 files changed, 33 insertions(+), 0 deletions(-)
diff --git a/test/dump-restore b/test/dump-restore
index 48caf4e..122de5c 100755
--- a/test/dump-restore
+++ b/test/dump-restore
@@ -94,4 +94,37 @@ notmuch dump --format=sup -- from:cworth | tr -d \(\) >
EXPECTED.$test_count
notmuch dump --format=notmuch -- from:cworth > OUTPUT.$test_count
test_expect_equal_file OUTPUT.$test_count EXPECTED.$test_count
+test_begin_subtest "format=notmuch, # round-trip"
+notmuch dump --format=sup | sort > EXPECTED.$test_count
+notmuch dump --format=notmuch | notmuch restore --format=notmuch
+notmuch dump --format=sup | sort > OUTPUT.$test_count
+test_expect_equal_file EXPECTED.$test_count OUTPUT.$test_count
+
+tag1='comic_swear=$&^%$^%\\//-+$^%$'
+enc1=$($TEST_DIRECTORY/hex-xcode e "$tag1")
+
+tag2=$(printf 'this\n tag\t has\n spaces')
+enc2=$($TEST_DIRECTORY/hex-xcode e "$tag2")
+
+notmuch dump --format=notmuch > BACKUP
+
+notmuch tag +"$tag1" +"$tag2" -inbox -unread "*"
+
+test_begin_subtest 'format=notmuch, round trip with strange tags'
+ notmuch dump --format=notmuch > EXPECTED.$test_count
+ notmuch dump --format=notmuch | notmuch restore --format=notmuch
+ notmuch dump --format=notmuch > OUTPUT.$test_count
+test_expect_equal_file EXPECTED.$test_count OUTPUT.$test_count
+
+
+test_begin_subtest 'format=notmuch, checking encoded output'
+cp /dev/null EXPECTED.$test_count
+notmuch dump --format=notmuch -- from:cworth |\
+awk "{ print \$1 \" $enc1 $enc2\" }" > EXPECTED.$test_count
+
+notmuch dump --format=notmuch -- from:cworth > OUTPUT.$test_count
+test_expect_equal_file EXPECTED.$test_count OUTPUT.$test_count
+
+notmuch restore --format=notmuch < BACKUP
+
test_done
--
1.7.5.4
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch
[Alpha PATCH 1/6] util/hex-escape.[ch]: encoding/decoding strings into restricted character set
From: David Bremner
The character set is chosen to be suitable for pathnames, and the same as that
used by contrib/nmbug
---
util/Makefile.local |2 +-
util/hex-escape.c | 150 +++
util/hex-escape.h | 15 +
3 files changed, 166 insertions(+), 1 deletions(-)
create mode 100644 util/hex-escape.c
create mode 100644 util/hex-escape.h
diff --git a/util/Makefile.local b/util/Makefile.local
index 26e4c3f..2e63932 100644
--- a/util/Makefile.local
+++ b/util/Makefile.local
@@ -3,7 +3,7 @@
dir := util
extra_cflags += -I$(srcdir)/$(dir)
-libutil_c_srcs := $(dir)/xutil.c $(dir)/error_util.c
+libutil_c_srcs := $(dir)/xutil.c $(dir)/error_util.c $(dir)/hex-escape.c
libutil_modules := $(libutil_c_srcs:.c=.o)
diff --git a/util/hex-escape.c b/util/hex-escape.c
new file mode 100644
index 000..3e08465
--- /dev/null
+++ b/util/hex-escape.c
@@ -0,0 +1,150 @@
+/* pathname.c - Manage encoding and decoding of byte strings into path names
+ *
+ * Copyright (c) 2011 David Bremner
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 3 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see http://www.gnu.org/licenses/ .
+ *
+ * Author: David Bremner
+ */
+
+#include
+#include
+#include "error_util.h"
+#include "hex-escape.h"
+
+static const size_t default_buf_size=1024;
+
+static const char* output_charset=
+"ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+-_@=.:,";
+
+static const char escape_char='%';
+
+static int
+is_output (char c) {
+return (strchr (output_charset, c) != NULL);
+}
+
+
+static int
+maybe_realloc(void *ctx, size_t needed, char **out, size_t *out_size)
+{
+if (*out_size < needed) {
+
+ if (*out == NULL)
+ *out = talloc_size(ctx,needed);
+ else
+ *out = talloc_realloc(ctx,*out,char,needed);
+
+ if (*out == NULL)
+ return 0;
+
+ *out_size = needed;
+}
+return 1;
+}
+
+
+hex_status_t
+hex_encode (void *ctx, const char *in, char **out, size_t *out_size)
+{
+
+const char *p;
+char *q;
+
+int escape_count=0;
+size_t needed;
+
+for (p = in; *p; p++)
+ escape_count += (! is_output (*p));
+
+needed = strlen (in) + 2*escape_count + 1;
+
+if (*out == NULL)
+ *out_size=0;
+
+if (!maybe_realloc (ctx, needed, out, out_size))
+ return HEX_OUT_OF_MEMORY;
+
+q = *out;
+p = in;
+
+while (*p) {
+ if (is_output (*p)) {
+ *q++ = *p++;
+ } else {
+ sprintf (q, "%%%02x", *p++);
+ q += 3;
+ }
+}
+
+*q = '\0';
+return HEX_SUCCESS;
+}
+
+
+hex_status_t
+hex_decode (void *ctx, const char *in, char **out, size_t *out_size) {
+
+char buf[3];
+
+const char *p;
+char *q;
+
+size_t escape_count = 0;
+size_t needed = 0;
+
+size_t len = strlen (in);
+
+for (p = in; *p; p++)
+ escape_count += (*p == escape_char);
+
+needed = len - escape_count*2 +1;
+
+if (!maybe_realloc(ctx, needed, out, out_size))
+ return HEX_OUT_OF_MEMORY;
+
+p = in;
+q = *out;
+buf[2]=0;
+
+
+while (*p) {
+
+ if (*p == escape_char) {
+
+ char *endp;
+
+ if (len < 3)
+ return HEX_SYNTAX_ERROR;
+
+ buf[0]=p[1];
+ buf[1]=p[2];
+
+ *q = strtol(buf, &endp, 16);
+
+ if (endp != buf+2)
+ return HEX_SYNTAX_ERROR;
+
+ len -= 3;
+ p += 3;
+ q++;
+ } else {
+ *q++ = *p++;
+ }
+}
+
+*q = '\0';
+
+return HEX_SUCCESS;
+}
diff --git a/util/hex-escape.h b/util/hex-escape.h
new file mode 100644
index 000..98ecbe0
--- /dev/null
+++ b/util/hex-escape.h
@@ -0,0 +1,15 @@
+#ifndef _HEX_ESCAPE_H
+#define _HEX_ESCAPE_H
+
+typedef enum hex_status {
+HEX_SUCCESS = 0,
+HEX_SYNTAX_ERROR,
+HEX_OUT_OF_MEMORY
+} hex_status_t;
+
+hex_status_t
+hex_encode (void *talloc_ctx, const char *in, char **out, size_t *out_size);
+
+hex_status_t
+hex_decode (void *talloc_ctx, const char *in, char **out, size_t *out_size);
+#endif
--
1.7.5.4
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch
New Dump/Restore Format
Hi All; There are some style/doc issues remaining, but because bugs in dump and restore really suck, I thought I would ask for early feedback on functionality. I'm particularly interested in how the new dump format works for weird message-ids (spaces and so on). If you have public messages with tricky message-id's, I'd appreciate adding those messages to the test suite. Things to bikeshed now: name(s) of the formats; sup and notmuch are maybe not ideal. The format itself? The encoding format? The latter is chosen for compatibility with nmbug, but we could discussing using a bigger character set. Things I know about - not enough tests - no man page, online docs. - no API docs for hex_encode/blah. I think the code in hex-escape.[ch] is otherwise ready for (second) review; I'll probably do another review of the code in notmuch-(dump|restore).c myself for clarity, so you might want to wait for the next round before diving in. If you prefer pull from git, you can get these patches on branch "new-dump" at git://pivot.cs.unb.ca/notmuch.git ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
[Alpha PATCH 4/6] test: add test for dump --format=notmuch
From: David Bremner The first test is really to test our assumptions about the corpus, namely that a certain set of message-id's is safe (i.e. doesn't change under hex-escaping). We then check dump output as best we can without functionality-to-come in notmuch-restore. --- test/dump-restore | 12 1 files changed, 12 insertions(+), 0 deletions(-) diff --git a/test/dump-restore b/test/dump-restore index 439e998..48caf4e 100755 --- a/test/dump-restore +++ b/test/dump-restore @@ -82,4 +82,16 @@ test_begin_subtest "dump outfile -- from:cworth" notmuch dump dump-outfile-dash-inbox.actual -- from:cworth test_expect_equal_file dump-cworth.expected dump-outfile-dash-inbox.actual +test_begin_subtest "Check for a safe set of message-ids" +notmuch search --output=messages from:cworth > EXPECTED.$test_count +notmuch search --output=messages from:cworth |\ + $TEST_DIRECTORY/hex-xcode e > OUTPUT.$test_count +test_expect_equal_file OUTPUT.$test_count EXPECTED.$test_count + +# we have observed that cworth has sane message-ids, and hopefully sane tags. +test_begin_subtest "dump --format=notmuch -- from:cworth" +notmuch dump --format=sup -- from:cworth | tr -d \(\) > EXPECTED.$test_count +notmuch dump --format=notmuch -- from:cworth > OUTPUT.$test_count +test_expect_equal_file OUTPUT.$test_count EXPECTED.$test_count + test_done -- 1.7.5.4 ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
[Alpha PATCH 2/6] test: add test for hex_(encode|decode)
From: David Bremner
The test binary hex-xcode can also be used to check for memory leaks.
---
test/.gitignore |1 +
test/Makefile.local |6 +++-
test/basic |2 +-
test/hex-escaping | 20 +
test/hex-xcode.c| 76 +++
test/notmuch-test |1 +
6 files changed, 104 insertions(+), 2 deletions(-)
create mode 100755 test/hex-escaping
create mode 100644 test/hex-xcode.c
diff --git a/test/.gitignore b/test/.gitignore
index e63c689..be7ab5e 100644
--- a/test/.gitignore
+++ b/test/.gitignore
@@ -3,4 +3,5 @@ corpus.mail
smtp-dummy
symbol-test
arg-test
+hex-xcode
tmp.*
diff --git a/test/Makefile.local b/test/Makefile.local
index 6cb6c82..d27826a 100644
--- a/test/Makefile.local
+++ b/test/Makefile.local
@@ -13,6 +13,9 @@ smtp_dummy_modules = $(smtp_dummy_srcs:.c=.o)
$(dir)/arg-test: $(dir)/arg-test.o command-line-arguments.o util/libutil.a
$(call quiet,CC) -I. $^ -o $@
+$(dir)/hex-xcode: $(dir)/hex-xcode.o util/libutil.a
+ $(call quiet,CC) -I. $^ -o $@ -ltalloc
+
$(dir)/smtp-dummy: $(smtp_dummy_modules)
$(call quiet,CC) $^ -o $@
@@ -21,7 +24,8 @@ $(dir)/symbol-test: $(dir)/symbol-test.o
.PHONY: test check
-test-binaries: $(dir)/arg-test $(dir)/smtp-dummy $(dir)/symbol-test
+test-binaries: $(dir)/arg-test $(dir)/hex-xcode \
+$(dir)/smtp-dummy $(dir)/symbol-test
test: all test-binaries
@${dir}/notmuch-test $(OPTIONS)
diff --git a/test/basic b/test/basic
index d6aed24..af57026 100755
--- a/test/basic
+++ b/test/basic
@@ -54,7 +54,7 @@ test_begin_subtest 'Ensure that all available tests will be
run by notmuch-test'
eval $(sed -n -e '/^TESTS="$/,/^"$/p' $TEST_DIRECTORY/notmuch-test)
tests_in_suite=$(for i in $TESTS; do echo $i; done | sort)
available=$(find "$TEST_DIRECTORY" -maxdepth 1 -type f -executable -printf
'%f\n' | \
-sed -r -e
"/^(aggregate-results.sh|notmuch-test|smtp-dummy|test-verbose|symbol-test|arg-test)$/d"
| \
+sed -r -e
"/^(aggregate-results.sh|notmuch-test|smtp-dummy|test-verbose|symbol-test|arg-test|hex-xcode)$/d"
| \
sort)
test_expect_equal "$tests_in_suite" "$available"
diff --git a/test/hex-escaping b/test/hex-escaping
new file mode 100755
index 000..d0a993e
--- /dev/null
+++ b/test/hex-escaping
@@ -0,0 +1,20 @@
+#!/usr/bin/env bash
+test_description="hex encoding and decoding"
+. ./test-lib.sh
+
+test_begin_subtest "round trip"
+find $TEST_DIRECTORY/corpus -type f -print | sort | xargs cat > EXPECTED
+$TEST_DIRECTORY/hex-xcode e < EXPECTED | $TEST_DIRECTORY/hex-xcode d > OUTPUT
+test_expect_equal_file OUTPUT EXPECTED
+
+test_begin_subtest "punctuation"
+tag1='comic_swear=$&^%$^%\\//-+$^%$'
+tag_enc1=$($TEST_DIRECTORY/hex-xcode e "$tag1")
+test_expect_equal "$tag_enc1"
"comic_swear=%24%26%5e%25%24%5e%25%5c%5c%2f%2f-+%24%5e%25%24"
+
+test_begin_subtest "round trip newlines"
+printf 'this\n tag\t has\n spaces\n' > EXPECTED.$test_count
+$TEST_DIRECTORY/hex-xcode e < EXPECTED.$test_count |\
+ $TEST_DIRECTORY/hex-xcode d > OUTPUT.$test_count
+test_expect_equal_file EXPECTED.$test_count OUTPUT.$test_count
+test_done
diff --git a/test/hex-xcode.c b/test/hex-xcode.c
new file mode 100644
index 000..fff3e01
--- /dev/null
+++ b/test/hex-xcode.c
@@ -0,0 +1,76 @@
+/* No, nothing to to with IDE from Apple Inc.
+ testbed for ../util/hex-escape.c.
+
+ usage: hex-xcode (e|d) < foo
+
+ e is for encode
+ d is for decode
+ */
+
+#include "notmuch-client.h"
+#include "hex-escape.h"
+#include
+
+static int
+xcode(void *ctx, char dir, char *in, char **buf_p, size_t *size_p)
+{
+hex_status_t status;
+
+if (dir == 'e')
+ status = hex_encode (ctx, in, buf_p, size_p);
+else
+ status = hex_decode (ctx, in, buf_p, size_p);
+
+if (status == HEX_SUCCESS)
+ puts(*buf_p);
+
+return status;
+}
+
+
+int main (int argc, char **argv){
+
+assert(argc > 1 && argv[1]);
+
+char dir=argv[1][0];
+
+void *ctx = talloc_new(NULL);
+
+char *line = NULL;
+size_t line_size;
+ssize_t line_len;
+
+char *buffer=NULL;
+size_t buf_size=0;
+
+int arg_index=2;
+notmuch_bool_t read_stdin=TRUE;
+
+for (arg_index=2; arg_index < argc; arg_index++) {
+
+ if (xcode (ctx, dir, argv[arg_index],
+ &buffer, &buf_size) != HEX_SUCCESS)
+ return 1;
+
+ read_stdin=FALSE;
+}
+
+if (!read_stdin)
+ return 0;
+
+while ((line_len = getline (&line, &line_size, stdin)) != -1) {
+
+ chomp_newline (line);
+
+ if (xcode (ctx, dir, line, &buffer, &buf_size) != HEX_SUCCESS)
+ return 1;
+
+}
+
+if (line)
+ free(line);
+
+talloc_free(ctx);
+
+return 0;
+}
diff --git a/test/notmuch-test b/test/notmuch-test
index ded79e8..af72c73 100755
--- a/test/notmuch-test
+++ b/test/notmuch-test
@@ -50,6 +50,7 @@ TESTS="
python
hooks
argument-parsing
+ hex-escaping
"
TESTS=${NOTMUCH_TESTS:=$TESTS}
Re: [PATCH] test: optionally print subtest number
On Tue, 13 Dec 2011 15:56:47 -0400, David Bremner wrote:
> From: David Bremner
>
> The idea is that $test_count could be used in tests to label
> intermediate files. The output enabled by this patch (and --debug)
> helps figure out which OUTPUT.nn file belongs to which test in case
> several subtests write to OUTPUT.$test_count
> ---
>
> Is there something that depends on the test format? I find it pretty
> handy to have the subtest numbers, but I don't want to break some
> other tools. I followed the existing style of conditionally defining
> functions, but maybe someone with more bash-fu can improve that.
>
Looks good to me. Except for tabs taking too much space. Perhaps the
following would be better?
printf "%-4s" "[$(($test_count - 1))]"
Regards,
Dmitry
> test/test-lib.sh | 12
> 1 files changed, 12 insertions(+), 0 deletions(-)
>
> diff --git a/test/test-lib.sh b/test/test-lib.sh
> index 6be93fe..f5c 100644
> --- a/test/test-lib.sh
> +++ b/test/test-lib.sh
> @@ -116,6 +116,16 @@ do
> esac
> done
>
> +if test -n "$debug"; then
> +print_subtest () {
> + printf "\t[%d]\t" $(($test_count - 1))
> +}
> +else
> +print_subtest () {
> + true
> +}
> +fi
> +
> if test -n "$color"; then
> say_color () {
> (
> @@ -132,6 +142,7 @@ if test -n "$color"; then
> printf " "
> printf "$@"
> tput sgr0
> + print_subtest
> )
> }
> else
> @@ -140,6 +151,7 @@ else
> shift
> printf " "
> printf "$@"
> + print_subtest
> }
> fi
>
> --
> 1.7.5.4
>
> ___
> notmuch mailing list
> notmuch@notmuchmail.org
> http://notmuchmail.org/mailman/listinfo/notmuch
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch
[PATCH] test: optionally print subtest number
From: David Bremner
The idea is that $test_count could be used in tests to label
intermediate files. The output enabled by this patch (and --debug)
helps figure out which OUTPUT.nn file belongs to which test in case
several subtests write to OUTPUT.$test_count
---
Is there something that depends on the test format? I find it pretty
handy to have the subtest numbers, but I don't want to break some
other tools. I followed the existing style of conditionally defining
functions, but maybe someone with more bash-fu can improve that.
test/test-lib.sh | 12
1 files changed, 12 insertions(+), 0 deletions(-)
diff --git a/test/test-lib.sh b/test/test-lib.sh
index 6be93fe..f5c 100644
--- a/test/test-lib.sh
+++ b/test/test-lib.sh
@@ -116,6 +116,16 @@ do
esac
done
+if test -n "$debug"; then
+print_subtest () {
+ printf "\t[%d]\t" $(($test_count - 1))
+}
+else
+print_subtest () {
+ true
+}
+fi
+
if test -n "$color"; then
say_color () {
(
@@ -132,6 +142,7 @@ if test -n "$color"; then
printf " "
printf "$@"
tput sgr0
+ print_subtest
)
}
else
@@ -140,6 +151,7 @@ else
shift
printf " "
printf "$@"
+ print_subtest
}
fi
--
1.7.5.4
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch
[PATCH 2/2] test/test-lib.sh: launch smtp-dummy with --background and finally kill(1) it.
Take the new --background option of smtp-dummy to use so that it is known
there is smtpd listener ready when it is needed. As the smtp-dummy instance
is no longer child process of the script sending SIGKILL to it is the only
way to make sure the instance exits when required.
---
Resent after whitespace-cleanup in patch 1/2 (the other one).
test/test-lib.sh | 12 +++-
1 files changed, 7 insertions(+), 5 deletions(-)
diff --git a/test/test-lib.sh b/test/test-lib.sh
index 6be93fe..a857afe 100644
--- a/test/test-lib.sh
+++ b/test/test-lib.sh
@@ -382,8 +382,9 @@ emacs_deliver_message ()
shift 2
# before we can send a message, we have to prepare the FCC maildir
mkdir -p "$MAIL_DIR"/sent/{cur,new,tmp}
-$TEST_DIRECTORY/smtp-dummy sent_message &
-smtp_dummy_pid=$!
+
+smtp_dummy_pid=$($TEST_DIRECTORY/smtp-dummy --background sent_message) \
+ || return
test_emacs \
"(let ((message-send-mail-function 'message-smtpmail-send-it)
(smtpmail-smtp-server \"localhost\")
@@ -398,9 +399,10 @@ emacs_deliver_message ()
(insert \"${body}\")
$@
(message-send-and-exit))"
-# opportunistically quit smtp-dummy in case above fails.
-{ echo QUIT > /dev/tcp/localhost/25025; } 2>/dev/null
-wait ${smtp_dummy_pid}
+# Cannot wait, not our child. In case message was sent properly, client
+# waits for confirmation before exiting and resuming control here;
therefore
+# making sure that server exits by sending (KILL) signal to it is safe.
+kill -9 ${smtp_dummy_pid}
notmuch new >/dev/null
}
--
1.7.7.3
[PATCH 1/2] test/smtp-dummy: add --background option for going background after listen(2)
To avoid the possibility that smtp-dummy doesn't have chance to bind
its listening socket until something tries to send message to it this
option makes caller wait until socket is already listening for connections.
In case this --background option is used, the pid of running smtp-dummy
is printed on stdout.
---
Resent after whitespace-cleanup in patch 1/2 (this patch).
test/smtp-dummy.c | 35 ++-
1 files changed, 34 insertions(+), 1 deletions(-)
diff --git a/test/smtp-dummy.c b/test/smtp-dummy.c
index 3801a5e..9126c00 100644
--- a/test/smtp-dummy.c
+++ b/test/smtp-dummy.c
@@ -124,9 +124,21 @@ main (int argc, char *argv[])
struct hostent *hostinfo;
socklen_t peer_addr_len;
int reuse;
+ int bg;
+
+ /* XXX Quick implementation -- fix if more functionality is desired. */
+ if (argc >= 2 && strcmp(argv[1], "--background") == 0) {
+ argc--;
+ argv[1] = argv[0];
+ argv++;
+ bg = 1;
+ }
+ else
+ bg = 0;
if (argc != 2) {
- fprintf (stderr, "Usage: %s \n", argv[0]);
+ fprintf (stderr, "Usage: %s [--background] \n",
+argv[0]);
return 1;
}
@@ -179,7 +191,27 @@ main (int argc, char *argv[])
return 1;
}
+ if (bg) {
+ int pid = fork ();
+ if (pid > 0) {
+ printf ("%d\n", pid);
+ return 0;
+ }
+ if (pid < 0) {
+ fprintf (stderr, "Error: fork() failed: %s\n",
+strerror (errno));
+ close (sock);
+ return 1;
+ }
+ /* Reached if pid == 0. */
+ /* Close stdout so that the one interested in pid value will
+ also get EOF. */
+ close (1);
+ /* dup2() will re-reserve fd 1 (opportunistically, in case fd 2
+ is open. If that was not open we don't care fd 1 either.) */
+ dup2 (2, 1);
+ }
+
peer_addr_len = sizeof (peer_addr);
peer = accept (sock, (struct sockaddr *) &peer_addr, &peer_addr_len);
if (peer == -1) {
--
1.7.7.3
[alot] announcing v0.2 aka. "robot santa"
Thx Adrian & Jamie! It would be great to get it into Debian, yes. I will have a look at Debian packaging if we don't find a more capable volunteer in the next few weeks. Thanks for your initial work on this. While we're at it: I heard that there are some build scripts for Arch and Gentoo around. If you authored one of these step forward: I'd gladly include them to the git repo. Quoting Jameson Graef Rollins (2011-12-12 00:14:10) >Hey, Patrick. I'm guessing this isn't actually an alot issue, but when >I start up alot I get the following message to the terminal: > >/usr/lib/python2.7/dist-packages/zope/__init__.py:3: UserWarning: Module >argparse was already imported from /usr/lib/python2.7/argparse.pyc, but >/usr/lib/python2.7/dist-packages is being added to sys.path > import pkg_resources > >Any idea what's going on there? I have seen this before yes. It seems to be a problem with the python distribution on debian. Apparently, you have two different versions of the argparse module installed: As of 2.7, argparse is part of the standard lib, but debian ships a separate package python-argparse. What happens is that I import argparse to alot, which happens to pick the one from "/usr/lib/python2.7/argparse.pyc". Now zope (a 2nd order dependency) explicitly imports "/usr/lib/python2.7/dist-packages/argparse.pyc" and warns that its overloading this namespace. Also see https://bugs.launchpad.net/debian/+source/distribute/+bug/810019 for a similar problem with python-distribute, particularly comment #9. Best, /p
Re: [PATCH] notmuch: unref charset_filter (once more) to fix one memory leak
On Tue, 13 Dec 2011 20:18:48 +0200, tomi.oll...@iki.fi wrote:
> From: Tomi Ollila
>
> In my test case added g_object_unref(charset_filter) reduces memory
> consumption over 90% when 'notmuch show --format=text "*"' is
> executed (~11000 messages, RES ~330M -> ~25M).
> ---
> Thanks Dmitry. I did not realize unref unreferences, does not deallocate
> memory/data structures. g_mime_stream_filter_add() takes an additional
> reference to the charset_filter object.
Looks good to me.
Regards,
Dmitry
> notmuch-show.c |5 -
> 1 files changed, 4 insertions(+), 1 deletions(-)
>
> diff --git a/notmuch-show.c b/notmuch-show.c
> index 873a7c4..8da3295 100644
> --- a/notmuch-show.c
> +++ b/notmuch-show.c
> @@ -471,9 +471,12 @@ show_text_part_content (GMimeObject *part, GMimeStream
> *stream_out)
> /* This result can be NULL for things like "unknown-8bit".
>* Don't set a NULL filter as that makes GMime print
>* annoying assertion-failure messages on stderr. */
> - if (charset_filter)
> + if (charset_filter) {
> g_mime_stream_filter_add (GMIME_STREAM_FILTER (stream_filter),
> charset_filter);
> + g_object_unref (charset_filter);
> + }
> +
> }
>
> wrapper = g_mime_part_get_content_object (GMIME_PART (part));
> --
> 1.7.6.1
>
> ___
> notmuch mailing list
> notmuch@notmuchmail.org
> http://notmuchmail.org/mailman/listinfo/notmuch
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch
[PATCH] notmuch: unref charset_filter (once more) to fix one memory leak
From: Tomi Ollila
In my test case added g_object_unref(charset_filter) reduces memory
consumption over 90% when 'notmuch show --format=text "*"' is
executed (~11000 messages, RES ~330M -> ~25M).
---
Thanks Dmitry. I did not realize unref unreferences, does not deallocate
memory/data structures. g_mime_stream_filter_add() takes an additional
reference to the charset_filter object.
notmuch-show.c |5 -
1 files changed, 4 insertions(+), 1 deletions(-)
diff --git a/notmuch-show.c b/notmuch-show.c
index 873a7c4..8da3295 100644
--- a/notmuch-show.c
+++ b/notmuch-show.c
@@ -471,9 +471,12 @@ show_text_part_content (GMimeObject *part, GMimeStream
*stream_out)
/* This result can be NULL for things like "unknown-8bit".
* Don't set a NULL filter as that makes GMime print
* annoying assertion-failure messages on stderr. */
- if (charset_filter)
+ if (charset_filter) {
g_mime_stream_filter_add (GMIME_STREAM_FILTER (stream_filter),
charset_filter);
+ g_object_unref (charset_filter);
+ }
+
}
wrapper = g_mime_part_get_content_object (GMIME_PART (part));
--
1.7.6.1
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch
Re: [PATCH v3 0/4] Several minor Emacs enhancements
On Tue, 13 Dec 2011 18:32:08 +0100, Thomas Jost wrote: > Here is a rebased version of this patch series, with more descriptive > commit messages. Hi, Thomas. Thanks for resending these. To be clear, this is not really a patch series as the four patches are completely independent, don't depend on each other, and are fairly unrelated. In any event, they all look good to me. I have been using the emacs "choose where to compose new mails" patch for a couple of months now and I love it. So that one at least gets my vote to push. jamie. pgphg8zV6EDGo.pgp Description: PGP signature ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
[PATCH v3 0/4] Several minor Emacs enhancements
On Tue, 13 Dec 2011 18:32:08 +0100, Thomas Jost wrote: > Here is a rebased version of this patch series, with more descriptive > commit messages. Hi, Thomas. Thanks for resending these. To be clear, this is not really a patch series as the four patches are completely independent, don't depend on each other, and are fairly unrelated. In any event, they all look good to me. I have been using the emacs "choose where to compose new mails" patch for a couple of months now and I love it. So that one at least gets my vote to push. jamie. -- next part -- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: not available URL: <http://notmuchmail.org/pipermail/notmuch/attachments/20111213/c995404c/attachment.pgp>
Re: [PATCH v3 4/4] emacs: add notmuch-hello-hook
On Tue, 13 Dec 2011 18:32:12 +0100, Thomas Jost wrote: > This hook is called every time the notmuch-hello buffer is updated. > --- > emacs/notmuch-hello.el |9 - > 1 files changed, 8 insertions(+), 1 deletions(-) > > diff --git a/emacs/notmuch-hello.el b/emacs/notmuch-hello.el > index 0fe9c1d..112b40b 100644 > --- a/emacs/notmuch-hello.el > +++ b/emacs/notmuch-hello.el > @@ -131,6 +131,11 @@ So: > (integer :tag "Number of characters") > (float :tag "Fraction of window"))) > > +(defcustom notmuch-hello-hook nil > + "Functions called after populating a `notmuch-hello' buffer." > + :type 'hook > + :group 'notmuch) > + > (defcustom notmuch-thousands-separator "," >"The string used as a thousands separator. > > @@ -579,7 +584,9 @@ Complete list of currently available key bindings: > (widget-forward 1))) > >(unless (widget-at) > - (notmuch-hello-goto-search) > + (notmuch-hello-goto-search)) > + > + (run-hooks 'notmuch-hello-hook > I spent some time finding out why run-hooks are not on the top level. Turns out it is inside two let statements. Can we move to the top level of `notmuch-hello' to make it clear that it is always run (i.e. there are no ifs or whens)? BTW this would replace one of the oldest custom patches which I have in my master branch :) Regards, Dmitry > (defun notmuch-folder () >"Deprecated function for invoking notmuch---calling `notmuch' is preferred > now." > -- > 1.7.8 > > ___ > notmuch mailing list > notmuch@notmuchmail.org > http://notmuchmail.org/mailman/listinfo/notmuch ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
[PATCH v3 4/4] emacs: add notmuch-hello-hook
This hook is called every time the notmuch-hello buffer is updated. --- emacs/notmuch-hello.el |9 - 1 files changed, 8 insertions(+), 1 deletions(-) diff --git a/emacs/notmuch-hello.el b/emacs/notmuch-hello.el index 0fe9c1d..112b40b 100644 --- a/emacs/notmuch-hello.el +++ b/emacs/notmuch-hello.el @@ -131,6 +131,11 @@ So: (integer :tag "Number of characters") (float :tag "Fraction of window"))) +(defcustom notmuch-hello-hook nil + "Functions called after populating a `notmuch-hello' buffer." + :type 'hook + :group 'notmuch) + (defcustom notmuch-thousands-separator "," "The string used as a thousands separator. @@ -579,7 +584,9 @@ Complete list of currently available key bindings: (widget-forward 1))) (unless (widget-at) - (notmuch-hello-goto-search) + (notmuch-hello-goto-search)) + + (run-hooks 'notmuch-hello-hook (defun notmuch-folder () "Deprecated function for invoking notmuch---calling `notmuch' is preferred now." -- 1.7.8 ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
[PATCH v3 2/4] emacs: Add a face for crypto parts headers
Commit cb841878 introduced new parts handlers for crypto parts, but also hardcoded values for their headers face. This replaces these hardcoded values with a customizable face. --- emacs/notmuch-crypto.el |5 + emacs/notmuch-show.el |4 ++-- 2 files changed, 7 insertions(+), 2 deletions(-) diff --git a/emacs/notmuch-crypto.el b/emacs/notmuch-crypto.el index 44fccae..67c26af 100644 --- a/emacs/notmuch-crypto.el +++ b/emacs/notmuch-crypto.el @@ -37,6 +37,11 @@ mode." :group 'notmuch :type 'boolean) +(defface notmuch-crypto-part-header + '((t (:foreground "blue"))) + "Face used for crypto parts headers." + :group 'notmuch) + (defface notmuch-crypto-signature-good '((t (:background "green" :foreground "black"))) "Face used for good signatures." diff --git a/emacs/notmuch-show.el b/emacs/notmuch-show.el index 33ee3d8..ec9c52c 100644 --- a/emacs/notmuch-show.el +++ b/emacs/notmuch-show.el @@ -457,7 +457,7 @@ current buffer, if possible." (defun notmuch-show-insert-part-multipart/signed (msg part content-type nth depth declared-type) (let ((button (notmuch-show-insert-part-header nth declared-type content-type nil))) -(button-put button 'face '(:foreground "blue")) +(button-put button 'face 'notmuch-crypto-part-header) ;; add signature status button if sigstatus provided (if (plist-member part :sigstatus) (let* ((from (notmuch-show-get-header :From msg)) @@ -479,7 +479,7 @@ current buffer, if possible." (defun notmuch-show-insert-part-multipart/encrypted (msg part content-type nth depth declared-type) (let ((button (notmuch-show-insert-part-header nth declared-type content-type nil))) -(button-put button 'face '(:foreground "blue")) +(button-put button 'face 'notmuch-crypto-part-header) ;; add encryption status button if encstatus specified (if (plist-member part :encstatus) (let ((encstatus (car (plist-get part :encstatus -- 1.7.8 ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
[PATCH v3 3/4] emacs: rename notmuch-decimal-separator to notmuch-thousands-separator
In 123,456.78, "." is the decimal separator, but "," is the thousands separator. This commit also mentions the space being used as thousands separator in several European countries. --- emacs/notmuch-hello.el |8 1 files changed, 4 insertions(+), 4 deletions(-) diff --git a/emacs/notmuch-hello.el b/emacs/notmuch-hello.el index 0582cae..0fe9c1d 100644 --- a/emacs/notmuch-hello.el +++ b/emacs/notmuch-hello.el @@ -131,10 +131,10 @@ So: (integer :tag "Number of characters") (float :tag "Fraction of window"))) -(defcustom notmuch-decimal-separator "," - "The string used as a decimal separator. +(defcustom notmuch-thousands-separator "," + "The string used as a thousands separator. -Typically \",\" in the US and UK and \".\" in Europe." +Typically \",\" in the US and UK and \".\" or \" \" in Europe." :group 'notmuch :type 'string) @@ -159,7 +159,7 @@ Typically \",\" in the US and UK and \".\" in Europe." (apply #'concat (number-to-string (car result)) (mapcar (lambda (elem) - (format "%s%03d" notmuch-decimal-separator elem)) + (format "%s%03d" notmuch-thousands-separator elem)) (cdr result) (defun notmuch-hello-trim (search) -- 1.7.8 ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
[PATCH v3 1/4] emacs: Let the user choose where to compose new mails
Reusing the current window to compose a new mail may be troublesome for the
user. This patch introduces a new customizable variable, notmuch-mua-compose-in,
which lets the user choose where to create the mail buffer: in the current
window (current and default behaviour), in a new window, or in a new frame.
---
emacs/notmuch-mua.el | 40 ++--
1 files changed, 38 insertions(+), 2 deletions(-)
diff --git a/emacs/notmuch-mua.el b/emacs/notmuch-mua.el
index 8824b08..90834d6 100644
--- a/emacs/notmuch-mua.el
+++ b/emacs/notmuch-mua.el
@@ -31,6 +31,21 @@
:group 'notmuch
:type 'hook)
+(defcustom notmuch-mua-compose-in 'current-window
+ "Where to create the mail buffer used to compose a new message.
+ Possible values are `current-window' (default), `new-window'
+ and `new-frame'. If set to `current-window', the mail buffer
+ will be displayed in the current window, so the old buffer will
+ be restored when the mail buffer is killed. If set to
+ `new-window' or `new-frame', the mail buffer will be displayed
+ in a new window/frame that will be destroyed when the buffer is
+ killed. You may want to customize `message-kill-buffer-on-exit'
+ accordingly."
+ :group 'notmuch
+ :type '(choice (const :tag "Compose in the current window" current-window)
+(const :tag "Compose mail in a new window" new-window)
+(const :tag "Compose mail in a new frame" new-frame)))
+
(defcustom notmuch-mua-user-agent-function 'notmuch-mua-user-agent-full
"Function used to generate a `User-Agent:' string. If this is
`nil' then no `User-Agent:' will be generated."
@@ -48,6 +63,23 @@ list."
;;
+(defun notmuch-mua-get-switch-function ()
+ "Get a switch function according to `notmuch-mua-compose-in'."
+ (cond ((eq notmuch-mua-compose-in 'current-window)
+'switch-to-buffer)
+ ((eq notmuch-mua-compose-in 'new-window)
+'switch-to-buffer-other-window)
+ ((eq notmuch-mua-compose-in 'new-frame)
+'switch-to-buffer-other-frame)
+ (t (error "Invalid value for `notmuch-mua-compose-in'"
+
+(defun notmuch-mua-maybe-set-window-dedicated ()
+ "Set the selected window as dedicated according to
+`notmuch-mua-compose-in'."
+ (when (or (eq notmuch-mua-compose-in 'new-frame)
+ (eq notmuch-mua-compose-in 'new-window))
+(set-window-dedicated-p (selected-window) t)))
+
(defun notmuch-mua-user-agent-full ()
"Generate a `User-Agent:' string suitable for notmuch."
(concat (notmuch-mua-user-agent-notmuch)
@@ -99,7 +131,8 @@ list."
((same-window-regexps '("\\*mail .*")))
(notmuch-mua-mail (mail-header 'to headers)
(mail-header 'subject headers)
- (message-headers-to-generate headers t '(to subject
+ (message-headers-to-generate headers t '(to subject))
+ nil (notmuch-mua-get-switch-function)))
;; insert the message body - but put it in front of the signature
;; if one is present
(goto-char (point-max))
@@ -112,6 +145,7 @@ list."
(message-goto-body))
(defun notmuch-mua-forward-message ()
+ (funcall (notmuch-mua-get-switch-function) (current-buffer))
(message-forward)
(when notmuch-mua-user-agent-function
@@ -121,6 +155,7 @@ list."
(message-sort-headers)
(message-hide-headers)
(set-buffer-modified-p nil)
+ (notmuch-mua-maybe-set-window-dedicated)
(message-goto-to))
@@ -143,6 +178,7 @@ list."
(message-sort-headers)
(message-hide-headers)
(set-buffer-modified-p nil)
+ (notmuch-mua-maybe-set-window-dedicated)
(message-goto-to))
@@ -199,7 +235,7 @@ the From: address first."
(let ((other-headers
(when (or prompt-for-sender notmuch-always-prompt-for-sender)
(list (cons 'from (notmuch-mua-prompt-for-sender))
-(notmuch-mua-mail nil nil other-headers)))
+(notmuch-mua-mail nil nil other-headers nil
(notmuch-mua-get-switch-function
(defun notmuch-mua-new-forward-message (&optional prompt-for-sender)
"Invoke the notmuch message forwarding window.
--
1.7.8
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch
[PATCH v3 0/4] Several minor Emacs enhancements
Hi David et al., Here is a rebased version of this patch series, with more descriptive commit messages. Please tell me if there are still ways to enhance them. English is not my native language and it's sometimes hard to figure out what's relevant and how to word it :) Best regards, Thomas Thomas Jost (4): emacs: Let the user choose where to compose new mails emacs: Add a face for crypto parts headers emacs: rename notmuch-decimal-separator to notmuch-thousands-separator emacs: add notmuch-hello-hook emacs/notmuch-crypto.el |5 + emacs/notmuch-hello.el | 17 - emacs/notmuch-mua.el| 40 ++-- emacs/notmuch-show.el |4 ++-- 4 files changed, 57 insertions(+), 9 deletions(-) -- 1.7.8 ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
Re: [PATCH 1/2] test: add a function to run Python tests
On Sun, 11 Dec 2011 18:58:18 +0400, Dmitry Kurochkin wrote: > Perhaps we should have a test-lib.py for test-specific stuff like this > (similar to test-lib.el)? I think it would be cleaner and makes it easy > to add more Python test auxiliary functions later. Well, right now that would probably be overkill: the stdout-to-file redirection takes 3 lines at most. Besides we would still need to set some environment variables to run python with the correct directories. But of course if we need to add more helper functions for running python tests, then a test-lib.py would be nice. Maybe we could even think about using one of Python's unit test libs for that: doctest, unittest (both in the standard library) and nose seem to be the most popular ones. Regards, -- Thomas/Schnouki pgpBDBDxFNlN2.pgp Description: PGP signature ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
[PATCH 5/5] python: add get_mtime() to the Message class
--- bindings/python/notmuch/message.py | 20 1 files changed, 20 insertions(+), 0 deletions(-) diff --git a/bindings/python/notmuch/message.py b/bindings/python/notmuch/message.py index ce8e718..56f56c2 100644 --- a/bindings/python/notmuch/message.py +++ b/bindings/python/notmuch/message.py @@ -293,6 +293,10 @@ class Message(object): _get_date.argtypes = [NotmuchMessageP] _get_date.restype = c_long +_get_mtime = nmlib.notmuch_message_get_mtime +_get_mtime.argtypes = [NotmuchMessageP] +_get_mtime.restype = c_long + _get_header = nmlib.notmuch_message_get_header _get_header.argtypes = [NotmuchMessageP, c_char_p] _get_header.restype = c_char_p @@ -401,6 +405,22 @@ class Message(object): raise NotmuchError(STATUS.NOT_INITIALIZED) return Message._get_date(self._msg) +def get_mtime(self): +"""Returns time_t of the message mtime + +The mtime is the timestamp of the last time the message was modified, +e.g. the time it was added to the database or the last time a tag was +added or removed. + +:returns: A time_t timestamp. +:rtype: c_unit64 +:exception: :exc:`NotmuchError` STATUS.NOT_INITIALIZED if the message +is not initialized. +""" +if self._msg is None: +raise NotmuchError(STATUS.NOT_INITIALIZED) +return Message._get_mtime(self._msg) + def get_header(self, header): """Get the value of the specified header. -- 1.7.8 ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
[PATCH 4/5] show: include mtime in JSON output
This could be used by a UI implementation somehow.
---
notmuch-show.c |7 ---
1 files changed, 4 insertions(+), 3 deletions(-)
diff --git a/notmuch-show.c b/notmuch-show.c
index 873a7c4..7279601 100644
--- a/notmuch-show.c
+++ b/notmuch-show.c
@@ -202,17 +202,18 @@ format_message_json (const void *ctx, notmuch_message_t
*message, unused (int in
notmuch_tags_t *tags;
int first = 1;
void *ctx_quote = talloc_new (ctx);
-time_t date;
+time_t date, mtime;
const char *relative_date;
date = notmuch_message_get_date (message);
relative_date = notmuch_time_relative_date (ctx, date);
+mtime = notmuch_message_get_mtime (message);
-printf ("\"id\": %s, \"match\": %s, \"filename\": %s, \"timestamp\": %ld,
\"date_relative\": \"%s\", \"tags\": [",
+printf ("\"id\": %s, \"match\": %s, \"filename\": %s, \"timestamp\": %ld,
\"date_relative\": \"%s\", \"mtime\": %ld, \"tags\": [",
json_quote_str (ctx_quote, notmuch_message_get_message_id
(message)),
notmuch_message_get_flag (message, NOTMUCH_MESSAGE_FLAG_MATCH) ?
"true" : "false",
json_quote_str (ctx_quote, notmuch_message_get_filename (message)),
- date, relative_date);
+ date, relative_date, mtime);
for (tags = notmuch_message_get_tags (message);
notmuch_tags_valid (tags);
--
1.7.8
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch
[PATCH 3/5] lib: Make MTIME values searchable
Tag modification times are now searchable as ranges (just like regular message
dates) with the "mtime:" prefix.
---
lib/database-private.h |1 +
lib/database.cc|3 +++
notmuch.1 | 14 --
notmuch.c | 13 ++---
4 files changed, 26 insertions(+), 5 deletions(-)
diff --git a/lib/database-private.h b/lib/database-private.h
index 88532d5..e71c8e4 100644
--- a/lib/database-private.h
+++ b/lib/database-private.h
@@ -52,6 +52,7 @@ struct _notmuch_database {
Xapian::QueryParser *query_parser;
Xapian::TermGenerator *term_gen;
Xapian::ValueRangeProcessor *value_range_processor;
+Xapian::ValueRangeProcessor *mtime_value_range_processor;
};
/* Return the list of terms from the given iterator matching a prefix.
diff --git a/lib/database.cc b/lib/database.cc
index 6dc6f73..cc970c1 100644
--- a/lib/database.cc
+++ b/lib/database.cc
@@ -677,12 +677,14 @@ notmuch_database_open (const char *path,
notmuch->term_gen = new Xapian::TermGenerator;
notmuch->term_gen->set_stemmer (Xapian::Stem ("english"));
notmuch->value_range_processor = new Xapian::NumberValueRangeProcessor
(NOTMUCH_VALUE_TIMESTAMP);
+ notmuch->mtime_value_range_processor = new
Xapian::NumberValueRangeProcessor (NOTMUCH_VALUE_MTIME, "mtime:");
notmuch->query_parser->set_default_op (Xapian::Query::OP_AND);
notmuch->query_parser->set_database (*notmuch->xapian_db);
notmuch->query_parser->set_stemmer (Xapian::Stem ("english"));
notmuch->query_parser->set_stemming_strategy
(Xapian::QueryParser::STEM_SOME);
notmuch->query_parser->add_valuerangeprocessor
(notmuch->value_range_processor);
+ notmuch->query_parser->add_valuerangeprocessor
(notmuch->mtime_value_range_processor);
for (i = 0; i < ARRAY_SIZE (BOOLEAN_PREFIX_EXTERNAL); i++) {
prefix_t *prefix = &BOOLEAN_PREFIX_EXTERNAL[i];
@@ -726,6 +728,7 @@ notmuch_database_close (notmuch_database_t *notmuch)
delete notmuch->query_parser;
delete notmuch->xapian_db;
delete notmuch->value_range_processor;
+delete notmuch->mtime_value_range_processor;
talloc_free (notmuch);
}
diff --git a/notmuch.1 b/notmuch.1
index 3dbd67e..2235096 100644
--- a/notmuch.1
+++ b/notmuch.1
@@ -644,6 +644,8 @@ terms to match against specific portions of an email, (where
folder:
+ mtime:
+
The
.B from:
prefix is used to match the name or address of the sender of an email
@@ -707,8 +709,8 @@ operators, but will have to be protected from
interpretation by the
shell, (such as by putting quotation marks around any parenthesized
expression).
-Finally, results can be restricted to only messages within a
-particular time range, (based on the Date: header) with a syntax of:
+Results can be restricted to only messages within a particular time range,
+(based on the Date: header) with a syntax of:
..
@@ -721,6 +723,14 @@ specify a date range to return messages from 2009\-10\-01
until the
current time:
$(date +%s \-d 2009\-10\-01)..$(date +%s)
+
+Finally, the
+.B mtime:
+prefix can be used to search for messages which were modified (e.g. tags were
+added or removed) within a particular time range, with the same syntax as
+before:
+
+ mtime:..
.SH HOOKS
Hooks are scripts (or arbitrary executables or symlinks to such) that notmuch
invokes before and after certain actions. These scripts reside in
diff --git a/notmuch.c b/notmuch.c
index c0ce026..443cf59 100644
--- a/notmuch.c
+++ b/notmuch.c
@@ -71,6 +71,7 @@ static const char search_terms_help[] =
"\t\tid:\n"
"\t\tthread:\n"
"\t\tfolder:\n"
+"\t\tmtime:\n"
"\n"
"\tThe from: prefix is used to match the name or address of\n"
"\tthe sender of an email message.\n"
@@ -112,8 +113,8 @@ static const char search_terms_help[] =
"\tinterpretation by the shell, (such as by putting quotation\n"
"\tmarks around any parenthesized expression).\n"
"\n"
-"\tFinally, results can be restricted to only messages within a\n"
-"\tparticular time range, (based on the Date: header) with:\n"
+"\tResults can be restricted to only messages within a particular\n"
+"\ttime range, (based on the Date: header) with:\n"
"\n"
"\t\t..\n"
"\n"
@@ -125,7 +126,13 @@ static const char search_terms_help[] =
"\tfollowing syntax would specify a date range to return messages\n"
"\tfrom 2009-10-01 until the current time:\n"
"\n"
-"\t\t$(date +%%s -d 2009-10-01)..$(date +%%s)\n\n";
+"\t\t$(date +%%s -d 2009-10-01)..$(date +%%s)\n\n"
+"\n"
+"\tFinally, the mtime: prefix can be used to search for messages\n"
+"\twhich were modified (e.g. tags were added or removed) within a\n"
+"\tparticular time range, with the same syntax as before:\n"
+"\n"
+"\t\tmtime:..\n";
static const char hooks_help[] =
"\tHooks are scripts (or arbitrary executables or symlin
[PATCH 2/5] lib: Add a MTIME value to every mail document
This is a time_t value, similar to the message date (TIMESTAMP). It is first set
when the message is added to the database, and is then updated every time a tag
is added or removed. It can thus be used for doing incremental dumps of the
database or for synchronizing it between several computers.
This value can be read freely (with notmuch_message_get_mtime()) but for now it
can't be set to an arbitrary value: it can only be set to "now" when updated.
There's no specific reason for this except that I don't really see a real use
case for setting it to an arbitrary value.
---
lib/database.cc |7 ++-
lib/message.cc| 32
lib/notmuch-private.h |6 +-
lib/notmuch.h |4
4 files changed, 47 insertions(+), 2 deletions(-)
diff --git a/lib/database.cc b/lib/database.cc
index 2025189..6dc6f73 100644
--- a/lib/database.cc
+++ b/lib/database.cc
@@ -81,7 +81,7 @@ typedef struct {
* STRING is the name of a file within that
* directory for this mail message.
*
- *A mail document also has four values:
+ *A mail document also has five values:
*
* TIMESTAMP: The time_t value corresponding to the message's
* Date header.
@@ -92,6 +92,9 @@ typedef struct {
*
* SUBJECT:The value of the "Subject" header
*
+ * MTIME: The time_t value corresponding to the last time
+ * a tag was added or removed on the message.
+ *
* In addition, terms from the content of the message are added with
* "from", "to", "attachment", and "subject" prefixes for use by the
* user in searching. Similarly, terms from the path of the mail
@@ -1735,6 +1738,8 @@ notmuch_database_add_message (notmuch_database_t *notmuch,
date = notmuch_message_file_get_header (message_file, "date");
_notmuch_message_set_header_values (message, date, from, subject);
+_notmuch_message_update_mtime (message);
+
_notmuch_message_index_file (message, filename);
} else {
ret = NOTMUCH_STATUS_DUPLICATE_MESSAGE_ID;
diff --git a/lib/message.cc b/lib/message.cc
index 0075425..0c98589 100644
--- a/lib/message.cc
+++ b/lib/message.cc
@@ -830,6 +830,34 @@ _notmuch_message_set_header_values (notmuch_message_t
*message,
message->doc.add_value (NOTMUCH_VALUE_SUBJECT, subject);
}
+/* Get the message mtime, i.e. when it was added or the last time a tag was
+ * added/removed. */
+time_t
+notmuch_message_get_mtime (notmuch_message_t *message)
+{
+std::string value;
+
+try {
+ value = message->doc.get_value (NOTMUCH_VALUE_MTIME);
+} catch (Xapian::Error &error) {
+ INTERNAL_ERROR ("Failed to read mtime value from document.");
+ return 0;
+}
+
+return Xapian::sortable_unserialise (value);
+}
+
+/* Set the message mtime to "now". */
+void
+_notmuch_message_update_mtime (notmuch_message_t *message)
+{
+time_t time_value;
+
+time_value = time (NULL);
+message->doc.add_value (NOTMUCH_VALUE_MTIME,
+Xapian::sortable_serialise (time_value));
+}
+
/* Synchronize changes made to message->doc out into the database. */
void
_notmuch_message_sync (notmuch_message_t *message)
@@ -994,6 +1022,8 @@ notmuch_message_add_tag (notmuch_message_t *message, const
char *tag)
private_status);
}
+_notmuch_message_update_mtime (message);
+
if (! message->frozen)
_notmuch_message_sync (message);
@@ -1022,6 +1052,8 @@ notmuch_message_remove_tag (notmuch_message_t *message,
const char *tag)
private_status);
}
+_notmuch_message_update_mtime (message);
+
if (! message->frozen)
_notmuch_message_sync (message);
diff --git a/lib/notmuch-private.h b/lib/notmuch-private.h
index 60a932f..9859872 100644
--- a/lib/notmuch-private.h
+++ b/lib/notmuch-private.h
@@ -95,7 +95,8 @@ typedef enum {
NOTMUCH_VALUE_TIMESTAMP = 0,
NOTMUCH_VALUE_MESSAGE_ID,
NOTMUCH_VALUE_FROM,
-NOTMUCH_VALUE_SUBJECT
+NOTMUCH_VALUE_SUBJECT,
+NOTMUCH_VALUE_MTIME
} notmuch_value_t;
/* Xapian (with flint backend) complains if we provide a term longer
@@ -276,6 +277,9 @@ _notmuch_message_set_header_values (notmuch_message_t
*message,
const char *from,
const char *subject);
void
+_notmuch_message_update_mtime (notmuch_message_t *message);
+
+void
_notmuch_message_sync (notmuch_message_t *message);
notmuch_status_t
diff --git a/lib/notmuch.h b/lib/notmuch.h
index 9f23a10..643ebce 100644
--- a/lib/notmuch.h
+++ b/lib/notmuch.h
@@ -910,6 +910,10 @@ notmuch_message_set_flag (notmuch_message_t *message,
time_t
notmuch_message_get_date (notmuch_message_t *message);
+/* Get the mtime of 'message' as a time_t value. */
+time_t
+notmuch_message_get_mtime (notmuch_message_t *message);
+
[PATCH 0/5] Store message modification times in the DB
Hello world, This is a patch series I've been working on for some time in order to be able to sync my tags on several computers. I'm posting it now, but please consider it as a RFC rather than something that is ready to be pushed. The basic idea is to the last time each message was modified, i.e. "the message was added to the DB", "a tag was added" or "a tag was removed". This mtime is accessible through a library function and in the JSON output of "notmuch show". It is also searchable with the "mtime:" prefix and with timestamp ranges, like for searching messages by date: notmuch search mtime:$(date +%s 2011-12-01)..$(date +%s) This can then be used in scripts or helper programs to do incremental dumps or tags synchronization. (I already have a script to do incremental backups, but it needs some cleaning, and I'm still working on something for sync'ing tags, but it's starting to work really well; I'll post them later). This can be seen as an alternative to David Bremner's jlog branch, but with several differences: + no external dependency + everything is stored in the notmuch DB: atomicity for free! - when a message is removed we lose everything about it, which makes the sync process more complicated - for a human, it's harder to manipulate timestamps than log messages - this can store much less data than a proper log system On IRC amdragon suggested using a simple sequence number instead of a timestamp. This would indeed eliminate the need for proper time synchronization between computers one would want to keep in sync, and it would reduce the risk of time-going-backward problems, but IMHO it would cause more problems: no global clock --> no simple way to tell if DB #A is more recent than DB #B. So, here are the patches: - first a little fix to the comments describing the DB schema (not specific to this patch series at all, I just noticed it when rebasing this series) - the second commit adds the MTIME value to the database schema, and creates the functions used to update and access this value. - the third commit makes the MTIME value searchable with a range syntax. - the fourth commit adds the MTIME to the JSON output of "notmuch show". - the fifth and last commit adds Message.get_mtime() to the Python bindings. Please tell me what you think of this. Best regards, Thomas Thomas Jost (5): Fix comments about what is stored in the database lib: Add a MTIME value to every mail document lib: Make MTIME values searchable show: include mtime in JSON output python: add get_mtime() to the Message class bindings/python/notmuch/message.py | 20 lib/database-private.h |1 + lib/database.cc| 14 +- lib/message.cc | 32 lib/notmuch-private.h |6 +- lib/notmuch.h |4 notmuch-show.c |7 --- notmuch.1 | 14 -- notmuch.c | 13 ++--- 9 files changed, 101 insertions(+), 10 deletions(-) -- 1.7.8 ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
[PATCH 1/5] Fix comments about what is stored in the database
Commit 567bcbc2 introduced two new values for each message (content of the
"From" and "Subject" headers), but the comments about the database schema had
not been updated accordingly.
---
lib/database.cc |6 +-
1 files changed, 5 insertions(+), 1 deletions(-)
diff --git a/lib/database.cc b/lib/database.cc
index 98f101e..2025189 100644
--- a/lib/database.cc
+++ b/lib/database.cc
@@ -81,13 +81,17 @@ typedef struct {
* STRING is the name of a file within that
* directory for this mail message.
*
- *A mail document also has two values:
+ *A mail document also has four values:
*
* TIMESTAMP: The time_t value corresponding to the message's
* Date header.
*
* MESSAGE_ID: The unique ID of the mail mess (see "id" above)
*
+ * FROM: The value of the "From" header
+ *
+ * SUBJECT:The value of the "Subject" header
+ *
* In addition, terms from the content of the message are added with
* "from", "to", "attachment", and "subject" prefixes for use by the
* user in searching. Similarly, terms from the path of the mail
--
1.7.8
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch
Re: [alot] announcing v0.2 aka. "robot santa"
On Tue, 13 Dec 2011 10:38:46 +, Patrick Totzke wrote: > I have seen this before yes. It seems to be a problem with the python > distribution on > debian. Apparently, you have two different versions of the argparse module > installed: As > of 2.7, argparse is part of the standard lib, but debian ships a separate > package > python-argparse. I just filed a Debian bug against python-argparse: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=651962 jamie. pgpNtgdzizn9v.pgp Description: PGP signature ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
[alot] announcing v0.2 aka. "robot santa"
On Tue, 13 Dec 2011 10:38:46 +, Patrick Totzke wrote: > I have seen this before yes. It seems to be a problem with the python > distribution on > debian. Apparently, you have two different versions of the argparse module > installed: As > of 2.7, argparse is part of the standard lib, but debian ships a separate > package > python-argparse. I just filed a Debian bug against python-argparse: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=651962 jamie. -- next part -- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: not available URL: <http://notmuchmail.org/pipermail/notmuch/attachments/20111213/7b26689e/attachment.pgp>
Re: [PATCH] notmuch: unref charset_filter to fix one memory leak
On Tue, 13 Dec 2011 14:03:33 +0200, Tomi Ollila wrote:
> In my use case g_object_unref(charset_filter) reduces memory
> consumption over 90% when 'notmuch show --format=text "*"' is
> executed (~11000 messages, RES ~330M -> ~25M).
> ---
> notmuch-show.c |4 +++-
> 1 files changed, 3 insertions(+), 1 deletions(-)
>
> diff --git a/notmuch-show.c b/notmuch-show.c
> index 873a7c4..23d7368 100644
> --- a/notmuch-show.c
> +++ b/notmuch-show.c
> @@ -450,6 +450,7 @@ show_text_part_content (GMimeObject *part, GMimeStream
> *stream_out)
> {
> GMimeContentType *content_type = g_mime_object_get_content_type
> (GMIME_OBJECT (part));
> GMimeStream *stream_filter = NULL;
> +GMimeFilter *charset_filter = NULL;
> GMimeDataWrapper *wrapper;
> const char *charset;
>
> @@ -466,7 +467,6 @@ show_text_part_content (GMimeObject *part, GMimeStream
> *stream_out)
>
> charset = g_mime_object_get_content_type_parameter (part, "charset");
> if (charset) {
> - GMimeFilter *charset_filter;
> charset_filter = g_mime_filter_charset_new (charset, "UTF-8");
> /* This result can be NULL for things like "unknown-8bit".
>* Don't set a NULL filter as that makes GMime print
> @@ -479,6 +479,8 @@ show_text_part_content (GMimeObject *part, GMimeStream
> *stream_out)
> wrapper = g_mime_part_get_content_object (GMIME_PART (part));
> if (wrapper && stream_filter)
> g_mime_data_wrapper_write_to_stream (wrapper, stream_filter);
> +if (charset_filter)
> + g_object_unref (charset_filter);
Why can't we do this inside the if (charset) block?
Regards,
Dmitry
> if (stream_filter)
> g_object_unref(stream_filter);
> }
> --
> 1.7.6.1
> ___
> notmuch mailing list
> notmuch@notmuchmail.org
> http://notmuchmail.org/mailman/listinfo/notmuch
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch
[PATCH] notmuch: unref charset_filter to fix one memory leak
In my use case g_object_unref(charset_filter) reduces memory
consumption over 90% when 'notmuch show --format=text "*"' is
executed (~11000 messages, RES ~330M -> ~25M).
---
notmuch-show.c |4 +++-
1 files changed, 3 insertions(+), 1 deletions(-)
diff --git a/notmuch-show.c b/notmuch-show.c
index 873a7c4..23d7368 100644
--- a/notmuch-show.c
+++ b/notmuch-show.c
@@ -450,6 +450,7 @@ show_text_part_content (GMimeObject *part, GMimeStream
*stream_out)
{
GMimeContentType *content_type = g_mime_object_get_content_type
(GMIME_OBJECT (part));
GMimeStream *stream_filter = NULL;
+GMimeFilter *charset_filter = NULL;
GMimeDataWrapper *wrapper;
const char *charset;
@@ -466,7 +467,6 @@ show_text_part_content (GMimeObject *part, GMimeStream
*stream_out)
charset = g_mime_object_get_content_type_parameter (part, "charset");
if (charset) {
- GMimeFilter *charset_filter;
charset_filter = g_mime_filter_charset_new (charset, "UTF-8");
/* This result can be NULL for things like "unknown-8bit".
* Don't set a NULL filter as that makes GMime print
@@ -479,6 +479,8 @@ show_text_part_content (GMimeObject *part, GMimeStream
*stream_out)
wrapper = g_mime_part_get_content_object (GMIME_PART (part));
if (wrapper && stream_filter)
g_mime_data_wrapper_write_to_stream (wrapper, stream_filter);
+if (charset_filter)
+ g_object_unref (charset_filter);
if (stream_filter)
g_object_unref(stream_filter);
}
--
1.7.6.1
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch
Re: [alot] announcing v0.2 aka. "robot santa"
Thx Adrian & Jamie! It would be great to get it into Debian, yes. I will have a look at Debian packaging if we don't find a more capable volunteer in the next few weeks. Thanks for your initial work on this. While we're at it: I heard that there are some build scripts for Arch and Gentoo around. If you authored one of these step forward: I'd gladly include them to the git repo. Quoting Jameson Graef Rollins (2011-12-12 00:14:10) >Hey, Patrick. I'm guessing this isn't actually an alot issue, but when >I start up alot I get the following message to the terminal: > >/usr/lib/python2.7/dist-packages/zope/__init__.py:3: UserWarning: Module >argparse was already imported from /usr/lib/python2.7/argparse.pyc, but >/usr/lib/python2.7/dist-packages is being added to sys.path > import pkg_resources > >Any idea what's going on there? I have seen this before yes. It seems to be a problem with the python distribution on debian. Apparently, you have two different versions of the argparse module installed: As of 2.7, argparse is part of the standard lib, but debian ships a separate package python-argparse. What happens is that I import argparse to alot, which happens to pick the one from "/usr/lib/python2.7/argparse.pyc". Now zope (a 2nd order dependency) explicitly imports "/usr/lib/python2.7/dist-packages/argparse.pyc" and warns that its overloading this namespace. Also see https://bugs.launchpad.net/debian/+source/distribute/+bug/810019 for a similar problem with python-distribute, particularly comment #9. Best, /p ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
[PATCH v3] emacs: support "notmuch new" as a notmuch-poll-script
On Mon, 12 Dec 2011 16:24:51 -0500, Austin Clements wrote: > Quoth Jani Nikula on Dec 12 at 11:13 pm: > > On Tue, 13 Dec 2011 00:53:05 +0400, Dmitry Kurochkin > gmail.com> wrote: > > > On Mon, 12 Dec 2011 22:50:04 +0200, Jani Nikula > > > wrote: > > > > +If set to nil (the default), new mail is processed by invoking > > > > +\"notmuch new\". Otherwise, this should be set to a string that > > > > +gives the name of an external script that processes new mail. If > > > > +set to the empty string, no command will be run. > > > > > > I think this should be "an empty string". But I may be mistaking. > > > > Shameless copy paste from a native speaker, who am I to argue? :) > > Austin? > > Either way is grammatically correct. Really, this is a philosophical > question. Can two empty strings have different identities? Or is > there only one empty string in the universe? > > (eq "" "") => t > (eq "" (string)) => t > (eq "" (make-string 0 ?a)) => t > (eq "" (substring "a" 1)) => t > > It would appear Elisp is squarely in the "there is one empty string" > camp, so "the empty string" would be more correct. Fine with me then :) Regards, Dmitry
[PATCH 1/2] test/smtp-dummy: add --background option for going background after listen(2)
To avoid the possibility that smtp-dummy doesn't have chance to bind
its listening socket until something tries to send message to it this
option makes caller wait until socket is already listening for connections.
In case this --background option is used, the pid of running smtp-dummy
is printed on stdout.
---
Resent after whitespace-cleanup in patch 1/2 (this patch).
test/smtp-dummy.c | 35 ++-
1 files changed, 34 insertions(+), 1 deletions(-)
diff --git a/test/smtp-dummy.c b/test/smtp-dummy.c
index 3801a5e..9126c00 100644
--- a/test/smtp-dummy.c
+++ b/test/smtp-dummy.c
@@ -124,9 +124,21 @@ main (int argc, char *argv[])
struct hostent *hostinfo;
socklen_t peer_addr_len;
int reuse;
+ int bg;
+
+ /* XXX Quick implementation -- fix if more functionality is desired. */
+ if (argc >= 2 && strcmp(argv[1], "--background") == 0) {
+ argc--;
+ argv[1] = argv[0];
+ argv++;
+ bg = 1;
+ }
+ else
+ bg = 0;
if (argc != 2) {
- fprintf (stderr, "Usage: %s \n", argv[0]);
+ fprintf (stderr, "Usage: %s [--background] \n",
+argv[0]);
return 1;
}
@@ -179,7 +191,27 @@ main (int argc, char *argv[])
return 1;
}
+ if (bg) {
+ int pid = fork ();
+ if (pid > 0) {
+ printf ("%d\n", pid);
+ return 0;
+ }
+ if (pid < 0) {
+ fprintf (stderr, "Error: fork() failed: %s\n",
+strerror (errno));
+ close (sock);
+ return 1;
+ }
+ /* Reached if pid == 0. */
+ /* Close stdout so that the one interested in pid value will
+ also get EOF. */
+ close (1);
+ /* dup2() will re-reserve fd 1 (opportunistically, in case fd 2
+ is open. If that was not open we don't care fd 1 either.) */
+ dup2 (2, 1);
+ }
+
peer_addr_len = sizeof (peer_addr);
peer = accept (sock, (struct sockaddr *) &peer_addr, &peer_addr_len);
if (peer == -1) {
--
1.7.7.3
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch
[PATCH 2/2] test/test-lib.sh: launch smtp-dummy with --background and finally kill(1) it.
Take the new --background option of smtp-dummy to use so that it is known
there is smtpd listener ready when it is needed. As the smtp-dummy instance
is no longer child process of the script sending SIGKILL to it is the only
way to make sure the instance exits when required.
---
Resent after whitespace-cleanup in patch 1/2 (the other one).
test/test-lib.sh | 12 +++-
1 files changed, 7 insertions(+), 5 deletions(-)
diff --git a/test/test-lib.sh b/test/test-lib.sh
index 6be93fe..a857afe 100644
--- a/test/test-lib.sh
+++ b/test/test-lib.sh
@@ -382,8 +382,9 @@ emacs_deliver_message ()
shift 2
# before we can send a message, we have to prepare the FCC maildir
mkdir -p "$MAIL_DIR"/sent/{cur,new,tmp}
-$TEST_DIRECTORY/smtp-dummy sent_message &
-smtp_dummy_pid=$!
+
+smtp_dummy_pid=$($TEST_DIRECTORY/smtp-dummy --background sent_message) \
+ || return
test_emacs \
"(let ((message-send-mail-function 'message-smtpmail-send-it)
(smtpmail-smtp-server \"localhost\")
@@ -398,9 +399,10 @@ emacs_deliver_message ()
(insert \"${body}\")
$@
(message-send-and-exit))"
-# opportunistically quit smtp-dummy in case above fails.
-{ echo QUIT > /dev/tcp/localhost/25025; } 2>/dev/null
-wait ${smtp_dummy_pid}
+# Cannot wait, not our child. In case message was sent properly, client
+# waits for confirmation before exiting and resuming control here;
therefore
+# making sure that server exits by sending (KILL) signal to it is safe.
+kill -9 ${smtp_dummy_pid}
notmuch new >/dev/null
}
--
1.7.7.3
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch
[PATCH v3] emacs: support "notmuch new" as a notmuch-poll-script
On Mon, 12 Dec 2011 22:50:04 +0200, Jani Nikula wrote: > Support nil value for notmuch-poll-script to run "notmuch new" instead of > an external script, and make this the new default. "notmuch new" is run > using the configured notmuch-command. > > This allows taking better advantage of the "notmuch new" hooks from emacs > without intermediate scripts. > > Signed-off-by: Jani Nikula > > --- > > v3: only documentation changes suggested by Austin and Dmitry. > --- > emacs/notmuch.el | 35 +-- > 1 files changed, 25 insertions(+), 10 deletions(-) > > diff --git a/emacs/notmuch.el b/emacs/notmuch.el > index 8936149..675a110 100644 > --- a/emacs/notmuch.el > +++ b/emacs/notmuch.el > @@ -965,28 +965,43 @@ same relative position within the new buffer." > (notmuch-search query oldest-first target-thread target-line > continuation) > (goto-char (point-min > > -(defcustom notmuch-poll-script "" > +(defcustom notmuch-poll-script nil >"An external script to incorporate new mail into the notmuch database. > > -If this variable is non empty, then it should name a script to be > -invoked by `notmuch-search-poll-and-refresh-view' and > +This variable controls the action invoked by > +`notmuch-search-poll-and-refresh-view' and > `notmuch-hello-poll-and-update' (each have a default keybinding > -of 'G'). The script could do any of the following depending on > +of 'G') to incorporate new mail into the notmuch database. > + > +If set to nil (the default), new mail is processed by invoking > +\"notmuch new\". Otherwise, this should be set to a string that > +gives the name of an external script that processes new mail. If > +set to the empty string, no command will be run. I think this should be "an empty string". But I may be mistaking. Regards, Dmitry > + > +The external script could do any of the following depending on > the user's needs: > > 1. Invoke a program to transfer mail to the local mail store > 2. Invoke \"notmuch new\" to incorporate the new mail > -3. Invoke one or more \"notmuch tag\" commands to classify the mail" > - :type 'string > +3. Invoke one or more \"notmuch tag\" commands to classify the mail > + > +Note that the recommended way of achieving the same is using > +\"notmuch new\" hooks." > + :type '(choice (const :tag "notmuch new" nil) > + (const :tag "Disabled" "") > + (string :tag "Custom script")) >:group 'notmuch) > > (defun notmuch-poll () > - "Run external script to import mail. > + "Run \"notmuch new\" or an external script to import mail. > > -Invokes `notmuch-poll-script' if it is not set to an empty string." > +Invokes `notmuch-poll-script', \"notmuch new\", or does nothing > +depending on the value of `notmuch-poll-script'." >(interactive) > - (if (not (string= notmuch-poll-script "")) > - (call-process notmuch-poll-script nil nil))) > + (if (stringp notmuch-poll-script) > + (if (not (string= notmuch-poll-script "")) > + (call-process notmuch-poll-script nil nil)) > +(call-process notmuch-command nil nil nil "new"))) > > (defun notmuch-search-poll-and-refresh-view () >"Invoke `notmuch-poll' to import mail, then refresh the current view." > -- > 1.7.5.4 > > ___ > notmuch mailing list > notmuch at notmuchmail.org > http://notmuchmail.org/mailman/listinfo/notmuch
[PATCH 2/2] test/test-lib.sh: launch smtp-dummy with --background and finally kill(1) it.
Take the new --background option of smtp-dummy to use so that it is known
there is smtpd listener ready when it is needed. As the smtp-dummy instance
is no longer child process of the script sending SIGKILL to it is the only
way to make sure the instance exits when required.
---
test/test-lib.sh | 12 +++-
1 files changed, 7 insertions(+), 5 deletions(-)
diff --git a/test/test-lib.sh b/test/test-lib.sh
index 6be93fe..a857afe 100644
--- a/test/test-lib.sh
+++ b/test/test-lib.sh
@@ -382,8 +382,9 @@ emacs_deliver_message ()
shift 2
# before we can send a message, we have to prepare the FCC maildir
mkdir -p "$MAIL_DIR"/sent/{cur,new,tmp}
-$TEST_DIRECTORY/smtp-dummy sent_message &
-smtp_dummy_pid=$!
+
+smtp_dummy_pid=$($TEST_DIRECTORY/smtp-dummy --background sent_message) \
+ || return
test_emacs \
"(let ((message-send-mail-function 'message-smtpmail-send-it)
(smtpmail-smtp-server \"localhost\")
@@ -398,9 +399,10 @@ emacs_deliver_message ()
(insert \"${body}\")
$@
(message-send-and-exit))"
-# opportunistically quit smtp-dummy in case above fails.
-{ echo QUIT > /dev/tcp/localhost/25025; } 2>/dev/null
-wait ${smtp_dummy_pid}
+# Cannot wait, not our child. In case message was sent properly, client
+# waits for confirmation before exiting and resuming control here;
therefore
+# making sure that server exits by sending (KILL) signal to it is safe.
+kill -9 ${smtp_dummy_pid}
notmuch new >/dev/null
}
--
1.7.7.3
[PATCH 1/2] test/smtp-dummy: add --background option for going background after listen(2)
To avoid the possibility that smtp-dummy doesn't have chance to bind
its listening socket until something tries to send message to it this
option makes caller wait until socket is already listening for connections.
In case this --background option is used, the pid of running smtp-dummy
is printed on stdout.
---
test/smtp-dummy.c | 35 ++-
1 files changed, 34 insertions(+), 1 deletions(-)
diff --git a/test/smtp-dummy.c b/test/smtp-dummy.c
index 3801a5e..9126c00 100644
--- a/test/smtp-dummy.c
+++ b/test/smtp-dummy.c
@@ -124,9 +124,21 @@ main (int argc, char *argv[])
struct hostent *hostinfo;
socklen_t peer_addr_len;
int reuse;
+ int bg;
+
+ /* XXX Quick implementation -- fix if more functionality is desired. */
+ if (argc >= 2 && strcmp(argv[1], "--background") == 0) {
+ argc--;
+ argv[1] = argv[0];
+ argv++;
+ bg = 1;
+ }
+ else
+ bg = 0;
if (argc != 2) {
- fprintf (stderr, "Usage: %s \n", argv[0]);
+ fprintf (stderr, "Usage: %s [--background] \n",
+argv[0]);
return 1;
}
@@ -179,6 +191,27 @@ main (int argc, char *argv[])
return 1;
}
+ if (bg) {
+ int pid = fork ();
+ if (pid > 0) {
+ printf ("%d\n", pid);
+ return 0;
+ }
+ if (pid < 0) {
+ fprintf (stderr, "Error: fork() failed: %s\n",
+strerror (errno));
+ close (sock);
+ return 1;
+ }
+ /* Reached if pid == 0. */
+ /* Close stdout so that the one interested in pid value will
+ also get EOF. */
+ close (1);
+ /* dup2() will re-reserve fd 1 (opportunistically, in case fd 2
+ is open. If that was not open we don't care fd 1 either.) */
+ dup2 (2, 1);
+ }
+
peer_addr_len = sizeof (peer_addr);
peer = accept (sock, (struct sockaddr *) &peer_addr, &peer_addr_len);
if (peer == -1) {
--
1.7.7.3
[PATCH v2] emacs: support "notmuch new" as a notmuch-poll-script
Hi Jani. On Mon, 12 Dec 2011 21:57:28 +0200, Jani Nikula wrote: > Support nil value for notmuch-poll-script to run "notmuch new" instead of > an external script, and make this the new default. "notmuch new" is run > using the configured notmuch-command. > > This allows taking better advantage of the "notmuch new" hooks from emacs > without intermediate scripts. > > Signed-off-by: Jani Nikula Looks good to me. Few comments regarding documentation below. > --- > emacs/notmuch.el | 34 -- > 1 files changed, 24 insertions(+), 10 deletions(-) > > diff --git a/emacs/notmuch.el b/emacs/notmuch.el > index 8936149..5a8ab9d 100644 > --- a/emacs/notmuch.el > +++ b/emacs/notmuch.el > @@ -965,28 +965,42 @@ same relative position within the new buffer." > (notmuch-search query oldest-first target-thread target-line > continuation) > (goto-char (point-min > > -(defcustom notmuch-poll-script "" > +(defcustom notmuch-poll-script nil >"An external script to incorporate new mail into the notmuch database. > > -If this variable is non empty, then it should name a script to be > -invoked by `notmuch-search-poll-and-refresh-view' and > +This variable controls the action invoked by > +`notmuch-search-poll-and-refresh-view' and > `notmuch-hello-poll-and-update' (each have a default keybinding > -of 'G'). The script could do any of the following depending on > +of 'G') to incorporate new mail into the notmuch database. > + > +If this variable is non empty, then it should name an external Please s/non empty/non-empty string/ to make it more clear. > +script to be run. If set to an empty string, no action is > +invoked. Finally, if set to nil (the default), \"notmuch new\" is > +run using the command specified by `notmuch-command'. > + > +The external script could do any of the following depending on > the user's needs: > > 1. Invoke a program to transfer mail to the local mail store > 2. Invoke \"notmuch new\" to incorporate the new mail > -3. Invoke one or more \"notmuch tag\" commands to classify the mail" > - :type 'string > +3. Invoke one or more \"notmuch tag\" commands to classify the mail > + > +Note that the same can be achieved through \"notmuch new\" hooks." s/though/using/? Would be nice to mention that using hooks is the recommended way for this. > + :type '(choice (const :tag "Notmuch new" nil) Perhaps s/Notmuch/notmuch/ because we are talking about a command here? Regards, Dmitry > + (const :tag "Disabled" "") > + (string :tag "Custom script")) >:group 'notmuch) > > (defun notmuch-poll () > - "Run external script to import mail. > + "Run \"notmuch new\" or an external script to import mail. > > -Invokes `notmuch-poll-script' if it is not set to an empty string." > +Invokes `notmuch-poll-script', \"notmuch new\", or does nothing > +depending on the value of `notmuch-poll-script'." >(interactive) > - (if (not (string= notmuch-poll-script "")) > - (call-process notmuch-poll-script nil nil))) > + (if (stringp notmuch-poll-script) > + (if (not (string= notmuch-poll-script "")) > + (call-process notmuch-poll-script nil nil)) > +(call-process notmuch-command nil nil nil "new"))) > > (defun notmuch-search-poll-and-refresh-view () >"Invoke `notmuch-poll' to import mail, then refresh the current view." > -- > 1.7.5.4 > > ___ > notmuch mailing list > notmuch at notmuchmail.org > http://notmuchmail.org/mailman/listinfo/notmuch
