Re: [PATCH] after gzgets(), Z_STREAM_END means EOF, not error

[Olivier Taïbi]

On Tue, Apr 14, 2020 at 05:38:32PM -0300, David Bremner wrote:
> Olivier Taïbi  writes:
> > PS: out of curiosity, why bother with the --gzip feature in notmuch dump
> > and restore when the user can simply pipe to/from a gzip/bzip2/xz/...
> > command?
> 
> I believe the original motivation (in 2014) was to make the dumping
> process more atomic. I guess you could dig up the mailing list
> discussion from the  time if you were interested. I'd be reluctant to
> remove the feature after 6 six years.

Thanks for the explanation.
 
> > PPS: apart from dump and restore (and the indirect use of xapian), it
> > seems that the only other use of zlib in notmuch is in
> > format_part_mbox() in notmuch-show.c, which is able to read a compressed
> > email (it seems that dovecot has an option to write emails in maildir
> > format in this way to save space).>  Do I understand correctly that
> > notmuch does not support indexing compressed email, and if so what is
> > the point of using zlib in format_part_mbox()?
> 
> No, that's not correct. Notmuch does support indexing gzip compressed
> mail as of version 0.29. IIRC that part uses GMime streams to do the
> decompression (probably also using zlib indirectly).

Thanks, that's good to know.

> It would be helpful to move the discussion not intended to be part of
> the commit message after --- (see https://notmuchmail.org/contributing/
> for details).

Sorry about that, I read the "bugs" section of the website but did not
notice these guidelines.  I'll do my best to do better next time.
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: timezone in notmuch

[David Bremner]

David Bremner  writes:

>
> There's no user customizable variable for this. The code in this part of
> notmuch show is fairly simple, so someone could probably figure out how
> to pass (notmuch-show-get-timestamp) to the appropriate emacs function
> to format the date in the local timezone.

I should mention that having "notmuch-show-relative-dates" on hides this
problem, since it doesn't use timezones at all. Perhaps you would prefer
that (or perhaps not).

d
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: timezone in notmuch

[David Bremner]

di...@santanas.co.za writes:

> Greetings :)
>
> In notmuch-show-mode, some emails have a date like this: Date: Thu, 09
> Apr 2020 14:34:42 + while others like this Date: Sun, 12 Apr 2020
> 21:04:01 +0200
>
> Is this something I can change in notmuch to always show the date/time
> in my local timezone (SAST/+0200)?

There's no user customizable variable for this. The code in this part of
notmuch show is fairly simple, so someone could probably figure out how
to pass (notmuch-show-get-timestamp) to the appropriate emacs function
to format the date in the local timezone.

d
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH] gzerror() after gzclose_r() is a use after free

[David Bremner]

Olivier Taïbi  writes:

> As suggested by David Bremner in
> https://notmuchmail.org/pipermail/notmuch/2020/029288.html
> here is a separate patch for bug #2: calling gzerror() (indirectly via
> gzerror_str()) after gzclose_r is a use after free, according to zlib's 
> manual.
>

Looks OK. The commit message could be trimmed, but I can live with the
current version.

d
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH] after gzgets(), Z_STREAM_END means EOF, not error

[David Bremner]

Olivier Taïbi  writes:

> As suggested by David Bremner in
> https://notmuchmail.org/pipermail/notmuch/2020/029288.html
> here is the patch for bug #3: after gzgets() returns NULL (meaning EOF
> or error), the error code Z_STREAM_END means EOF and not error.

> PS: out of curiosity, why bother with the --gzip feature in notmuch dump
> and restore when the user can simply pipe to/from a gzip/bzip2/xz/...
> command?

I believe the original motivation (in 2014) was to make the dumping
process more atomic. I guess you could dig up the mailing list
discussion from the  time if you were interested. I'd be reluctant to
remove the feature after 6 six years.

> PPS: apart from dump and restore (and the indirect use of xapian), it
> seems that the only other use of zlib in notmuch is in
> format_part_mbox() in notmuch-show.c, which is able to read a compressed
> email (it seems that dovecot has an option to write emails in maildir
> format in this way to save space).>  Do I understand correctly that
> notmuch does not support indexing compressed email, and if so what is
> the point of using zlib in format_part_mbox()?

No, that's not correct. Notmuch does support indexing gzip compressed
mail as of version 0.29. IIRC that part uses GMime streams to do the
decompression (probably also using zlib indirectly).
>
> diff --git a/util/zlib-extra.c b/util/zlib-extra.c
> index 623f6d62..2d2d2414 100644
> --- a/util/zlib-extra.c
> +++ b/util/zlib-extra.c
> @@ -47,6 +47,7 @@ gz_getline (void *talloc_ctx, char **bufptr, ssize_t 
> *bytes_read, gzFile stream)
>   int zlib_status = 0;
>   (void) gzerror (stream, _status);
>   switch (zlib_status) {
> + case Z_STREAM_END:
>   case Z_OK:
>   /* no data read before EOF */
>   if (offset == 0)

As you say, the lib docs could be improved, but I guess the "zlib error
numbers" referred to as re the same as "Return codes for the
compression/decompression functions", in which case this makes sense.

It would be helpful to move the discussion not intended to be part of
the commit message after --- (see https://notmuchmail.org/contributing/
for details).

d
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH] after gzgets(), Z_STREAM_END means EOF, not error

[Olivier Taïbi]

As suggested by David Bremner in
https://notmuchmail.org/pipermail/notmuch/2020/029288.html
here is the patch for bug #3: after gzgets() returns NULL (meaning EOF
or error), the error code Z_STREAM_END means EOF and not error.

Context: I am compiling notmuch on OpenBSD which has a rather old zlib
1.2.3.  It seems that the behaviour of gzgets() changed slightly between
this version and more recent versions, but the manual does not reflect
that change.  Note that zlib's manual:
- does not specify which error code (Z_OK or Z_STREAM_END) is set when
  EOF is reached,
- does not indicate the meaning of Z_STREAM_END after gzgets(), but
  based on its meaning as a possible return value of inflate(), I would
  guess that it means EOF.

PS: out of curiosity, why bother with the --gzip feature in notmuch dump
and restore when the user can simply pipe to/from a gzip/bzip2/xz/...
command?  It seems that this feature adds complication, some of this
complication being due to the fact that the behaviour of zlib is not as
well-defined as stdio.  Moreover using pipes allows to check that the
compressed dump decompresses as it should, e.g.:
$ mkfifo pipe
$ dump.gz &
$ notmuch dump | tee pipe | cksum
$ wait && zcat dump.gz | cksum ; rm pipe
This could be simplified with process substitution for shells that have
this feature, and the checksum comparison can certainly be made
automatic in a backup script.
Disclosure: I am biased because I am currently patching notmuch-dump.c
to use stdio instead of zlib in order to port notmuch to OpenBSD, since
OpenBSD's older zlib does not have the "T" mode for gzopen(), so with
zlib the only choice would to compress the output.  Perhaps zlib will be
updated in OpenBSD in the future, but this is a short-term solution that
seems to not be too much trouble to maintain downstream.

PPS: apart from dump and restore (and the indirect use of xapian), it
seems that the only other use of zlib in notmuch is in
format_part_mbox() in notmuch-show.c, which is able to read a compressed
email (it seems that dovecot has an option to write emails in maildir
format in this way to save space).  Do I understand correctly that
notmuch does not support indexing compressed email, and if so what is
the point of using zlib in format_part_mbox()?


diff --git a/util/zlib-extra.c b/util/zlib-extra.c
index 623f6d62..2d2d2414 100644
--- a/util/zlib-extra.c
+++ b/util/zlib-extra.c
@@ -47,6 +47,7 @@ gz_getline (void *talloc_ctx, char **bufptr, ssize_t 
*bytes_read, gzFile stream)
int zlib_status = 0;
(void) gzerror (stream, _status);
switch (zlib_status) {
+   case Z_STREAM_END:
case Z_OK:
/* no data read before EOF */
if (offset == 0)
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH] gzerror() after gzclose_r() is a use after free

[Olivier Taïbi]

As suggested by David Bremner in
https://notmuchmail.org/pipermail/notmuch/2020/029288.html
here is a separate patch for bug #2: calling gzerror() (indirectly via
gzerror_str()) after gzclose_r is a use after free, according to zlib's manual.

diff --git a/notmuch-restore.c b/notmuch-restore.c
index 9a8b7fb5..e2dc3d45 100644
--- a/notmuch-restore.c
+++ b/notmuch-restore.c
@@ -237,6 +237,7 @@ notmuch_restore_command (notmuch_config_t *config, int 
argc, char *argv[])
 int opt_index;
 int include = 0;
 int input_format = DUMP_FORMAT_AUTO;
+int errnum;
 
 if (notmuch_database_open (notmuch_config_get_database_path (config),
   NOTMUCH_DATABASE_MODE_READ_WRITE, ))
@@ -448,10 +449,13 @@ notmuch_restore_command (notmuch_config_t *config, int 
argc, char *argv[])
 if (notmuch)
notmuch_database_destroy (notmuch);
 
-if (input && gzclose_r (input)) {
-   fprintf (stderr, "Error closing %s: %s\n",
-name_for_error, gzerror_str (input));
-   ret = EXIT_FAILURE;
+if (input) {
+   errnum = gzclose_r (input);
+   if (errnum) {
+   fprintf (stderr, "Error closing %s: %d\n",
+name_for_error, errnum);
+   ret = EXIT_FAILURE;
+   }
 }
 
 return ret ? EXIT_FAILURE : EXIT_SUCCESS;
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: easy (?) elisp project for notmuch [it's mostly DONE]

[Kyle Meyer]

Jonas Bernoulli  writes:

> (I cannot actually reply to "easy (?) elisp project for notmuch"
> because I wasn't subscribed yet when that was send.)

[ in case it's useful in the future ]

I've recently set up a public-inbox archive of notmuch here:

https://yhetil.org/notmuch/

>From there, you can easily download an mbox for a message or thread from
before you subscribed (e.g. to feed to 'notmuch insert').  As an
example, here's the one for your message:

https://yhetil.org/notmuch/875ze211gi@bernoul.li/raw

It might be possible to download a single message or thread from the
archive at , though looking
quickly it seems like downloads may only be available for messages over
a year time span.  (Also, FWIW the entire history is available at
.)
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: easy (?) elisp project for notmuch [it's mostly DONE]

[David Bremner]

Jonas Bernoulli  writes:

> I have already done this (and then some) a while ago.  I want to have
> another look before I submit it, but will try to get that done today.
>
> (I cannot actually reply to "easy (?) elisp project for notmuch"
> because I wasn't subscribed yet when that was send.)
>

Sounds good. Now that you've claimed the project, no need to push
yourself to get it done today ;).

cheers

d
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: easy (?) elisp project for notmuch [it's mostly DONE]

[Jonas Bernoulli]

> As of Emacs 27, Emacs will start issuing deprecation warnings for
> packages that load cl.el. I _think_ it's just a matter of replacing
> functions and macros from cl.el with cl- prefixed ones, but I
> haven't really investigated.

> If someone is looking for an easy way to contribute, this cleanup might
> be a good way to start.

Just a heads up to avoid duplicated work.

I have already done this (and then some) a while ago.  I want to have
another look before I submit it, but will try to get that done today.

(I cannot actually reply to "easy (?) elisp project for notmuch"
because I wasn't subscribed yet when that was send.)

  Cheers,
  Jonas
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

easy (?) elisp project for notmuch

[David Bremner]

As of Emacs 27, Emacs will start issuing deprecation warnings for
packages that load cl.el. I _think_ it's just a matter of replacing
functions and macros from cl.el with cl- prefixed ones, but I
haven't really investigated.

If someone is looking for an easy way to contribute, this cleanup might
be a good way to start.

d
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

timezone in notmuch

[divan]

Greetings :)

In notmuch-show-mode, some emails have a date like this: Date: Thu, 09
Apr 2020 14:34:42 + while others like this Date: Sun, 12 Apr 2020
21:04:01 +0200

Is this something I can change in notmuch to always show the date/time
in my local timezone (SAST/+0200)?

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH] emacs: introduce notmuch-search-by-tag

[David Edmondson]

On Monday, 2020-04-13 at 22:10:50 +02, Keegan Carruthers-Smith wrote:

> This is like notmuch-search-filter-by-tag, but creates a new search
> rather than filtering the current search. We add this to
> notmuch-common-keymap since this can be used by many contexts. We bind
> to the key "t", which is the same key used by
> notmuch-search-filter-by-tag in notmuch-search-mode-map. This is done
> intentionally since the keybinding for notmuch-search-mode-map can be
> seen as a specialization of creating a new search.
>
> This change was motivated for use in "notmuch-hello". It is a more
> convenient way to search a tag than expanding the list of all tags. I
> also noticed many saved searches people use are simply tags.

Reviewed-by: David Edmondson 

> ---
>  devel/emacs-keybindings.org | 2 +-
>  emacs/notmuch-lib.el| 1 +
>  emacs/notmuch.el| 6 ++
>  3 files changed, 8 insertions(+), 1 deletion(-)
>
> diff --git a/devel/emacs-keybindings.org b/devel/emacs-keybindings.org
> index 464b9467..65dfe0eb 100644
> --- a/devel/emacs-keybindings.org
> +++ b/devel/emacs-keybindings.org
> @@ -20,7 +20,7 @@
>  | q | notmuch-bury-or-kill-this-buffer   | 
> notmuch-bury-or-kill-this-buffer  | 
> notmuch-bury-or-kill-this-buffer|
>  | r | notmuch-search-reply-to-thread-sender  | 
> notmuch-show-reply-sender | 
> notmuch-show-reply-sender   |
>  | s | notmuch-search | notmuch-search
> | notmuch-search  |
> -| t | notmuch-search-filter-by-tag   | toggle-truncate-lines 
> | |
> +| t | notmuch-search-filter-by-tag   | toggle-truncate-lines 
> | notmuch-search-by-tag   |
>  | u ||   
> | |
>  | v ||   
> | notmuch-show-view-all-mime-parts|
>  | w || 
> notmuch-show-save-attachments | 
> notmuch-show-save-attachments   |
> diff --git a/emacs/notmuch-lib.el b/emacs/notmuch-lib.el
> index 73b165e4..e085a06b 100644
> --- a/emacs/notmuch-lib.el
> +++ b/emacs/notmuch-lib.el
> @@ -153,6 +153,7 @@ For example, if you wanted to remove an \"inbox\" tag and 
> add an
>  (define-key map "?" 'notmuch-help)
>  (define-key map "q" 'notmuch-bury-or-kill-this-buffer)
>  (define-key map "s" 'notmuch-search)
> +(define-key map "t" 'notmuch-search-by-tag)
>  (define-key map "z" 'notmuch-tree)
>  (define-key map "u" 'notmuch-unthreaded)
>  (define-key map "m" 'notmuch-mua-new-mail)
> diff --git a/emacs/notmuch.el b/emacs/notmuch.el
> index f4789b4f..f5f03244 100644
> --- a/emacs/notmuch.el
> +++ b/emacs/notmuch.el
> @@ -1077,6 +1077,12 @@ current search results AND that are tagged with the 
> given tag."
> (list (notmuch-select-tag-with-completion "Filter by tag: " 
> notmuch-search-query-string)))
>(notmuch-search (concat notmuch-search-query-string " and tag:" tag) 
> notmuch-search-oldest-first))
>  
> +(defun notmuch-search-by-tag (tag)
> +  "Display threads matching TAG in a notmuch-search buffer."
> +  (interactive
> +   (list (notmuch-select-tag-with-completion "Notmuch search tag: ")))
> +  (notmuch-search (concat "tag:" tag)))
> +
>  ;;;###autoload
>  (defun notmuch ()
>"Run notmuch and display saved searches, known tags, etc."
> -- 
> 2.26.0
> ___
> notmuch mailing list
> notmuch@notmuchmail.org
> https://notmuchmail.org/mailman/listinfo/notmuch

dme.
-- 
Why stay in college? Why go to night school?
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH] emacs: use def instead of initial-input for notmuch-show-browse-urls

[David Edmondson]

On Monday, 2020-04-13 at 21:58:19 +02, Keegan Carruthers-Smith wrote:

> This is the non-deprecated way to use completing-read. Additionally
> the old use was broken when using ivy for completing-read. For user's
> using completing-read-default they won't see the default URL now, but
> if they hit enter it will be visited. Alternatively they can select
> it with M-n.
>
> From the completing-read documentation for initial-input:
>
>  This feature is deprecated--it is best to pass nil for INITIAL-INPUT
>  and supply the default value DEF instead.  The user can yank the
>  default value into the minibuffer easily using M-n.
>
> Additionally collection is now all urls, rather than all but the
> first. I'm not sure why "(cdr urls)" was previously done.

Reviewed-by: David Edmondson 

> ---
>  emacs/notmuch-show.el | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/emacs/notmuch-show.el b/emacs/notmuch-show.el
> index 214e279f..079281c3 100644
> --- a/emacs/notmuch-show.el
> +++ b/emacs/notmuch-show.el
> @@ -2559,7 +2559,7 @@ browsing."
>   (prompt (if kill "Copy URL to kill ring: " "Browse URL: "))
>   (fn (if kill #'kill-new #'browse-url)))
>  (if urls
> - (funcall fn (completing-read prompt (cdr urls) nil nil (car urls)))
> + (funcall fn (completing-read prompt urls nil nil nil nil (car urls)))
>(message "No URLs found."
>  
>  (provide 'notmuch-show)
> -- 
> 2.26.0
>
> ___
> notmuch mailing list
> notmuch@notmuchmail.org
> https://notmuchmail.org/mailman/listinfo/notmuch

dme.
-- 
I used to worry, thought I was goin' mad in a hurry.
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch