exim pipe transport, notmuch insert and mbox-style messages
On Sb, 22 nov 14, 10:44:45, David Bremner wrote: > Edward Betts writes: > > > > I wonder if 'notmuch insert' could be modified to detect and drop the From_ > > line before writing the message to disk and index it. It could do that > > silently or with a warning. > > I don't know about the larger question(s), but I'd suggest just escaping > it to something like X-Envelope-From: . There may even be some > semi-standard header to use for this. I had the same problem with postfix + maildrop, as local(8) unconditionally adds the 'From ' header when delivering to an external command and upstream did not consider my suggestion to make this configurable. The work-arounds I came up with: 1. use 'reformail -f0' as suggested in maildrop(1) reformail(1): -f0 Any initial blank lines are removed. If the first non-blank line is a "From_" line, it gets converted to a "Return-Path:" header, and any existing "Return-Path:" header gets removed. If the message does not start with a "From_" line, the message remains unchanged. 2. adjust the 'maildrop' destination in postfix' master.cf to remove the 'F' flag. pipe(8): F Prepend a "From sender time_stamp" envelope header to the message content. This is expected by, for example, UUCP software. 3. Deliver from postfix to a Maildir, bypassing any filtering. Kind regards, Andrei -- If you can't explain it simply, you don't understand it well enough. (Albert Einstein) http://nuvreauspam.ro/gpg-transition.txt -- next part -- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: Digital signature URL: <http://notmuchmail.org/pipermail/notmuch/attachments/20141122/f9095b31/attachment-0001.pgp>
[PATCH v2 2/2] test: Add test for correct naming of author.
On Fri, 21 Nov 2014, Jesse Rosenthal wrote: > This is a new test file, since handling of unusual email addresses > doesn't seem to fit well in any of our existing tests. It would be nice to have this as patch 1/2 annotated with "known broken" and have the implementation patch fix it. It's nice to see that the patch actually changes things! ;) BR, Jani. > > Signed-off-by: Jesse Rosenthal > --- > test/T205-author-naming.sh | 12 > 1 file changed, 12 insertions(+) > create mode 100755 test/T205-author-naming.sh > > diff --git a/test/T205-author-naming.sh b/test/T205-author-naming.sh > new file mode 100755 > index 000..cb678ae > --- /dev/null > +++ b/test/T205-author-naming.sh > @@ -0,0 +1,12 @@ > +#!/usr/bin/env bash > +test_description="naming of authors with unusual addresses" > +. ./test-lib.sh > + > +test_begin_subtest "Add author with empty quoted real name" > +add_message '[subject]="author-naming: Initial thread subject"' \ > + '[date]="Fri, 05 Jan 2001 15:43:56 -"' \ > + '[from]="\"\" "' > +output=$(notmuch search --sort=oldest-first author-naming and tag:inbox | > notmuch_search_sanitize) > +test_expect_equal "$output" "thread:XXX 2001-01-05 [1/1] address at > example.com; author-naming: Initial thread subject (inbox unread)" > + > +test_done > -- > 2.1.3 > > ___ > notmuch mailing list > notmuch at notmuchmail.org > http://notmuchmail.org/mailman/listinfo/notmuch
[PATCH v2 1/2] lib: Use email address instead of empty real name.
On Fri, 21 Nov 2014, Jesse Rosenthal wrote:
> Currently, if a From-header is of the form:
>
> ""
>
> the empty string will be treated as a valid real-name, and the entry
> in the search results will be empty.
>
> The new behavior here is that we treat an empty real-name field as if
> it were null, so that the email address will be used in the search
> results instead.
Makes sense.
> Signed-off-by: Jesse Rosenthal
> ---
> lib/thread.cc | 3 ++-
> 1 file changed, 2 insertions(+), 1 deletion(-)
>
> diff --git a/lib/thread.cc b/lib/thread.cc
> index 8922403..68b2b94 100644
> --- a/lib/thread.cc
> +++ b/lib/thread.cc
> @@ -277,7 +277,8 @@ _thread_add_message (notmuch_thread_t *thread,
> address = internet_address_list_get_address (list, 0);
> if (address) {
> author = internet_address_get_name (address);
> - if (author == NULL) {
> + // We treat quoted empty names as if they were empty.
Nitpick, we don't use // style comments.
> + if (author == NULL || author[0]=='\0' ) {
Nitpick, please add spaces around "==" but no space before closing ")".
BR,
Jani.
> InternetAddressMailbox *mailbox;
> mailbox = INTERNET_ADDRESS_MAILBOX (address);
> author = internet_address_mailbox_get_addr (mailbox);
> --
> 2.1.3
>
> ___
> notmuch mailing list
> notmuch at notmuchmail.org
> http://notmuchmail.org/mailman/listinfo/notmuch
exim pipe transport, notmuch insert and mbox-style messages
Edward Betts writes: > I wonder if 'notmuch insert' could be modified to detect and drop the From_ > line before writing the message to disk and index it. It could do that > silently or with a warning. I don't know about the larger question(s), but I'd suggest just escaping it to something like X-Envelope-From: . There may even be some semi-standard header to use for this. d
exim pipe transport, notmuch insert and mbox-style messages
My mail arrives on a Debian machine running exim. It was being filtered
by procmail then piped into 'notmuch insert'. This was generating the
single-message mbox warning:
> Warning: ... is an mbox containing a single message,
> likely caused by misconfigured mail delivery. Support for single-message
> mboxes is deprecated and may be removed in the future.
I thought it was a problem with procmail delivering mbox style messages to
pipes, so I switched to maildrop. Same problem. This warning is produced
because there is an mbox-style From_ line before the first header line. Here
is an example:
> From MAILER-DAEMON Fri Jul 8 12:08:34 2011
The man page for maildrop says this style of message isn't supported, but it
doesn't detect and warn about them. It will pipe them into commands without
removing the From_ line.
Reading the exim documentation for pipe transports I found the message_prefix
option, which is by default is set like this:
> message_prefix = \
> From ${if def:return_path{$return_path}{MAILER-DAEMON}}\
> ${tod_bsdinbox}\n
This means that exim turns all messages into mbox style messages before
delivering them to a pipe. The justification given in the documentation is
"This is required by the commonly used /usr/bin/vacation program."
The fix is to unset the exim message_prefix option for delivery to maildrop.
Here is an example of the Debian exim maildrop_pipe transport with a blank
message_prefix to stop the From_ line being added before the message is piped
to maildrop.
> maildrop_pipe:
> debug_print = "T: maildrop_pipe for $local_part@$domain"
> driver = pipe
> path = "/bin:/usr/bin:/usr/local/bin"
> command = "/usr/bin/maildrop"
> message_prefix =
> message_suffix =
> return_path_add
> delivery_date_add
> envelope_to_add
The same fix could be applied for procmail pipe or a pipe from exim to
'notmuch insert'.
I submitted a Debian bug for exim4-config with my change as a patch. The
maintainer has excepted my patch and uploaded a release of exim4-config with
the fix to Debian experimental.
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=769396
I wonder if 'notmuch insert' could be modified to detect and drop the From_
line before writing the message to disk and index it. It could do that
silently or with a warning.
--
Edward.
Re: exim pipe transport, notmuch insert and mbox-style messages
On Sb, 22 nov 14, 10:44:45, David Bremner wrote: > Edward Betts writes: > > > > I wonder if 'notmuch insert' could be modified to detect and drop the From_ > > line before writing the message to disk and index it. It could do that > > silently or with a warning. > > I don't know about the larger question(s), but I'd suggest just escaping > it to something like X-Envelope-From: . There may even be some > semi-standard header to use for this. I had the same problem with postfix + maildrop, as local(8) unconditionally adds the 'From ' header when delivering to an external command and upstream did not consider my suggestion to make this configurable. The work-arounds I came up with: 1. use 'reformail -f0' as suggested in maildrop(1) reformail(1): -f0 Any initial blank lines are removed. If the first non-blank line is a "From_" line, it gets converted to a "Return-Path:" header, and any existing "Return-Path:" header gets removed. If the message does not start with a "From_" line, the message remains unchanged. 2. adjust the 'maildrop' destination in postfix' master.cf to remove the 'F' flag. pipe(8): F Prepend a "From sender time_stamp" envelope header to the message content. This is expected by, for example, UUCP software. 3. Deliver from postfix to a Maildir, bypassing any filtering. Kind regards, Andrei -- If you can't explain it simply, you don't understand it well enough. (Albert Einstein) http://nuvreauspam.ro/gpg-transition.txt signature.asc Description: Digital signature ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
[PATCH v3 2/2] lib: Use email address instead of empty real name.
Currently, if a From-header is of the form:
""
the empty string will be treated as a valid real-name, and the entry
in the search results will be empty.
The new behavior here is that we treat an empty real-name field as if
it were null, so that the email address will be used in the search
results instead.
Signed-off-by: Jesse Rosenthal
---
lib/thread.cc | 3 ++-
test/T205-author-naming.sh | 1 -
2 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/lib/thread.cc b/lib/thread.cc
index 8922403..79c3e9b 100644
--- a/lib/thread.cc
+++ b/lib/thread.cc
@@ -277,7 +277,8 @@ _thread_add_message (notmuch_thread_t *thread,
address = internet_address_list_get_address (list, 0);
if (address) {
author = internet_address_get_name (address);
- if (author == NULL) {
+ /* We treat quoted empty names as if they were empty. */
+ if (author == NULL || author[0] == '\0') {
InternetAddressMailbox *mailbox;
mailbox = INTERNET_ADDRESS_MAILBOX (address);
author = internet_address_mailbox_get_addr (mailbox);
diff --git a/test/T205-author-naming.sh b/test/T205-author-naming.sh
index 18819dd..cb678ae 100755
--- a/test/T205-author-naming.sh
+++ b/test/T205-author-naming.sh
@@ -3,7 +3,6 @@ test_description="naming of authors with unusual addresses"
. ./test-lib.sh
test_begin_subtest "Add author with empty quoted real name"
-test_subtest_known_broken
add_message '[subject]="author-naming: Initial thread subject"' \
'[date]="Fri, 05 Jan 2001 15:43:56 -"' \
'[from]="\"\" "'
--
2.1.3
[PATCH v3 1/2] test: Add known-broken test for empty author name
We test for whether a quoted empty email address "" will show up as the address, instead of the empty string. This is marked as known-broken, since the current behavior is to use the empty string. This is a new test file, since handling of unusual email addresses doesn't seem to fit well in any of our existing tests. Signed-off-by: Jesse Rosenthal --- test/T205-author-naming.sh | 13 + 1 file changed, 13 insertions(+) create mode 100755 test/T205-author-naming.sh diff --git a/test/T205-author-naming.sh b/test/T205-author-naming.sh new file mode 100755 index 000..18819dd --- /dev/null +++ b/test/T205-author-naming.sh @@ -0,0 +1,13 @@ +#!/usr/bin/env bash +test_description="naming of authors with unusual addresses" +. ./test-lib.sh + +test_begin_subtest "Add author with empty quoted real name" +test_subtest_known_broken +add_message '[subject]="author-naming: Initial thread subject"' \ + '[date]="Fri, 05 Jan 2001 15:43:56 -"' \ + '[from]="\"\" "' +output=$(notmuch search --sort=oldest-first author-naming and tag:inbox | notmuch_search_sanitize) +test_expect_equal "$output" "thread:XXX 2001-01-05 [1/1] address at example.com; author-naming: Initial thread subject (inbox unread)" + +test_done -- 2.1.3
[PATCH v3 0/2] Use email address instead of empty real name.
This version obsoletes [1]. The changes in this version are: * Commit test first with known-broken annotation, and then remove that annotation in the second commit (when the test passes) * Style fixes in the thread.cc to conform with notmuch style. [1] id:1416585462-24558-1-git-send-email-jrosenthal at jhu.edu Jesse Rosenthal (2): test: Add known-broken test for empty author name lib: Use email address instead of empty real name. lib/thread.cc | 3 ++- test/T205-author-naming.sh | 12 2 files changed, 14 insertions(+), 1 deletion(-) create mode 100755 test/T205-author-naming.sh -- 2.1.3
[PATCH v3 2/2] lib: Use email address instead of empty real name.
Currently, if a From-header is of the form:
""
the empty string will be treated as a valid real-name, and the entry
in the search results will be empty.
The new behavior here is that we treat an empty real-name field as if
it were null, so that the email address will be used in the search
results instead.
Signed-off-by: Jesse Rosenthal
---
lib/thread.cc | 3 ++-
test/T205-author-naming.sh | 1 -
2 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/lib/thread.cc b/lib/thread.cc
index 8922403..79c3e9b 100644
--- a/lib/thread.cc
+++ b/lib/thread.cc
@@ -277,7 +277,8 @@ _thread_add_message (notmuch_thread_t *thread,
address = internet_address_list_get_address (list, 0);
if (address) {
author = internet_address_get_name (address);
- if (author == NULL) {
+ /* We treat quoted empty names as if they were empty. */
+ if (author == NULL || author[0] == '\0') {
InternetAddressMailbox *mailbox;
mailbox = INTERNET_ADDRESS_MAILBOX (address);
author = internet_address_mailbox_get_addr (mailbox);
diff --git a/test/T205-author-naming.sh b/test/T205-author-naming.sh
index 18819dd..cb678ae 100755
--- a/test/T205-author-naming.sh
+++ b/test/T205-author-naming.sh
@@ -3,7 +3,6 @@ test_description="naming of authors with unusual addresses"
. ./test-lib.sh
test_begin_subtest "Add author with empty quoted real name"
-test_subtest_known_broken
add_message '[subject]="author-naming: Initial thread subject"' \
'[date]="Fri, 05 Jan 2001 15:43:56 -"' \
'[from]="\"\" "'
--
2.1.3
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch
[PATCH v3 0/2] Use email address instead of empty real name.
This version obsoletes [1]. The changes in this version are: * Commit test first with known-broken annotation, and then remove that annotation in the second commit (when the test passes) * Style fixes in the thread.cc to conform with notmuch style. [1] id:1416585462-24558-1-git-send-email-jrosent...@jhu.edu Jesse Rosenthal (2): test: Add known-broken test for empty author name lib: Use email address instead of empty real name. lib/thread.cc | 3 ++- test/T205-author-naming.sh | 12 2 files changed, 14 insertions(+), 1 deletion(-) create mode 100755 test/T205-author-naming.sh -- 2.1.3 ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
[PATCH v3 1/2] test: Add known-broken test for empty author name
We test for whether a quoted empty email address "" will show up as the address, instead of the empty string. This is marked as known-broken, since the current behavior is to use the empty string. This is a new test file, since handling of unusual email addresses doesn't seem to fit well in any of our existing tests. Signed-off-by: Jesse Rosenthal --- test/T205-author-naming.sh | 13 + 1 file changed, 13 insertions(+) create mode 100755 test/T205-author-naming.sh diff --git a/test/T205-author-naming.sh b/test/T205-author-naming.sh new file mode 100755 index 000..18819dd --- /dev/null +++ b/test/T205-author-naming.sh @@ -0,0 +1,13 @@ +#!/usr/bin/env bash +test_description="naming of authors with unusual addresses" +. ./test-lib.sh + +test_begin_subtest "Add author with empty quoted real name" +test_subtest_known_broken +add_message '[subject]="author-naming: Initial thread subject"' \ + '[date]="Fri, 05 Jan 2001 15:43:56 -"' \ + '[from]="\"\" "' +output=$(notmuch search --sort=oldest-first author-naming and tag:inbox | notmuch_search_sanitize) +test_expect_equal "$output" "thread:XXX 2001-01-05 [1/1] addr...@example.com; author-naming: Initial thread subject (inbox unread)" + +test_done -- 2.1.3 ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
Re: [PATCH v2 2/2] test: Add test for correct naming of author.
On Fri, 21 Nov 2014, Jesse Rosenthal wrote: > This is a new test file, since handling of unusual email addresses > doesn't seem to fit well in any of our existing tests. It would be nice to have this as patch 1/2 annotated with "known broken" and have the implementation patch fix it. It's nice to see that the patch actually changes things! ;) BR, Jani. > > Signed-off-by: Jesse Rosenthal > --- > test/T205-author-naming.sh | 12 > 1 file changed, 12 insertions(+) > create mode 100755 test/T205-author-naming.sh > > diff --git a/test/T205-author-naming.sh b/test/T205-author-naming.sh > new file mode 100755 > index 000..cb678ae > --- /dev/null > +++ b/test/T205-author-naming.sh > @@ -0,0 +1,12 @@ > +#!/usr/bin/env bash > +test_description="naming of authors with unusual addresses" > +. ./test-lib.sh > + > +test_begin_subtest "Add author with empty quoted real name" > +add_message '[subject]="author-naming: Initial thread subject"' \ > + '[date]="Fri, 05 Jan 2001 15:43:56 -"' \ > + '[from]="\"\" "' > +output=$(notmuch search --sort=oldest-first author-naming and tag:inbox | > notmuch_search_sanitize) > +test_expect_equal "$output" "thread:XXX 2001-01-05 [1/1] > addr...@example.com; author-naming: Initial thread subject (inbox unread)" > + > +test_done > -- > 2.1.3 > > ___ > notmuch mailing list > notmuch@notmuchmail.org > http://notmuchmail.org/mailman/listinfo/notmuch ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
Re: [PATCH v2 1/2] lib: Use email address instead of empty real name.
On Fri, 21 Nov 2014, Jesse Rosenthal wrote:
> Currently, if a From-header is of the form:
>
> ""
>
> the empty string will be treated as a valid real-name, and the entry
> in the search results will be empty.
>
> The new behavior here is that we treat an empty real-name field as if
> it were null, so that the email address will be used in the search
> results instead.
Makes sense.
> Signed-off-by: Jesse Rosenthal
> ---
> lib/thread.cc | 3 ++-
> 1 file changed, 2 insertions(+), 1 deletion(-)
>
> diff --git a/lib/thread.cc b/lib/thread.cc
> index 8922403..68b2b94 100644
> --- a/lib/thread.cc
> +++ b/lib/thread.cc
> @@ -277,7 +277,8 @@ _thread_add_message (notmuch_thread_t *thread,
> address = internet_address_list_get_address (list, 0);
> if (address) {
> author = internet_address_get_name (address);
> - if (author == NULL) {
> + // We treat quoted empty names as if they were empty.
Nitpick, we don't use // style comments.
> + if (author == NULL || author[0]=='\0' ) {
Nitpick, please add spaces around "==" but no space before closing ")".
BR,
Jani.
> InternetAddressMailbox *mailbox;
> mailbox = INTERNET_ADDRESS_MAILBOX (address);
> author = internet_address_mailbox_get_addr (mailbox);
> --
> 2.1.3
>
> ___
> notmuch mailing list
> notmuch@notmuchmail.org
> http://notmuchmail.org/mailman/listinfo/notmuch
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch
Re: exim pipe transport, notmuch insert and mbox-style messages
Edward Betts writes: > I wonder if 'notmuch insert' could be modified to detect and drop the From_ > line before writing the message to disk and index it. It could do that > silently or with a warning. I don't know about the larger question(s), but I'd suggest just escaping it to something like X-Envelope-From: . There may even be some semi-standard header to use for this. d ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
exim pipe transport, notmuch insert and mbox-style messages
My mail arrives on a Debian machine running exim. It was being filtered
by procmail then piped into 'notmuch insert'. This was generating the
single-message mbox warning:
> Warning: ... is an mbox containing a single message,
> likely caused by misconfigured mail delivery. Support for single-message
> mboxes is deprecated and may be removed in the future.
I thought it was a problem with procmail delivering mbox style messages to
pipes, so I switched to maildrop. Same problem. This warning is produced
because there is an mbox-style From_ line before the first header line. Here
is an example:
> From MAILER-DAEMON Fri Jul 8 12:08:34 2011
The man page for maildrop says this style of message isn't supported, but it
doesn't detect and warn about them. It will pipe them into commands without
removing the From_ line.
Reading the exim documentation for pipe transports I found the message_prefix
option, which is by default is set like this:
> message_prefix = \
> From ${if def:return_path{$return_path}{MAILER-DAEMON}}\
> ${tod_bsdinbox}\n
This means that exim turns all messages into mbox style messages before
delivering them to a pipe. The justification given in the documentation is
"This is required by the commonly used /usr/bin/vacation program."
The fix is to unset the exim message_prefix option for delivery to maildrop.
Here is an example of the Debian exim maildrop_pipe transport with a blank
message_prefix to stop the From_ line being added before the message is piped
to maildrop.
> maildrop_pipe:
> debug_print = "T: maildrop_pipe for $local_part@$domain"
> driver = pipe
> path = "/bin:/usr/bin:/usr/local/bin"
> command = "/usr/bin/maildrop"
> message_prefix =
> message_suffix =
> return_path_add
> delivery_date_add
> envelope_to_add
The same fix could be applied for procmail pipe or a pipe from exim to
'notmuch insert'.
I submitted a Debian bug for exim4-config with my change as a patch. The
maintainer has excepted my patch and uploaded a release of exim4-config with
the fix to Debian experimental.
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=769396
I wonder if 'notmuch insert' could be modified to detect and drop the From_
line before writing the message to disk and index it. It could do that
silently or with a warning.
--
Edward.
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch
