Re: notmuch new: emails with BOM are ignored

[David Bremner]

Andreas Amann  writes:

> Hi,
>
> I recently received some spam mails, which have a utf-16 byte order mark
> (BOM) U+FEFF as the first character in one of their "Received:"
> lines. When I run "notmuch new" I get the following:
>
> Note: Ignoring non-mail file: /home/user/Mail/new/path_to_email_with_BOM
>
> Could this be a bug in notmuch? Possible emails with BOM are not "legal"
> anyhow, but in my opinion it would be better to simply ignore the BOM.
>
> I am using notmuch 0.18.
>
> Andreas
>

I couldn't duplicate this old report with current notmuch. I replaced
"Received: " with "Received: " but the message was picked up by
notmuch new.

d
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

notmuch new: emails with BOM are ignored

[Andreas Amann]

Jani Nikula  writes:
> My quick guess is that gmime stops header processing at that point, and
> all the headers we require are after that invalid header. 

I can confirm that.  I adapted one of the example programs which comes
with gmime to read out the "From" header, and it indeed was not found,
because it came after the BOM.  Therefore the root of the problem is
indeed with gmime not notmuch.  I will send an email to the gmime-devel
list, to ask them if they consider this to be a bug. 

> If it were a real email and not spam, I'd be interested... this could be
> intentional to trick a spam filter.

quite possible.  Still a bit annoying ...

Andreas

notmuch new: emails with BOM are ignored

[Jani Nikula]

On Wed, 28 May 2014, David Bremner  wrote:
> I've tagged this as a wishlist bug / feature request for now, until
> somebody has a chance to track down where exactly the issue is, and how
> hard it would be to fix.

My quick guess is that gmime stops header processing at that point, and
all the headers we require are after that invalid header. Our old header
parser was a bit more relaxed in that sense, and probably just skipped
the header.

If it were a real email and not spam, I'd be interested... this could be
intentional to trick a spam filter.


BR,
Jani.

Re: notmuch new: emails with BOM are ignored

[Jani Nikula]

On Wed, 28 May 2014, David Bremner da...@tethera.net wrote:
 I've tagged this as a wishlist bug / feature request for now, until
 somebody has a chance to track down where exactly the issue is, and how
 hard it would be to fix.

My quick guess is that gmime stops header processing at that point, and
all the headers we require are after that invalid header. Our old header
parser was a bit more relaxed in that sense, and probably just skipped
the header.

If it were a real email and not spam, I'd be interested... this could be
intentional to trick a spam filter.


BR,
Jani.
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: notmuch new: emails with BOM are ignored

[Andreas Amann]

Jani Nikula j...@nikula.org writes:
 My quick guess is that gmime stops header processing at that point, and
 all the headers we require are after that invalid header. 

I can confirm that.  I adapted one of the example programs which comes
with gmime to read out the From header, and it indeed was not found,
because it came after the BOM.  Therefore the root of the problem is
indeed with gmime not notmuch.  I will send an email to the gmime-devel
list, to ask them if they consider this to be a bug. 

 If it were a real email and not spam, I'd be interested... this could be
 intentional to trick a spam filter.

quite possible.  Still a bit annoying ...

Andreas


___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

notmuch new: emails with BOM are ignored

[Andreas Amann]

Hi,

I recently received some spam mails, which have a utf-16 byte order mark
(BOM) U+FEFF as the first character in one of their "Received:"
lines. When I run "notmuch new" I get the following:

Note: Ignoring non-mail file: /home/user/Mail/new/path_to_email_with_BOM

Could this be a bug in notmuch? Possible emails with BOM are not "legal"
anyhow, but in my opinion it would be better to simply ignore the BOM.

I am using notmuch 0.18.

Andreas



P.S.: 

looking quickly at the code as a non-expert, it seems that the error
message comes from function "add_file" in "notmuch-new.c", because it
gets a status of "NOTMUCH_STATUS_FILE_NOT_EMAIL" from
"notmuch_database_add_message". This latter function is defined in
"lib/database.cc". It returns that status in the case, when
"notmuch_message_file_get_header" does not work (i.e. return NULL or "")
for the headers "from", "subject" or "to".  In the message at hand all
of these headers only come after the BOM, so my guess is that
"notmuch_message_file_get_header" somehow chokes when it hits on the
BOM?

notmuch new: emails with BOM are ignored

[Andreas Amann]

Hi,

I recently received some spam mails, which have a utf-16 byte order mark
(BOM) U+FEFF as the first character in one of their Received:
lines. When I run notmuch new I get the following:

Note: Ignoring non-mail file: /home/user/Mail/new/path_to_email_with_BOM

Could this be a bug in notmuch? Possible emails with BOM are not legal
anyhow, but in my opinion it would be better to simply ignore the BOM.

I am using notmuch 0.18.

Andreas



P.S.: 

looking quickly at the code as a non-expert, it seems that the error
message comes from function add_file in notmuch-new.c, because it
gets a status of NOTMUCH_STATUS_FILE_NOT_EMAIL from
notmuch_database_add_message. This latter function is defined in
lib/database.cc. It returns that status in the case, when
notmuch_message_file_get_header does not work (i.e. return NULL or )
for the headers from, subject or to.  In the message at hand all
of these headers only come after the BOM, so my guess is that
notmuch_message_file_get_header somehow chokes when it hits on the
BOM?





___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: notmuch new: emails with BOM are ignored

[David Bremner]

Andreas Amann a.am...@ucc.ie writes:


 Could this be a bug in notmuch? Possible emails with BOM are not legal
 anyhow, but in my opinion it would be better to simply ignore the BOM.

 I am using notmuch 0.18.

As of 0.18 notmuch is using the gmime mail parser exclusively. It could be a
bug/feature of the gmime library that it rejects such files.

I've tagged this as a wishlist bug / feature request for now, until
somebody has a chance to track down where exactly the issue is, and how
hard it would be to fix.

d
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch