privacy problem: text/html parts pull in network resources

2015-01-27 Thread Daniel Kahn Gillmor
On Sun 2015-01-25 12:51:43 -0500, David Bremner wrote:
> Daniel Kahn Gillmor  writes:
>
>> If i send a message with a text/html part (either it's only text/html,
>> or all parts are rendered, or it's multipart/alternative with only a
>> text/html subpart) and that HTML has > src="http://example.org/test.png"/> in it, then notmuch will make a
>> network request for that image.
>>
>> This is a privacy disaster, because it enables an e-mail sender to use
>> "web bugs" to tell when a given notmuch user has opened their e-mail.
>
> I've just pushed Austin's shr related series to master, so this problem
> should be fixed as of commit b74ed1c. One tradeoff that we should at
> least remark in NEWS, if not actually fix, is that I think there is now
> no way to view such images in notmuch.  I don't know offhand what other
> html renderers will do.

thanks for this, David and Austin!

Other html-rendering mail clients that are privacy-conscious will often
provide a button or mechanism to indicate that some remote resources
were requested by the page but weren't fetched (e.g. a button saying
something like [Load Remote Images...]).  I have no idea who actually
clicks on those buttons (or why), though, and even if we wanted them,
we'd only want to add a button on an image that actually had remote
network resources to load, and i don't know how we'd get that
information propagated back up the rendering stack to make such a
display decision.   So i'm fine with leaving it this way for now.

--dkg


[PATCH 5/5] completion: complete addresses in from:/to: search terms

2015-01-27 Thread Jani Nikula
Use the new notmuch address command to do completion for addresses in
from: and to:.

---

This patch is more of an RFC. The to: completion is slw because
typically there are more matches to begin with, and then producing the
results requires reading the messages. Maybe it would be better to use
the same mechanism as from: for both, even if it's not accurate for
to:?
---
 completion/notmuch-completion.bash | 30 +-
 1 file changed, 25 insertions(+), 5 deletions(-)

diff --git a/completion/notmuch-completion.bash 
b/completion/notmuch-completion.bash
index e0498903f22f..db49294fc58c 100644
--- a/completion/notmuch-completion.bash
+++ b/completion/notmuch-completion.bash
@@ -27,10 +27,30 @@
 # on completion.
 #

-_notmuch_user_emails()
+# $1: current input of the form prefix:partialinput, where prefix is
+# to or from.
+_notmuch_email()
 {
-notmuch config get user.primary_email
-notmuch config get user.other_email
+local output prefix cur
+
+prefix="${1%%:*}"
+cur="${1#*:}"
+
+# Cut the input to be completed at punctuation because
+# (apparently) Xapian does not support the trailing wildcard '*'
+# operator for input with punctuation. We let compgen handle the
+# extra filtering required.
+cur="${cur%%[^a-zA-Z0-9]*}"
+
+case "$prefix" in
+   to) output=recipients;;
+   from) output=sender;;
+   *) return;;
+esac
+
+# Only emit plain, lower case, unique addresses.
+notmuch address --output=$output $prefix:"${cur}*" | \
+   sed 's/[^<]*<\([^>]*\)>/\1/' | tr "[:upper:]" "[:lower:]" | sort -u
 }

 _notmuch_search_terms()
@@ -44,10 +64,10 @@ _notmuch_search_terms()
COMPREPLY=( $(compgen -P "tag:" -W "`notmuch search --output=tags 
\*`" -- ${cur##tag:}) )
;;
to:*)
-   COMPREPLY=( $(compgen -P "to:" -W "`_notmuch_user_emails`" -- 
${cur##to:}) )
+   COMPREPLY=( $(compgen -P "to:" -W "`_notmuch_email ${cur}`" -- 
${cur##to:}) )
;;
from:*)
-   COMPREPLY=( $(compgen -P "from:" -W "`_notmuch_user_emails`" -- 
${cur##from:}) )
+   COMPREPLY=( $(compgen -P "from:" -W "`_notmuch_email ${cur}`" -- 
${cur##from:}) )
;;
path:*)
local path=`notmuch config get database.path`
-- 
2.1.4



[PATCH 4/5] completion: complete notmuch dump --gzip option

2015-01-27 Thread Jani Nikula
---
 completion/notmuch-completion.bash | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/completion/notmuch-completion.bash 
b/completion/notmuch-completion.bash
index 39320f8d8107..e0498903f22f 100644
--- a/completion/notmuch-completion.bash
+++ b/completion/notmuch-completion.bash
@@ -169,7 +169,7 @@ _notmuch_dump()
 ! $split &&
 case "${cur}" in
-*)
-   local options="--format= --output="
+   local options="--gzip --format= --output="
compopt -o nospace
COMPREPLY=( $(compgen -W "$options" -- ${cur}) )
;;
-- 
2.1.4



[PATCH 3/5] doc: document notmuch-restore --input=filename option

2015-01-27 Thread Jani Nikula
Document all options for completeness.
---
 doc/man1/notmuch-restore.rst | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/doc/man1/notmuch-restore.rst b/doc/man1/notmuch-restore.rst
index 936b1383e045..362e2629e574 100644
--- a/doc/man1/notmuch-restore.rst
+++ b/doc/man1/notmuch-restore.rst
@@ -50,6 +50,9 @@ Supported options for **restore** include
 format, this heuristic, based the fact that batch-tag format
 contains no parentheses, should be accurate.

+``--input=``\ 
+Read input from given file instead of stdin.
+
 GZIPPED INPUT
 =

-- 
2.1.4



[PATCH 2/5] doc: document notmuch-dump --output=filename option

2015-01-27 Thread Jani Nikula
Document all options for completeness.
---
 doc/man1/notmuch-dump.rst | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/doc/man1/notmuch-dump.rst b/doc/man1/notmuch-dump.rst
index 3a6d0b34f505..a37c337c72e6 100644
--- a/doc/man1/notmuch-dump.rst
+++ b/doc/man1/notmuch-dump.rst
@@ -71,6 +71,8 @@ Supported options for **dump** include
 characters. Note also that tags with spaces will not be
 correctly restored with this format.

+``--output=``\ 
+Write output to given file instead of stdout.

 SEE ALSO
 
-- 
2.1.4



[PATCH 1/5] doc: bring notmuch-dump manual page in line with the rest

2015-01-27 Thread Jani Nikula
Fix indentation of options. Move search terms description before
options. Fix synopsis.
---
 doc/man1/notmuch-dump.rst | 100 --
 1 file changed, 53 insertions(+), 47 deletions(-)

diff --git a/doc/man1/notmuch-dump.rst b/doc/man1/notmuch-dump.rst
index d94cb4f84496..3a6d0b34f505 100644
--- a/doc/man1/notmuch-dump.rst
+++ b/doc/man1/notmuch-dump.rst
@@ -5,7 +5,7 @@ notmuch-dump
 SYNOPSIS
 

-**notmuch** **dump** [--format=(batch-tag|sup)] [--] [--output=<*file*>] [--] 
[<*search-term*> ...]
+**notmuch** **dump** [--gzip] [--format=(batch-tag|sup)] [--output=<*file*>] 
[--] [<*search-term*> ...]

 DESCRIPTION
 ===
@@ -19,52 +19,58 @@ recreated from the messages themselves. The output of 
notmuch dump is
 therefore the only critical thing to backup (and much more friendly to
 incremental backup than the native database files.)

-``--gzip``
-Compress the output in a format compatible with **gzip(1)**.
-
-``--format=(sup|batch-tag)``
-Notmuch restore supports two plain text dump formats, both with one
-message-id per line, followed by a list of tags.
-
-**batch-tag**
-The default **batch-tag** dump format is intended to more robust
-against malformed message-ids and tags containing whitespace or
-non-\ **ascii(7)** characters. Each line has the form
-
-+<*encoded-tag*\ > +<*encoded-tag*\ > ... --
-id:<*quoted-message-id*\ >
-
-Tags are hex-encoded by replacing every byte not matching the
-regex **[A-Za-z0-9@=.,\_+-]** with **%nn** where nn is the two
-digit hex encoding. The message ID is a valid Xapian query,
-quoted using Xapian boolean term quoting rules: if the ID
-contains whitespace or a close paren or starts with a double
-quote, it must be enclosed in double quotes and double quotes
-inside the ID must be doubled. The astute reader will notice
-this is a special case of the batch input format for
-**notmuch-tag(1)**; note that the single message-id query is
-mandatory for **notmuch-restore(1)**.
-
-**sup**
-The **sup** dump file format is specifically chosen to be
-compatible with the format of files produced by sup-dump. So if
-you've previously been using sup for mail, then the **notmuch
-restore** command provides you a way to import all of your tags
-(or labels as sup calls them). Each line has the following form
-
-<*message-id*\ > **(** <*tag*\ > ... **)**
-
-with zero or more tags are separated by spaces. Note that
-(malformed) message-ids may contain arbitrary non-null
-characters. Note also that tags with spaces will not be
-correctly restored with this format.
-
-With no search terms, a dump of all messages in the database will be
-generated. A "--" argument instructs notmuch that the remaining
-arguments are search terms.
-
-See **notmuch-search-terms(7)** for details of the supported syntax
-for .
+See **notmuch-search-terms(7)** for details of the supported syntax
+for . With no search terms, a dump of all messages in
+the database will be generated. A "--" argument instructs notmuch that
+the remaining arguments are search terms.
+
+Supported options for **dump** include
+
+``--gzip``
+Compress the output in a format compatible with **gzip(1)**.
+
+``--format=(sup|batch-tag)``
+Notmuch restore supports two plain text dump formats, both with one
+message-id per line, followed by a list of tags.
+
+**batch-tag**
+
+The default **batch-tag** dump format is intended to more
+robust against malformed message-ids and tags containing
+whitespace or non-\ **ascii(7)** characters. Each line has
+the form
+
++<*encoded-tag*\ > +<*encoded-tag*\ > ... --
+id:<*quoted-message-id*\ >
+
+Tags are hex-encoded by replacing every byte not matching
+the regex **[A-Za-z0-9@=.,\_+-]** with **%nn** where nn is
+the two digit hex encoding. The message ID is a valid
+Xapian query, quoted using Xapian boolean term quoting
+rules: if the ID contains whitespace or a close paren or
+starts with a double quote, it must be enclosed in double
+quotes and double quotes inside the ID must be
+doubled. The astute reader will notice this is a special
+case of the batch input format for **notmuch-tag(1)**;
+note that the single message-id query is mandatory for
+**notmuch-restore(1)**.
+
+**sup**
+
+The **sup** dump file format is specifically chosen to be
+compatible with the format of files produced by
+sup-dump. So if you've previously been using sup for mail,
+then the **notmuch restore** command provides you a 

privacy problem: text/html parts pull in network resources

2015-01-27 Thread Jinwoo Lee
On Tue, Jan 27, 2015 at 07:47 PM, Daniel Kahn Gillmor  wrote:
> On Sun 2015-01-25 12:51:43 -0500, David Bremner wrote:
>> Daniel Kahn Gillmor  writes:
>>
>>> If i send a message with a text/html part (either it's only text/html,
>>> or all parts are rendered, or it's multipart/alternative with only a
>>> text/html subpart) and that HTML has >> src="http://example.org/test.png"/> in it, then notmuch will make a
>>> network request for that image.
>>>
>>> This is a privacy disaster, because it enables an e-mail sender to use
>>> "web bugs" to tell when a given notmuch user has opened their e-mail.
>>
>> I've just pushed Austin's shr related series to master, so this problem
>> should be fixed as of commit b74ed1c. One tradeoff that we should at
>> least remark in NEWS, if not actually fix, is that I think there is now
>> no way to view such images in notmuch.  I don't know offhand what other
>> html renderers will do.
>
> thanks for this, David and Austin!
>
> Other html-rendering mail clients that are privacy-conscious will often
> provide a button or mechanism to indicate that some remote resources
> were requested by the page but weren't fetched (e.g. a button saying
> something like [Load Remote Images...]).  I have no idea who actually
> clicks on those buttons (or why), though, and even if we wanted them,
> we'd only want to add a button on an image that actually had remote
> network resources to load, and i don't know how we'd get that
> information propagated back up the rendering stack to make such a
> display decision.   So i'm fine with leaving it this way for now.

Well, most promotional emails contain remote images and their contents
are incomprehensible without those images.  I ignore most of them but I
do read a few of those promotional emails.  It would be great to have a
UI for loading remote resources.

>
> --dkg
> ___
> notmuch mailing list
> notmuch at notmuchmail.org
> http://notmuchmail.org/mailman/listinfo/notmuch


Unable to search for emails with content with a given MIME type

2015-01-27 Thread Todd

Petter,

The feature you wanted was committed a few days ago (searching with a
"mimetype:" prefix).  Searching for "mimetype:calendar" would get any
"text/calendar" or "ics/calendar" attachments.

(Replying to the list so it will appear in the list archive as well.)

Thanks,
Todd

-- next part --
Petter Reinholdtsen  writes:

> Not quite sure where to send a feature request, so I try to follow the
> instructions for bug reporst.
>
> I would like to be able to search for emails with content with a given
> MIME type.  The latest use case I ran into was my need to locate Notes
> and Exchange emails with text/calendar attachments, but I've also needed
> to find application/pdf and application/vnd.oasis.opendocument.text
> emails.
>
> Please consider extending notmuch to be able to search for specific mime
> types, for example using attachment:text/calendar or mime:text/calendar
> or something like that.  Or perhaps just index up the mime content type
> value, to allow me to search for "tag:attachment and text/calendar".
>
> PS: Please CC me on replies, as I am not subscribed to the list.
> -- 
> Happy hacking
> Petter Reinholdtsen
-- next part --
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 818 bytes
Desc: not available
URL: 
<http://notmuchmail.org/pipermail/notmuch/attachments/20150127/d9a572d8/attachment.pgp>


[PATCH 1/5] doc: bring notmuch-dump manual page in line with the rest

2015-01-27 Thread Jani Nikula
Fix indentation of options. Move search terms description before
options. Fix synopsis.
---
 doc/man1/notmuch-dump.rst | 100 --
 1 file changed, 53 insertions(+), 47 deletions(-)

diff --git a/doc/man1/notmuch-dump.rst b/doc/man1/notmuch-dump.rst
index d94cb4f84496..3a6d0b34f505 100644
--- a/doc/man1/notmuch-dump.rst
+++ b/doc/man1/notmuch-dump.rst
@@ -5,7 +5,7 @@ notmuch-dump
 SYNOPSIS
 
 
-**notmuch** **dump** [--format=(batch-tag|sup)] [--] [--output=*file*] [--] 
[*search-term* ...]
+**notmuch** **dump** [--gzip] [--format=(batch-tag|sup)] [--output=*file*] 
[--] [*search-term* ...]
 
 DESCRIPTION
 ===
@@ -19,52 +19,58 @@ recreated from the messages themselves. The output of 
notmuch dump is
 therefore the only critical thing to backup (and much more friendly to
 incremental backup than the native database files.)
 
-``--gzip``
-Compress the output in a format compatible with **gzip(1)**.
-
-``--format=(sup|batch-tag)``
-Notmuch restore supports two plain text dump formats, both with one
-message-id per line, followed by a list of tags.
-
-**batch-tag**
-The default **batch-tag** dump format is intended to more robust
-against malformed message-ids and tags containing whitespace or
-non-\ **ascii(7)** characters. Each line has the form
-
-+*encoded-tag*\  +*encoded-tag*\  ... --
-id:*quoted-message-id*\ 
-
-Tags are hex-encoded by replacing every byte not matching the
-regex **[A-Za-z0-9@=.,\_+-]** with **%nn** where nn is the two
-digit hex encoding. The message ID is a valid Xapian query,
-quoted using Xapian boolean term quoting rules: if the ID
-contains whitespace or a close paren or starts with a double
-quote, it must be enclosed in double quotes and double quotes
-inside the ID must be doubled. The astute reader will notice
-this is a special case of the batch input format for
-**notmuch-tag(1)**; note that the single message-id query is
-mandatory for **notmuch-restore(1)**.
-
-**sup**
-The **sup** dump file format is specifically chosen to be
-compatible with the format of files produced by sup-dump. So if
-you've previously been using sup for mail, then the **notmuch
-restore** command provides you a way to import all of your tags
-(or labels as sup calls them). Each line has the following form
-
-*message-id*\  **(** *tag*\  ... **)**
-
-with zero or more tags are separated by spaces. Note that
-(malformed) message-ids may contain arbitrary non-null
-characters. Note also that tags with spaces will not be
-correctly restored with this format.
-
-With no search terms, a dump of all messages in the database will be
-generated. A -- argument instructs notmuch that the remaining
-arguments are search terms.
-
-See **notmuch-search-terms(7)** for details of the supported syntax
-for search-terms.
+See **notmuch-search-terms(7)** for details of the supported syntax
+for search-terms. With no search terms, a dump of all messages in
+the database will be generated. A -- argument instructs notmuch that
+the remaining arguments are search terms.
+
+Supported options for **dump** include
+
+``--gzip``
+Compress the output in a format compatible with **gzip(1)**.
+
+``--format=(sup|batch-tag)``
+Notmuch restore supports two plain text dump formats, both with one
+message-id per line, followed by a list of tags.
+
+**batch-tag**
+
+The default **batch-tag** dump format is intended to more
+robust against malformed message-ids and tags containing
+whitespace or non-\ **ascii(7)** characters. Each line has
+the form
+
++*encoded-tag*\  +*encoded-tag*\  ... --
+id:*quoted-message-id*\ 
+
+Tags are hex-encoded by replacing every byte not matching
+the regex **[A-Za-z0-9@=.,\_+-]** with **%nn** where nn is
+the two digit hex encoding. The message ID is a valid
+Xapian query, quoted using Xapian boolean term quoting
+rules: if the ID contains whitespace or a close paren or
+starts with a double quote, it must be enclosed in double
+quotes and double quotes inside the ID must be
+doubled. The astute reader will notice this is a special
+case of the batch input format for **notmuch-tag(1)**;
+note that the single message-id query is mandatory for
+**notmuch-restore(1)**.
+
+**sup**
+
+The **sup** dump file format is specifically chosen to be
+compatible with the format of files produced by
+sup-dump. So if you've previously been using sup for mail,
+then the **notmuch restore** command provides you a 

[PATCH 3/5] doc: document notmuch-restore --input=filename option

2015-01-27 Thread Jani Nikula
Document all options for completeness.
---
 doc/man1/notmuch-restore.rst | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/doc/man1/notmuch-restore.rst b/doc/man1/notmuch-restore.rst
index 936b1383e045..362e2629e574 100644
--- a/doc/man1/notmuch-restore.rst
+++ b/doc/man1/notmuch-restore.rst
@@ -50,6 +50,9 @@ Supported options for **restore** include
 format, this heuristic, based the fact that batch-tag format
 contains no parentheses, should be accurate.
 
+``--input=``\ filename
+Read input from given file instead of stdin.
+
 GZIPPED INPUT
 =
 
-- 
2.1.4

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Re: Unable to search for emails with content with a given MIME type

2015-01-27 Thread Todd

Petter,

The feature you wanted was committed a few days ago (searching with a
mimetype: prefix).  Searching for mimetype:calendar would get any
text/calendar or ics/calendar attachments.

(Replying to the list so it will appear in the list archive as well.)

Thanks,
Todd

Petter Reinholdtsen p...@hungry.com writes:

 Not quite sure where to send a feature request, so I try to follow the
 instructions for bug reporst.

 I would like to be able to search for emails with content with a given
 MIME type.  The latest use case I ran into was my need to locate Notes
 and Exchange emails with text/calendar attachments, but I've also needed
 to find application/pdf and application/vnd.oasis.opendocument.text
 emails.

 Please consider extending notmuch to be able to search for specific mime
 types, for example using attachment:text/calendar or mime:text/calendar
 or something like that.  Or perhaps just index up the mime content type
 value, to allow me to search for tag:attachment and text/calendar.

 PS: Please CC me on replies, as I am not subscribed to the list.
 -- 
 Happy hacking
 Petter Reinholdtsen


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


[PATCH 2/5] doc: document notmuch-dump --output=filename option

2015-01-27 Thread Jani Nikula
Document all options for completeness.
---
 doc/man1/notmuch-dump.rst | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/doc/man1/notmuch-dump.rst b/doc/man1/notmuch-dump.rst
index 3a6d0b34f505..a37c337c72e6 100644
--- a/doc/man1/notmuch-dump.rst
+++ b/doc/man1/notmuch-dump.rst
@@ -71,6 +71,8 @@ Supported options for **dump** include
 characters. Note also that tags with spaces will not be
 correctly restored with this format.
 
+``--output=``\ filename
+Write output to given file instead of stdout.
 
 SEE ALSO
 
-- 
2.1.4

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


[PATCH 4/5] completion: complete notmuch dump --gzip option

2015-01-27 Thread Jani Nikula
---
 completion/notmuch-completion.bash | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/completion/notmuch-completion.bash 
b/completion/notmuch-completion.bash
index 39320f8d8107..e0498903f22f 100644
--- a/completion/notmuch-completion.bash
+++ b/completion/notmuch-completion.bash
@@ -169,7 +169,7 @@ _notmuch_dump()
 ! $split 
 case ${cur} in
-*)
-   local options=--format= --output=
+   local options=--gzip --format= --output=
compopt -o nospace
COMPREPLY=( $(compgen -W $options -- ${cur}) )
;;
-- 
2.1.4

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


[PATCH 5/5] completion: complete addresses in from:/to: search terms

2015-01-27 Thread Jani Nikula
Use the new notmuch address command to do completion for addresses in
from: and to:.

---

This patch is more of an RFC. The to: completion is slw because
typically there are more matches to begin with, and then producing the
results requires reading the messages. Maybe it would be better to use
the same mechanism as from: for both, even if it's not accurate for
to:?
---
 completion/notmuch-completion.bash | 30 +-
 1 file changed, 25 insertions(+), 5 deletions(-)

diff --git a/completion/notmuch-completion.bash 
b/completion/notmuch-completion.bash
index e0498903f22f..db49294fc58c 100644
--- a/completion/notmuch-completion.bash
+++ b/completion/notmuch-completion.bash
@@ -27,10 +27,30 @@
 # on completion.
 #
 
-_notmuch_user_emails()
+# $1: current input of the form prefix:partialinput, where prefix is
+# to or from.
+_notmuch_email()
 {
-notmuch config get user.primary_email
-notmuch config get user.other_email
+local output prefix cur
+
+prefix=${1%%:*}
+cur=${1#*:}
+
+# Cut the input to be completed at punctuation because
+# (apparently) Xapian does not support the trailing wildcard '*'
+# operator for input with punctuation. We let compgen handle the
+# extra filtering required.
+cur=${cur%%[^a-zA-Z0-9]*}
+
+case $prefix in
+   to) output=recipients;;
+   from) output=sender;;
+   *) return;;
+esac
+
+# Only emit plain, lower case, unique addresses.
+notmuch address --output=$output $prefix:${cur}* | \
+   sed 's/[^]*\([^]*\)/\1/' | tr [:upper:] [:lower:] | sort -u
 }
 
 _notmuch_search_terms()
@@ -44,10 +64,10 @@ _notmuch_search_terms()
COMPREPLY=( $(compgen -P tag: -W `notmuch search --output=tags 
\*` -- ${cur##tag:}) )
;;
to:*)
-   COMPREPLY=( $(compgen -P to: -W `_notmuch_user_emails` -- 
${cur##to:}) )
+   COMPREPLY=( $(compgen -P to: -W `_notmuch_email ${cur}` -- 
${cur##to:}) )
;;
from:*)
-   COMPREPLY=( $(compgen -P from: -W `_notmuch_user_emails` -- 
${cur##from:}) )
+   COMPREPLY=( $(compgen -P from: -W `_notmuch_email ${cur}` -- 
${cur##from:}) )
;;
path:*)
local path=`notmuch config get database.path`
-- 
2.1.4

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Re: privacy problem: text/html parts pull in network resources

2015-01-27 Thread Jinwoo Lee
On Tue, Jan 27, 2015 at 07:47 PM, Daniel Kahn Gillmor d...@fifthhorseman.net 
wrote:
 On Sun 2015-01-25 12:51:43 -0500, David Bremner wrote:
 Daniel Kahn Gillmor d...@fifthhorseman.net writes:

 If i send a message with a text/html part (either it's only text/html,
 or all parts are rendered, or it's multipart/alternative with only a
 text/html subpart) and that HTML has img
 src=http://example.org/test.png/ in it, then notmuch will make a
 network request for that image.

 This is a privacy disaster, because it enables an e-mail sender to use
 web bugs to tell when a given notmuch user has opened their e-mail.

 I've just pushed Austin's shr related series to master, so this problem
 should be fixed as of commit b74ed1c. One tradeoff that we should at
 least remark in NEWS, if not actually fix, is that I think there is now
 no way to view such images in notmuch.  I don't know offhand what other
 html renderers will do.

 thanks for this, David and Austin!

 Other html-rendering mail clients that are privacy-conscious will often
 provide a button or mechanism to indicate that some remote resources
 were requested by the page but weren't fetched (e.g. a button saying
 something like [Load Remote Images...]).  I have no idea who actually
 clicks on those buttons (or why), though, and even if we wanted them,
 we'd only want to add a button on an image that actually had remote
 network resources to load, and i don't know how we'd get that
 information propagated back up the rendering stack to make such a
 display decision.   So i'm fine with leaving it this way for now.

Well, most promotional emails contain remote images and their contents
are incomprehensible without those images.  I ignore most of them but I
do read a few of those promotional emails.  It would be great to have a
UI for loading remote resources.


 --dkg
 ___
 notmuch mailing list
 notmuch@notmuchmail.org
 http://notmuchmail.org/mailman/listinfo/notmuch
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Re: [Patch v4 5/5] test: add broken test for SMIME decryption with notmuch CLI

2015-01-27 Thread David Bremner
David Bremner da...@tethera.net writes:

 David Bremner da...@tethera.net writes:

 The test JSON here is not correct, but the larger problem is thatit
 seems like no actual decryption is being done.

 I played with this some more, and it seems like Jamie's code (and the
 gmime sample code [1] expects the top level part to be
 multipart/encrypted.

By repeated bludgeoning I convinced notmuch show to actually run the
decryption code, but then I hit another problem: there isn't an obvious
high level way to decrypt an application/(x)-pkcs7-mime part (and the
current code only works for multipart/encrypted). It should
be possible up GMimeStreams and use g_mime_crypto_context_decrypt, but
that seems like quite a bit more work than calling
g_mime_multipart_encrypted_decrypt.

diff --git a/mime-node.c b/mime-node.c
index fd9e4a4..7019be7 100644
--- a/mime-node.c
+++ b/mime-node.c
@@ -54,6 +54,20 @@ _mime_node_context_free (mime_node_context_t *res)
 return 0;
 }
 
+static
+notmuch_bool_t
+_is_smime_encrypted_part (GMimeObject *part) {
+
+GMimeContentType *content_type = g_mime_object_get_content_type(part);
+if (content_type) {
+	return g_mime_content_type_is_type (content_type, application,
+	   pkcs7-mime) ||
+	g_mime_content_type_is_type (content_type, application,
+	 x-pkcs7-mime);
+}
+return FALSE;
+}
+
 notmuch_status_t
 mime_node_open (const void *ctx, notmuch_message_t *message,
 		notmuch_crypto_t *crypto, mime_node_t **root_out)
@@ -323,22 +337,33 @@ _mime_node_create (mime_node_t *parent, GMimeObject *part)
 	return NULL;
 }
 
-if ((GMIME_IS_MULTIPART_ENCRYPTED (part)  node-ctx-crypto-decrypt)
+if (((GMIME_IS_MULTIPART_ENCRYPTED (part) || _is_smime_encrypted_part (part))
+	  node-ctx-crypto-decrypt)
 	|| (GMIME_IS_MULTIPART_SIGNED (part)  node-ctx-crypto-verify)) {
 	GMimeContentType *content_type = g_mime_object_get_content_type (part);
 	const char *protocol = g_mime_content_type_get_parameter (content_type, protocol);
+	if (!protocol) {
+	if (_is_smime_encrypted_part (part)) {
+		protocol = application/pkcs7-encrypted;
+	}
+	}
 	cryptoctx = notmuch_crypto_get_context (node-ctx-crypto, protocol);
 }
 
-/* Handle PGP/MIME parts */
-if (GMIME_IS_MULTIPART_ENCRYPTED (part)  node-ctx-crypto-decrypt  cryptoctx) {
-	if (node-nchildren != 2) {
-	/* this violates RFC 3156 section 4, so we won't bother with it. */
-	fprintf (stderr, Error: %d part(s) for a multipart/encrypted 
-		 message (must be exactly 2)\n,
-		 node-nchildren);
-	} else {
+/* Are we ready and able to decrypt something ? */
+if (node-ctx-crypto-decrypt  cryptoctx) {
+	if (_is_smime_encrypted_part (part)) {
 	node_decrypt_and_verify (node, part, cryptoctx);
+	} else if (GMIME_IS_MULTIPART_ENCRYPTED (part)) {
+	/* Handle PGP/MIME parts */
+	if (node-nchildren != 2) {
+		/* this violates RFC 3156 section 4, so we won't bother with it. */
+		fprintf (stderr, Error: %d part(s) for a multipart/encrypted 
+			 message (must be exactly 2)\n,
+			 node-nchildren);
+	} else {
+		node_decrypt_and_verify (node, part, cryptoctx);
+	}
 	}
 } else if (GMIME_IS_MULTIPART_SIGNED (part)  node-ctx-crypto-verify  cryptoctx) {
 	if (node-nchildren != 2) {
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Re: privacy problem: text/html parts pull in network resources

2015-01-27 Thread Daniel Kahn Gillmor
On Sun 2015-01-25 12:51:43 -0500, David Bremner wrote:
 Daniel Kahn Gillmor d...@fifthhorseman.net writes:

 If i send a message with a text/html part (either it's only text/html,
 or all parts are rendered, or it's multipart/alternative with only a
 text/html subpart) and that HTML has img
 src=http://example.org/test.png/ in it, then notmuch will make a
 network request for that image.

 This is a privacy disaster, because it enables an e-mail sender to use
 web bugs to tell when a given notmuch user has opened their e-mail.

 I've just pushed Austin's shr related series to master, so this problem
 should be fixed as of commit b74ed1c. One tradeoff that we should at
 least remark in NEWS, if not actually fix, is that I think there is now
 no way to view such images in notmuch.  I don't know offhand what other
 html renderers will do.

thanks for this, David and Austin!

Other html-rendering mail clients that are privacy-conscious will often
provide a button or mechanism to indicate that some remote resources
were requested by the page but weren't fetched (e.g. a button saying
something like [Load Remote Images...]).  I have no idea who actually
clicks on those buttons (or why), though, and even if we wanted them,
we'd only want to add a button on an image that actually had remote
network resources to load, and i don't know how we'd get that
information propagated back up the rendering stack to make such a
display decision.   So i'm fine with leaving it this way for now.

--dkg
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch