Re: [PATCH v2 5/5] cli/show: enable --decrypt=stash

[David Bremner]

Daniel Kahn Gillmor  writes:

> +
> +Note: ``--decrypt=stash`` requires a writable database.
> +Otherwise, ``notmuch show`` operates entirely in read-only mode.

I would rephrase this as "requires write access to the database";
otherwise it sounds like "writable" (or lack) is persistent property of
databases.


> +# show the message using stashing decryption
> +test_begin_subtest "stash decryption during show"
> +output=$(notmuch show --decrypt=stash tag:encrypted subject:002 | awk 
> '/^\014part}/{ f=0 }; { if (f) { print $0 } } /^\014part{ ID: 3/{ f=1 }')
> +expected='This is a test encrypted message with a wumpus.'
> +test_expect_equal \
> +"$output" \
> +"$expected"
> +

This is a bit hard to follow. I think it would be better to isolate this
kind of parsing in a function in test-lib.sh; then at least the name
would suggest the intent.

> +test_begin_subtest "search should now show the contents"

I think the point is not that it _shows_ the contents, but that it finds
them

> +output=$(notmuch search wumpus)
> +expected='thread:0003   2000-01-01 [1/1] Notmuch Test Suite; 
> test encrypted message for cleartext index 002 (encrypted inbox unread)'
> +if [ $NOTMUCH_HAVE_GMIME_SESSION_KEYS -eq 0 ]; then
> +test_subtest_known_broken
> +fi
> +test_expect_equal \
> +"$output" \
> +"$expected"
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH v2 3/5] cli: write session keys to database, if asked to do so

[David Bremner]

Daniel Kahn Gillmor  writes:

> +
> +#if HAVE_GMIME_SESSION_KEYS
> + if (node->ctx->crypto->decrypt == NOTMUCH_DECRYPT_TRUE && message) {
> + notmuch_database_t *db = notmuch_message_get_database (message);
> + const char *sk = g_mime_decrypt_result_get_session_key 
> (decrypt_result);
> + if (db && sk) {
> + notmuch_status_t status;
> + status = notmuch_message_add_property (message, "session-key", 
> sk);
> + if (status)
> + fprintf (stderr, "Failed to stash session key in the 
> database (%d) %s\n",
> +  status, notmuch_status_to_string (status));
> + }
> + }
> +#endif

As a nit, I don't really like sk as a variable name.

It might be worth definining a "print_status_message", along the lines
of print_status_query in status.c and using it here and in the next
commit. That would also handle any use of _notmuch_database_log.
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH v2 4/5] cli/show: reindex when we learned new session keys about a message

[David Bremner]

Daniel Kahn Gillmor  writes:

> +
> +if (params->crypto.decrypt == NOTMUCH_DECRYPT_TRUE && 
> session_key_count_error == NOTMUCH_STATUS_SUCCESS) {
> + unsigned int new_session_keys = 0;
> + if (notmuch_message_count_properties (message, "session-key", 
> _session_keys) == NOTMUCH_STATUS_SUCCESS &&
> + new_session_keys > session_keys) {
> + /* try a quiet re-indexing */
> + notmuch_indexopts_t *indexopts = 
> notmuch_database_get_default_indexopts (notmuch_message_get_database 
> (message));
> + if (indexopts) {
> + notmuch_indexopts_set_decrypt_policy (indexopts, 
> NOTMUCH_DECRYPT_AUTO);
> + status = notmuch_message_reindex (message, indexopts);
> + if (status)
> + fprintf (stderr, "Error re-indexing message with 
> --decrypt=stash. (%d) %s\n", status, notmuch_status_to_string (status));
> + }
> + }
> +}

I'm wondering about the lack of #if HAVE_GMIME_SESSION_KEYS here.  Are
you relying here on the number of session keys not increasing when
running a binary without session key support? Is there some advantage to
doing it this way? It seems a bit harder to reason about.

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH] python: add bindings for notmuch_message_get_propert(y/ies)

[David Bremner]

Ruben Pollan  writes:

> Message.get_property (prop) returns a string with the value of the property 
> and
> Message.get_properties (prop, exact=False) yields key, value pairs
> ---
>  bindings/python/docs/source/message.rst |  4 ++
>  bindings/python/notmuch/globals.py  |  5 +++
>  bindings/python/notmuch/message.py  | 80 
> -
>  3 files changed, 88 insertions(+), 1 deletion(-)
>

This version passes the first test (after fixing the format, as you
noted), but it looks like get_properties is returning pairs of
bytestrings.

FAIL   [15] msg.get_properties (python)
--- T610-message-property.16.EXPECTED   2018-05-02 00:02:11.160028179 
+
+++ T610-message-property.16.OUTPUT 2018-05-02 00:02:11.164028171 
+
@@ -1,4 +1,4 @@
-testkey1 = alice
-testkey1 = bob
-testkey1 = testvalue1
-testkey1 = testvalue2
+b'testkey1' = b'alice'
+b'testkey1' = b'bob'
+b'testkey1' = b'testvalue1'
+b'testkey1' = b'testvalue2'

I don't _think_ that's what we want. We had some discussion before and
decided that it was reasonable to only support utf-8 properties, so
converting to strings should be OK?

here's the proposed tests. 

diff --git a/test/T610-message-property.sh b/test/T610-message-property.sh
index 74b3f5a1..c903b2b6 100755
--- a/test/T610-message-property.sh
+++ b/test/T610-message-property.sh
@@ -256,4 +256,34 @@ id:4efc743a.3060...@april.org
 EOF
 test_expect_equal_file EXPECTED OUTPUT
 
+test_begin_subtest "msg.get_property (python)"
+test_python <<'EOF'
+import notmuch
+db = notmuch.Database(mode=notmuch.Database.MODE.READ_WRITE)
+msg = db.find_message("4efc743a.3060...@april.org")
+print("testkey1 = {0}".format(msg.get_property("testkey1")))
+print("testkey3 = {0}".format(msg.get_property("testkey3")))
+EOF
+cat <<'EOF' > EXPECTED
+testkey1 = alice
+testkey3 = alice3
+EOF
+test_expect_equal_file EXPECTED OUTPUT
+
+test_begin_subtest "msg.get_properties (python)"
+test_python <<'EOF'
+import notmuch
+db = notmuch.Database(mode=notmuch.Database.MODE.READ_WRITE)
+msg = db.find_message("4efc743a.3060...@april.org")
+for (key,val) in msg.get_properties("testkey1"):
+print("{0} = {1}".format(key,val))
+EOF
+cat <<'EOF' > EXPECTED
+testkey1 = alice
+testkey1 = bob
+testkey1 = testvalue1
+testkey1 = testvalue2
+EOF
+test_expect_equal_file EXPECTED OUTPUT
+
 test_done
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH 4/4] doc: document notmuch new --full-scan

[Tomi Ollila]

On Sun, Apr 29 2018, David Bremner wrote:

> ---
>  doc/man1/notmuch-new.rst | 6 ++
>  1 file changed, 6 insertions(+)
>
> diff --git a/doc/man1/notmuch-new.rst b/doc/man1/notmuch-new.rst
> index 16af5492..b0016ccc 100644
> --- a/doc/man1/notmuch-new.rst
> +++ b/doc/man1/notmuch-new.rst
> @@ -59,6 +59,12 @@ Supported options for **new** include
>  
>  See also ``index.decrypt`` in **notmuch-config(1)**.
>  
> +``--full-scan``
> +By default notmuch-new uses directory modification times (mtimes)
> +to optimize the scanning of directories for new mail. This option
> +allows turning that optimization off e.g. for testing or
> +debugging.
> +

Perhaps just (the last sentence): "This option turns that optimization off."

Tomi


>  EXIT STATUS
>  ===
>  
> -- 
> 2.17.0
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH 1/5] crypto: prepare for decryption of inline PGP encrypted messages

[Daniel Kahn Gillmor]

On Mon 2018-04-30 08:42:24 -0300, David Bremner wrote:
> David Bremner  writes:
>
>> Daniel Kahn Gillmor  writes:
>>
>>> We make use here of GMime's optimization function for detecting the
>>> presence of inline PGP encrypted content, which is only found in GMime
>>> 3.0 or later.
>>
>> We nominally support gmime-2.6 still. Maybe we shouldn't anymore, but
>> that's a different discussion.  Does this series compile and fail
>> gracefully with gmime 2.6?
>
> Looking at the rest of the series, it looks like it at least tries
> to. So maybe it's just the commit message of the first commit that is
> confusing, since the commit has nothing that looks like it handles gmime
> 2.6.

I think the point was that this is making use of features only found in
gmime 3.0.  so if you build against 2.6, the functionality is absent,
but it shouldn't cause breakage.

--dkg
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH] python: add bindings for notmuch_message_get_propert(y/ies)

[Ruben Pollan]

Message.get_property (prop) returns a string with the value of the property and
Message.get_properties (prop, exact=False) yields key, value pairs
---
 bindings/python/docs/source/message.rst |  4 ++
 bindings/python/notmuch/globals.py  |  5 +++
 bindings/python/notmuch/message.py  | 80 -
 3 files changed, 88 insertions(+), 1 deletion(-)

diff --git a/bindings/python/docs/source/message.rst 
b/bindings/python/docs/source/message.rst
index 1a6cc3d5..b0033924 100644
--- a/bindings/python/docs/source/message.rst
+++ b/bindings/python/docs/source/message.rst
@@ -33,6 +33,10 @@
 
.. automethod:: get_tags
 
+   .. automethod:: get_property
+
+   .. automethod:: get_properties
+
.. automethod:: maildir_flags_to_tags
 
.. automethod:: tags_to_maildir_flags
diff --git a/bindings/python/notmuch/globals.py 
b/bindings/python/notmuch/globals.py
index 97413996..11e328b7 100644
--- a/bindings/python/notmuch/globals.py
+++ b/bindings/python/notmuch/globals.py
@@ -75,6 +75,11 @@ class NotmuchMessageS(Structure):
 NotmuchMessageP = POINTER(NotmuchMessageS)
 
 
+class NotmuchMessagePropertiesS(Structure):
+pass
+NotmuchMessagePropertiesP = POINTER(NotmuchMessagePropertiesS)
+
+
 class NotmuchTagsS(Structure):
 pass
 NotmuchTagsP = POINTER(NotmuchTagsS)
diff --git a/bindings/python/notmuch/message.py 
b/bindings/python/notmuch/message.py
index 1b1f2174..cc945037 100644
--- a/bindings/python/notmuch/message.py
+++ b/bindings/python/notmuch/message.py
@@ -19,7 +19,7 @@ Copyright 2010 Sebastian Spaeth 
 """
 
 
-from ctypes import c_char_p, c_long, c_uint, c_int
+from ctypes import c_char_p, c_long, c_uint, c_int, POINTER, byref
 from datetime import date
 from .globals import (
 nmlib,
@@ -29,6 +29,7 @@ from .globals import (
 NotmuchTagsP,
 NotmuchMessageP,
 NotmuchMessagesP,
+NotmuchMessagePropertiesP,
 NotmuchFilenamesP,
 )
 from .errors import (
@@ -113,6 +114,36 @@ class Message(Python3StringMixIn):
 _maildir_flags_to_tags.argtypes = [NotmuchMessageP]
 _maildir_flags_to_tags.restype = c_int
 
+"""notmuch_message_get_property"""
+_get_property = nmlib.notmuch_message_get_property
+_get_property.argtypes = [NotmuchMessageP, c_char_p, POINTER(c_char_p)]
+_get_property.restype = c_int
+
+"""notmuch_message_get_properties"""
+_get_properties = nmlib.notmuch_message_get_properties
+_get_properties.argtypes = [NotmuchMessageP, c_char_p, c_int]
+_get_properties.restype = NotmuchMessagePropertiesP
+
+"""notmuch_message_properties_valid"""
+_properties_valid = nmlib.notmuch_message_properties_valid
+_properties_valid.argtypes = [NotmuchMessagePropertiesP]
+_properties_valid.restype = bool
+
+"""notmuch_message_properties_value"""
+_properties_value = nmlib.notmuch_message_properties_value
+_properties_value.argtypes = [NotmuchMessagePropertiesP]
+_properties_value.restype = c_char_p
+
+"""notmuch_message_properties_key"""
+_properties_key = nmlib.notmuch_message_properties_key
+_properties_key.argtypes = [NotmuchMessagePropertiesP]
+_properties_key.restype = c_char_p
+
+"""notmuch_message_properties_move_to_next"""
+_properties_move_to_next = nmlib.notmuch_message_properties_move_to_next
+_properties_move_to_next.argtypes = [NotmuchMessagePropertiesP]
+_properties_move_to_next.restype = None
+
 #Constants: Flags that can be set/get with set_flag
 FLAG = Enum(['MATCH'])
 
@@ -433,6 +464,53 @@ class Message(Python3StringMixIn):
 _freeze.argtypes = [NotmuchMessageP]
 _freeze.restype = c_uint
 
+def get_property(self, prop):
+""" Retrieve the value for a single property key
+
+:param prop: The name of the property to get.
+:returns: String with the property value or None if there is no such
+  key. In the case of multiple values for the given key, the
+  first one is retrieved.
+:raises: :exc:`NotInitializedError` if message has not been
+ initialized
+"""
+if not self._msg:
+raise NotInitializedError()
+
+value = c_char_p()
+status = Message._get_property(self._msg, _str(prop), byref(value))
+if status != 0:
+raise NotmuchError(status)
+
+return value.value.decode('utf-8')
+
+def get_properties(self, prop="", exact=False):
+""" Get the properties of the message, returning a generator of
+name, value pairs.
+
+The generator will yield once per value. There might be more than one
+value on each name, so the generator might yield the same name several
+times.
+
+:param prop: The name of the property to get. Otherwise it will return
+ the full list of properties of the message.
+:param exact: if True, require exact match with key. Otherwise
+  treat as prefix.
+

Re: [PATCH] python: add bindings for notmuch_message_get_propert(y/ies)

[meskio]

Quoting David Bremner (2018-05-01 15:06:38)
> running the test below, I get
> 
> Traceback (most recent call last):
>   File "", line 4, in 
>   File 
> "/home/bremner/software/upstream/notmuch/bindings/python/notmuch/message.py", 
> line 480, in get_property
> value = c_char_p("")
> TypeError: bytes or integer address expected instead of str instance

Ups, I was using python2 to test it. I see it doesn't work with python three.  
I'll send an update.

> diff --git a/test/T610-message-property.sh b/test/T610-message-property.sh
> index 74b3f5a1..f7220565 100755
> --- a/test/T610-message-property.sh
> +++ b/test/T610-message-property.sh
> @@ -89,6 +89,18 @@ testkey2 = NULL
>  EOF
>  test_expect_equal_file EXPECTED OUTPUT
>  
> +test_begin_subtest "msg.get_property (python)"
> +test_python <<'EOF'
> +import notmuch
> +db = notmuch.Database(mode=notmuch.Database.MODE.READ_WRITE)
> +msg = db.find_message("4efc743a.3060...@april.org")
> +print("testkey1[1] = %s\n".format(msg.get_property("testkey1")))

I think this should be (notice the {0} instead of %s):
print("testkey1[1] = {0}".format(msg.get_property("testkey1")))


-- 
meskio | http://meskio.net/
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
 My contact info: http://meskio.net/crypto.txt
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Nos vamos a Croatan.


signature.asc
Description: signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH] python: add bindings for notmuch_message_get_propert(y/ies)

[David Bremner]

Ruben Pollan  writes:

> Message.get_property (prop) returns a string with the value of the property 
> and
> Message.get_properties (prop, exact=False) yields key, value pairs
> ---
>  bindings/python/docs/source/message.rst |  4 ++
>  bindings/python/notmuch/globals.py  |  5 +++
>  bindings/python/notmuch/message.py  | 80 
> -
>  3 files changed, 88 insertions(+), 1 deletion(-)

I started to write some simple tests for this, but I didn't get very
far.

running the test below, I get

Traceback (most recent call last):
  File "", line 4, in 
  File 
"/home/bremner/software/upstream/notmuch/bindings/python/notmuch/message.py", 
line 480, in get_property
value = c_char_p("")
TypeError: bytes or integer address expected instead of str instance


diff --git a/test/T610-message-property.sh b/test/T610-message-property.sh
index 74b3f5a1..f7220565 100755
--- a/test/T610-message-property.sh
+++ b/test/T610-message-property.sh
@@ -89,6 +89,18 @@ testkey2 = NULL
 EOF
 test_expect_equal_file EXPECTED OUTPUT
 
+test_begin_subtest "msg.get_property (python)"
+test_python <<'EOF'
+import notmuch
+db = notmuch.Database(mode=notmuch.Database.MODE.READ_WRITE)
+msg = db.find_message("4efc743a.3060...@april.org")
+print("testkey1[1] = %s\n".format(msg.get_property("testkey1")))
+EOF
+cat <<'EOF' > EXPECTED
+testkey1[1] = testvalue1
+EOF
+test_expect_equal_file EXPECTED OUTPUT
+
 test_begin_subtest "notmuch_message_remove_all_properties"
 cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR}
 EXPECT0(notmuch_message_remove_all_properties (message, NULL));
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch