Re: [PATCH v3 3/4] cli: Extend the search command for --output={sender, recipients}
On Thu, Oct 23 2014, Mark Walters markwalters1...@gmail.com wrote: On Sun, 12 Oct 2014, Michal Sojka sojk...@fel.cvut.cz wrote: The new outputs allow printing senders, recipients or both of matching messages. The --output option is converted from keyword argument to flags argument, which means that the user can use --output=sender and --output=recipients simultaneously, to print both. Other combinations produce an error. ... +static void +print_address_list (const search_options_t *o, InternetAddressList *list) +{ +InternetAddress *address; +int i; + +for (i = 0; i internet_address_list_length (list); i++) { +address = internet_address_list_get_address (list, i); +if (INTERNET_ADDRESS_IS_GROUP (address)) { +InternetAddressGroup *group; +InternetAddressList *group_list; + +group = INTERNET_ADDRESS_GROUP (address); +group_list = internet_address_group_get_members (group); +if (group_list == NULL) +continue; + +print_address_list (o, group_list); +} else { +InternetAddressMailbox *mailbox; +const char *name; +const char *addr; +char *full_address; + +mailbox = INTERNET_ADDRESS_MAILBOX (address); + +name = internet_address_get_name (address); +addr = internet_address_mailbox_get_addr (mailbox); + +if (name *name) +full_address = talloc_asprintf (o-format, %s %s, name, addr); +else +full_address = talloc_strdup (o-format, addr); + +if (!full_address) { +fprintf (stderr, Error: out of memory\n); +break; +} +o-format-string (o-format, full_address); +o-format-separator (o-format); + +talloc_free (full_address); Thinking about this some more how about printing the name and address as a structured pair/map (at least for all cases except text/text0 output): something like (in JSON) [name: John Doe address: john@example.com] It seems wrong to me to go to the effort of separating them in the C and then combining them in the output. This could also help with the questions about uniqueness. If the client can get the data ready parsed into name/address then it can deal with much of the uniqueness itself. In that case client can also filter based on some substring, reducing the memory requirements... My preference would be for the default to print one line for each distinct full_address, and then any filter-by options to refine from there. Hmm, now I cannot decide whether this or just print out all addresses of messages, or do this distinct full_address output -- it looks like all other --output options prints unique lines, but there is potential of quite a lot of memory usage there... ... probably the memory usage is not problem there, OOM-killer eventually does it's job if necessary (!) (but machine may be slow (and trashing) for a while (just thinking out loud)) (!) but could we have general filter option for search to drop data before it is even considered for caching! -- maybe later ? One other advantage of structuring the output is that it is extensible: for example, at some later stage, we could include a count in the map allowing the client can pick the most popular variant. , and in this case notmuch cannot print any output until the full address list is gathered... :D Best wishes Mark Tomi +} +} +} + +static void +print_address_string (const search_options_t *o, const char *recipients) +{ +InternetAddressList *list; + +if (recipients == NULL) +return; + +list = internet_address_list_parse_string (recipients); +if (list == NULL) +return; + +print_address_list (o, list); +} + static int do_search_messages (search_options_t *o) { @@ -266,11 +330,29 @@ do_search_messages (search_options_t *o) notmuch_filenames_destroy( filenames ); -} else { /* output == OUTPUT_MESSAGES */ +} else if (o-output == OUTPUT_MESSAGES) { format-set_prefix (format, id); format-string (format, notmuch_message_get_message_id (message)); format-separator (format); +} else { +if (o-output OUTPUT_SENDER) { +const char *addrs; + +addrs = notmuch_message_get_header (message, from); +print_address_string (o, addrs); +} + +if (o-output OUTPUT_RECIPIENTS) { +const char *hdrs[] = { to, cc, bcc }; +const char *addrs; +size_t j; + +for (j = 0; j ARRAY_SIZE (hdrs); j++) { +addrs = notmuch_message_get_header (message, hdrs[j]); +print_address_string (o, addrs); +} +} } notmuch_message_destroy (message); @@ -337,7 +419,7 @@ notmuch_search_command (notmuch_config_t
Re: [PATCH v3 3/4] cli: Extend the search command for --output={sender, recipients}
On Thu, Oct 23 2014, Mark Walters wrote: Thinking about this some more how about printing the name and address as a structured pair/map (at least for all cases except text/text0 output): something like (in JSON) [name: John Doe address: john@example.com] It seems wrong to me to go to the effort of separating them in the C and then combining them in the output. Agreed, this would be convenient. ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
Re: [PATCH v3 3/4] cli: Extend the search command for --output={sender, recipients}
Hi Mark, I mostly agree with your points mentioned in this and other your emails. I'll prepare v4 based on that. On Wed, Oct 22 2014, Mark Walters wrote: On Sun, 12 Oct 2014, Michal Sojka sojk...@fel.cvut.cz wrote: The new outputs allow printing senders, recipients or both of matching messages. The --output option is converted from keyword argument to flags argument, which means that the user can use --output=sender and --output=recipients simultaneously, to print both. Other combinations produce an error. This code based on a patch from Jani Nikula. --- completion/notmuch-completion.bash | 2 +- completion/notmuch-completion.zsh | 3 +- doc/man1/notmuch-search.rst| 22 +++- notmuch-search.c | 110 ++--- test/T090-search-output.sh | 64 + 5 files changed, 189 insertions(+), 12 deletions(-) diff --git a/completion/notmuch-completion.bash b/completion/notmuch-completion.bash index 0571dc9..cfbd389 100644 --- a/completion/notmuch-completion.bash +++ b/completion/notmuch-completion.bash @@ -294,7 +294,7 @@ _notmuch_search() return ;; --output) -COMPREPLY=( $( compgen -W summary threads messages files tags -- ${cur} ) ) +COMPREPLY=( $( compgen -W summary threads messages files tags sender recipients -- ${cur} ) ) return ;; --sort) diff --git a/completion/notmuch-completion.zsh b/completion/notmuch-completion.zsh index 67a9aba..3e52a00 100644 --- a/completion/notmuch-completion.zsh +++ b/completion/notmuch-completion.zsh @@ -52,7 +52,8 @@ _notmuch_search() _arguments -s : \ '--max-threads=[display only the first x threads from the search results]:number of threads to show: ' \ '--first=[omit the first x threads from the search results]:number of threads to omit: ' \ -'--sort=[sort results]:sorting:((newest-first\:reverse chronological order oldest-first\:chronological order))' +'--sort=[sort results]:sorting:((newest-first\:reverse chronological order oldest-first\:chronological order))' \ +'--output=[select what to output]:output:((summary threads messages files tags sender recipients))' } _notmuch() diff --git a/doc/man1/notmuch-search.rst b/doc/man1/notmuch-search.rst index 90160f2..c9d38b1 100644 --- a/doc/man1/notmuch-search.rst +++ b/doc/man1/notmuch-search.rst @@ -35,7 +35,7 @@ Supported options for **search** include intended for programs that invoke **notmuch(1)** internally. If omitted, the latest supported version will be used. -``--output=(summary|threads|messages|files|tags)`` +``--output=(summary|threads|messages|files|tags|sender|recipients)`` **summary** Output a summary of each thread with any message matching @@ -78,6 +78,26 @@ Supported options for **search** include by null characters (--format=text0), as a JSON array (--format=json), or as an S-Expression list (--format=sexp). +**sender** +Output all addresses from the *From* header that appear on +any message matching the search terms, either one per line +(--format=text), separated by null characters +(--format=text0), as a JSON array (--format=json), or as +an S-Expression list (--format=sexp). + +Note: Searching for **sender** should be much faster than +searching for **recipients**, because sender addresses are +cached directly in the database whereas other addresses +need to be fetched from message files. + +**recipients** +Like **sender** but for addresses from *To*, *Cc* and +*Bcc* headers. + +This option can be given multiple times to combine different +outputs. Curently, this is only supported for **sender** and +**recipients** outputs. + ``--sort=``\ (**newest-first**\ \|\ **oldest-first**) This option can be used to present results in either chronological order (**oldest-first**) or reverse chronological diff --git a/notmuch-search.c b/notmuch-search.c index 5ac2a26..74588f8 100644 --- a/notmuch-search.c +++ b/notmuch-search.c @@ -23,11 +23,14 @@ #include string-util.h typedef enum { -OUTPUT_SUMMARY, -OUTPUT_THREADS, -OUTPUT_MESSAGES, -OUTPUT_FILES, -OUTPUT_TAGS +OUTPUT_SUMMARY = 1 0, +OUTPUT_THREADS = 1 1, +OUTPUT_MESSAGES = 1 2, +OUTPUT_FILES= 1 3, +OUTPUT_TAGS = 1 4, +OUTPUT_SENDER = 1 5, +OUTPUT_RECIPIENTS = 1 6, +OUTPUT_ADDRESSES= OUTPUT_SENDER | OUTPUT_RECIPIENTS, I think I would drop the OUTPUT_ADDRESSES enum as the parser no longer uses it (and replace the one use by OUTPUT_SENDER | OUTPUT_RECIPIENTS below). As mentioned elsewhere, this is required to suppress the following warning.
Re: [PATCH v3 3/4] cli: Extend the search command for --output={sender, recipients}
On Sun, 12 Oct 2014, Michal Sojka sojk...@fel.cvut.cz wrote: The new outputs allow printing senders, recipients or both of matching messages. The --output option is converted from keyword argument to flags argument, which means that the user can use --output=sender and --output=recipients simultaneously, to print both. Other combinations produce an error. This code based on a patch from Jani Nikula. --- completion/notmuch-completion.bash | 2 +- completion/notmuch-completion.zsh | 3 +- doc/man1/notmuch-search.rst| 22 +++- notmuch-search.c | 110 ++--- test/T090-search-output.sh | 64 + 5 files changed, 189 insertions(+), 12 deletions(-) diff --git a/completion/notmuch-completion.bash b/completion/notmuch-completion.bash index 0571dc9..cfbd389 100644 --- a/completion/notmuch-completion.bash +++ b/completion/notmuch-completion.bash @@ -294,7 +294,7 @@ _notmuch_search() return ;; --output) - COMPREPLY=( $( compgen -W summary threads messages files tags -- ${cur} ) ) + COMPREPLY=( $( compgen -W summary threads messages files tags sender recipients -- ${cur} ) ) return ;; --sort) diff --git a/completion/notmuch-completion.zsh b/completion/notmuch-completion.zsh index 67a9aba..3e52a00 100644 --- a/completion/notmuch-completion.zsh +++ b/completion/notmuch-completion.zsh @@ -52,7 +52,8 @@ _notmuch_search() _arguments -s : \ '--max-threads=[display only the first x threads from the search results]:number of threads to show: ' \ '--first=[omit the first x threads from the search results]:number of threads to omit: ' \ -'--sort=[sort results]:sorting:((newest-first\:reverse chronological order oldest-first\:chronological order))' +'--sort=[sort results]:sorting:((newest-first\:reverse chronological order oldest-first\:chronological order))' \ +'--output=[select what to output]:output:((summary threads messages files tags sender recipients))' } _notmuch() diff --git a/doc/man1/notmuch-search.rst b/doc/man1/notmuch-search.rst index 90160f2..c9d38b1 100644 --- a/doc/man1/notmuch-search.rst +++ b/doc/man1/notmuch-search.rst @@ -35,7 +35,7 @@ Supported options for **search** include intended for programs that invoke **notmuch(1)** internally. If omitted, the latest supported version will be used. -``--output=(summary|threads|messages|files|tags)`` +``--output=(summary|threads|messages|files|tags|sender|recipients)`` **summary** Output a summary of each thread with any message matching @@ -78,6 +78,26 @@ Supported options for **search** include by null characters (--format=text0), as a JSON array (--format=json), or as an S-Expression list (--format=sexp). + **sender** +Output all addresses from the *From* header that appear on +any message matching the search terms, either one per line +(--format=text), separated by null characters +(--format=text0), as a JSON array (--format=json), or as +an S-Expression list (--format=sexp). + + Note: Searching for **sender** should be much faster than + searching for **recipients**, because sender addresses are + cached directly in the database whereas other addresses + need to be fetched from message files. + + **recipients** +Like **sender** but for addresses from *To*, *Cc* and + *Bcc* headers. + + This option can be given multiple times to combine different + outputs. Curently, this is only supported for **sender** and + **recipients** outputs. + ``--sort=``\ (**newest-first**\ \|\ **oldest-first**) This option can be used to present results in either chronological order (**oldest-first**) or reverse chronological diff --git a/notmuch-search.c b/notmuch-search.c index 5ac2a26..74588f8 100644 --- a/notmuch-search.c +++ b/notmuch-search.c @@ -23,11 +23,14 @@ #include string-util.h typedef enum { -OUTPUT_SUMMARY, -OUTPUT_THREADS, -OUTPUT_MESSAGES, -OUTPUT_FILES, -OUTPUT_TAGS +OUTPUT_SUMMARY = 1 0, +OUTPUT_THREADS = 1 1, +OUTPUT_MESSAGES = 1 2, +OUTPUT_FILES = 1 3, +OUTPUT_TAGS = 1 4, +OUTPUT_SENDER= 1 5, +OUTPUT_RECIPIENTS= 1 6, +OUTPUT_ADDRESSES = OUTPUT_SENDER | OUTPUT_RECIPIENTS, } output_t; typedef struct { @@ -220,6 +223,67 @@ do_search_threads (search_options_t *o) return 0; } +static void +print_address_list (const search_options_t *o, InternetAddressList *list) +{ +InternetAddress *address; +int i; + +for (i = 0; i internet_address_list_length (list); i++) { + address =
Re: [PATCH v3 3/4] cli: Extend the search command for --output={sender, recipients}
On Sun, 12 Oct 2014, Michal Sojka sojk...@fel.cvut.cz wrote: The new outputs allow printing senders, recipients or both of matching messages. The --output option is converted from keyword argument to flags argument, which means that the user can use --output=sender and --output=recipients simultaneously, to print both. Other combinations produce an error. This code based on a patch from Jani Nikula. --- completion/notmuch-completion.bash | 2 +- completion/notmuch-completion.zsh | 3 +- doc/man1/notmuch-search.rst| 22 +++- notmuch-search.c | 110 ++--- test/T090-search-output.sh | 64 + 5 files changed, 189 insertions(+), 12 deletions(-) diff --git a/completion/notmuch-completion.bash b/completion/notmuch-completion.bash index 0571dc9..cfbd389 100644 --- a/completion/notmuch-completion.bash +++ b/completion/notmuch-completion.bash @@ -294,7 +294,7 @@ _notmuch_search() return ;; --output) - COMPREPLY=( $( compgen -W summary threads messages files tags -- ${cur} ) ) + COMPREPLY=( $( compgen -W summary threads messages files tags sender recipients -- ${cur} ) ) return ;; --sort) diff --git a/completion/notmuch-completion.zsh b/completion/notmuch-completion.zsh index 67a9aba..3e52a00 100644 --- a/completion/notmuch-completion.zsh +++ b/completion/notmuch-completion.zsh @@ -52,7 +52,8 @@ _notmuch_search() _arguments -s : \ '--max-threads=[display only the first x threads from the search results]:number of threads to show: ' \ '--first=[omit the first x threads from the search results]:number of threads to omit: ' \ -'--sort=[sort results]:sorting:((newest-first\:reverse chronological order oldest-first\:chronological order))' +'--sort=[sort results]:sorting:((newest-first\:reverse chronological order oldest-first\:chronological order))' \ +'--output=[select what to output]:output:((summary threads messages files tags sender recipients))' } _notmuch() diff --git a/doc/man1/notmuch-search.rst b/doc/man1/notmuch-search.rst index 90160f2..c9d38b1 100644 --- a/doc/man1/notmuch-search.rst +++ b/doc/man1/notmuch-search.rst @@ -35,7 +35,7 @@ Supported options for **search** include intended for programs that invoke **notmuch(1)** internally. If omitted, the latest supported version will be used. -``--output=(summary|threads|messages|files|tags)`` +``--output=(summary|threads|messages|files|tags|sender|recipients)`` **summary** Output a summary of each thread with any message matching @@ -78,6 +78,26 @@ Supported options for **search** include by null characters (--format=text0), as a JSON array (--format=json), or as an S-Expression list (--format=sexp). + **sender** +Output all addresses from the *From* header that appear on +any message matching the search terms, either one per line +(--format=text), separated by null characters +(--format=text0), as a JSON array (--format=json), or as +an S-Expression list (--format=sexp). + + Note: Searching for **sender** should be much faster than + searching for **recipients**, because sender addresses are + cached directly in the database whereas other addresses + need to be fetched from message files. + + **recipients** +Like **sender** but for addresses from *To*, *Cc* and + *Bcc* headers. + + This option can be given multiple times to combine different + outputs. Curently, this is only supported for **sender** and + **recipients** outputs. + ``--sort=``\ (**newest-first**\ \|\ **oldest-first**) This option can be used to present results in either chronological order (**oldest-first**) or reverse chronological diff --git a/notmuch-search.c b/notmuch-search.c index 5ac2a26..74588f8 100644 --- a/notmuch-search.c +++ b/notmuch-search.c @@ -23,11 +23,14 @@ #include string-util.h typedef enum { -OUTPUT_SUMMARY, -OUTPUT_THREADS, -OUTPUT_MESSAGES, -OUTPUT_FILES, -OUTPUT_TAGS +OUTPUT_SUMMARY = 1 0, +OUTPUT_THREADS = 1 1, +OUTPUT_MESSAGES = 1 2, +OUTPUT_FILES = 1 3, +OUTPUT_TAGS = 1 4, +OUTPUT_SENDER= 1 5, +OUTPUT_RECIPIENTS= 1 6, +OUTPUT_ADDRESSES = OUTPUT_SENDER | OUTPUT_RECIPIENTS, I think I would drop the OUTPUT_ADDRESSES enum as the parser no longer uses it (and replace the one use by OUTPUT_SENDER | OUTPUT_RECIPIENTS below). } output_t; typedef struct { @@ -220,6 +223,67 @@ do_search_threads (search_options_t *o) return 0; } +static void +print_address_list (const search_options_t *o, InternetAddressList *list) +{
Re: [PATCH v3 3/4] cli: Extend the search command for --output={sender, recipients}
Hi Michal, been busy lately so I haven't got into this... ... I'll look into it in a bit more detail a bit later, On Tue, Oct 14 2014, Michal Sojka sojk...@fel.cvut.cz wrote: I don't know. You seem to think about this in the opposite way than how it is implemented. The implementation really filters things out whereas you specify what not to filter. My feeling is that if you would implement your proposal, the code would be more complex than in my patch, because the mapping between command line options and the actual algorithm would require some extra code. And in a previous comment, you preferred simplicity. Yes, maybe the choice of reusing --duplicate was bad, and perhaps took thought from the main point -- what kind output options should be possible... Hopefully, you consider the above as healthy criticism. Yes, I do -- I hope we can get something decent out in this in near future, with duplicate filtering that can have useful options to be used directly by MUAs and no duplicate filtering option which I can use in nottoomuch-addresses in 2016 ;D Thanks for quick review. -Michal Tomi ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
Re: [PATCH v3 3/4] cli: Extend the search command for --output={sender, recipients}
On Mon, Oct 13 2014, Michal Sojka sojk...@fel.cvut.cz wrote: The new outputs allow printing senders, recipients or both of matching messages. The --output option is converted from keyword argument to flags argument, which means that the user can use --output=sender and --output=recipients simultaneously, to print both. Other combinations produce an error. This code based on a patch from Jani Nikula. --- completion/notmuch-completion.bash | 2 +- completion/notmuch-completion.zsh | 3 +- doc/man1/notmuch-search.rst| 22 +++- notmuch-search.c | 110 ++--- test/T090-search-output.sh | 64 + 5 files changed, 189 insertions(+), 12 deletions(-) diff --git a/completion/notmuch-completion.bash b/completion/notmuch-completion.bash index 0571dc9..cfbd389 100644 --- a/completion/notmuch-completion.bash +++ b/completion/notmuch-completion.bash @@ -294,7 +294,7 @@ _notmuch_search() return ;; --output) - COMPREPLY=( $( compgen -W summary threads messages files tags -- ${cur} ) ) + COMPREPLY=( $( compgen -W summary threads messages files tags sender recipients -- ${cur} ) ) return ;; --sort) diff --git a/completion/notmuch-completion.zsh b/completion/notmuch-completion.zsh index 67a9aba..3e52a00 100644 --- a/completion/notmuch-completion.zsh +++ b/completion/notmuch-completion.zsh @@ -52,7 +52,8 @@ _notmuch_search() _arguments -s : \ '--max-threads=[display only the first x threads from the search results]:number of threads to show: ' \ '--first=[omit the first x threads from the search results]:number of threads to omit: ' \ -'--sort=[sort results]:sorting:((newest-first\:reverse chronological order oldest-first\:chronological order))' +'--sort=[sort results]:sorting:((newest-first\:reverse chronological order oldest-first\:chronological order))' \ +'--output=[select what to output]:output:((summary threads messages files tags sender recipients))' } _notmuch() diff --git a/doc/man1/notmuch-search.rst b/doc/man1/notmuch-search.rst index 90160f2..c9d38b1 100644 --- a/doc/man1/notmuch-search.rst +++ b/doc/man1/notmuch-search.rst @@ -35,7 +35,7 @@ Supported options for **search** include intended for programs that invoke **notmuch(1)** internally. If omitted, the latest supported version will be used. -``--output=(summary|threads|messages|files|tags)`` +``--output=(summary|threads|messages|files|tags|sender|recipients)`` **summary** Output a summary of each thread with any message matching @@ -78,6 +78,26 @@ Supported options for **search** include by null characters (--format=text0), as a JSON array (--format=json), or as an S-Expression list (--format=sexp). + **sender** +Output all addresses from the *From* header that appear on +any message matching the search terms, either one per line +(--format=text), separated by null characters +(--format=text0), as a JSON array (--format=json), or as +an S-Expression list (--format=sexp). + + Note: Searching for **sender** should be much faster than + searching for **recipients**, because sender addresses are + cached directly in the database whereas other addresses + need to be fetched from message files. + + **recipients** +Like **sender** but for addresses from *To*, *Cc* and + *Bcc* headers. + + This option can be given multiple times to combine different + outputs. Curently, this is only supported for **sender** and + **recipients** outputs. + ``--sort=``\ (**newest-first**\ \|\ **oldest-first**) This option can be used to present results in either chronological order (**oldest-first**) or reverse chronological diff --git a/notmuch-search.c b/notmuch-search.c index 5ac2a26..74588f8 100644 --- a/notmuch-search.c +++ b/notmuch-search.c @@ -23,11 +23,14 @@ #include string-util.h typedef enum { -OUTPUT_SUMMARY, -OUTPUT_THREADS, -OUTPUT_MESSAGES, -OUTPUT_FILES, -OUTPUT_TAGS +OUTPUT_SUMMARY = 1 0, +OUTPUT_THREADS = 1 1, +OUTPUT_MESSAGES = 1 2, +OUTPUT_FILES = 1 3, +OUTPUT_TAGS = 1 4, +OUTPUT_SENDER= 1 5, +OUTPUT_RECIPIENTS= 1 6, +OUTPUT_ADDRESSES = OUTPUT_SENDER | OUTPUT_RECIPIENTS, leftover, like mentioned below (this comment added just before sending) } output_t; typedef struct { @@ -220,6 +223,67 @@ do_search_threads (search_options_t *o) return 0; } +static void +print_address_list (const search_options_t *o, InternetAddressList *list) +{ +InternetAddress *address; +int i; + +for (i = 0; i