Re: moving the config into the database [was: Re: [PATCH] Display extra headers for emacs-mua - db config option]

[Daniel Kahn Gillmor]

On Wed 2019-12-11 10:00:07 -0400, David Bremner wrote:
> David Edmondson  writes:
>
>> It seems that only “notmuch new” is currently prepared to create a new
>> database, but it will also import all of my email before I have a chance
>> to set the above parameters.
>>
>> It seems that the obvious answer is “notmuch config” will create a
>> database if necessary. Is that the plan?
>
> Wouldn't it make sense to have "notmuch setup" do this?

It has always puzzled me that "notmuch setup" did not actually set up
the database in the first place.  So yes, i think this is the right
answer.

--dkg


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: moving the config into the database [was: Re: [PATCH] Display extra headers for emacs-mua - db config option]

[Jorge P . de Morais Neto]

Hello.

Em [2019-12-10 ter 12:01:06-0500], Daniel Kahn Gillmor escreveu:

> Perhaps you would be satisfied with some sort of "post-config-update" hook?

Yes, I think that would fulfill my needs.  And it would satisfy not only
people who use Unison, but also people who use some "realtime" (don't
know if that is the right word) synchronization tool like Netxcloud.

Regards
-- 
- 
- I am Brazilian.  I hope my English is correct and I welcome feedback.
- Please adopt free formats like PDF, ODF, Org, LaTeX, Opus, WebM and 7z.
- Free/libre software for Replicant, LineageOS and Android: https://f-droid.org
- [[https://www.gnu.org/philosophy/free-sw.html][What is free software?]]
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: moving the config into the database [was: Re: [PATCH] Display extra headers for emacs-mua - db config option]

[Daniel Kahn Gillmor]

On Tue 2019-12-10 09:46:14 -0300, Jorge P. de Morais Neto wrote:
> I hope you don't mind my insistence

Insistence is fine -- it sounds like you have a real need for your
backup/sync strategy, and if we can figure out a way to support it, that
would be good.  perhaps we should talk about different ways to get what
you're looking for, though.

Perhaps you would be satisfied with some sort of "post-config-update" hook?

--dkg


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: moving the config into the database [was: Re: [PATCH] Display extra headers for emacs-mua - db config option]

[David Edmondson]

On Wednesday, 2019-12-11 at 10:00:07 -04, David Bremner wrote:

> David Edmondson  writes:
>
>> It seems that only “notmuch new” is currently prepared to create a new
>> database, but it will also import all of my email before I have a chance
>> to set the above parameters.
>>
>> It seems that the obvious answer is “notmuch config” will create a
>> database if necessary. Is that the plan?
>
> Wouldn't it make sense to have "notmuch setup" do this?

Oh, yes, of course. I had forgotten that existed :-)

dme.
-- 
Tell her I'll be waiting, in the usual place.
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: moving the config into the database [was: Re: [PATCH] Display extra headers for emacs-mua - db config option]

[David Bremner]

David Edmondson  writes:

> It seems that only “notmuch new” is currently prepared to create a new
> database, but it will also import all of my email before I have a chance
> to set the above parameters.
>
> It seems that the obvious answer is “notmuch config” will create a
> database if necessary. Is that the plan?

Wouldn't it make sense to have "notmuch setup" do this?

d
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: moving the config into the database [was: Re: [PATCH] Display extra headers for emacs-mua - db config option]

[David Edmondson]

On Tuesday, 2019-12-10 at 08:11:56 -08, Jameson Graef Rollins wrote:

> Fwiw I support dkg's idea to store the config in the database itself.  I
> think that's a clean simplification that makes a lot of sense.

I'm not inherently against this, but I'm unsure exactly how it would
work.

For example, I set new.tags and new.ignore in my current
.notmuch-config. If they move into the database, how will I set them
before I have a database?

It seems that only “notmuch new” is currently prepared to create a new
database, but it will also import all of my email before I have a chance
to set the above parameters.

It seems that the obvious answer is “notmuch config” will create a
database if necessary. Is that the plan?

dme.
-- 
They like the smell of it in Hollywood.
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: moving the config into the database [was: Re: [PATCH] Display extra headers for emacs-mua - db config option]

[Jameson Graef Rollins]

On Sun, Dec 08 2019, Jorge P. de Morais Neto  wrote:
> Em [2019-12-08 dom 09:12:55-0800], Jameson Graef Rollins escreveu:
>
>> You can already use 'notmuch config list' to dump every configuration
>> item to stdout.  Would that be sufficient for personal synchronization
>> purposes.
>
> But then I would have to remember invoking
> ~notmuch config list > ~/notmuch-config~
> every time I changed Notmuch configuration.

Rather than remember, why not just have your synchronization script
retrieve the config at sync time using notmuch config/dump.  That way
you don't have to remember to do anything, and you can be agnostic about
how the configuration info is stored on disk.

Fwiw I support dkg's idea to store the config in the database itself.  I
think that's a clean simplification that makes a lot of sense.

jamie.
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: moving the config into the database [was: Re: [PATCH] Display extra headers for emacs-mua - db config option]

[Jorge P . de Morais Neto]

Hello.

Em [2019-12-09 seg 13:31:22-0500], Daniel Kahn Gillmor escreveu:

> One problem with such a "perennial text file" is that people will want
> to edit it.  When they do, what should happen?

Well, the file could be named "notmuch-config-view" and the contents
could include comments explaining it is just a view.  Besides, the
generation of such a file could be opt-in.

I hope you don't mind my insistence (it is my personality) and, of
course, there is still the problem of slightly increasing Notmuch's
complexity to provide a small benefit for relatively few people.  It is
(obviously) your call, since you understand much more about the Notmuch
codebase (and free/libre software development in general) than me.
Besides, it is not me who am going to actually implement this (I am
unfortunately unable to contribute C code to Notmuch at this moment).

Regards
-- 
- 
- I am Brazilian.  I hope my English is correct and I welcome feedback.
- Please adopt free formats like PDF, ODF, Org, LaTeX, Opus, WebM and 7z.
- Free/libre software for Replicant, LineageOS and Android: https://f-droid.org
- [[https://www.gnu.org/philosophy/free-sw.html][What is free software?]]

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: moving the config into the database [was: Re: [PATCH] Display extra headers for emacs-mua - db config option]

[Daniel Kahn Gillmor]

On Sun 2019-12-08 15:19:38 -0300, Jorge P. de Morais Neto wrote:
> Em [2019-12-08 dom 09:12:55-0800], Jameson Graef Rollins escreveu:
>
>> You can already use 'notmuch config list' to dump every configuration
>> item to stdout.  Would that be sufficient for personal synchronization
>> purposes.
>
> But then I would have to remember invoking
> ~notmuch config list > ~/notmuch-config~
> every time I changed Notmuch configuration.  It would be more convenient
> and less error-prone for Notmuch to keep a perennial text file
> reflecting the configuration in the database.  Oh, I that file should be
> updated not only by ~notmuch edit config~, but also by ~notmuch config
> set~.

One problem with such a "perennial text file" is that people will want
to edit it.  When they do, what should happen?

So the proposal to maintain such a file seems like we would be adding an
interface to notmuch that will not be used by many people, and will
eventually hurt (or at least confuse) users in surprising ways.

I'd rather keep notmuch itself simpler, and expect users who are syncing
to sync things like the output of "notmuch dump" (which already includes
all db-stored config).

--dkg


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: moving the config into the database [was: Re: [PATCH] Display extra headers for emacs-mua - db config option]

[Jorge P . de Morais Neto]

Em [2019-12-08 dom 09:12:55-0800], Jameson Graef Rollins escreveu:

> You can already use 'notmuch config list' to dump every configuration
> item to stdout.  Would that be sufficient for personal synchronization
> purposes.

But then I would have to remember invoking
~notmuch config list > ~/notmuch-config~
every time I changed Notmuch configuration.  It would be more convenient
and less error-prone for Notmuch to keep a perennial text file
reflecting the configuration in the database.  Oh, I that file should be
updated not only by ~notmuch edit config~, but also by ~notmuch config
set~.

Regards
-- 
- 
- I am Brazilian.  I hope my English is correct and I welcome feedback.
- Please adopt free formats like PDF, ODF, Org, LaTeX, Opus, WebM and 7z.
- Free/libre software for Replicant, LineageOS and Android: https://f-droid.org
- [[https://www.gnu.org/philosophy/free-sw.html][What is free software?]]
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: moving the config into the database [was: Re: [PATCH] Display extra headers for emacs-mua - db config option]

[Jameson Graef Rollins]

On Sun, Dec 08 2019, Jorge P. de Morais Neto  wrote:
> Em [2019-11-22 sex 10:43:40+0800], Daniel Kahn Gillmor escreveu:
>
>> Better than documenting, i'd be happy if we would add a "notmuch config
>> edit" subcommand, which handles the above sequence for you, invoking
>> $EDITOR at the appropriate time.
>>
>> The only caveat i see there is if the end user wants to inject comments
>> in the config file, which would then be stripped out in between these
>> invocations.  perhaps someone who finds these comments in config files
>> super important could propose a way to stash them in the db and recover
>> them during "notmuch config edit" as well :)
>
> I sync my two Notmuch configuration files -- personal notebook and
> workplace desktop -- over Unison via a USB flash drive.  This way, when
> I want to reconfigure Notmuch in one machine, I have the other machine's
> configuration as reference.  However, I do not sync the entire Notmuch
> databases because, since they are big and change frequently, I suppose
> they would wear the USB flash drive and also increase the
> synchronization time.
>
> Thus for my use case it would be useful if "notmuch config edit" could
> automatically keep a perennial text file reflecting the configuration in
> the database.  That text file would then be synchronized by Unison.

You can already use 'notmuch config list' to dump every configuration
item to stdout.  Would that be sufficient for personal synchronization
purposes.

jamie.
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: moving the config into the database [was: Re: [PATCH] Display extra headers for emacs-mua - db config option]

[Jorge P . de Morais Neto]

Em [2019-11-22 sex 10:43:40+0800], Daniel Kahn Gillmor escreveu:

> Better than documenting, i'd be happy if we would add a "notmuch config
> edit" subcommand, which handles the above sequence for you, invoking
> $EDITOR at the appropriate time.
>
> The only caveat i see there is if the end user wants to inject comments
> in the config file, which would then be stripped out in between these
> invocations.  perhaps someone who finds these comments in config files
> super important could propose a way to stash them in the db and recover
> them during "notmuch config edit" as well :)

I sync my two Notmuch configuration files -- personal notebook and
workplace desktop -- over Unison via a USB flash drive.  This way, when
I want to reconfigure Notmuch in one machine, I have the other machine's
configuration as reference.  However, I do not sync the entire Notmuch
databases because, since they are big and change frequently, I suppose
they would wear the USB flash drive and also increase the
synchronization time.

Thus for my use case it would be useful if "notmuch config edit" could
automatically keep a perennial text file reflecting the configuration in
the database.  That text file would then be synchronized by Unison.

Regards
-- 
- 
- I am Brazilian.  I hope my English is correct and I welcome feedback.
- Please adopt free formats like PDF, ODF, Org, LaTeX, Opus, WebM and 7z.
- Free/libre software for Replicant, LineageOS and Android: https://f-droid.org
- [[https://www.gnu.org/philosophy/free-sw.html][What is free software?]]

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH] Display extra headers for emacs-mua - db config option

[Carl Worth]

On Thu, Nov 21 2019, Johan Parin wrote:
> Johan Parin  writes:
>> Daniel Kahn Gillmor  writes:
>>>
>>> Is it that much worse to pass around the notmuch_database_t *?

My opinion is that we should not use the static notmuch_database_t
pointer. I think any desire to reach for that is, at best, papering over
some other problem that would be better fixed itself.

>> It has a lot of code impact, it really propagates to a lot of
>> places.
...
> Here is a taste (not fully tested, but seems to work).
...
>  static int
> -do_reply (notmuch_config_t *config,
> +do_reply (notmuch_database_t *notmuch,
> +   notmuch_config_t *config,

This passing around of the pair of notmuch_database_t* and a
notmuch_config_t* all over the place looks like an anti-pattern to me,
(and something that was just being hidden by the static
notmuch_database_t* that would be better to be fixed properly).

One option for reducing that pair to a single pointer would be to move
the configuration into the database, (as Daniel has been arguing for a
while).

I've argued against that in the past, (and obviously, I came up with the
current approach originally). But the historical situation has already
led to some unpleasant mixing of configuration state, (where some state
is in the configuration file and other state is in the database and it's
hard to predict which is where and which things can be controlled in the
configuration file).

And I think a patch like the above with the "notmuch, config" all over
the place would be another unpleasant result of the historical
convention.

So, I wouldn't be opposed to having the configuration move into the
database entirely. I would definitely like to see a tool that allows for
a dump/restore operation of the configuration state, (so users can still
edit configuration parameters with a text editor and with
fully-documented parameters). Bonus points if the users can also capture
their own commentary/explanation of values they are setting (as they can
do with the current configuration file).

And if things do move, existing configuration files should be updated
with a comment describing what that new dump/restore mechanism is and
that the current configuration file will no longer be used, (though, at
the transition point, obviously configuration parameters should be
copied from the configuration file into the database).

-Carl


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

moving the config into the database [was: Re: [PATCH] Display extra headers for emacs-mua - db config option]

[Daniel Kahn Gillmor]

On Thu 2019-11-21 23:38:06 +0200, Tomi Ollila wrote:
> How can library open the database if it doesn't read the config file
> -- the config file defines where database is located =D

The *only* thing i think worth keeping in the config file is ultimately
the location of the database, though i would eventually prefer that we
introduce a NOTMUCH_DATABASE env var, and use it if we can't find a
config.

This is the moral equivalent of the NOTMUCH_CONFIG env var, as far as
i'm concerned, but more rationalized.

and *eventually eventually* drop the config file entirely, yes.

> If the library can somehow guess the location of the database without
> reading the config file ;D I think dumping -- editing -- restoring
> the database is tolerable solution -- provided it is clearly documented
> for people like me who wants to edit configuration files w/ text
> editor(*),

Better than documenting, i'd be happy if we would add a "notmuch config
edit" subcommand, which handles the above sequence for you, invoking
$EDITOR at the appropriate time.

The only caveat i see there is if the end user wants to inject comments
in the config file, which would then be stripped out in between these
invocations.  perhaps someone who finds these comments in config files
super important could propose a way to stash them in the db and recover
them during "notmuch config edit" as well :)

> (+) if we dropped the configuration file, then notmuch, and library could
> open database from ~/mail/notmuch/ by default, or from location pointed
> by e.g. NOTMUCH_DATABASE_DIR -- as an additional benefit(?) notmuch
> would pollute user's $HOME by one file less -- the `.notmuch-config`.

You named the env var differently than i did, but i'd be happy to go
with your name, since it seems we arrived at the same design
independently :)

 --dkg


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH] Display extra headers for emacs-mua - db config option

[Daniel Kahn Gillmor]

On Thu 2019-11-21 22:56:04 +0100, Johan Parin wrote:
> Here is a taste (not fully tested, but seems to work).

Oof, i see what you mean :(

I haven't tried to implement this a different way, so i don't know
whether there isn't a shorter cut to what we want, but yow it is a lot.

Are we doing something more deeply, architecturally wrong that saddles
us these two rather displeasing choices?

> diff --git a/doc/man1/notmuch-config.rst b/doc/man1/notmuch-config.rst
> index 28487079..0eb59883 100644
> --- a/doc/man1/notmuch-config.rst
> +++ b/doc/man1/notmuch-config.rst
> @@ -204,6 +204,12 @@ The available configuration items are described below.
>  supported. See **notmuch-search-terms(7)** for a list of existing
>  prefixes, and an explanation of probabilistic prefixes.
>  
> +**show.extra_headers**
> +A list of extra headers that will be output by **notmuch show**
> +with ``--format=sexp``, if present in the message.
> +
> +Default: empty list.
> +
>  **built_with.**
>  Compile time feature . Current possibilities include
>  "compact" (see **notmuch-compact(1)**) and "field_processor" (see

Given the implementation choices, i think you want **[STORED IN
DATABASE]** here (see the definitions for query. and
index.header. and index.decrypt).

Thanks for taking the time to sort this out!

   --dkg


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH] Display extra headers for emacs-mua - db config option

[Johan Parin]

Johan Parin  writes:

> Daniel Kahn Gillmor  writes:
>>
>> Is it that much worse to pass around the notmuch_database_t *?
>>
>
> It has a lot of code impact, it really propagates to a lot of
> places. For instance it also impacts the json and text api, because
> those need to have the same signatures as the json api. Also the
> database needs to be opened in places where it's currently not opened,
> such as in notmuch_search_command().
>

Here is a taste (not fully tested, but seems to work).

/Johan

>From 706a0f784a101b05ac82a462ff6c19c0cef550af Mon Sep 17 00:00:00 2001
From: Johan Parin 
Date: Sun, 17 Nov 2019 13:15:39 +0100
Subject: [PATCH] Display extra headers for emacs-mua - db config option

Modify format_headers_sprinter so that it returns some additional headers in the
sexp, instead of a small fixed set of headers.

The extra header list is configured by the database config option
`show.extra_headers'.

This is required in order for the elisp variable
`notmuch-message-headers' to work.

See this bug report:

  https://notmuchmail.org/pipermail/notmuch/2017/026069.html

This version avoids the file global notmuch database variable in
notmuch-show.c by passing around the database pointer in a lot of
function calls.
---
 doc/man1/notmuch-config.rst |   6 ++
 notmuch-client.h|  12 ++--
 notmuch-config.c|   7 ++-
 notmuch-reply.c |  13 ++--
 notmuch-search.c|  29 ++---
 notmuch-show.c  | 114 
 sprinter-json.c |   3 +-
 sprinter-sexp.c |   3 +-
 sprinter-text.c |  10 +++-
 sprinter.h  |  12 ++--
 10 files changed, 155 insertions(+), 54 deletions(-)

diff --git a/doc/man1/notmuch-config.rst b/doc/man1/notmuch-config.rst
index 28487079..0eb59883 100644
--- a/doc/man1/notmuch-config.rst
+++ b/doc/man1/notmuch-config.rst
@@ -204,6 +204,12 @@ The available configuration items are described below.
 supported. See **notmuch-search-terms(7)** for a list of existing
 prefixes, and an explanation of probabilistic prefixes.
 
+**show.extra_headers**
+A list of extra headers that will be output by **notmuch show**
+with ``--format=sexp``, if present in the message.
+
+Default: empty list.
+
 **built_with.**
 Compile time feature . Current possibilities include
 "compact" (see **notmuch-compact(1)**) and "field_processor" (see
diff --git a/notmuch-client.h b/notmuch-client.h
index 74690054..d4402231 100644
--- a/notmuch-client.h
+++ b/notmuch-client.h
@@ -64,8 +64,10 @@ struct sprinter;
 struct notmuch_show_params;
 
 typedef struct notmuch_show_format {
-struct sprinter *(*new_sprinter)(const void *ctx, FILE *stream);
-notmuch_status_t (*part)(const void *ctx, struct sprinter *sprinter,
+struct sprinter *(*new_sprinter)(notmuch_database_t *notmuch,
+ const void *ctx, FILE *stream);
+notmuch_status_t (*part)(notmuch_database_t *notmuch,
+			 const void *ctx, struct sprinter *sprinter,
 			 struct mime_node *node, int indent,
 			 const struct notmuch_show_params *params);
 } notmuch_show_format_t;
@@ -227,12 +229,14 @@ notmuch_status_t
 show_one_part (const char *filename, int part);
 
 void
-format_part_sprinter (const void *ctx, struct sprinter *sp, mime_node_t *node,
+format_part_sprinter (notmuch_database_t *notmuch,
+		  const void *ctx, struct sprinter *sp, mime_node_t *node,
 		  bool output_body,
 		  bool include_html);
 
 void
-format_headers_sprinter (struct sprinter *sp, GMimeMessage *message,
+format_headers_sprinter (notmuch_database_t *notmuch,
+			 struct sprinter *sp, GMimeMessage *message,
 			 bool reply, const _notmuch_message_crypto_t *msg_crypto);
 
 typedef enum {
diff --git a/notmuch-config.c b/notmuch-config.c
index 1b079e85..6554ad9b 100644
--- a/notmuch-config.c
+++ b/notmuch-config.c
@@ -841,9 +841,10 @@ typedef struct config_key {
 
 static struct config_key
 config_key_table[] = {
-{ "index.decrypt",   true,   false,  NULL },
-{ "index.header.",   true,   true,   validate_field_name },
-{ "query.",  true,   true,   NULL },
+{ "index.decrypt",  true,   false,  NULL },
+{ "index.header.",  true,   true,   validate_field_name },
+{ "query.", true,   true,   NULL },
+{ "show.extra_headers", true,   false,  NULL }
 };
 
 static config_key_info_t *
diff --git a/notmuch-reply.c b/notmuch-reply.c
index 2c30f6f9..5d468c88 100644
--- a/notmuch-reply.c
+++ b/notmuch-reply.c
@@ -614,7 +614,8 @@ enum {
 };
 
 static int
-do_reply (notmuch_config_t *config,
+do_reply (notmuch_database_t *notmuch,
+	  notmuch_config_t *config,
 	  notmuch_query_t *query,
 	  notmuch_show_params_t *params,
 	  int format,
@@ -640,9 +641,9 @@ do_reply (notmuch_config_t *config,
 	}
 
 	if (format == FO

Re: [PATCH] Display extra headers for emacs-mua - db config option

[Tomi Ollila]

On Fri, Nov 22 2019, Daniel Kahn Gillmor wrote:

> On Thu 2019-11-21 08:27:04 -0400, David Bremner wrote:
>> Apologies for being late to the discussion of where to store the
>> configuration. So far we have only stored configuration in the database
>> where it affected the behaviour of the library API.
>
> While i'm being ambitious, i'd like also to eventually consider moving
> more of the existing CLI functionality into being accessible from the
> library.  I think this would help downstream MUAs that use notmuch who
> currently can't (or don't want to, for whatever reason) take advantage
> of the existing CLI, but can use the library.
>
> If we stuff more config in the config file, then the behavior of any
> future move to the library will have to grapple with the config move
> (currently the library never reads the config file, it just opens the
> database).

How can library open the database if it doesn't read the config file
-- the config file defines where database is located =D

>> I know some people (e.g. dkg) have suggested it would be better to
>> store all of the configuration in the database for consistency, while
>> others are disgruntled that some of the configuration is not editable
>> with text editor.
>
> It would still editable with a text editor -- you just need to edit the
> output of "notmuch dump --include config" and feed the result back into
> "notmuch restore" :)

If the library can somehow guess the location of the database without
reading the config file ;D I think dumping -- editing -- restoring
the database is tolerable solution -- provided it is clearly documented
for people like me who wants to edit configuration files w/ text editor(*),
for forgets these instructions easily. If, in the future, we still have
configuration file(+), notmuch could also write such instructions to
the config file.

>
>  --dkg

Tomi

(*) like someone may know I'm not too fond of editing configuration files
when there are alternatives available (like in this case) -- however
is someone else(tm) takes the time to create this solution and it uses
configuration files, I'll use it...

(+) if we dropped the configuration file, then notmuch, and library could
open database from ~/mail/notmuch/ by default, or from location pointed
by e.g. NOTMUCH_DATABASE_DIR -- as an additional benefit(?) notmuch
would pollute user's $HOME by one file less -- the `.notmuch-config`.
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH] Display extra headers for emacs-mua - db config option

[Daniel Kahn Gillmor]

On Thu 2019-11-21 08:27:04 -0400, David Bremner wrote:
> Apologies for being late to the discussion of where to store the
> configuration. So far we have only stored configuration in the database
> where it affected the behaviour of the library API.

While i'm being ambitious, i'd like also to eventually consider moving
more of the existing CLI functionality into being accessible from the
library.  I think this would help downstream MUAs that use notmuch who
currently can't (or don't want to, for whatever reason) take advantage
of the existing CLI, but can use the library.

If we stuff more config in the config file, then the behavior of any
future move to the library will have to grapple with the config move
(currently the library never reads the config file, it just opens the
database).

> I know some people (e.g. dkg) have suggested it would be better to
> store all of the configuration in the database for consistency, while
> others are disgruntled that some of the configuration is not editable
> with text editor.

It would still editable with a text editor -- you just need to edit the
output of "notmuch dump --include config" and feed the result back into
"notmuch restore" :)

 --dkg


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH] Display extra headers for emacs-mua - db config option

[Johan Parin]

Daniel Kahn Gillmor  writes:

>
> I'm a little weirded out by the move to a static notmuch_database_t
> *notmuch object.  Are we doing this because we don't want to pass
> around the database to internal functions?

Yes

> I know that the scope of nomtuch-show.c is basically "global scope",
> but i worry that it makes the code more difficult to read and
> maintain.
>

I agree it's not so nice with globals in general.

>
> Is it that much worse to pass around the notmuch_database_t *?
>

It has a lot of code impact, it really propagates to a lot of
places. For instance it also impacts the json and text api, because
those need to have the same signatures as the json api. Also the
database needs to be opened in places where it's currently not opened,
such as in notmuch_search_command().

I have started on such a patch and can certainly complete that if this
is the direction we want to move. I can warn you that there is a lot of
code changes (although they are cosmetic).


/Johan
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH] Display extra headers for emacs-mua - db config option

[David Bremner]

Johan Parin  writes:

> +
> +if (notmuch_database_get_config (notmuch, "show.extra_headers",
> +  _headers) != NOTMUCH_STATUS_SUCCESS)
> + return;

Apologies for being late to the discussion of where to store the
configuration. So far we have only stored configuration in the database
where it affected the behaviour of the library API. I know some people
(e.g. dkg) have suggested it would be better to store all of the
configuration in the database for consistency, while others are
disgruntled that some of the configuration is not editable with text
editor. The approach here would represent a change, which I'm not
necessarily against, but I think we need to think it through.

> +
> +header_list  = g_mime_object_get_header_list (GMIME_OBJECT(message));
> +
> +tofree = extra_headers;

It's somewhat a matter of taste, but I would rather introduce a variable
for the pointer to be mutated by strsep, and call  free (extra_headers) below
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH] Display extra headers for emacs-mua - db config option

[David Bremner]

Daniel Kahn Gillmor  writes:

>
> I'm a little weirded out by the move to a static notmuch_database_t
> *notmuch object.  Are we doing this because we don't want to pass around
> the database to internal functions?  I know that the scope of
> nomtuch-show.c is basically "global scope", but i worry that it makes
> the code more difficult to read and maintain.

I had a similar reaction.

>
> It's also not a common idiom in the rest of the codebase (at least not
> one that i've seen).
>
> Is it that much worse to pass around the notmuch_database_t *?

There are some call chains 3 or 4 deep that would need to be modified. I
_think_ that there is always a notmuch_config_t available, so one option
would be to stash the headers or the database there. We do have the
convention other places (mainly in  lib/) of objects having a pointer to
their "parent" database.
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH] Display extra headers for emacs-mua - db config option

[Daniel Kahn Gillmor]

On Sat 2019-11-16 17:27:23 +0100, Johan Parin wrote:
> Modify format_headers_sprinter so that it returns some additional headers in 
> the
> sexp, instead of a small fixed set of headers.
>
> The extra header list is configured by the database config option
> `show.extra_headers'.
>
> This is required in order for the elisp variable
> `notmuch-message-headers' to work.

Thanks for this work, Johan, and for your persistence on this
functionality.

> diff --git a/notmuch-show.c b/notmuch-show.c
> index 21792a57..4c77468f 100644
> --- a/notmuch-show.c
> +++ b/notmuch-show.c
> @@ -18,11 +18,16 @@
>   * Author: Carl Worth 
>   */
>  
> +#include 
> +
>  #include "notmuch-client.h"
>  #include "gmime-filter-reply.h"
>  #include "sprinter.h"
>  #include "zlib-extra.h"
>  
> +static notmuch_database_t *notmuch = NULL;
> +
> +
>  static const char *
>  _get_tags_as_string (const void *ctx, notmuch_message_t *message)
>  {
> @@ -195,6 +200,38 @@ _is_from_line (const char *line)
>   return 0;
>  }
>  
> +/* Output extra headers if configured with the `show.extra_headers'
> + * database configuration option
> + */
> +void
> +format_extra_headers_sprinter (sprinter_t *sp, GMimeMessage *message)
> +{
> +GMimeHeaderList *header_list;
> +GMimeHeader *header;
> +char *extra_headers, *tofree, *header_name;
> +
> +if (notmuch == NULL)
> + return;
> +
> +if (notmuch_database_get_config (notmuch, "show.extra_headers",
> +  _headers) != NOTMUCH_STATUS_SUCCESS)
> + return;
> +
> +header_list  = g_mime_object_get_header_list (GMIME_OBJECT(message));
> +
> +tofree = extra_headers;
> +while ( (header_name = strsep(_headers, ";")) != NULL) {
> +
> + header = g_mime_header_list_get_header (header_list, header_name);
> + if (header == NULL)
> + continue;
> +
> + sp->map_key (sp, g_mime_header_get_name(header));
> + sp->string (sp, g_mime_header_get_value(header));
> +}
> +free (tofree);
> +}
> +
>  void
>  format_headers_sprinter (sprinter_t *sp, GMimeMessage *message,
>bool reply, const _notmuch_message_crypto_t 
> *msg_crypto)
> @@ -253,6 +290,9 @@ format_headers_sprinter (sprinter_t *sp, GMimeMessage 
> *message,
>  } else {
>   sp->map_key (sp, "Date");
>   sp->string (sp, g_mime_message_get_date_string (sp, message));
> +
> + /* Output extra headers the user has configured in the database, if any 
> */
> + format_extra_headers_sprinter (sp, message);
>  }
>  
>  sp->end (sp);
> @@ -1152,7 +1192,6 @@ static const notmuch_show_format_t *formatters[] = {
>  int
>  notmuch_show_command (notmuch_config_t *config, int argc, char *argv[])
>  {
> -notmuch_database_t *notmuch;
>  notmuch_query_t *query;
>  char *query_string;
>  int opt_index, ret;

I'm a little weirded out by the move to a static notmuch_database_t
*notmuch object.  Are we doing this because we don't want to pass around
the database to internal functions?  I know that the scope of
nomtuch-show.c is basically "global scope", but i worry that it makes
the code more difficult to read and maintain.

It's also not a common idiom in the rest of the codebase (at least not
one that i've seen).

Is it that much worse to pass around the notmuch_database_t *?

 --dkg
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH] Display extra headers for emacs-mua - db config option

[Johan Parin]

This is another version of my attempt to get configurability on extra
headers to be displayed in the emacs MUA. It is a modification of a
previous patch where the extra headers returned by notmuch-show were
hard coded. In this version, the extra headers is configured by a
database config option `show.extra_headers'. As I see it the advantage
with this approach is:

- It gives full flexibility to choose any extra header to be
  displayed. The required steps are two (1) configure
  `show.extra_headers' in the db and (2) set `notmuch-message-headers'
  as desired.
- There is absolutely no performance concern.

The change is also very simple, low impact.

There is no API change (if that is considered a maintenance cost).

If a decision would later be taken to have format_headers_sprinter
return all headers, then users having adopted the configuration option
would still have the same functionality, albeit with an unused db config
entry.

I think this patch is complete, I also added an update to the config man
page. I guess this patch does not require updating the schemata (?)


/Johan
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH] Display extra headers for emacs-mua - db config option

[Tomi Ollila]

On Sat, Nov 16 2019, Johan Parin wrote:

> Modify format_headers_sprinter so that it returns some additional headers
> in the 
> sexp, instead of a small fixed set of headers.
>
> The extra header list is configured by the database config option
> `show.extra_headers'.
>
> This is required in order for the elisp variable
> `notmuch-message-headers' to work.

I did not look the patch, but if there is going to be configuration option
(either in config file, or in database), this way is (IMO) the most 
sensible alternative (add to the default options) -- no "full list" or 
"suppress header" options.

Tomi

>
> See this bug report:
>
>   https://notmuchmail.org/pipermail/notmuch/2017/026069.html
> ---
>  doc/man1/notmuch-config.rst |  6 ++
>  notmuch-config.c|  7 ---
>  notmuch-show.c  | 41 -
>  3 files changed, 50 insertions(+), 4 deletions(-)
>
> diff --git a/doc/man1/notmuch-config.rst b/doc/man1/notmuch-config.rst
> index 28487079..0eb59883 100644
> --- a/doc/man1/notmuch-config.rst
> +++ b/doc/man1/notmuch-config.rst
> @@ -204,6 +204,12 @@ The available configuration items are described below.
>  supported. See **notmuch-search-terms(7)** for a list of existing
>  prefixes, and an explanation of probabilistic prefixes.
>  
> +**show.extra_headers**
> +A list of extra headers that will be output by **notmuch show**
> +with ``--format=sexp``, if present in the message.
> +
> +Default: empty list.
> +
>  **built_with.**
>  Compile time feature . Current possibilities include
>  "compact" (see **notmuch-compact(1)**) and "field_processor" (see
> diff --git a/notmuch-config.c b/notmuch-config.c
> index 1b079e85..6554ad9b 100644
> --- a/notmuch-config.c
> +++ b/notmuch-config.c
> @@ -841,9 +841,10 @@ typedef struct config_key {
>  
>  static struct config_key
>  config_key_table[] = {
> -{ "index.decrypt",   true,   false,  NULL },
> -{ "index.header.",   true,   true,   validate_field_name },
> -{ "query.",  true,   true,   NULL },
> +{ "index.decrypt",  true,   false,  NULL },
> +{ "index.header.",  true,   true,   validate_field_name },
> +{ "query.", true,   true,   NULL },
> +{ "show.extra_headers", true,   false,  NULL }
>  };
>  
>  static config_key_info_t *
> diff --git a/notmuch-show.c b/notmuch-show.c
> index 21792a57..4c77468f 100644
> --- a/notmuch-show.c
> +++ b/notmuch-show.c
> @@ -18,11 +18,16 @@
>   * Author: Carl Worth 
>   */
>  
> +#include 
> +
>  #include "notmuch-client.h"
>  #include "gmime-filter-reply.h"
>  #include "sprinter.h"
>  #include "zlib-extra.h"
>  
> +static notmuch_database_t *notmuch = NULL;
> +
> +
>  static const char *
>  _get_tags_as_string (const void *ctx, notmuch_message_t *message)
>  {
> @@ -195,6 +200,38 @@ _is_from_line (const char *line)
>   return 0;
>  }
>  
> +/* Output extra headers if configured with the `show.extra_headers'
> + * database configuration option
> + */
> +void
> +format_extra_headers_sprinter (sprinter_t *sp, GMimeMessage *message)
> +{
> +GMimeHeaderList *header_list;
> +GMimeHeader *header;
> +char *extra_headers, *tofree, *header_name;
> +
> +if (notmuch == NULL)
> + return;
> +
> +if (notmuch_database_get_config (notmuch, "show.extra_headers",
> +  _headers) != NOTMUCH_STATUS_SUCCESS)
> + return;
> +
> +header_list  = g_mime_object_get_header_list (GMIME_OBJECT(message));
> +
> +tofree = extra_headers;
> +while ( (header_name = strsep(_headers, ";")) != NULL) {
> +
> + header = g_mime_header_list_get_header (header_list, header_name);
> + if (header == NULL)
> + continue;
> +
> + sp->map_key (sp, g_mime_header_get_name(header));
> + sp->string (sp, g_mime_header_get_value(header));
> +}
> +free (tofree);
> +}
> +
>  void
>  format_headers_sprinter (sprinter_t *sp, GMimeMessage *message,
>bool reply, const _notmuch_message_crypto_t 
> *msg_crypto)
> @@ -253,6 +290,9 @@ format_headers_sprinter (sprinter_t *sp, GMimeMessage 
> *message,
>  } else {
>   sp->map_key (sp, "Date");
>   sp->string (sp, g_mime_message_get_date_string (sp, message));
> +
> + /* Output extra headers the user has configured in the database, if any 
> */
> + format_extra_headers_sprinter (sp, message);
>  }
>  
>  sp->end (sp);
> @@ -1152,7 +1192,6 @@ static const notmuch_show_format_t *formatters[] = {
>  int
>  notmuch_show_command (notmuch_config_t *config, int argc, char *argv[])
>  {
> -notmuch_database_t *notmuch;
>  notmuch_query_t *query;
>  char *query_string;
>  int opt_index, ret;
> -- 
> 2.21.0 (Apple Git-122)
>
> ___
> notmuch mailing list
> notmuch@notmuchmail.org
> https://notmuchmail.org/mailman/listinfo/notmuch
___
notmuch mailing list

[PATCH] Display extra headers for emacs-mua - db config option

[Johan Parin]

Modify format_headers_sprinter so that it returns some additional headers in the
sexp, instead of a small fixed set of headers.

The extra header list is configured by the database config option
`show.extra_headers'.

This is required in order for the elisp variable
`notmuch-message-headers' to work.

See this bug report:

  https://notmuchmail.org/pipermail/notmuch/2017/026069.html
---
 doc/man1/notmuch-config.rst |  6 ++
 notmuch-config.c|  7 ---
 notmuch-show.c  | 41 -
 3 files changed, 50 insertions(+), 4 deletions(-)

diff --git a/doc/man1/notmuch-config.rst b/doc/man1/notmuch-config.rst
index 28487079..0eb59883 100644
--- a/doc/man1/notmuch-config.rst
+++ b/doc/man1/notmuch-config.rst
@@ -204,6 +204,12 @@ The available configuration items are described below.
 supported. See **notmuch-search-terms(7)** for a list of existing
 prefixes, and an explanation of probabilistic prefixes.
 
+**show.extra_headers**
+A list of extra headers that will be output by **notmuch show**
+with ``--format=sexp``, if present in the message.
+
+Default: empty list.
+
 **built_with.**
 Compile time feature . Current possibilities include
 "compact" (see **notmuch-compact(1)**) and "field_processor" (see
diff --git a/notmuch-config.c b/notmuch-config.c
index 1b079e85..6554ad9b 100644
--- a/notmuch-config.c
+++ b/notmuch-config.c
@@ -841,9 +841,10 @@ typedef struct config_key {
 
 static struct config_key
 config_key_table[] = {
-{ "index.decrypt",   true,   false,  NULL },
-{ "index.header.",   true,   true,   validate_field_name },
-{ "query.",  true,   true,   NULL },
+{ "index.decrypt",  true,   false,  NULL },
+{ "index.header.",  true,   true,   validate_field_name },
+{ "query.", true,   true,   NULL },
+{ "show.extra_headers", true,   false,  NULL }
 };
 
 static config_key_info_t *
diff --git a/notmuch-show.c b/notmuch-show.c
index 21792a57..4c77468f 100644
--- a/notmuch-show.c
+++ b/notmuch-show.c
@@ -18,11 +18,16 @@
  * Author: Carl Worth 
  */
 
+#include 
+
 #include "notmuch-client.h"
 #include "gmime-filter-reply.h"
 #include "sprinter.h"
 #include "zlib-extra.h"
 
+static notmuch_database_t *notmuch = NULL;
+
+
 static const char *
 _get_tags_as_string (const void *ctx, notmuch_message_t *message)
 {
@@ -195,6 +200,38 @@ _is_from_line (const char *line)
return 0;
 }
 
+/* Output extra headers if configured with the `show.extra_headers'
+ * database configuration option
+ */
+void
+format_extra_headers_sprinter (sprinter_t *sp, GMimeMessage *message)
+{
+GMimeHeaderList *header_list;
+GMimeHeader *header;
+char *extra_headers, *tofree, *header_name;
+
+if (notmuch == NULL)
+   return;
+
+if (notmuch_database_get_config (notmuch, "show.extra_headers",
+_headers) != NOTMUCH_STATUS_SUCCESS)
+   return;
+
+header_list  = g_mime_object_get_header_list (GMIME_OBJECT(message));
+
+tofree = extra_headers;
+while ( (header_name = strsep(_headers, ";")) != NULL) {
+
+   header = g_mime_header_list_get_header (header_list, header_name);
+   if (header == NULL)
+   continue;
+
+   sp->map_key (sp, g_mime_header_get_name(header));
+   sp->string (sp, g_mime_header_get_value(header));
+}
+free (tofree);
+}
+
 void
 format_headers_sprinter (sprinter_t *sp, GMimeMessage *message,
 bool reply, const _notmuch_message_crypto_t 
*msg_crypto)
@@ -253,6 +290,9 @@ format_headers_sprinter (sprinter_t *sp, GMimeMessage 
*message,
 } else {
sp->map_key (sp, "Date");
sp->string (sp, g_mime_message_get_date_string (sp, message));
+
+   /* Output extra headers the user has configured in the database, if any 
*/
+   format_extra_headers_sprinter (sp, message);
 }
 
 sp->end (sp);
@@ -1152,7 +1192,6 @@ static const notmuch_show_format_t *formatters[] = {
 int
 notmuch_show_command (notmuch_config_t *config, int argc, char *argv[])
 {
-notmuch_database_t *notmuch;
 notmuch_query_t *query;
 char *query_string;
 int opt_index, ret;
-- 
2.21.0 (Apple Git-122)

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch