[PATCH v3] Emacs: Add address completion mechanism implemented in elisp

2014-09-08 Thread Michal Sojka
On Mon, Sep 08 2014, David Edmondson wrote:
> On Mon, Aug 11 2014, Michal Sojka wrote:
>> Currently, notmuch has an address completion mechanism that requires
>> external command to provide completion candidates. This patch adds a
>> completion mechanism inspired by https://github.com/tjim/nevermore,
>> which is implemented in Emacs lisp only.
>>
>> The core of the new mechanism is the function notmuch-address-harvest
>> that collects the completion candidates from the notmuch database and
>> stores them in notmuch-address-completions variable.
>> notmuch-address-harvest is called on the first entry to message-mode
>> and runs asychnornously so that the user doesn't have to wait for it
>> to complete while composing the message. The
>> notmuch-address-completions variable is used in message-mode as a
>> source of completion candidates. Currently, there are two ways how the
>> notmuch-address-completions variable is used.
>>
>> First, preexisting address completion mechanism is extended to use
>> notmuch-address-completions in addition to the external command. This
>> new behavior is configured by setting notmuch-address-command to nil,
>> which is the new default. Note that this may *BREAK EXISTING SETUPS*
>> when the user used external command named "notmuch-addresses", i.e.
>> the previous default. The result will be that the user will use the
>> new mechanism instead of the his command. I believe that many users
>> may not even recognize this because the new mechanism works the same
>> as http://commonmeasure.org/~jkr/git/notmuch_addresses.git and perhaps
>> also as other commands suggested at
>> http://notmuchmail.org/emacstips/#address_completion.
>>
>> Second way of using notmuch-address-completions is notmuch-company.el.
>> This presents the possible completions in a nice popup box after a
>> short typing delay but requires company-mode to be installed.
>
> This looks great, thanks for doing it. It seems like a better approach
> than id:1409921969-65129-1-git-send-email-dme at dme.org. Some comments:
>
> - Adding the address collection to `message-mode-hook' means that it
>   runs every time I start to compose a message. If the address
>   collection is disk intensive, this might be bad for battery life. 

The actual harvesting starts only when notmuch-address-completions is
nil, i.e. when the message-mode is entered for the first time.

> The set of potential recipients doesn't change _that_ much over time
> for a typical person, I'd wager. Maybe the hook should only run once a
> day? (Tunable, of course.)

The current version of the patch has a drawback that harvesting is never
run again. Adding a tunable option for reharvesting might be a good
idea.

Since initial harvesting is very slow on non-SSD disk, I want to change
the implementation so that initially, only addresses matching the
entered prefix will be harvested, which should be reasonably fast. Then
full harvest will run on background and once it is finished,
prefix-based harvesting won't be used anymore.

Maybe prefix-based harvesting could be then used as a fallback when no
candidates are found in the data from full harvest. This could also be a
solution to the "reharvest" problem.

I've just returned from vacations so I plan to work on that this week.
Jani's --output=address patch also looks like something to play with.

Cheers,
-Michal

>
> - The addition of company mode support (which I haven't tried) should be
>   a separate patch in the series.
>
>> ---
>> Changes from v1:
>> - Use of notmuch-parser.el instead of the custom parser in the
>>   original code. The notmuch parser is slightly faster.
>> - Use of functions in notmuch-query.el instead of functions in the
>>   original code with almost the same functionality.
>> - Integrated with existing completion mechanism in notmuch.
>> - notmuch-company.el was moved from emacs/contrib to emacs and
>>   no-byte-compile directive was added to it.
>> - Aligned with notmuch naming conventions.
>> - Documented bugs found in notmuch-company.el
>>
>> Changes from v2:
>> - Updated Makefile.local to not conflict with current master
>> ---
>>  emacs/Makefile.local |  6 ++-
>>  emacs/notmuch-address.el | 95 
>> +++-
>>  emacs/notmuch-company.el | 69 +++
>>  emacs/notmuch-lib.el |  3 ++
>>  4 files changed, 163 insertions(+), 10 deletions(-)
>>  create mode 100644 emacs/notmuch-company.el
>>
>> diff --git a/emacs/Makefile.local b/emacs/Makefile.local
>> index 1109cfa..6c93e73 100644
>> --- a/emacs/Makefile.local
>> +++ b/emacs/Makefile.local
>> @@ -20,6 +20,7 @@ emacs_sources := \
>>  $(dir)/notmuch-print.el \
>>  $(dir)/notmuch-version.el \
>>  $(dir)/notmuch-jump.el \
>> +$(dir)/notmuch-company.el
>>  
>>  $(dir)/notmuch-version.el: $(dir)/Makefile.local version.stamp
>>  $(dir)/notmuch-version.el: $(srcdir)/$(dir)/notmuch-version.el.tmpl
>> @@ -30,7 +31,10 @@ $(dir)/notmuch-version.el: 
>> 

[PATCH v3] Emacs: Add address completion mechanism implemented in elisp

2014-09-08 Thread David Edmondson
On Mon, Sep 08 2014, Michal Sojka wrote:
> On Mon, Sep 08 2014, David Edmondson wrote:
>> On Mon, Aug 11 2014, Michal Sojka wrote:
>>> Currently, notmuch has an address completion mechanism that requires
>>> external command to provide completion candidates. This patch adds a
>>> completion mechanism inspired by https://github.com/tjim/nevermore,
>>> which is implemented in Emacs lisp only.
>>>
>>> The core of the new mechanism is the function notmuch-address-harvest
>>> that collects the completion candidates from the notmuch database and
>>> stores them in notmuch-address-completions variable.
>>> notmuch-address-harvest is called on the first entry to message-mode
>>> and runs asychnornously so that the user doesn't have to wait for it
>>> to complete while composing the message. The
>>> notmuch-address-completions variable is used in message-mode as a
>>> source of completion candidates. Currently, there are two ways how the
>>> notmuch-address-completions variable is used.
>>>
>>> First, preexisting address completion mechanism is extended to use
>>> notmuch-address-completions in addition to the external command. This
>>> new behavior is configured by setting notmuch-address-command to nil,
>>> which is the new default. Note that this may *BREAK EXISTING SETUPS*
>>> when the user used external command named "notmuch-addresses", i.e.
>>> the previous default. The result will be that the user will use the
>>> new mechanism instead of the his command. I believe that many users
>>> may not even recognize this because the new mechanism works the same
>>> as http://commonmeasure.org/~jkr/git/notmuch_addresses.git and perhaps
>>> also as other commands suggested at
>>> http://notmuchmail.org/emacstips/#address_completion.
>>>
>>> Second way of using notmuch-address-completions is notmuch-company.el.
>>> This presents the possible completions in a nice popup box after a
>>> short typing delay but requires company-mode to be installed.
>>
>> This looks great, thanks for doing it. It seems like a better approach
>> than id:1409921969-65129-1-git-send-email-dme at dme.org. Some comments:
>>
>> - Adding the address collection to `message-mode-hook' means that it
>>   runs every time I start to compose a message. If the address
>>   collection is disk intensive, this might be bad for battery life. 
>
> The actual harvesting starts only when notmuch-address-completions is
> nil, i.e. when the message-mode is entered for the first time.

Ah, sorry. I didn't read closely enough.

>> The set of potential recipients doesn't change _that_ much over time
>> for a typical person, I'd wager. Maybe the hook should only run once a
>> day? (Tunable, of course.)
>
> The current version of the patch has a drawback that harvesting is never
> run again. Adding a tunable option for reharvesting might be a good
> idea.
>
> Since initial harvesting is very slow on non-SSD disk, I want to change
> the implementation so that initially, only addresses matching the
> entered prefix will be harvested, which should be reasonably fast. Then
> full harvest will run on background and once it is finished,
> prefix-based harvesting won't be used anymore.
>
> Maybe prefix-based harvesting could be then used as a fallback when no
> candidates are found in the data from full harvest. This could also be a
> solution to the "reharvest" problem.
>
> I've just returned from vacations so I plan to work on that this week.
> Jani's --output=address patch also looks like something to play with.

Sounds great, thanks.


[PATCH v3] Emacs: Add address completion mechanism implemented in elisp

2014-09-08 Thread Mark Walters
On Mon, 08 Sep 2014, David Edmondson  wrote:
> On Mon, Aug 11 2014, Michal Sojka wrote:
>> Currently, notmuch has an address completion mechanism that requires
>> external command to provide completion candidates. This patch adds a
>> completion mechanism inspired by https://github.com/tjim/nevermore,
>> which is implemented in Emacs lisp only.
>>
>> The core of the new mechanism is the function notmuch-address-harvest
>> that collects the completion candidates from the notmuch database and
>> stores them in notmuch-address-completions variable.
>> notmuch-address-harvest is called on the first entry to message-mode
>> and runs asychnornously so that the user doesn't have to wait for it
>> to complete while composing the message. The
>> notmuch-address-completions variable is used in message-mode as a
>> source of completion candidates. Currently, there are two ways how the
>> notmuch-address-completions variable is used.
>>
>> First, preexisting address completion mechanism is extended to use
>> notmuch-address-completions in addition to the external command. This
>> new behavior is configured by setting notmuch-address-command to nil,
>> which is the new default. Note that this may *BREAK EXISTING SETUPS*
>> when the user used external command named "notmuch-addresses", i.e.
>> the previous default. The result will be that the user will use the
>> new mechanism instead of the his command. I believe that many users
>> may not even recognize this because the new mechanism works the same
>> as http://commonmeasure.org/~jkr/git/notmuch_addresses.git and perhaps
>> also as other commands suggested at
>> http://notmuchmail.org/emacstips/#address_completion.
>>
>> Second way of using notmuch-address-completions is notmuch-company.el.
>> This presents the possible completions in a nice popup box after a
>> short typing delay but requires company-mode to be installed.
>
> This looks great, thanks for doing it. It seems like a better approach
> than id:1409921969-65129-1-git-send-email-dme at dme.org. Some comments:
>
> - Adding the address collection to `message-mode-hook' means that it
>   runs every time I start to compose a message. If the address
>   collection is disk intensive, this might be bad for battery life. The
>   set of potential recipients doesn't change _that_ much over time for a
>   typical person, I'd wager. Maybe the hook should only run once a day?
>   (Tunable, of course.)

Just a meta comment really: Austin is very close to posting a patch
adding a timestamp to messages in the database (I think roughly last
modify time). Once that is in an address-harvest query of the form all
messages with any changes since the last harvest should be simple and
fast.

A second more general comment: if something like Jani's patch
id:1410021689-15901-1-git-send-email-jani at nikula.org goes in the
harvesting from addresses is hugely faster (circa 20 times, see
id:8761h0fxow.fsf at qmul.ac.uk ) than harvesting to,cc addresses so some
people might want to tweak/customize that.

Best wishes

Mark

>
> - The addition of company mode support (which I haven't tried) should be
>   a separate patch in the series.
>
>> ---
>> Changes from v1:
>> - Use of notmuch-parser.el instead of the custom parser in the
>>   original code. The notmuch parser is slightly faster.
>> - Use of functions in notmuch-query.el instead of functions in the
>>   original code with almost the same functionality.
>> - Integrated with existing completion mechanism in notmuch.
>> - notmuch-company.el was moved from emacs/contrib to emacs and
>>   no-byte-compile directive was added to it.
>> - Aligned with notmuch naming conventions.
>> - Documented bugs found in notmuch-company.el
>>
>> Changes from v2:
>> - Updated Makefile.local to not conflict with current master
>> ---
>>  emacs/Makefile.local |  6 ++-
>>  emacs/notmuch-address.el | 95 
>> +++-
>>  emacs/notmuch-company.el | 69 +++
>>  emacs/notmuch-lib.el |  3 ++
>>  4 files changed, 163 insertions(+), 10 deletions(-)
>>  create mode 100644 emacs/notmuch-company.el
>>
>> diff --git a/emacs/Makefile.local b/emacs/Makefile.local
>> index 1109cfa..6c93e73 100644
>> --- a/emacs/Makefile.local
>> +++ b/emacs/Makefile.local
>> @@ -20,6 +20,7 @@ emacs_sources := \
>>  $(dir)/notmuch-print.el \
>>  $(dir)/notmuch-version.el \
>>  $(dir)/notmuch-jump.el \
>> +$(dir)/notmuch-company.el
>>  
>>  $(dir)/notmuch-version.el: $(dir)/Makefile.local version.stamp
>>  $(dir)/notmuch-version.el: $(srcdir)/$(dir)/notmuch-version.el.tmpl
>> @@ -30,7 +31,10 @@ $(dir)/notmuch-version.el: 
>> $(srcdir)/$(dir)/notmuch-version.el.tmpl
>>  emacs_images := \
>>  $(srcdir)/$(dir)/notmuch-logo.png
>>  
>> -emacs_bytecode = $(emacs_sources:.el=.elc)
>> +# Do not try to install files that are not byte-compiled.
>> +emacs_no_byte_compile := $(dir)/notmuch-company.el
>> +
>> +emacs_bytecode = $(patsubst 

[PATCH v3] Emacs: Add address completion mechanism implemented in elisp

2014-09-08 Thread David Edmondson
On Mon, Aug 11 2014, Michal Sojka wrote:
> Currently, notmuch has an address completion mechanism that requires
> external command to provide completion candidates. This patch adds a
> completion mechanism inspired by https://github.com/tjim/nevermore,
> which is implemented in Emacs lisp only.
>
> The core of the new mechanism is the function notmuch-address-harvest
> that collects the completion candidates from the notmuch database and
> stores them in notmuch-address-completions variable.
> notmuch-address-harvest is called on the first entry to message-mode
> and runs asychnornously so that the user doesn't have to wait for it
> to complete while composing the message. The
> notmuch-address-completions variable is used in message-mode as a
> source of completion candidates. Currently, there are two ways how the
> notmuch-address-completions variable is used.
>
> First, preexisting address completion mechanism is extended to use
> notmuch-address-completions in addition to the external command. This
> new behavior is configured by setting notmuch-address-command to nil,
> which is the new default. Note that this may *BREAK EXISTING SETUPS*
> when the user used external command named "notmuch-addresses", i.e.
> the previous default. The result will be that the user will use the
> new mechanism instead of the his command. I believe that many users
> may not even recognize this because the new mechanism works the same
> as http://commonmeasure.org/~jkr/git/notmuch_addresses.git and perhaps
> also as other commands suggested at
> http://notmuchmail.org/emacstips/#address_completion.
>
> Second way of using notmuch-address-completions is notmuch-company.el.
> This presents the possible completions in a nice popup box after a
> short typing delay but requires company-mode to be installed.

This looks great, thanks for doing it. It seems like a better approach
than id:1409921969-65129-1-git-send-email-dme at dme.org. Some comments:

- Adding the address collection to `message-mode-hook' means that it
  runs every time I start to compose a message. If the address
  collection is disk intensive, this might be bad for battery life. The
  set of potential recipients doesn't change _that_ much over time for a
  typical person, I'd wager. Maybe the hook should only run once a day?
  (Tunable, of course.)

- The addition of company mode support (which I haven't tried) should be
  a separate patch in the series.

> ---
> Changes from v1:
> - Use of notmuch-parser.el instead of the custom parser in the
>   original code. The notmuch parser is slightly faster.
> - Use of functions in notmuch-query.el instead of functions in the
>   original code with almost the same functionality.
> - Integrated with existing completion mechanism in notmuch.
> - notmuch-company.el was moved from emacs/contrib to emacs and
>   no-byte-compile directive was added to it.
> - Aligned with notmuch naming conventions.
> - Documented bugs found in notmuch-company.el
>
> Changes from v2:
> - Updated Makefile.local to not conflict with current master
> ---
>  emacs/Makefile.local |  6 ++-
>  emacs/notmuch-address.el | 95 
> +++-
>  emacs/notmuch-company.el | 69 +++
>  emacs/notmuch-lib.el |  3 ++
>  4 files changed, 163 insertions(+), 10 deletions(-)
>  create mode 100644 emacs/notmuch-company.el
>
> diff --git a/emacs/Makefile.local b/emacs/Makefile.local
> index 1109cfa..6c93e73 100644
> --- a/emacs/Makefile.local
> +++ b/emacs/Makefile.local
> @@ -20,6 +20,7 @@ emacs_sources := \
>   $(dir)/notmuch-print.el \
>   $(dir)/notmuch-version.el \
>   $(dir)/notmuch-jump.el \
> + $(dir)/notmuch-company.el
>  
>  $(dir)/notmuch-version.el: $(dir)/Makefile.local version.stamp
>  $(dir)/notmuch-version.el: $(srcdir)/$(dir)/notmuch-version.el.tmpl
> @@ -30,7 +31,10 @@ $(dir)/notmuch-version.el: 
> $(srcdir)/$(dir)/notmuch-version.el.tmpl
>  emacs_images := \
>   $(srcdir)/$(dir)/notmuch-logo.png
>  
> -emacs_bytecode = $(emacs_sources:.el=.elc)
> +# Do not try to install files that are not byte-compiled.
> +emacs_no_byte_compile := $(dir)/notmuch-company.el
> +
> +emacs_bytecode = $(patsubst %.el,%.elc,$(filter-out 
> $(emacs_no_byte_compile),$(emacs_sources)))
>  
>  # Because of defmacro's and defsubst's, we have to account for load
>  # dependencies between Elisp files when byte compiling.  Otherwise,
> diff --git a/emacs/notmuch-address.el b/emacs/notmuch-address.el
> index fa65cd5..a50f4f4 100644
> --- a/emacs/notmuch-address.el
> +++ b/emacs/notmuch-address.el
> @@ -20,14 +20,18 @@
>  ;; Authors: David Edmondson 
>  
>  (require 'message)
> +(require 'notmuch-query)
> +(require 'notmuch-parser)
>  
>  ;;
>  
> -(defcustom notmuch-address-command "notmuch-addresses"
> -  "The command which generates possible addresses. It must take a
> -single argument and output a list of possible matches, one per
> -line."
> -  :type 'string
> +(defcustom 

Re: [PATCH v3] Emacs: Add address completion mechanism implemented in elisp

2014-09-08 Thread David Edmondson
On Mon, Aug 11 2014, Michal Sojka wrote:
 Currently, notmuch has an address completion mechanism that requires
 external command to provide completion candidates. This patch adds a
 completion mechanism inspired by https://github.com/tjim/nevermore,
 which is implemented in Emacs lisp only.

 The core of the new mechanism is the function notmuch-address-harvest
 that collects the completion candidates from the notmuch database and
 stores them in notmuch-address-completions variable.
 notmuch-address-harvest is called on the first entry to message-mode
 and runs asychnornously so that the user doesn't have to wait for it
 to complete while composing the message. The
 notmuch-address-completions variable is used in message-mode as a
 source of completion candidates. Currently, there are two ways how the
 notmuch-address-completions variable is used.

 First, preexisting address completion mechanism is extended to use
 notmuch-address-completions in addition to the external command. This
 new behavior is configured by setting notmuch-address-command to nil,
 which is the new default. Note that this may *BREAK EXISTING SETUPS*
 when the user used external command named notmuch-addresses, i.e.
 the previous default. The result will be that the user will use the
 new mechanism instead of the his command. I believe that many users
 may not even recognize this because the new mechanism works the same
 as http://commonmeasure.org/~jkr/git/notmuch_addresses.git and perhaps
 also as other commands suggested at
 http://notmuchmail.org/emacstips/#address_completion.

 Second way of using notmuch-address-completions is notmuch-company.el.
 This presents the possible completions in a nice popup box after a
 short typing delay but requires company-mode to be installed.

This looks great, thanks for doing it. It seems like a better approach
than id:1409921969-65129-1-git-send-email-...@dme.org. Some comments:

- Adding the address collection to `message-mode-hook' means that it
  runs every time I start to compose a message. If the address
  collection is disk intensive, this might be bad for battery life. The
  set of potential recipients doesn't change _that_ much over time for a
  typical person, I'd wager. Maybe the hook should only run once a day?
  (Tunable, of course.)

- The addition of company mode support (which I haven't tried) should be
  a separate patch in the series.

 ---
 Changes from v1:
 - Use of notmuch-parser.el instead of the custom parser in the
   original code. The notmuch parser is slightly faster.
 - Use of functions in notmuch-query.el instead of functions in the
   original code with almost the same functionality.
 - Integrated with existing completion mechanism in notmuch.
 - notmuch-company.el was moved from emacs/contrib to emacs and
   no-byte-compile directive was added to it.
 - Aligned with notmuch naming conventions.
 - Documented bugs found in notmuch-company.el

 Changes from v2:
 - Updated Makefile.local to not conflict with current master
 ---
  emacs/Makefile.local |  6 ++-
  emacs/notmuch-address.el | 95 
 +++-
  emacs/notmuch-company.el | 69 +++
  emacs/notmuch-lib.el |  3 ++
  4 files changed, 163 insertions(+), 10 deletions(-)
  create mode 100644 emacs/notmuch-company.el

 diff --git a/emacs/Makefile.local b/emacs/Makefile.local
 index 1109cfa..6c93e73 100644
 --- a/emacs/Makefile.local
 +++ b/emacs/Makefile.local
 @@ -20,6 +20,7 @@ emacs_sources := \
   $(dir)/notmuch-print.el \
   $(dir)/notmuch-version.el \
   $(dir)/notmuch-jump.el \
 + $(dir)/notmuch-company.el
  
  $(dir)/notmuch-version.el: $(dir)/Makefile.local version.stamp
  $(dir)/notmuch-version.el: $(srcdir)/$(dir)/notmuch-version.el.tmpl
 @@ -30,7 +31,10 @@ $(dir)/notmuch-version.el: 
 $(srcdir)/$(dir)/notmuch-version.el.tmpl
  emacs_images := \
   $(srcdir)/$(dir)/notmuch-logo.png
  
 -emacs_bytecode = $(emacs_sources:.el=.elc)
 +# Do not try to install files that are not byte-compiled.
 +emacs_no_byte_compile := $(dir)/notmuch-company.el
 +
 +emacs_bytecode = $(patsubst %.el,%.elc,$(filter-out 
 $(emacs_no_byte_compile),$(emacs_sources)))
  
  # Because of defmacro's and defsubst's, we have to account for load
  # dependencies between Elisp files when byte compiling.  Otherwise,
 diff --git a/emacs/notmuch-address.el b/emacs/notmuch-address.el
 index fa65cd5..a50f4f4 100644
 --- a/emacs/notmuch-address.el
 +++ b/emacs/notmuch-address.el
 @@ -20,14 +20,18 @@
  ;; Authors: David Edmondson d...@dme.org
  
  (require 'message)
 +(require 'notmuch-query)
 +(require 'notmuch-parser)
  
  ;;
  
 -(defcustom notmuch-address-command notmuch-addresses
 -  The command which generates possible addresses. It must take a
 -single argument and output a list of possible matches, one per
 -line.
 -  :type 'string
 +(defcustom notmuch-address-command nil
 +  The command which generates possible addresses for completion.
 +It 

Re: [PATCH v3] Emacs: Add address completion mechanism implemented in elisp

2014-09-08 Thread Michal Sojka
On Mon, Sep 08 2014, David Edmondson wrote:
 On Mon, Aug 11 2014, Michal Sojka wrote:
 Currently, notmuch has an address completion mechanism that requires
 external command to provide completion candidates. This patch adds a
 completion mechanism inspired by https://github.com/tjim/nevermore,
 which is implemented in Emacs lisp only.

 The core of the new mechanism is the function notmuch-address-harvest
 that collects the completion candidates from the notmuch database and
 stores them in notmuch-address-completions variable.
 notmuch-address-harvest is called on the first entry to message-mode
 and runs asychnornously so that the user doesn't have to wait for it
 to complete while composing the message. The
 notmuch-address-completions variable is used in message-mode as a
 source of completion candidates. Currently, there are two ways how the
 notmuch-address-completions variable is used.

 First, preexisting address completion mechanism is extended to use
 notmuch-address-completions in addition to the external command. This
 new behavior is configured by setting notmuch-address-command to nil,
 which is the new default. Note that this may *BREAK EXISTING SETUPS*
 when the user used external command named notmuch-addresses, i.e.
 the previous default. The result will be that the user will use the
 new mechanism instead of the his command. I believe that many users
 may not even recognize this because the new mechanism works the same
 as http://commonmeasure.org/~jkr/git/notmuch_addresses.git and perhaps
 also as other commands suggested at
 http://notmuchmail.org/emacstips/#address_completion.

 Second way of using notmuch-address-completions is notmuch-company.el.
 This presents the possible completions in a nice popup box after a
 short typing delay but requires company-mode to be installed.

 This looks great, thanks for doing it. It seems like a better approach
 than id:1409921969-65129-1-git-send-email-...@dme.org. Some comments:

 - Adding the address collection to `message-mode-hook' means that it
   runs every time I start to compose a message. If the address
   collection is disk intensive, this might be bad for battery life. 

The actual harvesting starts only when notmuch-address-completions is
nil, i.e. when the message-mode is entered for the first time.

 The set of potential recipients doesn't change _that_ much over time
 for a typical person, I'd wager. Maybe the hook should only run once a
 day? (Tunable, of course.)

The current version of the patch has a drawback that harvesting is never
run again. Adding a tunable option for reharvesting might be a good
idea.

Since initial harvesting is very slow on non-SSD disk, I want to change
the implementation so that initially, only addresses matching the
entered prefix will be harvested, which should be reasonably fast. Then
full harvest will run on background and once it is finished,
prefix-based harvesting won't be used anymore.

Maybe prefix-based harvesting could be then used as a fallback when no
candidates are found in the data from full harvest. This could also be a
solution to the reharvest problem.

I've just returned from vacations so I plan to work on that this week.
Jani's --output=address patch also looks like something to play with.

Cheers,
-Michal


 - The addition of company mode support (which I haven't tried) should be
   a separate patch in the series.

 ---
 Changes from v1:
 - Use of notmuch-parser.el instead of the custom parser in the
   original code. The notmuch parser is slightly faster.
 - Use of functions in notmuch-query.el instead of functions in the
   original code with almost the same functionality.
 - Integrated with existing completion mechanism in notmuch.
 - notmuch-company.el was moved from emacs/contrib to emacs and
   no-byte-compile directive was added to it.
 - Aligned with notmuch naming conventions.
 - Documented bugs found in notmuch-company.el

 Changes from v2:
 - Updated Makefile.local to not conflict with current master
 ---
  emacs/Makefile.local |  6 ++-
  emacs/notmuch-address.el | 95 
 +++-
  emacs/notmuch-company.el | 69 +++
  emacs/notmuch-lib.el |  3 ++
  4 files changed, 163 insertions(+), 10 deletions(-)
  create mode 100644 emacs/notmuch-company.el

 diff --git a/emacs/Makefile.local b/emacs/Makefile.local
 index 1109cfa..6c93e73 100644
 --- a/emacs/Makefile.local
 +++ b/emacs/Makefile.local
 @@ -20,6 +20,7 @@ emacs_sources := \
  $(dir)/notmuch-print.el \
  $(dir)/notmuch-version.el \
  $(dir)/notmuch-jump.el \
 +$(dir)/notmuch-company.el
  
  $(dir)/notmuch-version.el: $(dir)/Makefile.local version.stamp
  $(dir)/notmuch-version.el: $(srcdir)/$(dir)/notmuch-version.el.tmpl
 @@ -30,7 +31,10 @@ $(dir)/notmuch-version.el: 
 $(srcdir)/$(dir)/notmuch-version.el.tmpl
  emacs_images := \
  $(srcdir)/$(dir)/notmuch-logo.png
  
 -emacs_bytecode = $(emacs_sources:.el=.elc)
 +# Do not 

Re: [PATCH v3] Emacs: Add address completion mechanism implemented in elisp

2014-09-08 Thread Mark Walters
On Mon, 08 Sep 2014, David Edmondson d...@dme.org wrote:
 On Mon, Aug 11 2014, Michal Sojka wrote:
 Currently, notmuch has an address completion mechanism that requires
 external command to provide completion candidates. This patch adds a
 completion mechanism inspired by https://github.com/tjim/nevermore,
 which is implemented in Emacs lisp only.

 The core of the new mechanism is the function notmuch-address-harvest
 that collects the completion candidates from the notmuch database and
 stores them in notmuch-address-completions variable.
 notmuch-address-harvest is called on the first entry to message-mode
 and runs asychnornously so that the user doesn't have to wait for it
 to complete while composing the message. The
 notmuch-address-completions variable is used in message-mode as a
 source of completion candidates. Currently, there are two ways how the
 notmuch-address-completions variable is used.

 First, preexisting address completion mechanism is extended to use
 notmuch-address-completions in addition to the external command. This
 new behavior is configured by setting notmuch-address-command to nil,
 which is the new default. Note that this may *BREAK EXISTING SETUPS*
 when the user used external command named notmuch-addresses, i.e.
 the previous default. The result will be that the user will use the
 new mechanism instead of the his command. I believe that many users
 may not even recognize this because the new mechanism works the same
 as http://commonmeasure.org/~jkr/git/notmuch_addresses.git and perhaps
 also as other commands suggested at
 http://notmuchmail.org/emacstips/#address_completion.

 Second way of using notmuch-address-completions is notmuch-company.el.
 This presents the possible completions in a nice popup box after a
 short typing delay but requires company-mode to be installed.

 This looks great, thanks for doing it. It seems like a better approach
 than id:1409921969-65129-1-git-send-email-...@dme.org. Some comments:

 - Adding the address collection to `message-mode-hook' means that it
   runs every time I start to compose a message. If the address
   collection is disk intensive, this might be bad for battery life. The
   set of potential recipients doesn't change _that_ much over time for a
   typical person, I'd wager. Maybe the hook should only run once a day?
   (Tunable, of course.)

Just a meta comment really: Austin is very close to posting a patch
adding a timestamp to messages in the database (I think roughly last
modify time). Once that is in an address-harvest query of the form all
messages with any changes since the last harvest should be simple and
fast.

A second more general comment: if something like Jani's patch
id:1410021689-15901-1-git-send-email-j...@nikula.org goes in the
harvesting from addresses is hugely faster (circa 20 times, see
id:8761h0fxow@qmul.ac.uk ) than harvesting to,cc addresses so some
people might want to tweak/customize that.

Best wishes

Mark


 - The addition of company mode support (which I haven't tried) should be
   a separate patch in the series.

 ---
 Changes from v1:
 - Use of notmuch-parser.el instead of the custom parser in the
   original code. The notmuch parser is slightly faster.
 - Use of functions in notmuch-query.el instead of functions in the
   original code with almost the same functionality.
 - Integrated with existing completion mechanism in notmuch.
 - notmuch-company.el was moved from emacs/contrib to emacs and
   no-byte-compile directive was added to it.
 - Aligned with notmuch naming conventions.
 - Documented bugs found in notmuch-company.el

 Changes from v2:
 - Updated Makefile.local to not conflict with current master
 ---
  emacs/Makefile.local |  6 ++-
  emacs/notmuch-address.el | 95 
 +++-
  emacs/notmuch-company.el | 69 +++
  emacs/notmuch-lib.el |  3 ++
  4 files changed, 163 insertions(+), 10 deletions(-)
  create mode 100644 emacs/notmuch-company.el

 diff --git a/emacs/Makefile.local b/emacs/Makefile.local
 index 1109cfa..6c93e73 100644
 --- a/emacs/Makefile.local
 +++ b/emacs/Makefile.local
 @@ -20,6 +20,7 @@ emacs_sources := \
  $(dir)/notmuch-print.el \
  $(dir)/notmuch-version.el \
  $(dir)/notmuch-jump.el \
 +$(dir)/notmuch-company.el
  
  $(dir)/notmuch-version.el: $(dir)/Makefile.local version.stamp
  $(dir)/notmuch-version.el: $(srcdir)/$(dir)/notmuch-version.el.tmpl
 @@ -30,7 +31,10 @@ $(dir)/notmuch-version.el: 
 $(srcdir)/$(dir)/notmuch-version.el.tmpl
  emacs_images := \
  $(srcdir)/$(dir)/notmuch-logo.png
  
 -emacs_bytecode = $(emacs_sources:.el=.elc)
 +# Do not try to install files that are not byte-compiled.
 +emacs_no_byte_compile := $(dir)/notmuch-company.el
 +
 +emacs_bytecode = $(patsubst %.el,%.elc,$(filter-out 
 $(emacs_no_byte_compile),$(emacs_sources)))
  
  # Because of defmacro's and defsubst's, we have to account for load
  # dependencies between Elisp files 

Re: [PATCH v3] Emacs: Add address completion mechanism implemented in elisp

2014-09-08 Thread David Edmondson
On Mon, Sep 08 2014, Michal Sojka wrote:
 On Mon, Sep 08 2014, David Edmondson wrote:
 On Mon, Aug 11 2014, Michal Sojka wrote:
 Currently, notmuch has an address completion mechanism that requires
 external command to provide completion candidates. This patch adds a
 completion mechanism inspired by https://github.com/tjim/nevermore,
 which is implemented in Emacs lisp only.

 The core of the new mechanism is the function notmuch-address-harvest
 that collects the completion candidates from the notmuch database and
 stores them in notmuch-address-completions variable.
 notmuch-address-harvest is called on the first entry to message-mode
 and runs asychnornously so that the user doesn't have to wait for it
 to complete while composing the message. The
 notmuch-address-completions variable is used in message-mode as a
 source of completion candidates. Currently, there are two ways how the
 notmuch-address-completions variable is used.

 First, preexisting address completion mechanism is extended to use
 notmuch-address-completions in addition to the external command. This
 new behavior is configured by setting notmuch-address-command to nil,
 which is the new default. Note that this may *BREAK EXISTING SETUPS*
 when the user used external command named notmuch-addresses, i.e.
 the previous default. The result will be that the user will use the
 new mechanism instead of the his command. I believe that many users
 may not even recognize this because the new mechanism works the same
 as http://commonmeasure.org/~jkr/git/notmuch_addresses.git and perhaps
 also as other commands suggested at
 http://notmuchmail.org/emacstips/#address_completion.

 Second way of using notmuch-address-completions is notmuch-company.el.
 This presents the possible completions in a nice popup box after a
 short typing delay but requires company-mode to be installed.

 This looks great, thanks for doing it. It seems like a better approach
 than id:1409921969-65129-1-git-send-email-...@dme.org. Some comments:

 - Adding the address collection to `message-mode-hook' means that it
   runs every time I start to compose a message. If the address
   collection is disk intensive, this might be bad for battery life. 

 The actual harvesting starts only when notmuch-address-completions is
 nil, i.e. when the message-mode is entered for the first time.

Ah, sorry. I didn't read closely enough.

 The set of potential recipients doesn't change _that_ much over time
 for a typical person, I'd wager. Maybe the hook should only run once a
 day? (Tunable, of course.)

 The current version of the patch has a drawback that harvesting is never
 run again. Adding a tunable option for reharvesting might be a good
 idea.

 Since initial harvesting is very slow on non-SSD disk, I want to change
 the implementation so that initially, only addresses matching the
 entered prefix will be harvested, which should be reasonably fast. Then
 full harvest will run on background and once it is finished,
 prefix-based harvesting won't be used anymore.

 Maybe prefix-based harvesting could be then used as a fallback when no
 candidates are found in the data from full harvest. This could also be a
 solution to the reharvest problem.

 I've just returned from vacations so I plan to work on that this week.
 Jani's --output=address patch also looks like something to play with.

Sounds great, thanks.
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


[PATCH v3] Emacs: Add address completion mechanism implemented in elisp

2014-08-17 Thread Michal Sojka
On 11.8.2014 17:31, Michal Sojka wrote:
> Currently, notmuch has an address completion mechanism that requires
> external command to provide completion candidates. This patch adds a
> completion mechanism inspired by https://github.com/tjim/nevermore,
> which is implemented in Emacs lisp only.
>
> The core of the new mechanism is the function notmuch-address-harvest
> that collects the completion candidates from the notmuch database and
> stores them in notmuch-address-completions variable.
> notmuch-address-harvest is called on the first entry to message-mode
> and runs asychnornously so that the user doesn't have to wait for it
> to complete while composing the message. The
> notmuch-address-completions variable is used in message-mode as a
> source of completion candidates. Currently, there are two ways how the
> notmuch-address-completions variable is used.
This patch will need to be improved. I have just performed experiments 
on a system with rotating harddisk and the initial address harvesting 
takes about a minute (on SSD it's 6 seconds). This basically means that 
when writing a first message, address completion does not know about 
most completions candidates.

Probably, first invocation of address harvesting should take into 
account the initial text and search only for addresses matching this 
text. I'll try to implement this later.

-Michal


Re: [PATCH v3] Emacs: Add address completion mechanism implemented in elisp

2014-08-17 Thread Michal Sojka

On 11.8.2014 17:31, Michal Sojka wrote:

Currently, notmuch has an address completion mechanism that requires
external command to provide completion candidates. This patch adds a
completion mechanism inspired by https://github.com/tjim/nevermore,
which is implemented in Emacs lisp only.

The core of the new mechanism is the function notmuch-address-harvest
that collects the completion candidates from the notmuch database and
stores them in notmuch-address-completions variable.
notmuch-address-harvest is called on the first entry to message-mode
and runs asychnornously so that the user doesn't have to wait for it
to complete while composing the message. The
notmuch-address-completions variable is used in message-mode as a
source of completion candidates. Currently, there are two ways how the
notmuch-address-completions variable is used.
This patch will need to be improved. I have just performed experiments 
on a system with rotating harddisk and the initial address harvesting 
takes about a minute (on SSD it's 6 seconds). This basically means that 
when writing a first message, address completion does not know about 
most completions candidates.


Probably, first invocation of address harvesting should take into 
account the initial text and search only for addresses matching this 
text. I'll try to implement this later.


-Michal
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


[PATCH v3] Emacs: Add address completion mechanism implemented in elisp

2014-08-11 Thread Michal Sojka
Currently, notmuch has an address completion mechanism that requires
external command to provide completion candidates. This patch adds a
completion mechanism inspired by https://github.com/tjim/nevermore,
which is implemented in Emacs lisp only.

The core of the new mechanism is the function notmuch-address-harvest
that collects the completion candidates from the notmuch database and
stores them in notmuch-address-completions variable.
notmuch-address-harvest is called on the first entry to message-mode
and runs asychnornously so that the user doesn't have to wait for it
to complete while composing the message. The
notmuch-address-completions variable is used in message-mode as a
source of completion candidates. Currently, there are two ways how the
notmuch-address-completions variable is used.

First, preexisting address completion mechanism is extended to use
notmuch-address-completions in addition to the external command. This
new behavior is configured by setting notmuch-address-command to nil,
which is the new default. Note that this may *BREAK EXISTING SETUPS*
when the user used external command named "notmuch-addresses", i.e.
the previous default. The result will be that the user will use the
new mechanism instead of the his command. I believe that many users
may not even recognize this because the new mechanism works the same
as http://commonmeasure.org/~jkr/git/notmuch_addresses.git and perhaps
also as other commands suggested at
http://notmuchmail.org/emacstips/#address_completion.

Second way of using notmuch-address-completions is notmuch-company.el.
This presents the possible completions in a nice popup box after a
short typing delay but requires company-mode to be installed.

---
Changes from v1:
- Use of notmuch-parser.el instead of the custom parser in the
  original code. The notmuch parser is slightly faster.
- Use of functions in notmuch-query.el instead of functions in the
  original code with almost the same functionality.
- Integrated with existing completion mechanism in notmuch.
- notmuch-company.el was moved from emacs/contrib to emacs and
  no-byte-compile directive was added to it.
- Aligned with notmuch naming conventions.
- Documented bugs found in notmuch-company.el

Changes from v2:
- Updated Makefile.local to not conflict with current master
---
 emacs/Makefile.local |  6 ++-
 emacs/notmuch-address.el | 95 +++-
 emacs/notmuch-company.el | 69 +++
 emacs/notmuch-lib.el |  3 ++
 4 files changed, 163 insertions(+), 10 deletions(-)
 create mode 100644 emacs/notmuch-company.el

diff --git a/emacs/Makefile.local b/emacs/Makefile.local
index 1109cfa..6c93e73 100644
--- a/emacs/Makefile.local
+++ b/emacs/Makefile.local
@@ -20,6 +20,7 @@ emacs_sources := \
$(dir)/notmuch-print.el \
$(dir)/notmuch-version.el \
$(dir)/notmuch-jump.el \
+   $(dir)/notmuch-company.el

 $(dir)/notmuch-version.el: $(dir)/Makefile.local version.stamp
 $(dir)/notmuch-version.el: $(srcdir)/$(dir)/notmuch-version.el.tmpl
@@ -30,7 +31,10 @@ $(dir)/notmuch-version.el: 
$(srcdir)/$(dir)/notmuch-version.el.tmpl
 emacs_images := \
$(srcdir)/$(dir)/notmuch-logo.png

-emacs_bytecode = $(emacs_sources:.el=.elc)
+# Do not try to install files that are not byte-compiled.
+emacs_no_byte_compile := $(dir)/notmuch-company.el
+
+emacs_bytecode = $(patsubst %.el,%.elc,$(filter-out 
$(emacs_no_byte_compile),$(emacs_sources)))

 # Because of defmacro's and defsubst's, we have to account for load
 # dependencies between Elisp files when byte compiling.  Otherwise,
diff --git a/emacs/notmuch-address.el b/emacs/notmuch-address.el
index fa65cd5..a50f4f4 100644
--- a/emacs/notmuch-address.el
+++ b/emacs/notmuch-address.el
@@ -20,14 +20,18 @@
 ;; Authors: David Edmondson 

 (require 'message)
+(require 'notmuch-query)
+(require 'notmuch-parser)

 ;;

-(defcustom notmuch-address-command "notmuch-addresses"
-  "The command which generates possible addresses. It must take a
-single argument and output a list of possible matches, one per
-line."
-  :type 'string
+(defcustom notmuch-address-command nil
+  "The command which generates possible addresses for completion.
+It must take a single argument and output a list of possible
+matches, one per line. If set to nil, addresses are generated by
+a built-in completion mechanism."
+  :type '(radio (const :tag "No command: Use built-in completion" nil)
+(string :tag "Custom command" :value "notmuch-addresses"))
   :group 'notmuch-send
   :group 'notmuch-external)

@@ -42,6 +46,10 @@ to know how address selection is made by default."
   :group 'notmuch-send
   :group 'notmuch-external)

+(defvar notmuch-address-completions nil
+  "Hash of email addresses for completion during email composition.
+  This variable is set by calling `notmuch-address-harvest'.")
+
 (defun notmuch-address-selection-function (prompt collection initial-input)
   "Call