bug report: notmuch-hello 'All tags' view

2010-11-03 Thread Michal Sojka 
On Tue, 02 Nov 2010 20:17:56 +, Darren McGuicken  wrote:
> Wow, that was probably the most secure bug report in history - this time
> in plain for those of you who don't have access to Carl's private key!
> 
> I've noticed since rebasing to 0.4 that I'm seeing an empty entry in the
> 'All tags' view of notmuch-hello which appears to represent a search
> against 'tag:'.  
> 
> For some reason this search matches 1,233 of my now 22,905 messages,
> encompassing a date range of about five years and including an
> apparently random collection of both tagged and untagged threads.
> 
> Anyone else seeing anything similar?  What else do you need from me to
> track this one down?

I can confirm this with 0.4 debian package. notmuch search-tags outputs
first line as empty and emacs shows it in All tags view.

-Michal

notmuch release 0.4 now available

2010-11-03 Thread Scott Henson 
On Tue, Nov 2, 2010 at 5:49 PM, Carl Worth  wrote:
>
> > I'll also likely be submitting it to Fedora proper as I have time.
>  Thanks
> > everyone for all the work!
>
> I didn't mention it, but I've got the notmuch release process pushing
> things directly into Debian. If there's anything I can do to help with
> Fedora packaging I'd be happy to do that too.
>
>
I can have the package created and then add you to the packaging group.  I
figure that will be easier to not require you going through the review
process.



> We do have a notmuch.spec file in the tree already. Is it up to date?
> And I was all setup as a Fedora contributor at one point. I might be
> able to resurrect that. Would it be useful for the same "make release"
> that I run to also do something with Fedora down the road? If someone
> would like to set that up, I'd be happy to have that happen.
>

The spec file in the tree was a good starting place, but I had to modify it
quite a bit to get it to build properly.  I've got one version for building
from git and one version for building a release.  The release one is in the
src rpm I previously listed and I'll send a patch to the list tomorrow for
the git version.

-- 
Scott Henson
-- next part --
An HTML attachment was scrubbed...
URL: 
<http://notmuchmail.org/pipermail/notmuch/attachments/20101103/c6785c23/attachment.htm>

bug report: notmuch-hello 'All tags' view

2010-11-03 Thread Darren McGuicken 
On Tue, 02 Nov 2010 17:33:08 -0400, Jameson Rollins  wrote:
> On Tue, 02 Nov 2010 21:03:55 +, Darren McGuicken  fernseed.info> wrote:
> > A call to 'notmuch search-tags' from the command line does indeed
> > return an empty string as the first entry for me[1].
> 
> fwiw, I don't personally get any empty strings in the output of
> search-tags (with 0.4).  So I don't think it's anything inherent to
> that function.

Thanks for the confirmation Jamie - I had a potter with my data and sure
enough, a search for 'tag:""' returned one hit.  I was able to use the
emacs interface to remove the empty string tag from the matching message
('-' then return with no parameters).

So it's probably fair to say that we'll want to handle at least the
empty string scenario as part of our standard tag addition logic since
it can lead to unexpected side effects in searching.

Do we have any other 'known bad' scenarios in tag addition that we want
to catch before I try my hand at a patch?
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
URL: 
<http://notmuchmail.org/pipermail/notmuch/attachments/20101103/d599ca72/attachment.pgp>

bug report: notmuch-hello 'All tags' view

2010-11-03 Thread Jameson Rollins 
On Wed, 03 Nov 2010 22:57:55 +0100, Michal Sojka  wrote:
> I can confirm this with 0.4 debian package. notmuch search-tags outputs
> first line as empty and emacs shows it in All tags view.

Again, this is *not* what I am seeing.  Michael, are you sure you don't
also have messages tagged with the empty string or blank spaces, as did
Darren?

jamie.
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL: 
<http://notmuchmail.org/pipermail/notmuch/attachments/20101103/a7edb992/attachment.pgp>

Re: notmuch release 0.4 now available

2010-11-03 Thread Scott Henson 
On Tue, Nov 2, 2010 at 5:49 PM, Carl Worth  wrote:
>
> > I'll also likely be submitting it to Fedora proper as I have time.
>  Thanks
> > everyone for all the work!
>
> I didn't mention it, but I've got the notmuch release process pushing
> things directly into Debian. If there's anything I can do to help with
> Fedora packaging I'd be happy to do that too.
>
>
I can have the package created and then add you to the packaging group.  I
figure that will be easier to not require you going through the review
process.



> We do have a notmuch.spec file in the tree already. Is it up to date?
> And I was all setup as a Fedora contributor at one point. I might be
> able to resurrect that. Would it be useful for the same "make release"
> that I run to also do something with Fedora down the road? If someone
> would like to set that up, I'd be happy to have that happen.
>

The spec file in the tree was a good starting place, but I had to modify it
quite a bit to get it to build properly.  I've got one version for building
from git and one version for building a release.  The release one is in the
src rpm I previously listed and I'll send a patch to the list tomorrow for
the git version.

-- 
Scott Henson
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: bug report: notmuch-hello 'All tags' view

2010-11-03 Thread Jameson Rollins 
On Wed, 03 Nov 2010 22:57:55 +0100, Michal Sojka  wrote:
> I can confirm this with 0.4 debian package. notmuch search-tags outputs
> first line as empty and emacs shows it in All tags view.

Again, this is *not* what I am seeing.  Michael, are you sure you don't
also have messages tagged with the empty string or blank spaces, as did
Darren?

jamie.


pgpZs70IubCfy.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: bug report: notmuch-hello 'All tags' view

2010-11-03 Thread Michal Sojka 
On Tue, 02 Nov 2010 20:17:56 +, Darren McGuicken 
 wrote:
> Wow, that was probably the most secure bug report in history - this time
> in plain for those of you who don't have access to Carl's private key!
> 
> I've noticed since rebasing to 0.4 that I'm seeing an empty entry in the
> 'All tags' view of notmuch-hello which appears to represent a search
> against 'tag:'.  
> 
> For some reason this search matches 1,233 of my now 22,905 messages,
> encompassing a date range of about five years and including an
> apparently random collection of both tagged and untagged threads.
> 
> Anyone else seeing anything similar?  What else do you need from me to
> track this one down?

I can confirm this with 0.4 debian package. notmuch search-tags outputs
first line as empty and emacs shows it in All tags view.

-Michal
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH 3/4] Add rules to build notmuch.1 and notmuch-help.h from the pod file.

2010-11-03 Thread David Bremner 
On Wed,  3 Nov 2010 14:18:55 -0300, david at tethera.net wrote:
> +notmuch_help_files= \
> + notmuch-setup-help.h\
> + notmuch-new-help.h  \
> + notmuch-show-help.h \
> + notmuch-search-help.h   \
> + notmuch-count-help.h\
> + notmuch-reply-help.h\
> + notmuch-tag-help.h  \
> + notmuch-dump-help.h \
> + notmuch-restore-help.h  \
> + notmuch-part-help.h
> +

Oops. This bit is leftover from when I tried to do everything in the
Makefile. It is not needed now.

Also I'm sad that deleting a file made a 600 line diff, sorry about
that. Maybe there is  some option to tell it that the patch will be
applied by  git. I didn't see it right away.

d

[PATCH 4/4] notmuch.c: use help strings generated from notmuch.pod.

2010-11-03 Thread da...@tethera.net 
From: David Bremner 

It is quite possible this could be more automagically generated; there
is a certain amount of boilerplate in typing HELP_command_args,
HELP_command_desc, HELP_command_text. But, this way is less error
prone.
---
 notmuch.c |  388 +
 1 files changed, 54 insertions(+), 334 deletions(-)

diff --git a/notmuch.c b/notmuch.c
index 030e494..d1ae366 100644
--- a/notmuch.c
+++ b/notmuch.c
@@ -21,6 +21,7 @@
  */

 #include "notmuch-client.h"
+#include "notmuch-help.h"

 typedef int (*command_function_t) (void *ctx, int argc, char *argv[]);

@@ -36,354 +37,73 @@ static int
 notmuch_help_command (void *ctx, int argc, char *argv[]);

 static const char search_terms_help[] =
-"\tSeveral notmuch commands accept a comman syntax for search\n"
-"\tterms.\n"
-"\n"
-"\tThe search terms can consist of free-form text (and quoted\n"
-"\tphrases) which will match all messages that contain all of\n"
-"\tthe given terms/phrases in the body, the subject, or any of\n"
-"\tthe sender or recipient headers.\n"
-"\n"
-"\tAs a special case, a search string consisting of exactly a\n"
-"\tsingle asterisk (\"*\") will match all messages.\n"
-"\n"
-"\tIn addition to free text, the following prefixes can be used\n"
-"\tto force terms to match against specific portions of an email,\n"
-"\t(where  indicate user-supplied values):\n"
-"\n"
-"\t\tfrom:\n"
-"\t\tto:\n"
-"\t\tsubject:\n"
-"\t\tattachment:\n"
-"\t\ttag: (or is:)\n"
-"\t\tid:\n"
-"\t\tthread:\n"
-"\n"
-"\tThe from: prefix is used to match the name or address of\n"
-"\tthe sender of an email message.\n"
-"\n"
-"\tThe to: prefix is used to match the names or addresses of\n"
-"\tany recipient of an email message, (whether To, Cc, or Bcc).\n"
-"\n"
-"\tAny term prefixed with subject: will match only text from\n"
-"\tthe subject of an email. Quoted phrases are supported when\n"
-"\tsearching with: subject:\"this is a phrase\".\n"
-"\n"
-"\tFor tag: and is:, valid tag values include \"inbox\" and \"unread\"\n"
-"\tby default for new messages added by \"notmuch new\" as well\n"
-"\tas any other tag values added manually with \"notmuch tag\".\n"
-"\n"
-"\tFor id:, message ID values are the literal contents of the\n"
-"\tMessage-ID: header of email messages, but without the '<','>'\n"
-"\tdelimiters.\n"
-"\n"
-"\tThe thread: prefix can be used with the thread ID values that\n"
-"\tare generated internally by notmuch (and do not appear in email\n"
-"\tmessages). These thread ID values can be seen in the first\n"
-"\tcolumn of output from \"notmuch search\".\n"
-"\n"
-"\tIn addition to individual terms, multiple terms can be\n"
-"\tcombined with Boolean operators (\"and\", \"or\", \"not\", etc.).\n"
-"\tEach term in the query will be implicitly connected by a\n"
-"\tlogical AND if no explicit operator is provided, (except\n"
-"\tthat terms with a common prefix will be implicitly combined\n"
-"\twith OR until we get Xapian defect #402 fixed).\n"
-"\n"
-"\tParentheses can also be used to control the combination of\n"
-"\tthe Boolean operators, but will have to be protected from\n"
-"\tinterpretation by the shell, (such as by putting quotation\n"
-"\tmarks around any parenthesized expression).\n"
-"\n"
-"\tFinally, results can be restricted to only messages within a\n"
-"\tparticular time range, (based on the Date: header) with:\n"
-"\n"
-"\t\t..\n"
-"\n"
-"\tEach timestamp is a number representing the number of seconds\n"
-"\tsince 1970-01-01 00:00:00 UTC. This is not the most convenient\n"
-"\tmeans of expressing date ranges, but until notmuch is fixed to\n"
-"\taccept a more convenient form, one can use the date program to\n"
-"\tconstruct timestamps. For example, with the bash shell the\n"
-"\tfollowing syntax would specify a date range to return messages\n"
-"\tfrom 2009-10-01 until the current time:\n"
-"\n"
-"\t\t$(date +%%s -d 2009-10-01)..$(date +%%s)\n\n";
+  HELP_search_syntax_text;

 command_t commands[] = {
 { "setup", notmuch_setup_command,
-  NULL,
-  "Interactively setup notmuch for first use.",
-  "\tThe setup command will prompt for your full name, your primary\n"
-  "\temail address, any alternate email addresses you use, and the\n"
-  "\tdirectory containing your email archives. Your answers will be\n"
-  "\twritten to a configuration file in ${NOTMUCH_CONFIG} (if set)\n"
-  "\tor ${HOME}/.notmuch-config.\n"
-  "\n"
-  "\tThis configuration file will be created with descriptive\n"
-  "\tcomments, making it easy to edit by hand later to change the\n"
-  "\tconfiguration. Or you can run \"notmuch setup\" again.\n"
-  "\n"
-  "\tInvoking notmuch with no command arg

[PATCH 3/4] Add rules to build notmuch.1 and notmuch-help.h from the pod file.

2010-11-03 Thread da...@tethera.net 
From: David Bremner 

Add generated files to CLEAN. Remove notmuch.1 from git because we
auto generate it now.
---
 Makefile.local |   22 ++-
 notmuch.1  |  609 
 2 files changed, 21 insertions(+), 610 deletions(-)
 delete mode 100644 notmuch.1

diff --git a/Makefile.local b/Makefile.local
index 490265b..3e20d0c 100644
--- a/Makefile.local
+++ b/Makefile.local
@@ -251,6 +251,18 @@ notmuch_client_srcs =  \
show-message.c  \
json.c

+notmuch_help_files= \
+   notmuch-setup-help.h\
+   notmuch-new-help.h  \
+   notmuch-show-help.h \
+   notmuch-search-help.h   \
+   notmuch-count-help.h\
+   notmuch-reply-help.h\
+   notmuch-tag-help.h  \
+   notmuch-dump-help.h \
+   notmuch-restore-help.h  \
+   notmuch-part-help.h
+
 notmuch_client_modules = $(notmuch_client_srcs:.c=.o)

 notmuch: $(notmuch_client_modules) lib/libnotmuch.a
@@ -259,6 +271,14 @@ notmuch: $(notmuch_client_modules) lib/libnotmuch.a
 notmuch-shared: $(notmuch_client_modules) lib/$(LINKER_NAME)
$(call quiet,$(FINAL_NOTMUCH_LINKER) $(CFLAGS)) 
$(notmuch_client_modules) $(FINAL_NOTMUCH_LDFLAGS) -o $@

+notmuch.1: notmuch.pod
+   pod2man --stderr --center "User Commands" --release $(VERSION) $^ > $@
+
+notmuch.o: notmuch-help.h
+
+notmuch-help.h: notmuch.pod pod2help_h.pl
+   perl pod2help_h.pl < notmuch.pod > notmuch-help.h
+
 notmuch.1.gz: notmuch.1
gzip --stdout $^ > $@

@@ -294,4 +314,4 @@ install-desktop:
desktop-file-install --mode 0644 --dir $(DESTDIR)$(desktop_dir) 
notmuch.desktop

 SRCS  := $(SRCS) $(notmuch_client_srcs)
-CLEAN := $(CLEAN) notmuch notmuch-shared $(notmuch_client_modules) notmuch.elc 
notmuch.1.gz
+CLEAN := $(CLEAN) notmuch notmuch-shared $(notmuch_client_modules) notmuch.elc 
notmuch.1.gz notmuch.1 notmuch-help.h
diff --git a/notmuch.1 b/notmuch.1
deleted file mode 100644
index 2c33749..000
--- a/notmuch.1
+++ /dev/null
@@ -1,609 +0,0 @@
-.\" notmuch - Not much of an email program, (just index, search and tagging)
-.\"
-.\" Copyright ?? 2009 Carl Worth
-.\"
-.\" Notmuch is free software: you can redistribute it and/or modify
-.\" it under the terms of the GNU General Public License as published by
-.\" the Free Software Foundation, either version 3 of the License, or
-.\" (at your option) any later version.
-.\"
-.\" Notmuch is distributed in the hope that it will be useful,
-.\" but WITHOUT ANY WARRANTY; without even the implied warranty of
-.\" MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-.\" GNU General Public License for more details.
-.\"
-.\" You should have received a copy of the GNU General Public License
-.\" along with this program.  If not, see http://www.gnu.org/licenses/ .
-.\"
-.\" Author: Carl Worth 
-.TH NOTMUCH 1 2009-10-31 "Notmuch 0.1"
-.SH NAME
-notmuch \- thread-based email index, search, and tagging
-.SH SYNOPSIS
-.B notmuch
-.IR command " [" args " ...]"
-.SH DESCRIPTION
-Notmuch is a command-line based program for indexing, searching,
-reading, and tagging large collections of email messages.
-
-The quickest way to get started with Notmuch is to simply invoke the
-.B notmuch
-command with no arguments, which will interactively guide you through
-the process of indexing your mail.
-.SH NOTE
-While the command-line program
-.B notmuch
-provides powerful functionality, it does not provide the most
-convenient interface for that functionality. More sophisticated
-interfaces are expected to be built on top of either the command-line
-interface, or more likely, on top of the notmuch library
-interface. See http://notmuchmail.org for more about alternate
-interfaces to notmuch.
-.SH COMMANDS
-The
-.BR setup
-command is used to configure Notmuch for first use, (or to reconfigure
-it later).
-.RS 4
-.TP 4
-.B setup
-
-Interactively sets up notmuch for first use.
-
-The setup command will prompt for your full name, your primary email
-address, any alternate email addresses you use, and the directory
-containing your email archives. Your answers will be written to a
-configuration file in ${NOTMUCH_CONFIG} (if set) or
-${HOME}/.notmuch-config . This configuration file will be created with
-descriptive comments, making it easy to edit by hand later to change the
-configuration. Or you can run
-.B "notmuch setup"
-again to change the configuration.
-
-The mail directory you specify can contain any number of
-sub-directories and should primarily contain only files with individual
-email messages (eg. maildir or mh archives are perfect). If there are
-other, non-email files (such as indexes maintained by other email
-programs) then notmuch will do its best to detect those and ignore
-them.
-
-Mail storage that uses mbox format, (where one mbox file contains many
-messages), will not work with notmuch. If that's how your mail is
-currently stored, it is recommended you first convert it to maildir
-fo

[PATCH 2/4] Perl script to generate online help.

2010-11-03 Thread da...@tethera.net 
From: David Bremner 

This is a bit more complicated than expected, mainly because it needs
to split a chunk of the docs into 3 pieces corresponding to the
strings currently filled into a struct command_t.

To disable the ANSI escape codes, replace Pod::Text:Color with
Pod::Text. It would not be that much more work to generate colorized
and uncolorized help strings, and choose at runtime.
---
 pod2help_h.pl |  108 +
 1 files changed, 108 insertions(+), 0 deletions(-)
 create mode 100644 pod2help_h.pl

diff --git a/pod2help_h.pl b/pod2help_h.pl
new file mode 100644
index 000..99cfb96
--- /dev/null
+++ b/pod2help_h.pl
@@ -0,0 +1,108 @@
+#!/usr/bin/perl
+
+# Copyright ?? David Bremner, 2010
+# This file is distributed under the same terms as perl, Artistic or GPL 1+.
+#
+# Authors: David Bremner 
+
+use Pod::Select;
+use Pod::Text::Color;
+use strict;
+
+# Create customized version of Pod::Select, that uses Pod::Text to
+# format pieces of the file.
+
+package OurParser;
+our @ISA = qw(Pod::Select);
+
+sub new{
+  my $class = shift;
+  my $self =  { body => "" , name => undef};
+  bless $self, $class;
+  $self->initialize();
+  return $self;
+}
+
+# process verbatim blocks, err, verbatimly. Note that $self->{body} is
+# processed a second time by Pod::Text
+
+sub verbatim {
+  my ($self,$text,$line_num,$pod_para)=@_;
+
+  $self->{body} .= $text;
+}
+
+# process a normal text or command block. Either do some command
+# starting with =for, append to text to be process, or process the
+# current section.  Note that this has the weakness/bug that nested
+# sections are not supported as chunks of text for help.
+sub textblock{
+my ($self,$text,$line_num,$pod_para) = @_;
+
+my $cmd = $pod_para -> cmd_name;
+
+if ($cmd =~ /^head/){
+
+  $self->output() if defined($self->{name});
+
+  # grab name heuristically as the first word.
+  $text =~ m/\s*\w+\s+([\w-]+)/;
+  $self->{name} = $1;
+  $self->{name} =~s/-/_/;
+
+} elsif ($cmd =~ /^for/){
+
+  $text =~ s/^\s+//;
+  $text =~ s/\s+$//;
+
+  # work around a pod bug/misfeature that requires a blank line between
+  # =for paragraphs.
+
+  my @lines=split("\n",$text);
+  foreach my $line (@lines){
+
+   $line =~ s/^=for\s+help\s+//;
+
+   # override name if specified.
+   $self->{name} = $1 if ($line =~ m/^name\s+(.*)$/);
+
+   # other "subcommands" could be supported here.
+   if ($line =~ s/^(args|desc)\s+//) {
+ my $quote="\"";
+ my $subcmd=$1;
+
+ # special case NULL; we don't want "NULL"
+ $quote = "" if ($line =~ /^NULL/);
+
+ print ("#define HELP_$self->{name}_$subcmd $quote$line$quote\n");
+   }
+  }
+} else {
+  $self->{body} .= $text;
+}
+}
+
+sub output {
+  my $self = shift;
+  my $formatter = new Pod::Text::Color(margin=>4);
+  my $output;
+  $formatter -> output_string (\$output);
+  $formatter -> parse_string_document ("=pod\n\n".$self->{body});
+
+  # post-process to turn into a C string.
+  $output =~ s/"/\\"/g;
+  $output =~ s/\n/\\n"\\\n"/g;
+  $output =~ s/\e/\\e/g;
+
+  print "#define HELP_$self->{name}_text \\\n\"$output\"\n";
+  $self->{body} = "";
+
+}
+
+package main;
+
+my $parser =  new OurParser();
+
+$parser->select("Commands/..*","Search Syntax");
+$parser->parse_from_filehandle(\*STDIN);
+$parser->output();
-- 
1.7.1

[PATCH 1/4] notmuch.pod: pod version of documentation, converted by rman, massaged by hand.

2010-11-03 Thread da...@tethera.net 
From: David Bremner 

Some places I deleted a bit of the continuity text introducing a
command because I didn't see how to make it work with the slightly
more structured layout.

I also moved show in front of search, because it explains the output
formats.  Probably it would make sense to add a separate section
explaining common output formats.

The =for help lines are ignored for the man page, but used to generate
the online help.

I also added dummy sections for currently undocumented commands; or
more precisely, commands that were not documented that last time I
refreshed notmuch.pod.
---
 notmuch.pod |  409 +++
 1 files changed, 409 insertions(+), 0 deletions(-)
 create mode 100644 notmuch.pod

diff --git a/notmuch.pod b/notmuch.pod
new file mode 100644
index 000..f4127e0
--- /dev/null
+++ b/notmuch.pod
@@ -0,0 +1,409 @@
+=head1 Name
+
+notmuch - thread-based email index, search, and tagging
+
+=head1 Synopsis
+
+=over
+
+=item B I [I ...]
+
+=back
+
+=head1 Description
+
+Notmuch is a command-line based program for indexing, searching,
+reading, and tagging large collections of email messages.  The
+quickest way to get started with Notmuch is to simply invoke the
+B command with no arguments, which will interactively guide
+you through the process of indexing your mail.
+
+=head2 Using notmuch
+
+The B and B commands are used to query the email
+database.  The B command is the only command available for
+manipulating database contents.  Several of the notmuch commands
+accept search terms with a common syntax. See the B
+section below for more details on the supported syntax.
+
+=head2 Note
+
+While the command-line program B provides powerful
+functionality, it does not provide the most convenient interface for
+that functionality. More sophisticated interfaces are expected to be
+built on top of either the command-line interface, or more likely, on
+top of the notmuch library interface. See L for
+more about alternate interfaces to notmuch.
+
+=head1 Commands
+
+=head2 setup
+
+=for help args NULL
+
+=for help desc Interactively sets up notmuch for first use.
+
+The setup command will prompt for your full name, your primary email
+address, any alternate email addresses you use, and the directory
+containing your email archives. Your answers will be written to a
+configuration file in ${NOTMUCH_CONFIG} (if set) or
+${HOME}/.notmuch-config . This configuration file will be created with
+descriptive comments, making it easy to edit by hand later to change
+the configuration. Or you can run B again to change the
+configuration.
+
+The mail directory you specify can contain any number of
+sub-directories and should primarily contain only files with
+individual email messages (eg. maildir or mh archives are perfect). If
+there are other, non-email files (such as indexes maintained by other
+email programs) then notmuch will do its best to detect those and
+ignore them.
+
+Mail storage that uses mbox format, (where one mbox file contains many
+messages), will not work with notmuch. If that's how your mail is
+currently stored, it is recommended you first convert it to maildir
+format with a utility such as mb2md before running B
+
+Invoking B with no command argument will run B if the
+setup command has not previously been completed.
+
+=head2 new
+
+=for help args NULL
+
+=for help desc Find and import any new messages to the database.
+
+The B command scans all sub-directories of the database,
+performing full-text indexing on new messages that are found. Each new
+message will automatically be tagged with both the B and
+B tags.  You should run B once after first
+running B to create the initial database. The first run
+may take a long time if you have a significant amount of mail (several
+hundred thousand messages or more). Subsequently, you should run
+B whenever new mail is delivered and you wish to
+incorporate it into the database.  These subsequent runs will be much
+quicker than the initial run.
+
+Invoking B with no command argument will run B if
+B has previously been completed, but B has
+not previously been run.
+
+=head2 show [options...] ...
+
+=for help args [options...] ...
+=for help desc Shows all messages matching the search terms.
+
+The messages will be grouped and sorted based on the threading (all
+replies to a particular message will appear immediately after that
+message in date order). The output is not indented by default, but
+depth tags are printed so that proper indentation can be performed by
+a post-processor (such as the emacs interface to notmuch).  Supported
+options for B include
+
+=over
+
+=item B<--entire-thread>
+
+By default only those messages that
+match the search terms will be displayed. With this option, all messages
+in the same thread as any matched message will be displayed.
+
+=item B<--format=(json|text)>
+
+=over
+
+=item B
+
+The default plain-text format has text-cont

generate help from pod.

2010-11-03 Thread da...@tethera.net 
Hi Carl;

Here is the generation. I hope you don't hate perl too much.  I also
hope the syntax in notmuch.pod is self-explanatory.  For me, with perl
5.10.1, this only uses modules included with Perl.

Re: bug report: notmuch-hello 'All tags' view

2010-11-03 Thread Darren McGuicken 
On Tue, 02 Nov 2010 17:33:08 -0400, Jameson Rollins 
 wrote:
> On Tue, 02 Nov 2010 21:03:55 +, Darren McGuicken 
>  wrote:
> > A call to 'notmuch search-tags' from the command line does indeed
> > return an empty string as the first entry for me[1].
> 
> fwiw, I don't personally get any empty strings in the output of
> search-tags (with 0.4).  So I don't think it's anything inherent to
> that function.

Thanks for the confirmation Jamie - I had a potter with my data and sure
enough, a search for 'tag:""' returned one hit.  I was able to use the
emacs interface to remove the empty string tag from the matching message
('-' then return with no parameters).

So it's probably fair to say that we'll want to handle at least the
empty string scenario as part of our standard tag addition logic since
it can lead to unexpected side effects in searching.

Do we have any other 'known bad' scenarios in tag addition that we want
to catch before I try my hand at a patch?


pgp8v0LM0v5sI.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH 3/4] Add rules to build notmuch.1 and notmuch-help.h from the pod file.

2010-11-03 Thread David Bremner 
On Wed,  3 Nov 2010 14:18:55 -0300, da...@tethera.net wrote:
> +notmuch_help_files= \
> + notmuch-setup-help.h\
> + notmuch-new-help.h  \
> + notmuch-show-help.h \
> + notmuch-search-help.h   \
> + notmuch-count-help.h\
> + notmuch-reply-help.h\
> + notmuch-tag-help.h  \
> + notmuch-dump-help.h \
> + notmuch-restore-help.h  \
> + notmuch-part-help.h
> +

Oops. This bit is leftover from when I tried to do everything in the
Makefile. It is not needed now.

Also I'm sad that deleting a file made a 600 line diff, sorry about
that. Maybe there is  some option to tell it that the patch will be
applied by  git. I didn't see it right away.

d
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH 3/4] Add rules to build notmuch.1 and notmuch-help.h from the pod file.

2010-11-03 Thread david 
From: David Bremner 

Add generated files to CLEAN. Remove notmuch.1 from git because we
auto generate it now.
---
 Makefile.local |   22 ++-
 notmuch.1  |  609 
 2 files changed, 21 insertions(+), 610 deletions(-)
 delete mode 100644 notmuch.1

diff --git a/Makefile.local b/Makefile.local
index 490265b..3e20d0c 100644
--- a/Makefile.local
+++ b/Makefile.local
@@ -251,6 +251,18 @@ notmuch_client_srcs =  \
show-message.c  \
json.c
 
+notmuch_help_files= \
+   notmuch-setup-help.h\
+   notmuch-new-help.h  \
+   notmuch-show-help.h \
+   notmuch-search-help.h   \
+   notmuch-count-help.h\
+   notmuch-reply-help.h\
+   notmuch-tag-help.h  \
+   notmuch-dump-help.h \
+   notmuch-restore-help.h  \
+   notmuch-part-help.h
+
 notmuch_client_modules = $(notmuch_client_srcs:.c=.o)
 
 notmuch: $(notmuch_client_modules) lib/libnotmuch.a
@@ -259,6 +271,14 @@ notmuch: $(notmuch_client_modules) lib/libnotmuch.a
 notmuch-shared: $(notmuch_client_modules) lib/$(LINKER_NAME)
$(call quiet,$(FINAL_NOTMUCH_LINKER) $(CFLAGS)) 
$(notmuch_client_modules) $(FINAL_NOTMUCH_LDFLAGS) -o $@
 
+notmuch.1: notmuch.pod
+   pod2man --stderr --center "User Commands" --release $(VERSION) $^ > $@
+
+notmuch.o: notmuch-help.h
+
+notmuch-help.h: notmuch.pod pod2help_h.pl
+   perl pod2help_h.pl < notmuch.pod > notmuch-help.h
+
 notmuch.1.gz: notmuch.1
gzip --stdout $^ > $@
 
@@ -294,4 +314,4 @@ install-desktop:
desktop-file-install --mode 0644 --dir $(DESTDIR)$(desktop_dir) 
notmuch.desktop
 
 SRCS  := $(SRCS) $(notmuch_client_srcs)
-CLEAN := $(CLEAN) notmuch notmuch-shared $(notmuch_client_modules) notmuch.elc 
notmuch.1.gz
+CLEAN := $(CLEAN) notmuch notmuch-shared $(notmuch_client_modules) notmuch.elc 
notmuch.1.gz notmuch.1 notmuch-help.h
diff --git a/notmuch.1 b/notmuch.1
deleted file mode 100644
index 2c33749..000
--- a/notmuch.1
+++ /dev/null
@@ -1,609 +0,0 @@
-.\" notmuch - Not much of an email program, (just index, search and tagging)
-.\"
-.\" Copyright © 2009 Carl Worth
-.\"
-.\" Notmuch is free software: you can redistribute it and/or modify
-.\" it under the terms of the GNU General Public License as published by
-.\" the Free Software Foundation, either version 3 of the License, or
-.\" (at your option) any later version.
-.\"
-.\" Notmuch is distributed in the hope that it will be useful,
-.\" but WITHOUT ANY WARRANTY; without even the implied warranty of
-.\" MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-.\" GNU General Public License for more details.
-.\"
-.\" You should have received a copy of the GNU General Public License
-.\" along with this program.  If not, see http://www.gnu.org/licenses/ .
-.\"
-.\" Author: Carl Worth 
-.TH NOTMUCH 1 2009-10-31 "Notmuch 0.1"
-.SH NAME
-notmuch \- thread-based email index, search, and tagging
-.SH SYNOPSIS
-.B notmuch
-.IR command " [" args " ...]"
-.SH DESCRIPTION
-Notmuch is a command-line based program for indexing, searching,
-reading, and tagging large collections of email messages.
-
-The quickest way to get started with Notmuch is to simply invoke the
-.B notmuch
-command with no arguments, which will interactively guide you through
-the process of indexing your mail.
-.SH NOTE
-While the command-line program
-.B notmuch
-provides powerful functionality, it does not provide the most
-convenient interface for that functionality. More sophisticated
-interfaces are expected to be built on top of either the command-line
-interface, or more likely, on top of the notmuch library
-interface. See http://notmuchmail.org for more about alternate
-interfaces to notmuch.
-.SH COMMANDS
-The
-.BR setup
-command is used to configure Notmuch for first use, (or to reconfigure
-it later).
-.RS 4
-.TP 4
-.B setup
-
-Interactively sets up notmuch for first use.
-
-The setup command will prompt for your full name, your primary email
-address, any alternate email addresses you use, and the directory
-containing your email archives. Your answers will be written to a
-configuration file in ${NOTMUCH_CONFIG} (if set) or
-${HOME}/.notmuch-config . This configuration file will be created with
-descriptive comments, making it easy to edit by hand later to change the
-configuration. Or you can run
-.B "notmuch setup"
-again to change the configuration.
-
-The mail directory you specify can contain any number of
-sub-directories and should primarily contain only files with individual
-email messages (eg. maildir or mh archives are perfect). If there are
-other, non-email files (such as indexes maintained by other email
-programs) then notmuch will do its best to detect those and ignore
-them.
-
-Mail storage that uses mbox format, (where one mbox file contains many
-messages), will not work with notmuch. If that's how your mail is
-currently stored, it is recommended you first convert it to maildi

[PATCH 4/4] notmuch.c: use help strings generated from notmuch.pod.

2010-11-03 Thread david 
From: David Bremner 

It is quite possible this could be more automagically generated; there
is a certain amount of boilerplate in typing HELP_command_args,
HELP_command_desc, HELP_command_text. But, this way is less error
prone.
---
 notmuch.c |  388 +
 1 files changed, 54 insertions(+), 334 deletions(-)

diff --git a/notmuch.c b/notmuch.c
index 030e494..d1ae366 100644
--- a/notmuch.c
+++ b/notmuch.c
@@ -21,6 +21,7 @@
  */
 
 #include "notmuch-client.h"
+#include "notmuch-help.h"
 
 typedef int (*command_function_t) (void *ctx, int argc, char *argv[]);
 
@@ -36,354 +37,73 @@ static int
 notmuch_help_command (void *ctx, int argc, char *argv[]);
 
 static const char search_terms_help[] =
-"\tSeveral notmuch commands accept a comman syntax for search\n"
-"\tterms.\n"
-"\n"
-"\tThe search terms can consist of free-form text (and quoted\n"
-"\tphrases) which will match all messages that contain all of\n"
-"\tthe given terms/phrases in the body, the subject, or any of\n"
-"\tthe sender or recipient headers.\n"
-"\n"
-"\tAs a special case, a search string consisting of exactly a\n"
-"\tsingle asterisk (\"*\") will match all messages.\n"
-"\n"
-"\tIn addition to free text, the following prefixes can be used\n"
-"\tto force terms to match against specific portions of an email,\n"
-"\t(where  indicate user-supplied values):\n"
-"\n"
-"\t\tfrom:\n"
-"\t\tto:\n"
-"\t\tsubject:\n"
-"\t\tattachment:\n"
-"\t\ttag: (or is:)\n"
-"\t\tid:\n"
-"\t\tthread:\n"
-"\n"
-"\tThe from: prefix is used to match the name or address of\n"
-"\tthe sender of an email message.\n"
-"\n"
-"\tThe to: prefix is used to match the names or addresses of\n"
-"\tany recipient of an email message, (whether To, Cc, or Bcc).\n"
-"\n"
-"\tAny term prefixed with subject: will match only text from\n"
-"\tthe subject of an email. Quoted phrases are supported when\n"
-"\tsearching with: subject:\"this is a phrase\".\n"
-"\n"
-"\tFor tag: and is:, valid tag values include \"inbox\" and \"unread\"\n"
-"\tby default for new messages added by \"notmuch new\" as well\n"
-"\tas any other tag values added manually with \"notmuch tag\".\n"
-"\n"
-"\tFor id:, message ID values are the literal contents of the\n"
-"\tMessage-ID: header of email messages, but without the '<','>'\n"
-"\tdelimiters.\n"
-"\n"
-"\tThe thread: prefix can be used with the thread ID values that\n"
-"\tare generated internally by notmuch (and do not appear in email\n"
-"\tmessages). These thread ID values can be seen in the first\n"
-"\tcolumn of output from \"notmuch search\".\n"
-"\n"
-"\tIn addition to individual terms, multiple terms can be\n"
-"\tcombined with Boolean operators (\"and\", \"or\", \"not\", etc.).\n"
-"\tEach term in the query will be implicitly connected by a\n"
-"\tlogical AND if no explicit operator is provided, (except\n"
-"\tthat terms with a common prefix will be implicitly combined\n"
-"\twith OR until we get Xapian defect #402 fixed).\n"
-"\n"
-"\tParentheses can also be used to control the combination of\n"
-"\tthe Boolean operators, but will have to be protected from\n"
-"\tinterpretation by the shell, (such as by putting quotation\n"
-"\tmarks around any parenthesized expression).\n"
-"\n"
-"\tFinally, results can be restricted to only messages within a\n"
-"\tparticular time range, (based on the Date: header) with:\n"
-"\n"
-"\t\t..\n"
-"\n"
-"\tEach timestamp is a number representing the number of seconds\n"
-"\tsince 1970-01-01 00:00:00 UTC. This is not the most convenient\n"
-"\tmeans of expressing date ranges, but until notmuch is fixed to\n"
-"\taccept a more convenient form, one can use the date program to\n"
-"\tconstruct timestamps. For example, with the bash shell the\n"
-"\tfollowing syntax would specify a date range to return messages\n"
-"\tfrom 2009-10-01 until the current time:\n"
-"\n"
-"\t\t$(date +%%s -d 2009-10-01)..$(date +%%s)\n\n";
+  HELP_search_syntax_text;
 
 command_t commands[] = {
 { "setup", notmuch_setup_command,
-  NULL,
-  "Interactively setup notmuch for first use.",
-  "\tThe setup command will prompt for your full name, your primary\n"
-  "\temail address, any alternate email addresses you use, and the\n"
-  "\tdirectory containing your email archives. Your answers will be\n"
-  "\twritten to a configuration file in ${NOTMUCH_CONFIG} (if set)\n"
-  "\tor ${HOME}/.notmuch-config.\n"
-  "\n"
-  "\tThis configuration file will be created with descriptive\n"
-  "\tcomments, making it easy to edit by hand later to change the\n"
-  "\tconfiguration. Or you can run \"notmuch setup\" again.\n"
-  "\n"
-  "\tInvoking notmuch with no comman

[PATCH 1/4] notmuch.pod: pod version of documentation, converted by rman, massaged by hand.

2010-11-03 Thread david 
From: David Bremner 

Some places I deleted a bit of the continuity text introducing a
command because I didn't see how to make it work with the slightly
more structured layout.

I also moved show in front of search, because it explains the output
formats.  Probably it would make sense to add a separate section
explaining common output formats.

The =for help lines are ignored for the man page, but used to generate
the online help.

I also added dummy sections for currently undocumented commands; or
more precisely, commands that were not documented that last time I
refreshed notmuch.pod.
---
 notmuch.pod |  409 +++
 1 files changed, 409 insertions(+), 0 deletions(-)
 create mode 100644 notmuch.pod

diff --git a/notmuch.pod b/notmuch.pod
new file mode 100644
index 000..f4127e0
--- /dev/null
+++ b/notmuch.pod
@@ -0,0 +1,409 @@
+=head1 Name
+
+notmuch - thread-based email index, search, and tagging
+
+=head1 Synopsis
+
+=over
+
+=item B I [I ...]
+
+=back
+
+=head1 Description
+
+Notmuch is a command-line based program for indexing, searching,
+reading, and tagging large collections of email messages.  The
+quickest way to get started with Notmuch is to simply invoke the
+B command with no arguments, which will interactively guide
+you through the process of indexing your mail.
+
+=head2 Using notmuch
+
+The B and B commands are used to query the email
+database.  The B command is the only command available for
+manipulating database contents.  Several of the notmuch commands
+accept search terms with a common syntax. See the B
+section below for more details on the supported syntax.
+
+=head2 Note
+
+While the command-line program B provides powerful
+functionality, it does not provide the most convenient interface for
+that functionality. More sophisticated interfaces are expected to be
+built on top of either the command-line interface, or more likely, on
+top of the notmuch library interface. See L for
+more about alternate interfaces to notmuch.
+
+=head1 Commands
+
+=head2 setup
+
+=for help args NULL
+
+=for help desc Interactively sets up notmuch for first use.
+
+The setup command will prompt for your full name, your primary email
+address, any alternate email addresses you use, and the directory
+containing your email archives. Your answers will be written to a
+configuration file in ${NOTMUCH_CONFIG} (if set) or
+${HOME}/.notmuch-config . This configuration file will be created with
+descriptive comments, making it easy to edit by hand later to change
+the configuration. Or you can run B again to change the
+configuration.
+
+The mail directory you specify can contain any number of
+sub-directories and should primarily contain only files with
+individual email messages (eg. maildir or mh archives are perfect). If
+there are other, non-email files (such as indexes maintained by other
+email programs) then notmuch will do its best to detect those and
+ignore them.
+
+Mail storage that uses mbox format, (where one mbox file contains many
+messages), will not work with notmuch. If that's how your mail is
+currently stored, it is recommended you first convert it to maildir
+format with a utility such as mb2md before running B
+
+Invoking B with no command argument will run B if the
+setup command has not previously been completed.
+
+=head2 new
+
+=for help args NULL
+
+=for help desc Find and import any new messages to the database.
+
+The B command scans all sub-directories of the database,
+performing full-text indexing on new messages that are found. Each new
+message will automatically be tagged with both the B and
+B tags.  You should run B once after first
+running B to create the initial database. The first run
+may take a long time if you have a significant amount of mail (several
+hundred thousand messages or more). Subsequently, you should run
+B whenever new mail is delivered and you wish to
+incorporate it into the database.  These subsequent runs will be much
+quicker than the initial run.
+
+Invoking B with no command argument will run B if
+B has previously been completed, but B has
+not previously been run.
+
+=head2 show [options...] ...
+
+=for help args [options...] ...
+=for help desc Shows all messages matching the search terms.
+
+The messages will be grouped and sorted based on the threading (all
+replies to a particular message will appear immediately after that
+message in date order). The output is not indented by default, but
+depth tags are printed so that proper indentation can be performed by
+a post-processor (such as the emacs interface to notmuch).  Supported
+options for B include
+
+=over
+
+=item B<--entire-thread>
+
+By default only those messages that
+match the search terms will be displayed. With this option, all messages
+in the same thread as any matched message will be displayed.
+
+=item B<--format=(json|text)>
+
+=over
+
+=item B
+
+The default plain-text format has text-cont

[PATCH 2/4] Perl script to generate online help.

2010-11-03 Thread david 
From: David Bremner 

This is a bit more complicated than expected, mainly because it needs
to split a chunk of the docs into 3 pieces corresponding to the
strings currently filled into a struct command_t.

To disable the ANSI escape codes, replace Pod::Text:Color with
Pod::Text. It would not be that much more work to generate colorized
and uncolorized help strings, and choose at runtime.
---
 pod2help_h.pl |  108 +
 1 files changed, 108 insertions(+), 0 deletions(-)
 create mode 100644 pod2help_h.pl

diff --git a/pod2help_h.pl b/pod2help_h.pl
new file mode 100644
index 000..99cfb96
--- /dev/null
+++ b/pod2help_h.pl
@@ -0,0 +1,108 @@
+#!/usr/bin/perl
+
+# Copyright © David Bremner, 2010
+# This file is distributed under the same terms as perl, Artistic or GPL 1+.
+#
+# Authors: David Bremner 
+
+use Pod::Select;
+use Pod::Text::Color;
+use strict;
+
+# Create customized version of Pod::Select, that uses Pod::Text to
+# format pieces of the file.
+
+package OurParser;
+our @ISA = qw(Pod::Select);
+
+sub new{
+  my $class = shift;
+  my $self =  { body => "" , name => undef};
+  bless $self, $class;
+  $self->initialize();
+  return $self;
+}
+
+# process verbatim blocks, err, verbatimly. Note that $self->{body} is
+# processed a second time by Pod::Text
+
+sub verbatim {
+  my ($self,$text,$line_num,$pod_para)=...@_;
+
+  $self->{body} .= $text;
+}
+
+# process a normal text or command block. Either do some command
+# starting with =for, append to text to be process, or process the
+# current section.  Note that this has the weakness/bug that nested
+# sections are not supported as chunks of text for help.
+sub textblock{
+my ($self,$text,$line_num,$pod_para) = @_;
+
+my $cmd = $pod_para -> cmd_name;
+
+if ($cmd =~ /^head/){
+
+  $self->output() if defined($self->{name});
+
+  # grab name heuristically as the first word.
+  $text =~ m/\s*\w+\s+([\w-]+)/;
+  $self->{name} = $1;
+  $self->{name} =~s/-/_/;
+
+} elsif ($cmd =~ /^for/){
+
+  $text =~ s/^\s+//;
+  $text =~ s/\s+$//;
+
+  # work around a pod bug/misfeature that requires a blank line between
+  # =for paragraphs.
+
+  my @lines=split("\n",$text);
+  foreach my $line (@lines){
+
+   $line =~ s/^=for\s+help\s+//;
+
+   # override name if specified.
+   $self->{name} = $1 if ($line =~ m/^name\s+(.*)$/);
+
+   # other "subcommands" could be supported here.
+   if ($line =~ s/^(args|desc)\s+//) {
+ my $quote="\"";
+ my $subcmd=$1;
+
+ # special case NULL; we don't want "NULL"
+ $quote = "" if ($line =~ /^NULL/);
+
+ print ("#define HELP_$self->{name}_$subcmd $quote$line$quote\n");
+   }
+  }
+} else {
+  $self->{body} .= $text;
+}
+}
+
+sub output {
+  my $self = shift;
+  my $formatter = new Pod::Text::Color(margin=>4);
+  my $output;
+  $formatter -> output_string (\$output);
+  $formatter -> parse_string_document ("=pod\n\n".$self->{body});
+
+  # post-process to turn into a C string.
+  $output =~ s/"/\\"/g;
+  $output =~ s/\n/\\n"\\\n"/g;
+  $output =~ s/\e/\\e/g;
+
+  print "#define HELP_$self->{name}_text \\\n\"$output\"\n";
+  $self->{body} = "";
+
+}
+
+package main;
+
+my $parser =  new OurParser();
+
+$parser->select("Commands/..*","Search Syntax");
+$parser->parse_from_filehandle(\*STDIN);
+$parser->output();
-- 
1.7.1

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

generate help from pod.

2010-11-03 Thread david 
Hi Carl;

Here is the generation. I hope you don't hate perl too much.  I also
hope the syntax in notmuch.pod is self-explanatory.  For me, with perl
5.10.1, this only uses modules included with Perl.

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch