date:20110810

not much reply multipart/mixed

2011-08-10 Thread Brian May

Hello,

When I reply to a given email, instead of quoting the original text, I see:

=== cut ===
On Tue, 9 Aug 2011 01:39:30 -0600, tivoli_support at ecurep.ibm.com wrote:
Non-text part: multipart/mixed
Non-text part: text/html
=== cut ===

This is with the latest git master version.

Thanks
-- 
Brian May

Added messages / total files count difference.

2011-08-10 Thread Tomi Ollila

On Tue 09 Aug 2011 14:02, Tomi Ollila  writes:

> Hi
>
> I get this output:
>
> $ notmuch new --verbose
> Found 15559 total files (that's not much mail).
> Processed 15559 total files in 5m 53s (43 files/sec.).
> Added 15546 new messages to the database.
>
> $ find * -type f | wc
>   15559   15559  529027
>
> How can I determine which 13 files were dropped. All of those
> 15559 files should be mails. I tried to check through mail files that
> have no 'Subject:' header but those were (at least one) indexed. Could
> it be about duplicate Message-ID: or something ?
>
> $ notmuch --version
> notmuch 0.7-7-g68e8560

It is about duplicate Message-ID:s

It would be nice that 'notmuch new' printes information about this
if this were to happen (as I recall it does when new file found
is not (considered as) a mail file).

The steps I took to figure this out (not all iterations with & without
'wc':s shown) at the end of this email.

>
> Tomi

Tomi

--8<8<8<8<8<8<8<8<8<8<--

$ find ~/mail/mails/* -type f | sort >! filenames-fs
$ wc filenames-fs 
 15559  15559 855766 filenames-fs

$ cd /path/to/notmuch-git/bindings/python
$ cat > foo.py
import notmuch
db = notmuch.Database()
msgs = notmuch.Query(db,'').search_messages()

for f in msgs:
print f.get_filename()

$ PYTHONPATH=/path/to/python-json:`pwd` python foo.py | sort > filenames-db
$ wc filenames-db
 15546  15546 855037 filenames-db

$ diff filenames-db filenames-fs | grep mails | wc
 13  26 755

$ cd ~/mail
$ cat >midcheck.pl
use strict;
use warnings;

my %msgids;

foreach () {
my $fn = $_;
my $mid;
open I, '<', $fn or die $!;
while () {
$mid = $1, next if /^Message-ID:\s*(.*)/i;
last if /^$/;
}
close I;
unless ($mid) {
print "$fn: no Message-ID (in same line with header tag?)\n";
next;
}
my $fn0 = $msgids{$mid};
if (defined $fn0) {
print "Files '$fn0' and '$fn' have same msg id: $mid\n";
}
else {
$msgids{$mid} = $fn;
}
}

$ perl midcheck.pl | wc
 13 1172098
$ perl midcheck.pl | grep \^Files | wc
 13 1172098

[RFC PATCH 3/3] lib: add gnulib get_date() based date range search

2011-08-10 Thread Jani Nikula

Add a custom value range processor to handle "date:" using get_date() from
gnulib. This enables date (and time) searches of the form
date:since..until, where "since" and "until" are expressions understood by
get_date(), compatible with most GNU programs. For the date input formats,
see the GNU coreutils manual:
http://www.gnu.org/software/coreutils/manual/html_node/Date-input-formats.html

Open-ended ranges are supported (since Xapian 1.2.1), i.e. you can specify
date:..until or date:since.. to not limit the start or end date,
respectively.

Note: The get_date() function has been renamed to parse_datetime() in
recent gnulib.

EXAMPLES:

date:2-weeks-ago..
date:today00:00:00..
date:yesterday00:00:00..today00:00:00
date:07/14/2011..2011-07-15

BUGS/CAVEATS:

At the moment it seems search phrases with spaces in them are not supported
in notmuch. For date: this means you can't specify a date with an
expression with spaces in it. In many (but apparently not all) cases you
can work around this by replacing spaces with '-' or simply leaving out
whitespace. For example, date:2-days-ago..yesterday00:00:00.

For the purpose of searching mail, the get_date() implementation has some
surprising interpretations. For example:

* Any date specification without time, such as date:yesterday.. or
  date:2011-08-10.., means that day at the same time as now (instead of
  00:00:00).

* As a consequence, ranges such as date:yesterday..yesterday match just the
  messages at the same time as now, to the second. The date parser should
  optimally make different kind of interpretations depending on whether
  parsing "since" or "until". For example, date:today..today should cover
  the whole day from beginning to end, date:today00:00:00..today23:59:59.

* date:monday.. means date:next-monday.. (rather than date:last-monday..)

* date:this-week.. really means date:now.. (you probably want
  date:1-week-ago-next-monday00:00:00.. or similar).

However, there is value in being compatible with GNU programs, and the
input formats have been rather well documented. It would be more surprising
to deviate from that, and it would also take some effort to do so,
including testing.
---
 lib/Makefile.local |1 +
 lib/database-private.h |1 +
 lib/database.cc|4 
 lib/getdate-proc.cc|   32 
 lib/getdate-proc.h |   21 +
 5 files changed, 59 insertions(+), 0 deletions(-)
 create mode 100644 lib/getdate-proc.cc
 create mode 100644 lib/getdate-proc.h

diff --git a/lib/Makefile.local b/lib/Makefile.local
index a1c234f..b722343 100644
--- a/lib/Makefile.local
+++ b/lib/Makefile.local
@@ -63,6 +63,7 @@ libnotmuch_c_srcs =   \

 libnotmuch_cxx_srcs =  \
$(dir)/database.cc  \
+   $(dir)/getdate-proc.cc  \
$(dir)/directory.cc \
$(dir)/index.cc \
$(dir)/message.cc   \
diff --git a/lib/database-private.h b/lib/database-private.h
index f705009..d83ae3b 100644
--- a/lib/database-private.h
+++ b/lib/database-private.h
@@ -51,6 +51,7 @@ struct _notmuch_database {
 Xapian::QueryParser *query_parser;
 Xapian::TermGenerator *term_gen;
 Xapian::ValueRangeProcessor *value_range_processor;
+Xapian::ValueRangeProcessor *getdate_proc;
 };

 /* Return the list of terms from the given iterator matching a prefix.
diff --git a/lib/database.cc b/lib/database.cc
index 9c2f4ec..b1a0732 100644
--- a/lib/database.cc
+++ b/lib/database.cc
@@ -19,6 +19,7 @@
  */

 #include "database-private.h"
+#include "getdate-proc.h"

 #include 

@@ -667,12 +668,14 @@ notmuch_database_open (const char *path,
notmuch->term_gen = new Xapian::TermGenerator;
notmuch->term_gen->set_stemmer (Xapian::Stem ("english"));
notmuch->value_range_processor = new Xapian::NumberValueRangeProcessor 
(NOTMUCH_VALUE_TIMESTAMP);
+   notmuch->getdate_proc = new GetDateValueRangeProcessor 
(NOTMUCH_VALUE_TIMESTAMP, "date:", true);

notmuch->query_parser->set_default_op (Xapian::Query::OP_AND);
notmuch->query_parser->set_database (*notmuch->xapian_db);
notmuch->query_parser->set_stemmer (Xapian::Stem ("english"));
notmuch->query_parser->set_stemming_strategy 
(Xapian::QueryParser::STEM_SOME);
notmuch->query_parser->add_valuerangeprocessor 
(notmuch->value_range_processor);
+   notmuch->query_parser->add_valuerangeprocessor (notmuch->getdate_proc);

for (i = 0; i < ARRAY_SIZE (BOOLEAN_PREFIX_EXTERNAL); i++) {
prefix_t *prefix = _PREFIX_EXTERNAL[i];
@@ -716,6 +719,7 @@ notmuch_database_close (notmuch_database_t *notmuch)
 delete notmuch->query_parser;
 delete notmuch->xapian_db;
 delete notmuch->value_range_processor;
+delete notmuch->getdate_proc;
 talloc_free (notmuch);
 }

diff --git a/lib/getdate-proc.cc b/lib/getdate-proc.cc
new file mode 100644
index 000..c384e5a
--- /dev/null
+++ b/lib/getdate-proc.cc
@@ -0,0 +1,32 @@
+
+#include

[RFC PATCH 1/3] Import date/time parser from GNU coreutils

2011-08-10 Thread Jani Nikula

From: Michal Sojka 

This function have quite a lot dependencies. We may reduce them later it
it is a problem.
---
 lib/c-ctype.c  |  398 +++
 lib/c-ctype.h  |  297 +
 lib/getdate.c  | 3497 
 lib/getdate.h  |   22 +
 lib/getdate.y  | 1572 +
 lib/gettime.c  |   48 +
 lib/intprops.h |   83 ++
 lib/timespec.h |   39 +
 lib/verify.h   |  140 +++
 9 files changed, 6096 insertions(+), 0 deletions(-)
 create mode 100644 lib/c-ctype.c
 create mode 100644 lib/c-ctype.h
 create mode 100644 lib/getdate.c
 create mode 100644 lib/getdate.h
 create mode 100644 lib/getdate.y
 create mode 100644 lib/gettime.c
 create mode 100644 lib/gettime.h
 create mode 100644 lib/intprops.h
 create mode 100644 lib/timespec.h
 create mode 100644 lib/verify.h

diff --git a/lib/c-ctype.c b/lib/c-ctype.c
new file mode 100644
index 000..48baa72
--- /dev/null
+++ b/lib/c-ctype.c
@@ -0,0 +1,398 @@
+/* -*- buffer-read-only: t -*- vi: set ro: */
+/* DO NOT EDIT! GENERATED AUTOMATICALLY! */
+/* Character handling in C locale.
+
+   Copyright 2000-2003, 2006, 2009-2010 Free Software Foundation, Inc.
+
+This program is free software; you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation; either version 3 of the License, or
+(at your option) any later version.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program; if not, write to the Free Software Foundation,
+Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.  */
+
+#include 
+
+/* Specification.  */
+#define NO_C_CTYPE_MACROS
+#include "c-ctype.h"
+
+/* The function isascii is not locale dependent. Its use in EBCDIC is
+   questionable. */
+bool
+c_isascii (int c)
+{
+  return (c >= 0x00 && c <= 0x7f);
+}
+
+bool
+c_isalnum (int c)
+{
+#if C_CTYPE_CONSECUTIVE_DIGITS \
+&& C_CTYPE_CONSECUTIVE_UPPERCASE && C_CTYPE_CONSECUTIVE_LOWERCASE
+#if C_CTYPE_ASCII
+  return ((c >= '0' && c <= '9')
+  || ((c & ~0x20) >= 'A' && (c & ~0x20) <= 'Z'));
+#else
+  return ((c >= '0' && c <= '9')
+  || (c >= 'A' && c <= 'Z')
+  || (c >= 'a' && c <= 'z'));
+#endif
+#else
+  switch (c)
+{
+case '0': case '1': case '2': case '3': case '4': case '5':
+case '6': case '7': case '8': case '9':
+case 'A': case 'B': case 'C': case 'D': case 'E': case 'F':
+case 'G': case 'H': case 'I': case 'J': case 'K': case 'L':
+case 'M': case 'N': case 'O': case 'P': case 'Q': case 'R':
+case 'S': case 'T': case 'U': case 'V': case 'W': case 'X':
+case 'Y': case 'Z':
+case 'a': case 'b': case 'c': case 'd': case 'e': case 'f':
+case 'g': case 'h': case 'i': case 'j': case 'k': case 'l':
+case 'm': case 'n': case 'o': case 'p': case 'q': case 'r':
+case 's': case 't': case 'u': case 'v': case 'w': case 'x':
+case 'y': case 'z':
+  return 1;
+default:
+  return 0;
+}
+#endif
+}
+
+bool
+c_isalpha (int c)
+{
+#if C_CTYPE_CONSECUTIVE_UPPERCASE && C_CTYPE_CONSECUTIVE_LOWERCASE
+#if C_CTYPE_ASCII
+  return ((c & ~0x20) >= 'A' && (c & ~0x20) <= 'Z');
+#else
+  return ((c >= 'A' && c <= 'Z') || (c >= 'a' && c <= 'z'));
+#endif
+#else
+  switch (c)
+{
+case 'A': case 'B': case 'C': case 'D': case 'E': case 'F':
+case 'G': case 'H': case 'I': case 'J': case 'K': case 'L':
+case 'M': case 'N': case 'O': case 'P': case 'Q': case 'R':
+case 'S': case 'T': case 'U': case 'V': case 'W': case 'X':
+case 'Y': case 'Z':
+case 'a': case 'b': case 'c': case 'd': case 'e': case 'f':
+case 'g': case 'h': case 'i': case 'j': case 'k': case 'l':
+case 'm': case 'n': case 'o': case 'p': case 'q': case 'r':
+case 's': case 't': case 'u': case 'v': case 'w': case 'x':
+case 'y': case 'z':
+  return 1;
+default:
+  return 0;
+}
+#endif
+}
+
+bool
+c_isblank (int c)
+{
+  return (c == ' ' || c == '\t');
+}
+
+bool
+c_iscntrl (int c)
+{
+#if C_CTYPE_ASCII
+  return ((c & ~0x1f) == 0 || c == 0x7f);
+#else
+  switch (c)
+{
+case ' ': case '!': case '"': case '#': case '$': case '%':
+case '&': case '\'': case '(': case ')': case '*': case '+':
+case ',': case '-': case '.': case '/':
+case '0': case '1': case '2': case '3': case '4': case '5':
+case '6': case '7': case '8': case '9':
+case ':': case ';': case '<': case '=': case '>': case '?':
+case '@':
+case 'A': case 'B': case 'C': case 'D': case 'E': case 'F':
+case 'G': case 'H': case 'I': case 'J': case 'K': case 'L':
+case 'M': case 'N': case 'O': case 'P': case 'Q': case 'R':
+case 'S': case 'T': case 'U': case 'V': case 'W':

[RFC PATCH 0/3] add user friendly date range search

2011-08-10 Thread Jani Nikula

Hi, this RFC series adds user friendly date range searches in notmuch.

Patches 1 and 2 add the date/time parser from coreutils, and were posted to
the list by Michal Sojka in January [1]. I picked them up from the
date-parser branch of the git tree at [2]. These worked without
modifications, so I didn't touch them.

Patch 3 adds the actual date range searches on top of current
master. Michal also posted a date search implementation, but it was based
on Austin Clements' custom parser patches, which I haven't looked at. Also,
this one uses date: range syntax rather than before: and after:, and I
think Michal would agree on this one [3].

If the same parser is to be eventually included in notmuch, I'd prefer
using the parse-datetime module [3] (get_date() was renamed) directly from
gnulib [4], instead of coreutils or elsewhere that just imports from
gnulib. Also, I think the gnulib module should be placed in a subdirectory
rather than alongside all the notmuch lib files. However there are some
complications in using parse-datetime, as gnulib modules heavily rely on
autotools. Perhaps one option would be to build a separate library with
autotools that has what we need from gnulib?

Anyway, this is here now, and works (well, with caveats outlined in the
commit message for patch 3) so that people can try this and get a feel of
the date range search. I wouldn't put much efforts to cleanly importing
gnulib before there's some guarantee that the syntax and semantics are
desired in the first place.

Personally, I've been happy with it ever since I got used to the semantics
of the parser.


BR,
Jani.


[1] http://notmuchmail.org/pipermail/notmuch/2011/003749.html
[2] git://rtime.felk.cvut.cz/notmuch.git
[3] http://notmuchmail.org/pipermail/notmuch/2011/004096.html
[4] http://www.gnu.org/s/gnulib/MODULES.html#module=parse-datetime
[3] http://www.gnu.org/s/gnulib/


Jani Nikula (1):
  lib: add gnulib get_date() based date range search

Michal Sojka (2):
  Import date/time parser from GNU coreutils
  Compile the date/time parser into notmuch library

 Makefile.local |3 +
 configure  |8 +
 lib/Makefile.local |6 +-
 lib/c-ctype.c  |  398 ++
 lib/c-ctype.h  |  297 
 lib/config.h   |   45 +
 lib/database-private.h |1 +
 lib/database.cc|4 +
 lib/getdate-proc.cc|   32 +
 lib/getdate-proc.h |   21 +
 lib/getdate.c  | 3506 
 lib/getdate.h  |   31 +
 lib/getdate.y  | 1581 ++
 lib/gettime.c  |   48 +
 lib/intprops.h |   83 ++
 lib/timespec.h |   39 +
 lib/verify.h   |  140 ++
 17 files changed, 6242 insertions(+), 1 deletions(-)
 create mode 100644 lib/c-ctype.c
 create mode 100644 lib/c-ctype.h
 create mode 100644 lib/config.h
 create mode 100644 lib/getdate-proc.cc
 create mode 100644 lib/getdate-proc.h
 create mode 100644 lib/getdate.c
 create mode 100644 lib/getdate.h
 create mode 100644 lib/getdate.y
 create mode 100644 lib/gettime.c
 create mode 100644 lib/gettime.h
 create mode 100644 lib/intprops.h
 create mode 100644 lib/timespec.h
 create mode 100644 lib/verify.h

not much reply multipart/mixed

2011-08-10 Thread Jameson Graef Rollins

On Wed, 10 Aug 2011 12:07:00 +1000, Brian May  wrote:
> When I reply to a given email, instead of quoting the original text, I see:
> 
> === cut ===
> On Tue, 9 Aug 2011 01:39:30 -0600, tivoli_support at ecurep.ibm.com wrote:
> Non-text part: multipart/mixed
> Non-text part: text/html
> === cut ===
> 
> This is with the latest git master version.

Hi, Brian.  A couple of issues.

The "multipart/mixed" line should definitely not be output, and there
was a patch to the list to fix that a long time ago, which unfortunately
has yet to be applied.

The "text/html" line is because notmuch doesn't output non-text/plain
parts by default.  Notmuch doesn't link against any html parser, for
instance, so if notmuch were to output a text/html part it would just be
raw html, which of course wouldn't be any better.

It looks like you've received an email without any text/plain parts,
which is unfortunate.  The emacs interface will do some parsing of html
parts (with w3, I think), but there's not currently any way to do this
in reply.

jamie.

Re: Added messages / total files count difference.

2011-08-10 Thread Tomi Ollila

On Tue 09 Aug 2011 14:02, Tomi Ollila tomi.oll...@nixu.com writes:

 Hi

 I get this output:

 $ notmuch new --verbose
 Found 15559 total files (that's not much mail).
 Processed 15559 total files in 5m 53s (43 files/sec.).
 Added 15546 new messages to the database.

 $ find * -type f | wc
   15559   15559  529027

 How can I determine which 13 files were dropped. All of those
 15559 files should be mails. I tried to check through mail files that
 have no 'Subject:' header but those were (at least one) indexed. Could
 it be about duplicate Message-ID: or something ?

 $ notmuch --version
 notmuch 0.7-7-g68e8560

It is about duplicate Message-ID:s

It would be nice that 'notmuch new' printes information about this
if this were to happen (as I recall it does when new file found
is not (considered as) a mail file).

The steps I took to figure this out (not all iterations with  without
'wc':s shown) at the end of this email.


 Tomi

Tomi

--8888888888--

$ find ~/mail/mails/* -type f | sort ! filenames-fs
$ wc filenames-fs 
 15559  15559 855766 filenames-fs

$ cd /path/to/notmuch-git/bindings/python
$ cat  foo.py
import notmuch
db = notmuch.Database()
msgs = notmuch.Query(db,'').search_messages()

for f in msgs:
print f.get_filename()

$ PYTHONPATH=/path/to/python-json:`pwd` python foo.py | sort  filenames-db
$ wc filenames-db
 15546  15546 855037 filenames-db

$ diff filenames-db filenames-fs | grep mails | wc
 13  26 755

$ cd ~/mail
$ cat midcheck.pl
use strict;
use warnings;

my %msgids;

foreach (mails/*/*) {
my $fn = $_;
my $mid;
open I, '', $fn or die $!;
while (I) {
$mid = $1, next if /^Message-ID:\s*(.*)/i;
last if /^$/;
}
close I;
unless ($mid) {
print $fn: no Message-ID (in same line with header tag?)\n;
next;
}
my $fn0 = $msgids{$mid};
if (defined $fn0) {
print Files '$fn0' and '$fn' have same msg id: $mid\n;
}
else {
$msgids{$mid} = $fn;
}
}

$ perl midcheck.pl | wc
 13 1172098
$ perl midcheck.pl | grep \^Files | wc
 13 1172098
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[RFC PATCH 0/3] add user friendly date range search

2011-08-10 Thread Jani Nikula

Hi, this RFC series adds user friendly date range searches in notmuch.

Patches 1 and 2 add the date/time parser from coreutils, and were posted to
the list by Michal Sojka in January [1]. I picked them up from the
date-parser branch of the git tree at [2]. These worked without
modifications, so I didn't touch them.

Patch 3 adds the actual date range searches on top of current
master. Michal also posted a date search implementation, but it was based
on Austin Clements' custom parser patches, which I haven't looked at. Also,
this one uses date: range syntax rather than before: and after:, and I
think Michal would agree on this one [3].

If the same parser is to be eventually included in notmuch, I'd prefer
using the parse-datetime module [3] (get_date() was renamed) directly from
gnulib [4], instead of coreutils or elsewhere that just imports from
gnulib. Also, I think the gnulib module should be placed in a subdirectory
rather than alongside all the notmuch lib files. However there are some
complications in using parse-datetime, as gnulib modules heavily rely on
autotools. Perhaps one option would be to build a separate library with
autotools that has what we need from gnulib?

Anyway, this is here now, and works (well, with caveats outlined in the
commit message for patch 3) so that people can try this and get a feel of
the date range search. I wouldn't put much efforts to cleanly importing
gnulib before there's some guarantee that the syntax and semantics are
desired in the first place.

Personally, I've been happy with it ever since I got used to the semantics
of the parser.


BR,
Jani.


[1] http://notmuchmail.org/pipermail/notmuch/2011/003749.html
[2] git://rtime.felk.cvut.cz/notmuch.git
[3] http://notmuchmail.org/pipermail/notmuch/2011/004096.html
[4] http://www.gnu.org/s/gnulib/MODULES.html#module=parse-datetime
[3] http://www.gnu.org/s/gnulib/


Jani Nikula (1):
  lib: add gnulib get_date() based date range search

Michal Sojka (2):
  Import date/time parser from GNU coreutils
  Compile the date/time parser into notmuch library

 Makefile.local |3 +
 configure  |8 +
 lib/Makefile.local |6 +-
 lib/c-ctype.c  |  398 ++
 lib/c-ctype.h  |  297 
 lib/config.h   |   45 +
 lib/database-private.h |1 +
 lib/database.cc|4 +
 lib/getdate-proc.cc|   32 +
 lib/getdate-proc.h |   21 +
 lib/getdate.c  | 3506 
 lib/getdate.h  |   31 +
 lib/getdate.y  | 1581 ++
 lib/gettime.c  |   48 +
 lib/intprops.h |   83 ++
 lib/timespec.h |   39 +
 lib/verify.h   |  140 ++
 17 files changed, 6242 insertions(+), 1 deletions(-)
 create mode 100644 lib/c-ctype.c
 create mode 100644 lib/c-ctype.h
 create mode 100644 lib/config.h
 create mode 100644 lib/getdate-proc.cc
 create mode 100644 lib/getdate-proc.h
 create mode 100644 lib/getdate.c
 create mode 100644 lib/getdate.h
 create mode 100644 lib/getdate.y
 create mode 100644 lib/gettime.c
 create mode 100644 lib/gettime.h
 create mode 100644 lib/intprops.h
 create mode 100644 lib/timespec.h
 create mode 100644 lib/verify.h

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[RFC PATCH 2/3] Compile the date/time parser into notmuch library

2011-08-10 Thread Jani Nikula

From: Michal Sojka sojk...@fel.cvut.cz

---
 Makefile.local |3 +
 configure  |8 ++
 lib/Makefile.local |5 +-
 lib/config.h   |   45 +
 lib/getdate.c  |  185 +++-
 lib/getdate.h  |9 +++
 lib/getdate.y  |   15 -
 7 files changed, 178 insertions(+), 92 deletions(-)
 create mode 100644 lib/config.h

diff --git a/Makefile.local b/Makefile.local
index e3d4d03..3da9ad9 100644
--- a/Makefile.local
+++ b/Makefile.local
@@ -234,6 +234,9 @@ quiet ?= $($(shell echo $1 | sed -e s'/ .*//'))
 %.o: %.c $(global_deps)
$(call quiet,CC $(CFLAGS)) -c $(FINAL_CFLAGS) $ -o $@
 
+%.c: %.y
+   $(call quiet,YACC $(YFLAGS)) $(YFLAGS) $ -o $@
+
 .deps/%.d: %.c $(global_deps)
@set -e; rm -f $@; mkdir -p $$(dirname $@) ; \
$(CC) -M $(CPPFLAGS) $(FINAL_CFLAGS) $  $@. 2/dev/null ; \
diff --git a/configure b/configure
index 3999ce8..f385c79 100755
--- a/configure
+++ b/configure
@@ -25,9 +25,11 @@ fi
 # environment variables)
 CC=${CC:-gcc}
 CXX=${CXX:-g++}
+YACC=${YACC:-yacc}
 CFLAGS=${CFLAGS:--O2}
 CXXFLAGS=${CXXFLAGS:-\$(CFLAGS)}
 LDFLAGS=${LDFLAGS:-}
+YFLAGS=${YFLAGS}
 XAPIAN_CONFIG=${XAPIAN_CONFIG:-xapian-config}
 
 # We don't allow the EMACS or GZIP Makefile variables inherit values
@@ -571,6 +573,9 @@ CC = ${CC}
 # The C++ compiler to use
 CXX = ${CXX}
 
+# The parser generator to use
+YACC = ${YACC}
+
 # Command to execute emacs from Makefiles
 EMACS = emacs --quick
 
@@ -589,6 +594,9 @@ WARN_CXXFLAGS=${WARN_CXXFLAGS}
 # Flags to enable warnings when using the C compiler
 WARN_CFLAGS=${WARN_CFLAGS}
 
+# Default FLAGS for parser generator (can be overridden by user such as make 
YFLAGS=-y)
+YFLAGS = ${YFLAGS}
+
 # The prefix to which notmuch should be installed
 # Note: If you change this value here, be sure to ensure that the
 # LIBDIR_IN_LDCONFIG value below is still set correctly.
diff --git a/lib/Makefile.local b/lib/Makefile.local
index fbc2f6a..a1c234f 100644
--- a/lib/Makefile.local
+++ b/lib/Makefile.local
@@ -56,7 +56,10 @@ libnotmuch_c_srcs =  \
$(dir)/messages.c   \
$(dir)/sha1.c   \
$(dir)/tags.c   \
-   $(dir)/xutil.c
+   $(dir)/xutil.c  \
+   $(dir)/getdate.c\
+   $(dir)/c-ctype.c\
+   $(dir)/gettime.c
 
 libnotmuch_cxx_srcs =  \
$(dir)/database.cc  \
diff --git a/lib/config.h b/lib/config.h
new file mode 100644
index 000..e8bc5b7
--- /dev/null
+++ b/lib/config.h
@@ -0,0 +1,45 @@
+/* lib/config.h.  Generated from config.hin by configure.  */
+/* lib/config.hin.  Generated from configure.ac by autoheader.  */
+
+/* Define to 1 if you have the `clock_gettime' function. */
+#define HAVE_CLOCK_GETTIME 1
+
+/* Define if you have compound literals. */
+#define HAVE_COMPOUND_LITERALS 1
+
+/* Define to 1 if you have the declaration of `tzname', and to 0 if you don't.
+   */
+/* #undef HAVE_DECL_TZNAME */
+
+/* Define to 1 if you have the `nanotime' function. */
+/* #undef HAVE_NANOTIME */
+
+/* Define to 1 if `tm_zone' is a member of `struct tm'. */
+#define HAVE_STRUCT_TM_TM_ZONE 1
+
+/* Define if struct tm has the tm_gmtoff member. */
+#define HAVE_TM_GMTOFF 1
+
+/* Define to 1 if your `struct tm' has `tm_zone'. Deprecated, use
+   `HAVE_STRUCT_TM_TM_ZONE' instead. */
+#define HAVE_TM_ZONE 1
+
+/* Define to 1 if you don't have `tm_zone' but do have the external array
+   `tzname'. */
+/* #undef HAVE_TZNAME */
+
+/* Define to 1 if you have the `tzset' function. */
+#define HAVE_TZSET 1
+
+
+/* Define as a marker that can be attached to declarations that might not
+be used.  This helps to reduce warnings, such as from
+GCC -Wunused-parameter.  */
+#if __GNUC__ = 3 || (__GNUC__ == 2  __GNUC_MINOR__ = 7)
+# define _GL_UNUSED __attribute__ ((__unused__))
+#else
+# define _GL_UNUSED
+#endif
+/* The name _UNUSED_PARAMETER_ is an earlier spelling, although the name
+   is a misnomer outside of parameter lists.  */
+#define _UNUSED_PARAMETER_ _GL_UNUSED
diff --git a/lib/getdate.c b/lib/getdate.c
index 5b20eeb..57f33e9 100644
--- a/lib/getdate.c
+++ b/lib/getdate.c
@@ -68,7 +68,7 @@
 /* Copy the first part of user declarations.  */
 
 /* Line 189 of yacc.c  */
-#line 1 getdate.y
+#line 1 lib/getdate.y
 
 /* Parse a string into an internal time stamp.
 
@@ -102,6 +102,8 @@
 /* FIXME: Check for arithmetic overflow in all cases, not just
some of them.  */
 
+#include notmuch-private.h   /* For xmalloc() */
+
 #include config.h
 
 #include getdate.h
@@ -137,9 +139,6 @@
 #include stdlib.h
 #include string.h
 
-#include xalloc.h
-
-
 /* ISDIGIT differs from isdigit, as follows:
- Its arg may be any int or unsigned int; it need not be an unsigned char
  or EOF.
@@ -344,7 +343,7 @@ set_hhmmss (parser_control *pc, long int hour, long int 
minutes,
 
 
 /* Line 189 of yacc.c  */
-#line 348 getdate.c
+#line 347 lib/getdate.c
 
 /* Enabling traces.  */
 #ifndef YYDEBUG

[RFC PATCH 3/3] lib: add gnulib get_date() based date range search

2011-08-10 Thread Jani Nikula

Add a custom value range processor to handle date: using get_date() from
gnulib. This enables date (and time) searches of the form
date:since..until, where since and until are expressions understood by
get_date(), compatible with most GNU programs. For the date input formats,
see the GNU coreutils manual:
http://www.gnu.org/software/coreutils/manual/html_node/Date-input-formats.html

Open-ended ranges are supported (since Xapian 1.2.1), i.e. you can specify
date:..until or date:since.. to not limit the start or end date,
respectively.

Note: The get_date() function has been renamed to parse_datetime() in
recent gnulib.

EXAMPLES:

date:2-weeks-ago..
date:today00:00:00..
date:yesterday00:00:00..today00:00:00
date:07/14/2011..2011-07-15

BUGS/CAVEATS:

At the moment it seems search phrases with spaces in them are not supported
in notmuch. For date: this means you can't specify a date with an
expression with spaces in it. In many (but apparently not all) cases you
can work around this by replacing spaces with '-' or simply leaving out
whitespace. For example, date:2-days-ago..yesterday00:00:00.

For the purpose of searching mail, the get_date() implementation has some
surprising interpretations. For example:

* Any date specification without time, such as date:yesterday.. or
  date:2011-08-10.., means that day at the same time as now (instead of
  00:00:00).

* As a consequence, ranges such as date:yesterday..yesterday match just the
  messages at the same time as now, to the second. The date parser should
  optimally make different kind of interpretations depending on whether
  parsing since or until. For example, date:today..today should cover
  the whole day from beginning to end, date:today00:00:00..today23:59:59.

* date:monday.. means date:next-monday.. (rather than date:last-monday..)

* date:this-week.. really means date:now.. (you probably want
  date:1-week-ago-next-monday00:00:00.. or similar).

However, there is value in being compatible with GNU programs, and the
input formats have been rather well documented. It would be more surprising
to deviate from that, and it would also take some effort to do so,
including testing.
---
 lib/Makefile.local |1 +
 lib/database-private.h |1 +
 lib/database.cc|4 
 lib/getdate-proc.cc|   32 
 lib/getdate-proc.h |   21 +
 5 files changed, 59 insertions(+), 0 deletions(-)
 create mode 100644 lib/getdate-proc.cc
 create mode 100644 lib/getdate-proc.h

diff --git a/lib/Makefile.local b/lib/Makefile.local
index a1c234f..b722343 100644
--- a/lib/Makefile.local
+++ b/lib/Makefile.local
@@ -63,6 +63,7 @@ libnotmuch_c_srcs =   \
 
 libnotmuch_cxx_srcs =  \
$(dir)/database.cc  \
+   $(dir)/getdate-proc.cc  \
$(dir)/directory.cc \
$(dir)/index.cc \
$(dir)/message.cc   \
diff --git a/lib/database-private.h b/lib/database-private.h
index f705009..d83ae3b 100644
--- a/lib/database-private.h
+++ b/lib/database-private.h
@@ -51,6 +51,7 @@ struct _notmuch_database {
 Xapian::QueryParser *query_parser;
 Xapian::TermGenerator *term_gen;
 Xapian::ValueRangeProcessor *value_range_processor;
+Xapian::ValueRangeProcessor *getdate_proc;
 };
 
 /* Return the list of terms from the given iterator matching a prefix.
diff --git a/lib/database.cc b/lib/database.cc
index 9c2f4ec..b1a0732 100644
--- a/lib/database.cc
+++ b/lib/database.cc
@@ -19,6 +19,7 @@
  */
 
 #include database-private.h
+#include getdate-proc.h
 
 #include iostream
 
@@ -667,12 +668,14 @@ notmuch_database_open (const char *path,
notmuch-term_gen = new Xapian::TermGenerator;
notmuch-term_gen-set_stemmer (Xapian::Stem (english));
notmuch-value_range_processor = new Xapian::NumberValueRangeProcessor 
(NOTMUCH_VALUE_TIMESTAMP);
+   notmuch-getdate_proc = new GetDateValueRangeProcessor 
(NOTMUCH_VALUE_TIMESTAMP, date:, true);
 
notmuch-query_parser-set_default_op (Xapian::Query::OP_AND);
notmuch-query_parser-set_database (*notmuch-xapian_db);
notmuch-query_parser-set_stemmer (Xapian::Stem (english));
notmuch-query_parser-set_stemming_strategy 
(Xapian::QueryParser::STEM_SOME);
notmuch-query_parser-add_valuerangeprocessor 
(notmuch-value_range_processor);
+   notmuch-query_parser-add_valuerangeprocessor (notmuch-getdate_proc);
 
for (i = 0; i  ARRAY_SIZE (BOOLEAN_PREFIX_EXTERNAL); i++) {
prefix_t *prefix = BOOLEAN_PREFIX_EXTERNAL[i];
@@ -716,6 +719,7 @@ notmuch_database_close (notmuch_database_t *notmuch)
 delete notmuch-query_parser;
 delete notmuch-xapian_db;
 delete notmuch-value_range_processor;
+delete notmuch-getdate_proc;
 talloc_free (notmuch);
 }
 
diff --git a/lib/getdate-proc.cc b/lib/getdate-proc.cc
new file mode 100644
index 000..c384e5a
--- /dev/null
+++ b/lib/getdate-proc.cc
@@ -0,0 +1,32 @@
+
+#include database-private.h

not much reply multipart/mixed

Added messages / total files count difference.

[RFC PATCH 3/3] lib: add gnulib get_date() based date range search

[RFC PATCH 1/3] Import date/time parser from GNU coreutils

[RFC PATCH 0/3] add user friendly date range search

not much reply multipart/mixed

Re: Added messages / total files count difference.

[RFC PATCH 0/3] add user friendly date range search

[RFC PATCH 2/3] Compile the date/time parser into notmuch library

[RFC PATCH 3/3] lib: add gnulib get_date() based date range search

10 matches

Site Navigation

Mail list logo

Footer information