[notmuch] [PATCH] Make the date parser nicer
On Tue, 26 Jan 2010 09:55:00 -0800, Keith Packard wrote: > Very cool. Oh, if you've got commits that don't compile on their own, > you should squash them together (or fix it in some other way). Makes > bisecting easier in the future. Makes sense. I am still quite new to git, so excuse those beginner's lapses. Perhaps Carl could squash commits ec3c79a and 2565fc6 when (if?) pulling, that would make every step compile IMHO. > Also, cworth is on vacation this week, so we won't be seeing any > merging to master... No hurry :-). cworth will have to do quite some catching up when he returns. The one "disadvantage" my integration has over your original approach, is that we now always require "date:XXX..YYY". A 'date:lastmonth' won't work, it will need to be "date:lastmonth..today". The reason is that xapian only seems to invoke the RangeParser when something of the format 'A..B' is passed as a parameter. So while we could get "date:..2005" to work, "date:2005.." is not passed to the RangeParser handler, it seems. We could ditch the "date:" prefix, but imho it is more consistent with the other keywords to use it. I have no strong feelings about this. It also still has the same limitation, in that it will not find emails with a future timestamp (I use date:lastweek..5000 to get all mails with a future stamp). Sebastian
Re: [notmuch] [PATCH] Make the date parser nicer
On Tue, 26 Jan 2010 09:55:00 -0800, Keith Packard kei...@keithp.com wrote: Very cool. Oh, if you've got commits that don't compile on their own, you should squash them together (or fix it in some other way). Makes bisecting easier in the future. Makes sense. I am still quite new to git, so excuse those beginner's lapses. Perhaps Carl could squash commits ec3c79a and 2565fc6 when (if?) pulling, that would make every step compile IMHO. Also, cworth is on vacation this week, so we won't be seeing any merging to master... No hurry :-). cworth will have to do quite some catching up when he returns. The one disadvantage my integration has over your original approach, is that we now always require date:XXX..YYY. A 'date:lastmonth' won't work, it will need to be date:lastmonth..today. The reason is that xapian only seems to invoke the RangeParser when something of the format 'A..B' is passed as a parameter. So while we could get date:..2005 to work, date:2005.. is not passed to the RangeParser handler, it seems. We could ditch the date: prefix, but imho it is more consistent with the other keywords to use it. I have no strong feelings about this. It also still has the same limitation, in that it will not find emails with a future timestamp (I use date:lastweek..5000 to get all mails with a future stamp). Sebastian ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
[notmuch] [PATCH] Make the date parser nicer
On Mon, 25 Jan 2010 22:36:35 -0800, Keith Packard wrote:
> Here's some code which further improves date parsing by allowing lots of
> date formats, including things like "today", "thisweek", ISO and US date
> formats and month names. You can separate two dates with .. to make a
> range, or you can just use the default range ("lastmonth" is everything
> From the 1st of the previous month to the 1st of the current month).
>
> I think this fits nicely with your code.
It fit nicely indeed. I have just integrated your date parser into my
code and sent it as a series of 4 patches based on current cworth
master. (commits 2565fc6 and 96e11c3 will not compile on their own)
ec3c79a integrate keithp's date.c into the notmuch date parser and delete my
previous own
2565fc6 compile date.c as well
96e11c3 add date parser file from Keith
6ed2569 Make the date parser nicer.
The topic branch is here for those who don't want to apply mail patches:
http://github.com/spaetz/notmuch-all-feature/commits/dateparser3
Documentation of the new notmuch_parse_date function:
/* Parse a string into the first and last possible timestamps.
* It parses the possible formats and stops if one pattern matches.
* Keywords: 'today','yesterday','thisweek','lastweek','thismonth',
* 'lastmonth'
* Month-day : month[-day]] (month: January, Jan, or 1)\n"
* ISO format: year[-month[-day]]
* US format : month[/day[/year]]
*
* 'after' is used to fill in bits from context if left out, e.g. a
* 'date:2004..01' will find from 2004-01-01 through 2004-01-31
*
* Return values:
* NOTMUCH_STATUS_SUCCESS
* NOTMUCH_STATUS_INVALID_DATE: Error parsing the date string
Please pull :-).
Sebastian
[notmuch] [PATCH] Make the date parser nicer
On Mon, 25 Jan 2010 22:36:35 -0800, Keith Packard wrote:
> Here's some code which further improves date parsing by allowing lots of
> date formats, including things like "today", "thisweek", ISO and US date
> formats and month names. You can separate two dates with .. to make a
> range, or you can just use the default range ("lastmonth" is everything
> From the 1st of the previous month to the 1st of the current month).
>
> I think this fits nicely with your code.
Hey, cool. I tried to keep my patch as small as possible, thanks for a
full-fledged date string parser :). I'll see how I can integrate that
into my dateparser branch.
Thanks
Sebastian
[notmuch] [PATCH] Make the date parser nicer
On Tue, 26 Jan 2010 12:50:41 +0100, "Sebastian Spaeth" wrote:
> On Mon, 25 Jan 2010 22:36:35 -0800, Keith Packard
> wrote:
> > Here's some code which further improves date parsing by allowing lots of
> > date formats, including things like "today", "thisweek", ISO and US date
> > formats and month names. You can separate two dates with .. to make a
> > range, or you can just use the default range ("lastmonth" is everything
> > From the 1st of the previous month to the 1st of the current month).
> >
> > I think this fits nicely with your code.
>
> It fit nicely indeed. I have just integrated your date parser into my
> code and sent it as a series of 4 patches based on current cworth
> master. (commits 2565fc6 and 96e11c3 will not compile on their own)
Very cool. Oh, if you've got commits that don't compile on their own,
you should squash them together (or fix it in some other way). Makes
bisecting easier in the future.
Also, cworth is on vacation this week, so we won't be seeing any
merging to master...
--
keith.packard at intel.com
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
URL:
Re: [notmuch] [PATCH] Make the date parser nicer
On Mon, 25 Jan 2010 22:36:35 -0800, Keith Packard kei...@keithp.com wrote: Here's some code which further improves date parsing by allowing lots of date formats, including things like today, thisweek, ISO and US date formats and month names. You can separate two dates with .. to make a range, or you can just use the default range (lastmonth is everything From the 1st of the previous month to the 1st of the current month). I think this fits nicely with your code. Hey, cool. I tried to keep my patch as small as possible, thanks for a full-fledged date string parser :). I'll see how I can integrate that into my dateparser branch. Thanks Sebastian ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
Re: [notmuch] [PATCH] Make the date parser nicer
On Mon, 25 Jan 2010 22:36:35 -0800, Keith Packard kei...@keithp.com wrote: Here's some code which further improves date parsing by allowing lots of date formats, including things like today, thisweek, ISO and US date formats and month names. You can separate two dates with .. to make a range, or you can just use the default range (lastmonth is everything From the 1st of the previous month to the 1st of the current month). I think this fits nicely with your code. It fit nicely indeed. I have just integrated your date parser into my code and sent it as a series of 4 patches based on current cworth master. (commits 2565fc6 and 96e11c3 will not compile on their own) ec3c79a integrate keithp's date.c into the notmuch date parser and delete my previous own 2565fc6 compile date.c as well 96e11c3 add date parser file from Keith 6ed2569 Make the date parser nicer. The topic branch is here for those who don't want to apply mail patches: http://github.com/spaetz/notmuch-all-feature/commits/dateparser3 Documentation of the new notmuch_parse_date function: /* Parse a string into the first and last possible timestamps. * It parses the possible formats and stops if one pattern matches. * Keywords: 'today','yesterday','thisweek','lastweek','thismonth', * 'lastmonth' * Month-day : month[-day]] (month: January, Jan, or 1)\n * ISO format: year[-month[-day]] * US format : month[/day[/year]] * * 'after' is used to fill in bits from context if left out, e.g. a * 'date:2004..01' will find from 2004-01-01 through 2004-01-31 * * Return values: * NOTMUCH_STATUS_SUCCESS * NOTMUCH_STATUS_INVALID_DATE: Error parsing the date string Please pull :-). Sebastian ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
Re: [notmuch] [PATCH] Make the date parser nicer
On Tue, 26 Jan 2010 12:50:41 +0100, Sebastian Spaeth sebast...@sspaeth.de wrote: On Mon, 25 Jan 2010 22:36:35 -0800, Keith Packard kei...@keithp.com wrote: Here's some code which further improves date parsing by allowing lots of date formats, including things like today, thisweek, ISO and US date formats and month names. You can separate two dates with .. to make a range, or you can just use the default range (lastmonth is everything From the 1st of the previous month to the 1st of the current month). I think this fits nicely with your code. It fit nicely indeed. I have just integrated your date parser into my code and sent it as a series of 4 patches based on current cworth master. (commits 2565fc6 and 96e11c3 will not compile on their own) Very cool. Oh, if you've got commits that don't compile on their own, you should squash them together (or fix it in some other way). Makes bisecting easier in the future. Also, cworth is on vacation this week, so we won't be seeing any merging to master... -- keith.pack...@intel.com pgpXXzLGP8ACa.pgp Description: PGP signature ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
[notmuch] [PATCH] Make the date parser nicer
On Fri, 22 Jan 2010 16:26:11 +0100, Sebastian Spaeth
wrote:
> Currently we have to enter mail dates as timestamps. This approach
> does 2 things: it requires the prefix 'date:' and it allows timestamps
> to be specified as , MM or MMDD. So a notmuch show
> date:2005..20060512 will find all mails from 2005-01-01 until
> 2006-05-12. The code is probably not in a proper location yet and
> needs to be shoved around by someone more knowledgable than me. My C++
> skills are somewhat,... lacking...
Here's some code which further improves date parsing by allowing lots of
date formats, including things like "today", "thisweek", ISO and US date
formats and month names. You can separate two dates with .. to make a
range, or you can just use the default range ("lastmonth" is everything
[notmuch] [PATCH] Make the date parser nicer (v3 + 'now' keyword) (final mail)
Sorry, very last mail from me on this issue. I squashed the patches into 3 distinct ones, and inherited a tree from cworth's master tree for easier pulling (rather than basing it off my all-features branch). This branch is here: http://github.com/spaetz/notmuch-all-feature/commits/dateparser2 (git://github.com/spaetz/notmuch-all-feature.git then switch to branch 'dateparser2')
[notmuch] [PATCH] Make the date parser nicer (v3 + 'now' keyword)
The patch previously sent can be considered my final attempt to a nicer date parser. In addition to some testing, I have updated the documentation to reflect the new syntax, and I have also added the keyword 'now' as a possibility. So 'date:2002..now' (all mails from 2002 until now) or 'date:now..31' (all mails that McFly sent to you between now and the 31st of this month) should all work. Note that this is not a sincere: 'since' replacement, as 2001..now won't find mails with timestamps in the future. The relevant 4 patches are in my git tree, git://github.com/spaetz/notmuch-all-feature.git then switch to branch 'dateparser': 59e9b56 remove superfluous debug statements from date parsing 4bdb0b0 allow 'now' as keyword for now cf6c500 Adapt documentation to new date: syntax d8d3d0b Make the date parser nicer. This is v3 and considered to be final (but the documentation). Thanks, Sebastian
[notmuch] [PATCH] Make the date parser nicer. This is v3 and considered to be final (but the documentation).
Currently we have to enter mail dates as timestamps. This approach does 2
things:
1) it requires the prefix 'date:'
2) it allows dates to be specified in a flexible way. So a notmuch show
date:2005..2006-05-12 will find all mails from 2005-01-01 until 2006-05-12.
Possible time formats: -MM-DD, -MM (from/through that month) ,
(from/through that year), MM-DD (month-day in current year), DD (day in current
month).
Signed-off-by: Sebastian Spaeth
---
lib/database.cc | 80 ++-
1 files changed, 79 insertions(+), 1 deletions(-)
diff --git a/lib/database.cc b/lib/database.cc
index 5b12320..da2fda8 100644
--- a/lib/database.cc
+++ b/lib/database.cc
@@ -494,6 +494,84 @@ _notmuch_database_ensure_writable (notmuch_database_t
*notmuch)
return NOTMUCH_STATUS_SUCCESS;
}
+struct MaildateValueRangeProcessor : public Xapian::ValueRangeProcessor {
+MaildateValueRangeProcessor() {}
+
+ time_t
+ parsedate(std::string , bool early) {
+/* Parse the date to a 'time_t', return NULL on error*/
+/* possible time formats: -MM-DD, -MM, , */
+/* MM-DD (current month), DD (day in current month). */
+/* Uses start of time unit when 'early', end otherwise, e.g. */
+/* 2001:=2001-01-01:00:00:00 when 'early' or 2001-12-31:23:59:59 */
+struct tm *timeinfo;
+time_t timet;
+int year = 0, month = 0, day = 0;
+
+if (str.size() == 2) {
+ /* We got just current day in month, parse & remove it */
+ day = atoi(str.c_str());
+ str.erase(0,2);
+}
+
+if (str.size() == 4 or str.size() > 5) {
+ /* expect a year, parse & remove it */
+ year = atoi(str.c_str());
+ str.erase(0,5);
+}
+
+/* parse & remove month if there is sth left in the string */
+month = atoi(str.c_str());
+str.erase(0,3);
+
+/* Parse day if we have one left */
+if (str.size())
+ day = atoi(str.c_str());
+
+if (year == 0 && month == 0 && day == 0)
+ // no expected time format
+ return -1 ;
+
+timet = time(NULL);/* init timeinfo with current time */
+timeinfo = gmtime();
+/* add timeunit if !early (1 second too much, which we deduct later */
+if (!early) {
+ if (year && !month)++year; /* only year given */
+ if (year && month && !day) ++month; /* year & month given */
+}
+if (year) timeinfo -> tm_year = year - 1900;
+if (month) timeinfo -> tm_mon = month - 1;
+if (day) timeinfo -> tm_mday = (early ? day : ++day);
+else timeinfo -> tm_mday = 1;
+
+timeinfo -> tm_hour = 0;
+timeinfo -> tm_min = 0;
+timeinfo -> tm_sec = (early ? 0 : -1); /* -1 sec if !early */
+timet = mktime(timeinfo);
+
+return timet;
+ }
+
+Xapian::valueno operator()(std::string , std::string ) {
+ time_t begintime, endtime;
+
+ if (begin.substr(0, 5) != "date:")
+return Xapian::BAD_VALUENO;
+ begin.erase(0, 5);
+
+ begintime = parsedate(begin, true);
+ endtime = parsedate(end, false);
+ if ((begintime == -1) || (endtime == -1))
+ // parsedate failed, no valid time format
+ return Xapian::BAD_VALUENO;
+
+ begin.assign(Xapian::sortable_serialise(begintime));
+ end.assign(Xapian::sortable_serialise(endtime));
+
+ return NOTMUCH_VALUE_TIMESTAMP;
+}
+};
+
notmuch_database_t *
notmuch_database_open (const char *path,
notmuch_database_mode_t mode)
@@ -570,8 +648,7 @@ notmuch_database_open (const char *path,
notmuch->query_parser = new Xapian::QueryParser;
notmuch->term_gen = new Xapian::TermGenerator;
notmuch->term_gen->set_stemmer (Xapian::Stem ("english"));
- notmuch->value_range_processor = new Xapian::NumberValueRangeProcessor
(NOTMUCH_VALUE_TIMESTAMP);
+ notmuch->value_range_processor = new MaildateValueRangeProcessor();
notmuch->query_parser->set_default_op (Xapian::Query::OP_AND);
notmuch->query_parser->set_database (*notmuch->xapian_db);
--
1.6.3.3
Re: [notmuch] [PATCH] Make the date parser nicer (v3 + 'now' keyword)
The patch previously sent can be considered my final attempt to a nicer date parser. In addition to some testing, I have updated the documentation to reflect the new syntax, and I have also added the keyword 'now' as a possibility. So 'date:2002..now' (all mails from 2002 until now) or 'date:now..31' (all mails that McFly sent to you between now and the 31st of this month) should all work. Note that this is not a sincere: 'since' replacement, as 2001..now won't find mails with timestamps in the future. The relevant 4 patches are in my git tree, git://github.com/spaetz/notmuch-all-feature.git then switch to branch 'dateparser': 59e9b56 remove superfluous debug statements from date parsing 4bdb0b0 allow 'now' as keyword for now cf6c500 Adapt documentation to new date: syntax d8d3d0b Make the date parser nicer. This is v3 and considered to be final (but the documentation). Thanks, Sebastian ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
[notmuch] [PATCH] Make the date parser nicer
Currently we have to enter mail dates as timestamps. This approach does 2
things: 1) it requires the prefix 'date:'
2) it allows dates to be specified in some formats. So a notmuch show
date:2005..2006-05-12 will find all mails from 2005-01-01 until 2006-05-12.
The code is probably not in a proper location yet and needs to be shoved around
by someone more knowledgable than me.
My C++ skills are somewhat,... lacking...
Possible time formats: -MM-DD,-MM (in that month) , (in that year)
MM-DD (month-day in current year), DD (day in current month)
Signed-off-by: Sebastian Spaeth
---
lib/database.cc | 90 ++-
1 files changed, 89 insertions(+), 1 deletions(-)
diff --git a/lib/database.cc b/lib/database.cc
index 5b12320..9c2842d 100644
--- a/lib/database.cc
+++ b/lib/database.cc
@@ -494,6 +494,94 @@ _notmuch_database_ensure_writable (notmuch_database_t
*notmuch)
return NOTMUCH_STATUS_SUCCESS;
}
+struct MaildateValueRangeProcessor : public Xapian::ValueRangeProcessor {
+MaildateValueRangeProcessor() {}
+
+ time_t
+ parsedate(std::string , bool early) {
+ // Parse the begin date to time_t
+ // possible time formats:
+ // -MM-DD (size 10)
+ // -MM(size 7)
+ // (size 4)
+ // MM-DD (size 5)
+ // DD (size 2)
+// begin of time unit when 'early', end of when not
+ struct tm *timeinfo;
+ time_t timet;
+ //const char * startptr;
+ int year = 0, month = 0, day = 0;
+
+ if (str.size() == 2) {
+ // parse day, then remove it from the string
+ day = atoi(str.c_str());
+ str.erase(0,2);
+ }
+
+ if (str.size() == 4 or str.size() == 7 or str.size() == 10) {
+ // parse year, then remove it from the string
+ year = atoi(str.c_str());
+ str.erase(0,5);
+ }
+
+ month = atoi(str.c_str());
+ str.erase(0,3);
+
+ // Do we have a day component left?
+ if (str.size())
+ day = atoi(str.c_str());
+
+ if (year == 0 && month == 0 && day == 0)
+ // no expected time format
+ return NULL ;
+
+ timet = time(NULL);
+ timeinfo = gmtime( );
+ timeinfo -> tm_isdst = 0;
+ if (!early && !month) ++year;
+ if (year) timeinfo -> tm_year = year - 1900;
+
+ if (month) timeinfo -> tm_mon = month - 1;
+ //else if (year) timeinfo -> tm_mon = (early ? 0: 12);
+
+ if (day) timeinfo -> tm_mday = (early ? day : ++day);
+ else timeinfo -> tm_mday = 1;
+
+ timeinfo -> tm_hour = 0;
+ timeinfo -> tm_min = 0;
+ timeinfo -> tm_sec = 0;
+ timet = mktime ( timeinfo );
+
+if (!early) --timet;
+ if (timet == -1)
+ return NULL;
+ return timet;
+ }
+
+Xapian::valueno operator()(std::string , std::string ) {
+ time_t begintime, endtime;
+
+ if (begin.substr(0, 5) != "date:")
+return Xapian::BAD_VALUENO;
+ begin.erase(0, 5);
+
+ begintime = parsedate ( begin, true);
+ if (begintime == -1)
+ // no valid time format
+ return Xapian::BAD_VALUENO;
+
+ endtime = parsedate ( end, false);
+ if (endtime == -1)
+ // no valid time format
+ return Xapian::BAD_VALUENO;
+
+ begin.assign(Xapian::sortable_serialise(begintime));
+ end.assign(Xapian::sortable_serialise(endtime));
+
+ return NOTMUCH_VALUE_TIMESTAMP;
+}
+};
+
notmuch_database_t *
notmuch_database_open (const char *path,
notmuch_database_mode_t mode)
@@ -570,7 +658,7 @@ notmuch_database_open (const char *path,
notmuch->query_parser = new Xapian::QueryParser;
notmuch->term_gen = new Xapian::TermGenerator;
notmuch->term_gen->set_stemmer (Xapian::Stem ("english"));
- notmuch->value_range_processor = new Xapian::NumberValueRangeProcessor
(NOTMUCH_VALUE_TIMESTAMP, "date:", true);
+ notmuch->value_range_processor = new MaildateValueRangeProcessor();
notmuch->query_parser->set_default_op (Xapian::Query::OP_AND);
notmuch->query_parser->set_database (*notmuch->xapian_db);
--
1.6.3.3
[notmuch] [PATCH] Make the date parser nicer
On Fri, 22 Jan 2010 16:26:11 +0100, Sebastian Spaeth
wrot> + if (begin.size() == 8) {
> + int no_items;
> + no_items = sscanf(begin.c_str(), "%4i%2i%2i", , , );
> + if (no_items != 3)
> + return Xapian::BAD_VALUENO;
Also I have found that my sscanf skills are sourly lacking:
a date of 20060108 will lead to 2006 1 0 for (year, month, day)
I bet someone knows a nice way to parse those values in a good way.
Sebastian
[notmuch] [PATCH] Make the date parser nicer
Please find in the previous mail attached my patch (against my current all-feature branch, but should pretty much apply to current master too). It is a proof-of-concept to make the date parser nicer. The following searches work with this code: notmuch show... ... date:2001..2010 (from beginning of 2001 until end of 2010) ... date:20011201..200506 (from 1001-12-01 until 2005-06-31 The code will pretty surely need some cleaning up, as I can hardly code C, not to speak of C++. But at least it works and it is not very intrusive. (Ps. now that I think of it, I always use day 31 as last day, which is surely wrong, it also still accepts some obviously wrong dates like 20040231) We could also think about using a xapian DateValueRangeParser to do that, but that would require saving the timestamp as MMDD in the database (which we probably do not want). Feedback welcome, Sebastian
[notmuch] [PATCH] Make the date parser nicer
Currently we have to enter mail dates as timestamps. This approach does 2
things: it requires the prefix 'date:' and it allows timestamps to be specified
as , MM or MMDD. So a notmuch show date:2005..20060512 will find
all mails from 2005-01-01 until 2006-05-12. The code is probably not in a
proper location yet and needs to be shoved around by someone more knowledgable
than me. My C++ skills are somewhat,... lacking...
Signed-off-by: Sebastian Spaeth
---
lib/database.cc | 94 ++-
1 files changed, 93 insertions(+), 1 deletions(-)
diff --git a/lib/database.cc b/lib/database.cc
index 5b12320..102a6ff 100644
--- a/lib/database.cc
+++ b/lib/database.cc
@@ -494,6 +494,97 @@ _notmuch_database_ensure_writable (notmuch_database_t
*notmuch)
return NOTMUCH_STATUS_SUCCESS;
}
+struct MaildateValueRangeProcessor : public Xapian::ValueRangeProcessor {
+MaildateValueRangeProcessor() {}
+
+Xapian::valueno operator()(std::string , std::string ) {
+if (begin.substr(0, 5) != "date:")
+return Xapian::BAD_VALUENO;
+begin.erase(0, 5);
+
+ // Parse the begin date to time_t
+ struct tm *timeinfo;
+ time_t begintime, endtime;
+ //const char * startptr;
+ int year, month, day;
+
+ if (begin.size() == 8) {
+ int no_items;
+ no_items = sscanf(begin.c_str(), "%4i%2i%2i", , , );
+ if (no_items != 3)
+ return Xapian::BAD_VALUENO;
+ } else if (begin.size() == 6) {
+ int no_items;
+ day = 1;
+ no_items = sscanf(begin.c_str(), "%4i%2i", , );
+ if (no_items != 2)
+ return Xapian::BAD_VALUENO;
+ } else if (begin.size() == 4) {
+ int no_items;
+ day = 1;
+ month = 1;
+ no_items = sscanf(begin.c_str(), "%4i", );
+ if (no_items != 1)
+ return Xapian::BAD_VALUENO;
+ } else {
+ // no expected time format
+ return Xapian::BAD_VALUENO;
+ }
+
+ begintime = time(NULL);
+ timeinfo = localtime( );
+ timeinfo -> tm_year = year - 1900;
+ timeinfo -> tm_mon = month - 1;
+ fprintf (stderr, "Startdate %d %d %d\n",year,month,day);
+ timeinfo -> tm_mday = day;
+ begintime = mktime ( timeinfo );
+
+ if (begintime == -1)
+ // no valid time format
+ return Xapian::BAD_VALUENO;
+
+ if (end.size() == 8) {
+ int no_items;
+ no_items = sscanf(end.c_str(), "%4i%2i%2i", , , );
+ if (no_items != 3)
+ return Xapian::BAD_VALUENO;
+ } else if (end.size() == 6) {
+ int no_items;
+ day = 31;
+ no_items = sscanf(end.c_str(), "%4i%2i", , );
+ if (no_items != 2)
+ return Xapian::BAD_VALUENO;
+ } else if (end.size() == 4) {
+ int no_items;
+ day = 31;
+ month = 12;
+ no_items = sscanf(end.c_str(), "%4i", );
+ if (no_items != 1)
+ return Xapian::BAD_VALUENO;
+ } else {
+ // no expected time format
+ return Xapian::BAD_VALUENO;
+ }
+
+ timeinfo = localtime( );
+ timeinfo -> tm_year = year - 1900;
+ timeinfo -> tm_mon = month - 1;
+ fprintf (stderr, "Enddate %d %d %d\n",year,month,day);
+ timeinfo -> tm_mday = day;
+ endtime = mktime ( timeinfo );
+ //XXX: plus 1 day to make the last day inclusive??
+
+ if (endtime == -1)
+ // no valid time format
+ return Xapian::BAD_VALUENO;
+
+ begin.assign(Xapian::sortable_serialise(begintime));
+ end.assign(Xapian::sortable_serialise(endtime));
+
+return NOTMUCH_VALUE_TIMESTAMP;
+}
+};
+
notmuch_database_t *
notmuch_database_open (const char *path,
notmuch_database_mode_t mode)
@@ -570,7 +661,8 @@ notmuch_database_open (const char *path,
notmuch->query_parser = new Xapian::QueryParser;
notmuch->term_gen = new Xapian::TermGenerator;
notmuch->term_gen->set_stemmer (Xapian::Stem ("english"));
- notmuch->value_range_processor = new Xapian::NumberValueRangeProcessor
(NOTMUCH_VALUE_TIMESTAMP, "date:", true);
+ notmuch->value_range_processor = new MaildateValueRangeProcessor();
+ // (NOTMUCH_VALUE_TIMESTAMP);
notmuch->query_parser->set_default_op (Xapian::Query::OP_AND);
notmuch->query_parser->set_database (*notmuch->xapian_db);
--
1.6.3.3
[notmuch] [PATCH] Make the date parser nicer
Currently we have to enter mail dates as timestamps. This approach does 2
things: it requires the prefix 'date:' and it allows timestamps to be specified
as , MM or MMDD. So a notmuch show date:2005..20060512 will find
all mails from 2005-01-01 until 2006-05-12. The code is probably not in a
proper location yet and needs to be shoved around by someone more knowledgable
than me. My C++ skills are somewhat,... lacking...
Signed-off-by: Sebastian Spaeth sebast...@sspaeth.de
---
lib/database.cc | 94 ++-
1 files changed, 93 insertions(+), 1 deletions(-)
diff --git a/lib/database.cc b/lib/database.cc
index 5b12320..102a6ff 100644
--- a/lib/database.cc
+++ b/lib/database.cc
@@ -494,6 +494,97 @@ _notmuch_database_ensure_writable (notmuch_database_t
*notmuch)
return NOTMUCH_STATUS_SUCCESS;
}
+struct MaildateValueRangeProcessor : public Xapian::ValueRangeProcessor {
+MaildateValueRangeProcessor() {}
+
+Xapian::valueno operator()(std::string begin, std::string end) {
+if (begin.substr(0, 5) != date:)
+return Xapian::BAD_VALUENO;
+begin.erase(0, 5);
+
+ // Parse the begin date to time_t
+ struct tm *timeinfo;
+ time_t begintime, endtime;
+ //const char * startptr;
+ int year, month, day;
+
+ if (begin.size() == 8) {
+ int no_items;
+ no_items = sscanf(begin.c_str(), %4i%2i%2i, year, month, day);
+ if (no_items != 3)
+ return Xapian::BAD_VALUENO;
+ } else if (begin.size() == 6) {
+ int no_items;
+ day = 1;
+ no_items = sscanf(begin.c_str(), %4i%2i, year, month);
+ if (no_items != 2)
+ return Xapian::BAD_VALUENO;
+ } else if (begin.size() == 4) {
+ int no_items;
+ day = 1;
+ month = 1;
+ no_items = sscanf(begin.c_str(), %4i, year);
+ if (no_items != 1)
+ return Xapian::BAD_VALUENO;
+ } else {
+ // no expected time format
+ return Xapian::BAD_VALUENO;
+ }
+
+ begintime = time(NULL);
+ timeinfo = localtime( begintime );
+ timeinfo - tm_year = year - 1900;
+ timeinfo - tm_mon = month - 1;
+ fprintf (stderr, Startdate %d %d %d\n,year,month,day);
+ timeinfo - tm_mday = day;
+ begintime = mktime ( timeinfo );
+
+ if (begintime == -1)
+ // no valid time format
+ return Xapian::BAD_VALUENO;
+
+ if (end.size() == 8) {
+ int no_items;
+ no_items = sscanf(end.c_str(), %4i%2i%2i, year, month, day);
+ if (no_items != 3)
+ return Xapian::BAD_VALUENO;
+ } else if (end.size() == 6) {
+ int no_items;
+ day = 31;
+ no_items = sscanf(end.c_str(), %4i%2i, year, month);
+ if (no_items != 2)
+ return Xapian::BAD_VALUENO;
+ } else if (end.size() == 4) {
+ int no_items;
+ day = 31;
+ month = 12;
+ no_items = sscanf(end.c_str(), %4i, year);
+ if (no_items != 1)
+ return Xapian::BAD_VALUENO;
+ } else {
+ // no expected time format
+ return Xapian::BAD_VALUENO;
+ }
+
+ timeinfo = localtime( begintime );
+ timeinfo - tm_year = year - 1900;
+ timeinfo - tm_mon = month - 1;
+ fprintf (stderr, Enddate %d %d %d\n,year,month,day);
+ timeinfo - tm_mday = day;
+ endtime = mktime ( timeinfo );
+ //XXX: plus 1 day to make the last day inclusive??
+
+ if (endtime == -1)
+ // no valid time format
+ return Xapian::BAD_VALUENO;
+
+ begin.assign(Xapian::sortable_serialise(begintime));
+ end.assign(Xapian::sortable_serialise(endtime));
+
+return NOTMUCH_VALUE_TIMESTAMP;
+}
+};
+
notmuch_database_t *
notmuch_database_open (const char *path,
notmuch_database_mode_t mode)
@@ -570,7 +661,8 @@ notmuch_database_open (const char *path,
notmuch-query_parser = new Xapian::QueryParser;
notmuch-term_gen = new Xapian::TermGenerator;
notmuch-term_gen-set_stemmer (Xapian::Stem (english));
- notmuch-value_range_processor = new Xapian::NumberValueRangeProcessor
(NOTMUCH_VALUE_TIMESTAMP, date:, true);
+ notmuch-value_range_processor = new MaildateValueRangeProcessor();
+ // (NOTMUCH_VALUE_TIMESTAMP);
notmuch-query_parser-set_default_op (Xapian::Query::OP_AND);
notmuch-query_parser-set_database (*notmuch-xapian_db);
--
1.6.3.3
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch
Re: [notmuch] [PATCH] Make the date parser nicer
Please find in the previous mail attached my patch (against my current all-feature branch, but should pretty much apply to current master too). It is a proof-of-concept to make the date parser nicer. The following searches work with this code: notmuch show... ... date:2001..2010 (from beginning of 2001 until end of 2010) ... date:20011201..200506 (from 1001-12-01 until 2005-06-31 The code will pretty surely need some cleaning up, as I can hardly code C, not to speak of C++. But at least it works and it is not very intrusive. (Ps. now that I think of it, I always use day 31 as last day, which is surely wrong, it also still accepts some obviously wrong dates like 20040231) We could also think about using a xapian DateValueRangeParser to do that, but that would require saving the timestamp as MMDD in the database (which we probably do not want). Feedback welcome, Sebastian ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
