Re: [PATCH 5/5] lib: Add lastmod: queries for filtering by last modification

[Gaute Hope]

David Bremner writes on August 20, 2015 8:22:

David Bremner da...@tethera.net writes:


From: Austin Clements amdra...@mit.edu

The implementation is essentially the same as the date range search
prior to Jani's fancy date parser.


pushed the series. Those of you running master be prepared for a
database update next time you run notmuch new. This particular update is
relatively fast, it took about a minute for my 380k messages (on SSD).



Great!

-g

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH 5/5] lib: Add lastmod: queries for filtering by last modification

[David Bremner]

David Bremner da...@tethera.net writes:

 From: Austin Clements amdra...@mit.edu

 The implementation is essentially the same as the date range search
 prior to Jani's fancy date parser.

pushed the series. Those of you running master be prepared for a
database update next time you run notmuch new. This particular update is
relatively fast, it took about a minute for my 380k messages (on SSD).

d
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH 5/5] lib: Add "lastmod:" queries for filtering by last modification

[David Bremner]

From: Austin Clements 

The implementation is essentially the same as the date range search
prior to Jani's fancy date parser.
---
 doc/man7/notmuch-search-terms.rst |  8 
 lib/database-private.h|  1 +
 lib/database.cc   |  4 
 test/T570-revision-tracking.sh| 17 +
 4 files changed, 30 insertions(+)

diff --git a/doc/man7/notmuch-search-terms.rst 
b/doc/man7/notmuch-search-terms.rst
index 1d27ac1..e71a525 100644
--- a/doc/man7/notmuch-search-terms.rst
+++ b/doc/man7/notmuch-search-terms.rst
@@ -54,6 +54,8 @@ indicate user-supplied values):

 -  date:..

+-  lastmod:..
+
 The **from:** prefix is used to match the name or address of the sender
 of an email message.

@@ -124,6 +126,12 @@ The time range can also be specified using timestamps with 
a syntax of:
 Each timestamp is a number representing the number of seconds since
 1970-01-01 00:00:00 UTC.

+The **lastmod:** prefix can be used to restrict the result by the
+database revision number of when messages were last modified (tags
+were added/removed or filenames changed).  This is usually used in
+conjunction with the **--uuid** argument to **notmuch search**
+to find messages that have changed since an earlier query.
+
 Operators
 -

diff --git a/lib/database-private.h b/lib/database-private.h
index 4e93257..3fb10f7 100644
--- a/lib/database-private.h
+++ b/lib/database-private.h
@@ -176,6 +176,7 @@ struct _notmuch_database {
 Xapian::TermGenerator *term_gen;
 Xapian::ValueRangeProcessor *value_range_processor;
 Xapian::ValueRangeProcessor *date_range_processor;
+Xapian::ValueRangeProcessor *last_mod_range_processor;
 };

 /* Prior to database version 3, features were implied by the database
diff --git a/lib/database.cc b/lib/database.cc
index fc78769..bab3334 100644
--- a/lib/database.cc
+++ b/lib/database.cc
@@ -1000,6 +1000,7 @@ notmuch_database_open_verbose (const char *path,
notmuch->term_gen->set_stemmer (Xapian::Stem ("english"));
notmuch->value_range_processor = new Xapian::NumberValueRangeProcessor 
(NOTMUCH_VALUE_TIMESTAMP);
notmuch->date_range_processor = new ParseTimeValueRangeProcessor 
(NOTMUCH_VALUE_TIMESTAMP);
+   notmuch->last_mod_range_processor = new 
Xapian::NumberValueRangeProcessor (NOTMUCH_VALUE_LAST_MOD, "lastmod:");

notmuch->query_parser->set_default_op (Xapian::Query::OP_AND);
notmuch->query_parser->set_database (*notmuch->xapian_db);
@@ -1007,6 +1008,7 @@ notmuch_database_open_verbose (const char *path,
notmuch->query_parser->set_stemming_strategy 
(Xapian::QueryParser::STEM_SOME);
notmuch->query_parser->add_valuerangeprocessor 
(notmuch->value_range_processor);
notmuch->query_parser->add_valuerangeprocessor 
(notmuch->date_range_processor);
+   notmuch->query_parser->add_valuerangeprocessor 
(notmuch->last_mod_range_processor);

for (i = 0; i < ARRAY_SIZE (BOOLEAN_PREFIX_EXTERNAL); i++) {
prefix_t *prefix = _PREFIX_EXTERNAL[i];
@@ -1085,6 +1087,8 @@ notmuch_database_close (notmuch_database_t *notmuch)
 notmuch->value_range_processor = NULL;
 delete notmuch->date_range_processor;
 notmuch->date_range_processor = NULL;
+delete notmuch->last_mod_range_processor;
+notmuch->last_mod_range_processor = NULL;

 return status;
 }
diff --git a/test/T570-revision-tracking.sh b/test/T570-revision-tracking.sh
index 20b44cb..0936011 100755
--- a/test/T570-revision-tracking.sh
+++ b/test/T570-revision-tracking.sh
@@ -73,4 +73,21 @@ test_expect_success 'tag succeeds with correct uuid' \
 test_expect_code 1 'tag fails with incorrect uuid' \
 "notmuch tag --uuid=this-is-no-uuid '*' +test2"

+test_begin_subtest 'lastmod:0.. matches everything'
+total=$(notmuch count '*')
+modtotal=$(notmuch count lastmod:0..)
+test_expect_equal "$total" "$modtotal"
+
+test_begin_subtest 'lastmod:100.. matches nothing'
+modtotal=$(notmuch count lastmod:100..)
+test_expect_equal 0 "$modtotal"
+
+test_begin_subtest 'exclude one message using lastmod'
+lastmod=$(notmuch count --lastmod '*' | cut -f3)
+total=$(notmuch count '*')
+notmuch tag +4EFC743A.3060609 at april.org id:4EFC743A.3060609 at april.org
+subtotal=$(notmuch count lastmod:..$lastmod)
+result=$(($subtotal == $total-1))
+test_expect_equal 1 "$result"
+
 test_done
-- 
2.5.0

[PATCH 5/5] lib: Add lastmod: queries for filtering by last modification

[David Bremner]

From: Austin Clements amdra...@mit.edu

The implementation is essentially the same as the date range search
prior to Jani's fancy date parser.
---
 doc/man7/notmuch-search-terms.rst |  8 
 lib/database-private.h|  1 +
 lib/database.cc   |  4 
 test/T570-revision-tracking.sh| 17 +
 4 files changed, 30 insertions(+)

diff --git a/doc/man7/notmuch-search-terms.rst 
b/doc/man7/notmuch-search-terms.rst
index 1d27ac1..e71a525 100644
--- a/doc/man7/notmuch-search-terms.rst
+++ b/doc/man7/notmuch-search-terms.rst
@@ -54,6 +54,8 @@ indicate user-supplied values):
 
 -  date:since..until
 
+-  lastmod:since..until
+
 The **from:** prefix is used to match the name or address of the sender
 of an email message.
 
@@ -124,6 +126,12 @@ The time range can also be specified using timestamps with 
a syntax of:
 Each timestamp is a number representing the number of seconds since
 1970-01-01 00:00:00 UTC.
 
+The **lastmod:** prefix can be used to restrict the result by the
+database revision number of when messages were last modified (tags
+were added/removed or filenames changed).  This is usually used in
+conjunction with the **--uuid** argument to **notmuch search**
+to find messages that have changed since an earlier query.
+
 Operators
 -
 
diff --git a/lib/database-private.h b/lib/database-private.h
index 4e93257..3fb10f7 100644
--- a/lib/database-private.h
+++ b/lib/database-private.h
@@ -176,6 +176,7 @@ struct _notmuch_database {
 Xapian::TermGenerator *term_gen;
 Xapian::ValueRangeProcessor *value_range_processor;
 Xapian::ValueRangeProcessor *date_range_processor;
+Xapian::ValueRangeProcessor *last_mod_range_processor;
 };
 
 /* Prior to database version 3, features were implied by the database
diff --git a/lib/database.cc b/lib/database.cc
index fc78769..bab3334 100644
--- a/lib/database.cc
+++ b/lib/database.cc
@@ -1000,6 +1000,7 @@ notmuch_database_open_verbose (const char *path,
notmuch-term_gen-set_stemmer (Xapian::Stem (english));
notmuch-value_range_processor = new Xapian::NumberValueRangeProcessor 
(NOTMUCH_VALUE_TIMESTAMP);
notmuch-date_range_processor = new ParseTimeValueRangeProcessor 
(NOTMUCH_VALUE_TIMESTAMP);
+   notmuch-last_mod_range_processor = new 
Xapian::NumberValueRangeProcessor (NOTMUCH_VALUE_LAST_MOD, lastmod:);
 
notmuch-query_parser-set_default_op (Xapian::Query::OP_AND);
notmuch-query_parser-set_database (*notmuch-xapian_db);
@@ -1007,6 +1008,7 @@ notmuch_database_open_verbose (const char *path,
notmuch-query_parser-set_stemming_strategy 
(Xapian::QueryParser::STEM_SOME);
notmuch-query_parser-add_valuerangeprocessor 
(notmuch-value_range_processor);
notmuch-query_parser-add_valuerangeprocessor 
(notmuch-date_range_processor);
+   notmuch-query_parser-add_valuerangeprocessor 
(notmuch-last_mod_range_processor);
 
for (i = 0; i  ARRAY_SIZE (BOOLEAN_PREFIX_EXTERNAL); i++) {
prefix_t *prefix = BOOLEAN_PREFIX_EXTERNAL[i];
@@ -1085,6 +1087,8 @@ notmuch_database_close (notmuch_database_t *notmuch)
 notmuch-value_range_processor = NULL;
 delete notmuch-date_range_processor;
 notmuch-date_range_processor = NULL;
+delete notmuch-last_mod_range_processor;
+notmuch-last_mod_range_processor = NULL;
 
 return status;
 }
diff --git a/test/T570-revision-tracking.sh b/test/T570-revision-tracking.sh
index 20b44cb..0936011 100755
--- a/test/T570-revision-tracking.sh
+++ b/test/T570-revision-tracking.sh
@@ -73,4 +73,21 @@ test_expect_success 'tag succeeds with correct uuid' \
 test_expect_code 1 'tag fails with incorrect uuid' \
 notmuch tag --uuid=this-is-no-uuid '*' +test2
 
+test_begin_subtest 'lastmod:0.. matches everything'
+total=$(notmuch count '*')
+modtotal=$(notmuch count lastmod:0..)
+test_expect_equal $total $modtotal
+
+test_begin_subtest 'lastmod:100.. matches nothing'
+modtotal=$(notmuch count lastmod:100..)
+test_expect_equal 0 $modtotal
+
+test_begin_subtest 'exclude one message using lastmod'
+lastmod=$(notmuch count --lastmod '*' | cut -f3)
+total=$(notmuch count '*')
+notmuch tag +4efc743a.3060...@april.org id:4efc743a.3060...@april.org
+subtotal=$(notmuch count lastmod:..$lastmod)
+result=$(($subtotal == $total-1))
+test_expect_equal 1 $result
+
 test_done
-- 
2.5.0

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH 5/5] lib: Add "lastmod:" queries for filtering by last modification

[David Bremner]

From: Austin Clements 

The implementation is essentially the same as the date range search
prior to Jani's fancy date parser.
---
 doc/man7/notmuch-search-terms.rst |  8 
 lib/database-private.h|  1 +
 lib/database.cc   |  4 
 test/T570-revision-tracking.sh| 17 +
 4 files changed, 30 insertions(+)

diff --git a/doc/man7/notmuch-search-terms.rst 
b/doc/man7/notmuch-search-terms.rst
index 1d27ac1..e71a525 100644
--- a/doc/man7/notmuch-search-terms.rst
+++ b/doc/man7/notmuch-search-terms.rst
@@ -54,6 +54,8 @@ indicate user-supplied values):

 -  date:..

+-  lastmod:..
+
 The **from:** prefix is used to match the name or address of the sender
 of an email message.

@@ -124,6 +126,12 @@ The time range can also be specified using timestamps with 
a syntax of:
 Each timestamp is a number representing the number of seconds since
 1970-01-01 00:00:00 UTC.

+The **lastmod:** prefix can be used to restrict the result by the
+database revision number of when messages were last modified (tags
+were added/removed or filenames changed).  This is usually used in
+conjunction with the **--uuid** argument to **notmuch search**
+to find messages that have changed since an earlier query.
+
 Operators
 -

diff --git a/lib/database-private.h b/lib/database-private.h
index 4e93257..3fb10f7 100644
--- a/lib/database-private.h
+++ b/lib/database-private.h
@@ -176,6 +176,7 @@ struct _notmuch_database {
 Xapian::TermGenerator *term_gen;
 Xapian::ValueRangeProcessor *value_range_processor;
 Xapian::ValueRangeProcessor *date_range_processor;
+Xapian::ValueRangeProcessor *last_mod_range_processor;
 };

 /* Prior to database version 3, features were implied by the database
diff --git a/lib/database.cc b/lib/database.cc
index fc78769..bab3334 100644
--- a/lib/database.cc
+++ b/lib/database.cc
@@ -1000,6 +1000,7 @@ notmuch_database_open_verbose (const char *path,
notmuch->term_gen->set_stemmer (Xapian::Stem ("english"));
notmuch->value_range_processor = new Xapian::NumberValueRangeProcessor 
(NOTMUCH_VALUE_TIMESTAMP);
notmuch->date_range_processor = new ParseTimeValueRangeProcessor 
(NOTMUCH_VALUE_TIMESTAMP);
+   notmuch->last_mod_range_processor = new 
Xapian::NumberValueRangeProcessor (NOTMUCH_VALUE_LAST_MOD, "lastmod:");

notmuch->query_parser->set_default_op (Xapian::Query::OP_AND);
notmuch->query_parser->set_database (*notmuch->xapian_db);
@@ -1007,6 +1008,7 @@ notmuch_database_open_verbose (const char *path,
notmuch->query_parser->set_stemming_strategy 
(Xapian::QueryParser::STEM_SOME);
notmuch->query_parser->add_valuerangeprocessor 
(notmuch->value_range_processor);
notmuch->query_parser->add_valuerangeprocessor 
(notmuch->date_range_processor);
+   notmuch->query_parser->add_valuerangeprocessor 
(notmuch->last_mod_range_processor);

for (i = 0; i < ARRAY_SIZE (BOOLEAN_PREFIX_EXTERNAL); i++) {
prefix_t *prefix = _PREFIX_EXTERNAL[i];
@@ -1085,6 +1087,8 @@ notmuch_database_close (notmuch_database_t *notmuch)
 notmuch->value_range_processor = NULL;
 delete notmuch->date_range_processor;
 notmuch->date_range_processor = NULL;
+delete notmuch->last_mod_range_processor;
+notmuch->last_mod_range_processor = NULL;

 return status;
 }
diff --git a/test/T570-revision-tracking.sh b/test/T570-revision-tracking.sh
index 74e3ba9..55337e6 100755
--- a/test/T570-revision-tracking.sh
+++ b/test/T570-revision-tracking.sh
@@ -73,4 +73,21 @@ test_expect_success 'tag succeeds with correct uuid' \
 test_expect_code 1 'tag fails with incorrect uuid' \
 "notmuch tag --uuid=this-is-no-uuid '*' +test2"

+test_begin_subtest 'lastmod:0.. matches everything'
+total=$(notmuch count '*')
+modtotal=$(notmuch count lastmod:0..)
+test_expect_equal "$total" "$modtotal"
+
+test_begin_subtest 'lastmod:100.. matches nothing'
+modtotal=$(notmuch count lastmod:100..)
+test_expect_equal 0 "$modtotal"
+
+test_begin_subtest 'exclude one message using lastmod'
+lastmod=$(notmuch count --output=modifications '*' | cut -f2)
+total=$(notmuch count '*')
+notmuch tag +4EFC743A.3060609 at april.org id:4EFC743A.3060609 at april.org
+subtotal=$(notmuch count lastmod:..$lastmod)
+result=$(($subtotal == $total-1))
+test_expect_equal 1 "$result"
+
 test_done
-- 
2.1.4

[PATCH 5/5] lib: Add lastmod: queries for filtering by last modification

[David Bremner]

From: Austin Clements amdra...@mit.edu

The implementation is essentially the same as the date range search
prior to Jani's fancy date parser.
---
 doc/man7/notmuch-search-terms.rst |  8 
 lib/database-private.h|  1 +
 lib/database.cc   |  4 
 test/T570-revision-tracking.sh| 17 +
 4 files changed, 30 insertions(+)

diff --git a/doc/man7/notmuch-search-terms.rst 
b/doc/man7/notmuch-search-terms.rst
index 1d27ac1..e71a525 100644
--- a/doc/man7/notmuch-search-terms.rst
+++ b/doc/man7/notmuch-search-terms.rst
@@ -54,6 +54,8 @@ indicate user-supplied values):
 
 -  date:since..until
 
+-  lastmod:since..until
+
 The **from:** prefix is used to match the name or address of the sender
 of an email message.
 
@@ -124,6 +126,12 @@ The time range can also be specified using timestamps with 
a syntax of:
 Each timestamp is a number representing the number of seconds since
 1970-01-01 00:00:00 UTC.
 
+The **lastmod:** prefix can be used to restrict the result by the
+database revision number of when messages were last modified (tags
+were added/removed or filenames changed).  This is usually used in
+conjunction with the **--uuid** argument to **notmuch search**
+to find messages that have changed since an earlier query.
+
 Operators
 -
 
diff --git a/lib/database-private.h b/lib/database-private.h
index 4e93257..3fb10f7 100644
--- a/lib/database-private.h
+++ b/lib/database-private.h
@@ -176,6 +176,7 @@ struct _notmuch_database {
 Xapian::TermGenerator *term_gen;
 Xapian::ValueRangeProcessor *value_range_processor;
 Xapian::ValueRangeProcessor *date_range_processor;
+Xapian::ValueRangeProcessor *last_mod_range_processor;
 };
 
 /* Prior to database version 3, features were implied by the database
diff --git a/lib/database.cc b/lib/database.cc
index fc78769..bab3334 100644
--- a/lib/database.cc
+++ b/lib/database.cc
@@ -1000,6 +1000,7 @@ notmuch_database_open_verbose (const char *path,
notmuch-term_gen-set_stemmer (Xapian::Stem (english));
notmuch-value_range_processor = new Xapian::NumberValueRangeProcessor 
(NOTMUCH_VALUE_TIMESTAMP);
notmuch-date_range_processor = new ParseTimeValueRangeProcessor 
(NOTMUCH_VALUE_TIMESTAMP);
+   notmuch-last_mod_range_processor = new 
Xapian::NumberValueRangeProcessor (NOTMUCH_VALUE_LAST_MOD, lastmod:);
 
notmuch-query_parser-set_default_op (Xapian::Query::OP_AND);
notmuch-query_parser-set_database (*notmuch-xapian_db);
@@ -1007,6 +1008,7 @@ notmuch_database_open_verbose (const char *path,
notmuch-query_parser-set_stemming_strategy 
(Xapian::QueryParser::STEM_SOME);
notmuch-query_parser-add_valuerangeprocessor 
(notmuch-value_range_processor);
notmuch-query_parser-add_valuerangeprocessor 
(notmuch-date_range_processor);
+   notmuch-query_parser-add_valuerangeprocessor 
(notmuch-last_mod_range_processor);
 
for (i = 0; i  ARRAY_SIZE (BOOLEAN_PREFIX_EXTERNAL); i++) {
prefix_t *prefix = BOOLEAN_PREFIX_EXTERNAL[i];
@@ -1085,6 +1087,8 @@ notmuch_database_close (notmuch_database_t *notmuch)
 notmuch-value_range_processor = NULL;
 delete notmuch-date_range_processor;
 notmuch-date_range_processor = NULL;
+delete notmuch-last_mod_range_processor;
+notmuch-last_mod_range_processor = NULL;
 
 return status;
 }
diff --git a/test/T570-revision-tracking.sh b/test/T570-revision-tracking.sh
index 74e3ba9..55337e6 100755
--- a/test/T570-revision-tracking.sh
+++ b/test/T570-revision-tracking.sh
@@ -73,4 +73,21 @@ test_expect_success 'tag succeeds with correct uuid' \
 test_expect_code 1 'tag fails with incorrect uuid' \
 notmuch tag --uuid=this-is-no-uuid '*' +test2
 
+test_begin_subtest 'lastmod:0.. matches everything'
+total=$(notmuch count '*')
+modtotal=$(notmuch count lastmod:0..)
+test_expect_equal $total $modtotal
+
+test_begin_subtest 'lastmod:100.. matches nothing'
+modtotal=$(notmuch count lastmod:100..)
+test_expect_equal 0 $modtotal
+
+test_begin_subtest 'exclude one message using lastmod'
+lastmod=$(notmuch count --output=modifications '*' | cut -f2)
+total=$(notmuch count '*')
+notmuch tag +4efc743a.3060...@april.org id:4efc743a.3060...@april.org
+subtotal=$(notmuch count lastmod:..$lastmod)
+result=$(($subtotal == $total-1))
+test_expect_equal 1 $result
+
 test_done
-- 
2.1.4

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch