[PATCH v2 00/13] literal folder: prefix, new path: prefix
On Wed, Mar 05 2014, David Bremner wrote:
> Tomi Ollila writes:
>>
>> (IIRC openssl sha256 is available at least in Mac OS X, probably in other
>> BSD:s & Solaris too)
>>
>
> OTOH sha256sum is in coreutils, so everywhere with a gnu userland has
> it. Some might not have openssl (it's priority optional on Debian,
> e.g.).
Aaah these response times are a pleasure... :D
cd $TEST_DIRECTORY/test-databases
if hash sha256sum 2>/dev/null
then
sha256sum --quiet --check ${dbtarball}.sha256 || error "checksum of
database failed"
else
# opportunistically try to run openssl sha256 in case sha256sum is not
available
read sha256sum_exp filename < ${dbtarball}.sha256
sha256sum_act=`openssl sha256 "${dbtarball}" | sed 's/ .*//'`
test "$sha256sum_exp" = "$sha256sum_act" || error "checksum of database
failed"
fi
cd - >& /dev/null
>
> d
Tomi
[PATCH v2 00/13] literal folder: prefix, new path: prefix
Tomi Ollila writes: > > (IIRC openssl sha256 is available at least in Mac OS X, probably in other > BSD:s & Solaris too) > OTOH sha256sum is in coreutils, so everywhere with a gnu userland has it. Some might not have openssl (it's priority optional on Debian, e.g.). d
[PATCH v2 00/13] literal folder: prefix, new path: prefix
On Tue, Mar 04 2014, David Bremner wrote: > Jani Nikula writes: > >> >> I think we should use the prereq mechanism in the tests to check if a >> required test database is in place, skipping the test if not. This >> should be fairly straightforward to do, and does not care *how* the test >> database is to be fetched. (I'm undecided whether the test should verify >> the checksum, or the part that fetches the database. Minor detail.) > > Yep, prereqs sounds fine. SHA256 verification is pretty fast (on the > scale of test suite operations), so I'd lean to doing it late, but > I could be convinced otherwise. I decline commenting on prereqs... ;/ >> >> I know git annex in principle, but don't have any practical experience >> with it. I don't really have any ideas for that. > > Well, if you don't mind the rest of the setup (SHA256 sums checked, > tarballs fetched if needed), we can forge ahead for now. I can contribute script with interface ./fetch-and-check.sh [--verify-only] destdir url sha256sum > > d Tomi
[PATCH v2 00/13] literal folder: prefix, new path: prefix
Tomi Ollila writes:
>
> I can contribute script with interface
>
> ./fetch-and-check.sh [--verify-only] destdir url sha256sum
>
This script is basically embedded in
http://pivot.cs.unb.ca/git?p=notmuch.git;a=commitdiff;h=f93e32b11e517e160c755355de55ab465206dc62
except that I rebased away the verification in the actual test :(
Something like
diff --git a/test/T530-upgrade.sh b/test/T530-upgrade.sh
index 3c0134f..768242d 100755
--- a/test/T530-upgrade.sh
+++ b/test/T530-upgrade.sh
@@ -5,6 +5,10 @@ test_description="database upgrade"
dbtarball=database-v1.tar.xz
+cd $TEST_DIRECTORY/test-databases
+sha256sum --quiet --check ${dbtarball}.sha256 || error "checksum of database
failed"
+cd - >& /dev/null
+
tar Jxf $TEST_DIRECTORY/test-databases/${dbtarball} -C ${MAIL_DIR}
--strip-component
test_begin_subtest "folder: search does not work with old database version"
Re: [PATCH v2 00/13] literal folder: prefix, new path: prefix
On Tue, Mar 04 2014, David Bremner da...@tethera.net wrote: Jani Nikula j...@nikula.org writes: I think we should use the prereq mechanism in the tests to check if a required test database is in place, skipping the test if not. This should be fairly straightforward to do, and does not care *how* the test database is to be fetched. (I'm undecided whether the test should verify the checksum, or the part that fetches the database. Minor detail.) Yep, prereqs sounds fine. SHA256 verification is pretty fast (on the scale of test suite operations), so I'd lean to doing it late, but I could be convinced otherwise. I decline commenting on prereqs... ;/ I know git annex in principle, but don't have any practical experience with it. I don't really have any ideas for that. Well, if you don't mind the rest of the setup (SHA256 sums checked, tarballs fetched if needed), we can forge ahead for now. I can contribute script with interface ./fetch-and-check.sh [--verify-only] destdir url sha256sum d Tomi ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
Re: [PATCH v2 00/13] literal folder: prefix, new path: prefix
Tomi Ollila tomi.oll...@iki.fi writes:
I can contribute script with interface
./fetch-and-check.sh [--verify-only] destdir url sha256sum
This script is basically embedded in
http://pivot.cs.unb.ca/git?p=notmuch.git;a=commitdiff;h=f93e32b11e517e160c755355de55ab465206dc62
except that I rebased away the verification in the actual test :(
Something like
diff --git a/test/T530-upgrade.sh b/test/T530-upgrade.sh
index 3c0134f..768242d 100755
--- a/test/T530-upgrade.sh
+++ b/test/T530-upgrade.sh
@@ -5,6 +5,10 @@ test_description=database upgrade
dbtarball=database-v1.tar.xz
+cd $TEST_DIRECTORY/test-databases
+sha256sum --quiet --check ${dbtarball}.sha256 || error checksum of database
failed
+cd - /dev/null
+
tar Jxf $TEST_DIRECTORY/test-databases/${dbtarball} -C ${MAIL_DIR}
--strip-component
test_begin_subtest folder: search does not work with old database version
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch
Re: [PATCH v2 00/13] literal folder: prefix, new path: prefix
On Wed, Mar 05 2014, David Bremner da...@tethera.net wrote:
Tomi Ollila tomi.oll...@iki.fi writes:
I can contribute script with interface
./fetch-and-check.sh [--verify-only] destdir url sha256sum
This script is basically embedded in
http://pivot.cs.unb.ca/git?p=notmuch.git;a=commitdiff;h=f93e32b11e517e160c755355de55ab465206dc62
except that I rebased away the verification in the actual test :(
Something like
diff --git a/test/T530-upgrade.sh b/test/T530-upgrade.sh
index 3c0134f..768242d 100755
--- a/test/T530-upgrade.sh
+++ b/test/T530-upgrade.sh
@@ -5,6 +5,10 @@ test_description=database upgrade
dbtarball=database-v1.tar.xz
+cd $TEST_DIRECTORY/test-databases
+sha256sum --quiet --check ${dbtarball}.sha256 || error checksum of database
failed
+cd - /dev/null
more portable alternative:
read sha256sum_exp ${dbtarball}.sha256
#sha256sum_act=`openssl sha256 ${dbtarball} | sed 's/ .*//'`
sha256sum_act=`openssl sha256 ${dbtarball}`
test $sha256sum_exp = $sha256sum_act || error checksum of database failed
(IIRC openssl sha256 is available at least in Mac OS X, probably in other
BSD:s Solaris too)
+
tar Jxf $TEST_DIRECTORY/test-databases/${dbtarball} -C ${MAIL_DIR}
--strip-component
test_begin_subtest folder: search does not work with old database version
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch
Re: [PATCH v2 00/13] literal folder: prefix, new path: prefix
Tomi Ollila tomi.oll...@iki.fi writes: (IIRC openssl sha256 is available at least in Mac OS X, probably in other BSD:s Solaris too) OTOH sha256sum is in coreutils, so everywhere with a gnu userland has it. Some might not have openssl (it's priority optional on Debian, e.g.). d ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
Re: [PATCH v2 00/13] literal folder: prefix, new path: prefix
On Wed, Mar 05 2014, David Bremner da...@tethera.net wrote:
Tomi Ollila tomi.oll...@iki.fi writes:
(IIRC openssl sha256 is available at least in Mac OS X, probably in other
BSD:s Solaris too)
OTOH sha256sum is in coreutils, so everywhere with a gnu userland has
it. Some might not have openssl (it's priority optional on Debian,
e.g.).
Aaah these response times are a pleasure... :D
cd $TEST_DIRECTORY/test-databases
if hash sha256sum 2/dev/null
then
sha256sum --quiet --check ${dbtarball}.sha256 || error checksum of
database failed
else
# opportunistically try to run openssl sha256 in case sha256sum is not
available
read sha256sum_exp filename ${dbtarball}.sha256
sha256sum_act=`openssl sha256 ${dbtarball} | sed 's/ .*//'`
test $sha256sum_exp = $sha256sum_act || error checksum of database
failed
fi
cd - /dev/null
d
Tomi
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch
[PATCH v2 00/13] literal folder: prefix, new path: prefix
On Sun, 02 Mar 2014, David Bremner wrote: > Jani Nikula writes: >> >> I've dropped most of the content in patches 7 and 10 due to their >> size. The patches (and the whole series) are available in the >> boolean-folder-and-path-v2 branch at >> git://gitorious.org/jani/notmuch.git. Web interface at >> https://gitorious.org/jani/notmuch/commits/0b3dd2d1cc6c413ea07ea326883ac448499c0e79. >> > > Hi Jani; > > I put a some proposed changes at > > > http://pivot.cs.unb.ca/git?p=notmuch.git;a=shortlog;h=refs/heads/boolean-folder-and-path-v2 > > After playing with it a bit, I decided to > > - leave the source for the corpuses in the main tree. As we > discussed, we don't expect them to change much, and they are > text files. > > - store the binary database blobs externally. We could make the > same argument about infrequent changes here, except that even > the tiniest change will in practice require a whole new copy be > stored. > > - compute a checksum file > - test: add machinery to fetch and publish test databases > - test: commit database-v1.tar.xz checksum, ignore actual database > >- this is an independant bug fix that should probably be pushed anyway: > > - test: don't use $(dir) in recipes. > > - I also made some unrelated changes including switching to xz > compression. I intentionally didn't squash my patches into yours > so you see and decide > > - convert to XZ compression > - convert test to XZ > > Originally I thought metadata in the database tarball is > needed. It turns out not. I think it's harmless, but feel free > to drop it. > > - add some metadata to the database tarball I'm fine with all of the above. > - I struggled to find a way that would allow people to manage the > tarballs in a nice way while not requiring everyone to > install/learn new tools. My conclusion is in > > - test: add support for git annex managing test databases > > We still need to think about what to do for people installing from > tarballs. As it stands they will need to either have network access or > skip the relevant tests. Probably at least one followup patch is needed > here. The alternative would be to include the database tarballs in the > source; this will pretty quickly double the size of the source tarball. I think we should use the prereq mechanism in the tests to check if a required test database is in place, skipping the test if not. This should be fairly straightforward to do, and does not care *how* the test database is to be fetched. (I'm undecided whether the test should verify the checksum, or the part that fetches the database. Minor detail.) I know git annex in principle, but don't have any practical experience with it. I don't really have any ideas for that. BR, Jani.
[PATCH v2 00/13] literal folder: prefix, new path: prefix
Jani Nikula writes: > > I think we should use the prereq mechanism in the tests to check if a > required test database is in place, skipping the test if not. This > should be fairly straightforward to do, and does not care *how* the test > database is to be fetched. (I'm undecided whether the test should verify > the checksum, or the part that fetches the database. Minor detail.) Yep, prereqs sounds fine. SHA256 verification is pretty fast (on the scale of test suite operations), so I'd lean to doing it late, but I could be convinced otherwise. > > I know git annex in principle, but don't have any practical experience > with it. I don't really have any ideas for that. Well, if you don't mind the rest of the setup (SHA256 sums checked, tarballs fetched if needed), we can forge ahead for now. d
Re: [PATCH v2 00/13] literal folder: prefix, new path: prefix
On Sun, 02 Mar 2014, David Bremner da...@tethera.net wrote: Jani Nikula j...@nikula.org writes: I've dropped most of the content in patches 7 and 10 due to their size. The patches (and the whole series) are available in the boolean-folder-and-path-v2 branch at git://gitorious.org/jani/notmuch.git. Web interface at https://gitorious.org/jani/notmuch/commits/0b3dd2d1cc6c413ea07ea326883ac448499c0e79. Hi Jani; I put a some proposed changes at http://pivot.cs.unb.ca/git?p=notmuch.git;a=shortlog;h=refs/heads/boolean-folder-and-path-v2 After playing with it a bit, I decided to - leave the source for the corpuses in the main tree. As we discussed, we don't expect them to change much, and they are text files. - store the binary database blobs externally. We could make the same argument about infrequent changes here, except that even the tiniest change will in practice require a whole new copy be stored. - compute a checksum file - test: add machinery to fetch and publish test databases - test: commit database-v1.tar.xz checksum, ignore actual database - this is an independant bug fix that should probably be pushed anyway: - test: don't use $(dir) in recipes. - I also made some unrelated changes including switching to xz compression. I intentionally didn't squash my patches into yours so you see and decide - convert to XZ compression - convert test to XZ Originally I thought metadata in the database tarball is needed. It turns out not. I think it's harmless, but feel free to drop it. - add some metadata to the database tarball I'm fine with all of the above. - I struggled to find a way that would allow people to manage the tarballs in a nice way while not requiring everyone to install/learn new tools. My conclusion is in - test: add support for git annex managing test databases We still need to think about what to do for people installing from tarballs. As it stands they will need to either have network access or skip the relevant tests. Probably at least one followup patch is needed here. The alternative would be to include the database tarballs in the source; this will pretty quickly double the size of the source tarball. I think we should use the prereq mechanism in the tests to check if a required test database is in place, skipping the test if not. This should be fairly straightforward to do, and does not care *how* the test database is to be fetched. (I'm undecided whether the test should verify the checksum, or the part that fetches the database. Minor detail.) I know git annex in principle, but don't have any practical experience with it. I don't really have any ideas for that. BR, Jani. ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
Re: [PATCH v2 00/13] literal folder: prefix, new path: prefix
Jani Nikula j...@nikula.org writes: I think we should use the prereq mechanism in the tests to check if a required test database is in place, skipping the test if not. This should be fairly straightforward to do, and does not care *how* the test database is to be fetched. (I'm undecided whether the test should verify the checksum, or the part that fetches the database. Minor detail.) Yep, prereqs sounds fine. SHA256 verification is pretty fast (on the scale of test suite operations), so I'd lean to doing it late, but I could be convinced otherwise. I know git annex in principle, but don't have any practical experience with it. I don't really have any ideas for that. Well, if you don't mind the rest of the setup (SHA256 sums checked, tarballs fetched if needed), we can forge ahead for now. d ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
[PATCH v2 00/13] literal folder: prefix, new path: prefix
Jani Nikula writes: > > I've dropped most of the content in patches 7 and 10 due to their > size. The patches (and the whole series) are available in the > boolean-folder-and-path-v2 branch at > git://gitorious.org/jani/notmuch.git. Web interface at > https://gitorious.org/jani/notmuch/commits/0b3dd2d1cc6c413ea07ea326883ac448499c0e79. > Hi Jani; I put a some proposed changes at http://pivot.cs.unb.ca/git?p=notmuch.git;a=shortlog;h=refs/heads/boolean-folder-and-path-v2 After playing with it a bit, I decided to - leave the source for the corpuses in the main tree. As we discussed, we don't expect them to change much, and they are text files. - store the binary database blobs externally. We could make the same argument about infrequent changes here, except that even the tiniest change will in practice require a whole new copy be stored. - compute a checksum file - test: add machinery to fetch and publish test databases - test: commit database-v1.tar.xz checksum, ignore actual database - this is an independant bug fix that should probably be pushed anyway: - test: don't use $(dir) in recipes. - I also made some unrelated changes including switching to xz compression. I intentionally didn't squash my patches into yours so you see and decide - convert to XZ compression - convert test to XZ Originally I thought metadata in the database tarball is needed. It turns out not. I think it's harmless, but feel free to drop it. - add some metadata to the database tarball - I struggled to find a way that would allow people to manage the tarballs in a nice way while not requiring everyone to install/learn new tools. My conclusion is in - test: add support for git annex managing test databases We still need to think about what to do for people installing from tarballs. As it stands they will need to either have network access or skip the relevant tests. Probably at least one followup patch is needed here. The alternative would be to include the database tarballs in the source; this will pretty quickly double the size of the source tarball.
Re: [PATCH v2 00/13] literal folder: prefix, new path: prefix
Jani Nikula j...@nikula.org writes: I've dropped most of the content in patches 7 and 10 due to their size. The patches (and the whole series) are available in the boolean-folder-and-path-v2 branch at git://gitorious.org/jani/notmuch.git. Web interface at https://gitorious.org/jani/notmuch/commits/0b3dd2d1cc6c413ea07ea326883ac448499c0e79. Hi Jani; I put a some proposed changes at http://pivot.cs.unb.ca/git?p=notmuch.git;a=shortlog;h=refs/heads/boolean-folder-and-path-v2 After playing with it a bit, I decided to - leave the source for the corpuses in the main tree. As we discussed, we don't expect them to change much, and they are text files. - store the binary database blobs externally. We could make the same argument about infrequent changes here, except that even the tiniest change will in practice require a whole new copy be stored. - compute a checksum file - test: add machinery to fetch and publish test databases - test: commit database-v1.tar.xz checksum, ignore actual database - this is an independant bug fix that should probably be pushed anyway: - test: don't use $(dir) in recipes. - I also made some unrelated changes including switching to xz compression. I intentionally didn't squash my patches into yours so you see and decide - convert to XZ compression - convert test to XZ Originally I thought metadata in the database tarball is needed. It turns out not. I think it's harmless, but feel free to drop it. - add some metadata to the database tarball - I struggled to find a way that would allow people to manage the tarballs in a nice way while not requiring everyone to install/learn new tools. My conclusion is in - test: add support for git annex managing test databases We still need to think about what to do for people installing from tarballs. As it stands they will need to either have network access or skip the relevant tests. Probably at least one followup patch is needed here. The alternative would be to include the database tarballs in the source; this will pretty quickly double the size of the source tarball. ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
[PATCH v2 00/13] literal folder: prefix, new path: prefix
On Sun, Feb 23 2014, Mark Walters wrote:
> I have read most of this series, tested it and run the tests and LGTM +1.
>
> I read the C code fairly carefully, the tests rather less so but they
> looked sane, and I didn't really look at patch 9 for building old
> databases.
Well, I can add (at this point) that patch 9 is tolerable...
>
> Best wishes
>
> Mark
Tomi
>
>
>
>
> On Sat, 22 Feb 2014, Jani Nikula wrote:
>> Hi all, this is v2 of id:cover.1389304779.git.jani at nikula.org.
>>
>> The new path: prefix is a literal boolean prefix matching the paths,
>> relative from the maildir root, of the message files. There's no
>> interpretation of the maildir special cur/new folders, but a recursive
>> match is provided with "/**" suffix. See the patch for details.
>>
>> The folder: prefix becomes a literal boolean prefix, similar to path:,
>> except it matches the maildir cur/new folders in addition to the
>> specified path. There's no recursive version.
>>
>> Patches 1-5 add the above.
>>
>> Patches 6-8 change the test infrastructure to make it easier to add
>> multiple corpuses, and adds a new test for path: and folder:.
>>
>> Patches 9-11 add support for testing the database upgrade.
>>
>> Patches 12-13 update man pages.
>>
>>
>> I've dropped most of the content in patches 7 and 10 due to their
>> size. The patches (and the whole series) are available in the
>> boolean-folder-and-path-v2 branch at
>> git://gitorious.org/jani/notmuch.git. Web interface at
>> https://gitorious.org/jani/notmuch/commits/0b3dd2d1cc6c413ea07ea326883ac448499c0e79.
>>
>>
>> WARNING! The change requires a database format version bump, and a
>> database upgrade, which is automatically done on 'notmuch new'. The
>> upgrade is irreversible if you want to try this on your database! A
>> complete database rebuild is required for reverting the database format
>> version. Make sure your backups are in order!
>>
>>
>> BR,
>> Jani.
>>
>>
>> Jani Nikula (13):
>> lib: refactor folder term update after filename removal
>> lib: add support for path: prefix searches
>> test: make insert test use the path: prefix
>> lib: make folder: prefix literal
>> test: fix test for literal folder: search
>> test: make it possible to have several corpora
>> test: add new corpus with folders
>> test: add tests for the new boolean folder: and path: prefixes
>> devel: add script to generate test databases
>> test: add test database in format version 1
>> test: add database upgrade test from format version 1 to 2
>> man: update man pages for folder: and path: search terms
>> man: try to clarify the folder: and path: vs. --output=files confusion
>>
>> devel/gen-testdb.sh| 124
>> lib/database.cc| 45 -
>> lib/message.cc | 249 ---
>> lib/notmuch-private.h | 3 +
>> man/man1/notmuch-search.1 | 10 +-
>> man/man7/notmuch-search-terms.7| 28 ++-
>> test/.gitignore| 2 +-
>> test/Makefile.local| 2 +-
>> test/T070-insert.sh| 10 +-
>> test/T100-search-by-folder.sh | 24 ++-
>> test/T101-search-by-folder-and-path.sh | 83
>> test/T480-hex-escaping.sh | 4 +-
>> test/T530-upgrade.sh | 103 ++
>> test/corpus/{ => default}/cur/01:2,| 0
>> test/corpus/{ => default}/cur/02:2,| 0
>> test/corpus/{ => default}/cur/03:2,| 0
>> test/corpus/{ => default}/cur/04:2,| 0
>> test/corpus/{ => default}/cur/05:2,| 0
>> test/corpus/{ => default}/cur/06:2,| 0
>> test/corpus/{ => default}/cur/07:2,| 0
>> test/corpus/{ => default}/cur/08:2,| 0
>> test/corpus/{ => default}/cur/09:2,| 0
>> test/corpus/{ => default}/cur/10:2,| 0
>> test/corpus/{ => default}/cur/11:2,| 0
>> test/corpus/{ => default}/cur/12:2,| 0
>> test/corpus/{ => default}/cur/13:2,| 0
>> test/corpus/{ => default}/cur/14:2,| 0
>> test/corpus/{ => default}/cur/15:2,| 0
>> test/corpus/{ => default}/cur/16:2,| 0
>> test/corpus/{ => default}/cur/17:2,| 0
>> test/corpus/{ => default}/cur/18:2,| 0
>> test/corpus/{ => default}/cur/19:2,| 0
>> test/corpus/{ => default}/cur/20:2,| 0
>> test/corpus/{ => default}/cur/21:2,| 0
>> test/corpus/{ => default}/cur/22:2,| 0
>> test/corpus/{ => default}/cur/23:2,| 0
>> test/corpus/{ => default}/cur/24:2,| 0
>> test/corpus/{ => default}/cur/25:2,| 0
>> test/corpus/{ => default}/cur/26:2,| 0
>> test/corpus/{ => default}/cur/27:2,| 0
>> test/corpus/{ => default}/cur/28:2,| 0
>> test/corpus/{ => default}/cur/29:2,| 0
>> test/corpus/{ => default}/cur/30:2,| 0
>> test/corpus/{ => default}/cur/31:2,| 0
>> test/corpus/{ => default}/cur/32:2,| 0
>> test/corpus/{ => default}/cur/33:2,
[PATCH v2 00/13] literal folder: prefix, new path: prefix
Hi all, this is v2 of id:cover.1389304779.git.jani at nikula.org.
The new path: prefix is a literal boolean prefix matching the paths,
relative from the maildir root, of the message files. There's no
interpretation of the maildir special cur/new folders, but a recursive
match is provided with "/**" suffix. See the patch for details.
The folder: prefix becomes a literal boolean prefix, similar to path:,
except it matches the maildir cur/new folders in addition to the
specified path. There's no recursive version.
Patches 1-5 add the above.
Patches 6-8 change the test infrastructure to make it easier to add
multiple corpuses, and adds a new test for path: and folder:.
Patches 9-11 add support for testing the database upgrade.
Patches 12-13 update man pages.
I've dropped most of the content in patches 7 and 10 due to their
size. The patches (and the whole series) are available in the
boolean-folder-and-path-v2 branch at
git://gitorious.org/jani/notmuch.git. Web interface at
https://gitorious.org/jani/notmuch/commits/0b3dd2d1cc6c413ea07ea326883ac448499c0e79.
WARNING! The change requires a database format version bump, and a
database upgrade, which is automatically done on 'notmuch new'. The
upgrade is irreversible if you want to try this on your database! A
complete database rebuild is required for reverting the database format
version. Make sure your backups are in order!
BR,
Jani.
Jani Nikula (13):
lib: refactor folder term update after filename removal
lib: add support for path: prefix searches
test: make insert test use the path: prefix
lib: make folder: prefix literal
test: fix test for literal folder: search
test: make it possible to have several corpora
test: add new corpus with folders
test: add tests for the new boolean folder: and path: prefixes
devel: add script to generate test databases
test: add test database in format version 1
test: add database upgrade test from format version 1 to 2
man: update man pages for folder: and path: search terms
man: try to clarify the folder: and path: vs. --output=files confusion
devel/gen-testdb.sh| 124
lib/database.cc| 45 -
lib/message.cc | 249 ---
lib/notmuch-private.h | 3 +
man/man1/notmuch-search.1 | 10 +-
man/man7/notmuch-search-terms.7| 28 ++-
test/.gitignore| 2 +-
test/Makefile.local| 2 +-
test/T070-insert.sh| 10 +-
test/T100-search-by-folder.sh | 24 ++-
test/T101-search-by-folder-and-path.sh | 83
test/T480-hex-escaping.sh | 4 +-
test/T530-upgrade.sh | 103 ++
test/corpus/{ => default}/cur/01:2,| 0
test/corpus/{ => default}/cur/02:2,| 0
test/corpus/{ => default}/cur/03:2,| 0
test/corpus/{ => default}/cur/04:2,| 0
test/corpus/{ => default}/cur/05:2,| 0
test/corpus/{ => default}/cur/06:2,| 0
test/corpus/{ => default}/cur/07:2,| 0
test/corpus/{ => default}/cur/08:2,| 0
test/corpus/{ => default}/cur/09:2,| 0
test/corpus/{ => default}/cur/10:2,| 0
test/corpus/{ => default}/cur/11:2,| 0
test/corpus/{ => default}/cur/12:2,| 0
test/corpus/{ => default}/cur/13:2,| 0
test/corpus/{ => default}/cur/14:2,| 0
test/corpus/{ => default}/cur/15:2,| 0
test/corpus/{ => default}/cur/16:2,| 0
test/corpus/{ => default}/cur/17:2,| 0
test/corpus/{ => default}/cur/18:2,| 0
test/corpus/{ => default}/cur/19:2,| 0
test/corpus/{ => default}/cur/20:2,| 0
test/corpus/{ => default}/cur/21:2,| 0
test/corpus/{ => default}/cur/22:2,| 0
test/corpus/{ => default}/cur/23:2,| 0
test/corpus/{ => default}/cur/24:2,| 0
test/corpus/{ => default}/cur/25:2,| 0
test/corpus/{ => default}/cur/26:2,| 0
test/corpus/{ => default}/cur/27:2,| 0
test/corpus/{ => default}/cur/28:2,| 0
test/corpus/{ => default}/cur/29:2,| 0
test/corpus/{ => default}/cur/30:2,| 0
test/corpus/{ => default}/cur/31:2,| 0
test/corpus/{ => default}/cur/32:2,| 0
test/corpus/{ => default}/cur/33:2,| 0
test/corpus/{ => default}/cur/34:2,| 0
test/corpus/{ => default}/cur/35:2,| 0
test/corpus/{ => default}/cur/36:2,| 0
test/corpus/{ => default}/cur/37:2,| 0
test/corpus/{ => default}/cur/38:2,| 0
test/corpus/{ => default}/cur/39:2,| 0
test/corpus/{ => default}/cur/40:2,| 0
test/corpus/{ => default}/cur/41:2,| 0
test/corpus/{ => default}/cur/42:2,| 0
test/corpus/{ => default}/cur/43:2,| 0
test/corpus/{ => default}/cur/44:2,| 0
test/corpus/{ => default}/cur/45:2,| 0
test/corpus/{ => default}/cur/46:2,| 0
test/corpus/{ => default}/cur/47:2,| 0
test/corpus/{ => default}/cur/48:2,| 0
Re: [PATCH v2 00/13] literal folder: prefix, new path: prefix
On Sun, Feb 23 2014, Mark Walters markwalters1...@gmail.com wrote:
I have read most of this series, tested it and run the tests and LGTM +1.
I read the C code fairly carefully, the tests rather less so but they
looked sane, and I didn't really look at patch 9 for building old
databases.
Well, I can add (at this point) that patch 9 is tolerable...
Best wishes
Mark
Tomi
On Sat, 22 Feb 2014, Jani Nikula j...@nikula.org wrote:
Hi all, this is v2 of id:cover.1389304779.git.j...@nikula.org.
The new path: prefix is a literal boolean prefix matching the paths,
relative from the maildir root, of the message files. There's no
interpretation of the maildir special cur/new folders, but a recursive
match is provided with /** suffix. See the patch for details.
The folder: prefix becomes a literal boolean prefix, similar to path:,
except it matches the maildir cur/new folders in addition to the
specified path. There's no recursive version.
Patches 1-5 add the above.
Patches 6-8 change the test infrastructure to make it easier to add
multiple corpuses, and adds a new test for path: and folder:.
Patches 9-11 add support for testing the database upgrade.
Patches 12-13 update man pages.
I've dropped most of the content in patches 7 and 10 due to their
size. The patches (and the whole series) are available in the
boolean-folder-and-path-v2 branch at
git://gitorious.org/jani/notmuch.git. Web interface at
https://gitorious.org/jani/notmuch/commits/0b3dd2d1cc6c413ea07ea326883ac448499c0e79.
WARNING! The change requires a database format version bump, and a
database upgrade, which is automatically done on 'notmuch new'. The
upgrade is irreversible if you want to try this on your database! A
complete database rebuild is required for reverting the database format
version. Make sure your backups are in order!
BR,
Jani.
Jani Nikula (13):
lib: refactor folder term update after filename removal
lib: add support for path: prefix searches
test: make insert test use the path: prefix
lib: make folder: prefix literal
test: fix test for literal folder: search
test: make it possible to have several corpora
test: add new corpus with folders
test: add tests for the new boolean folder: and path: prefixes
devel: add script to generate test databases
test: add test database in format version 1
test: add database upgrade test from format version 1 to 2
man: update man pages for folder: and path: search terms
man: try to clarify the folder: and path: vs. --output=files confusion
devel/gen-testdb.sh| 124
lib/database.cc| 45 -
lib/message.cc | 249 ---
lib/notmuch-private.h | 3 +
man/man1/notmuch-search.1 | 10 +-
man/man7/notmuch-search-terms.7| 28 ++-
test/.gitignore| 2 +-
test/Makefile.local| 2 +-
test/T070-insert.sh| 10 +-
test/T100-search-by-folder.sh | 24 ++-
test/T101-search-by-folder-and-path.sh | 83
test/T480-hex-escaping.sh | 4 +-
test/T530-upgrade.sh | 103 ++
test/corpus/{ = default}/cur/01:2,| 0
test/corpus/{ = default}/cur/02:2,| 0
test/corpus/{ = default}/cur/03:2,| 0
test/corpus/{ = default}/cur/04:2,| 0
test/corpus/{ = default}/cur/05:2,| 0
test/corpus/{ = default}/cur/06:2,| 0
test/corpus/{ = default}/cur/07:2,| 0
test/corpus/{ = default}/cur/08:2,| 0
test/corpus/{ = default}/cur/09:2,| 0
test/corpus/{ = default}/cur/10:2,| 0
test/corpus/{ = default}/cur/11:2,| 0
test/corpus/{ = default}/cur/12:2,| 0
test/corpus/{ = default}/cur/13:2,| 0
test/corpus/{ = default}/cur/14:2,| 0
test/corpus/{ = default}/cur/15:2,| 0
test/corpus/{ = default}/cur/16:2,| 0
test/corpus/{ = default}/cur/17:2,| 0
test/corpus/{ = default}/cur/18:2,| 0
test/corpus/{ = default}/cur/19:2,| 0
test/corpus/{ = default}/cur/20:2,| 0
test/corpus/{ = default}/cur/21:2,| 0
test/corpus/{ = default}/cur/22:2,| 0
test/corpus/{ = default}/cur/23:2,| 0
test/corpus/{ = default}/cur/24:2,| 0
test/corpus/{ = default}/cur/25:2,| 0
test/corpus/{ = default}/cur/26:2,| 0
test/corpus/{ = default}/cur/27:2,| 0
test/corpus/{ = default}/cur/28:2,| 0
test/corpus/{ = default}/cur/29:2,| 0
test/corpus/{ = default}/cur/30:2,| 0
test/corpus/{ = default}/cur/31:2,| 0
test/corpus/{ = default}/cur/32:2,| 0
test/corpus/{ = default}/cur/33:2,| 0
test/corpus/{ = default}/cur/34:2,| 0
test/corpus/{ = default}/cur/35:2,| 0
test/corpus/{ = default}/cur/36:2,| 0
test/corpus/{ = default}/cur/37:2,| 0
test/corpus/{ =
[PATCH v2 00/13] literal folder: prefix, new path: prefix
I have read most of this series, tested it and run the tests and LGTM +1.
I read the C code fairly carefully, the tests rather less so but they
looked sane, and I didn't really look at patch 9 for building old
databases.
Best wishes
Mark
On Sat, 22 Feb 2014, Jani Nikula wrote:
> Hi all, this is v2 of id:cover.1389304779.git.jani at nikula.org.
>
> The new path: prefix is a literal boolean prefix matching the paths,
> relative from the maildir root, of the message files. There's no
> interpretation of the maildir special cur/new folders, but a recursive
> match is provided with "/**" suffix. See the patch for details.
>
> The folder: prefix becomes a literal boolean prefix, similar to path:,
> except it matches the maildir cur/new folders in addition to the
> specified path. There's no recursive version.
>
> Patches 1-5 add the above.
>
> Patches 6-8 change the test infrastructure to make it easier to add
> multiple corpuses, and adds a new test for path: and folder:.
>
> Patches 9-11 add support for testing the database upgrade.
>
> Patches 12-13 update man pages.
>
>
> I've dropped most of the content in patches 7 and 10 due to their
> size. The patches (and the whole series) are available in the
> boolean-folder-and-path-v2 branch at
> git://gitorious.org/jani/notmuch.git. Web interface at
> https://gitorious.org/jani/notmuch/commits/0b3dd2d1cc6c413ea07ea326883ac448499c0e79.
>
>
> WARNING! The change requires a database format version bump, and a
> database upgrade, which is automatically done on 'notmuch new'. The
> upgrade is irreversible if you want to try this on your database! A
> complete database rebuild is required for reverting the database format
> version. Make sure your backups are in order!
>
>
> BR,
> Jani.
>
>
> Jani Nikula (13):
> lib: refactor folder term update after filename removal
> lib: add support for path: prefix searches
> test: make insert test use the path: prefix
> lib: make folder: prefix literal
> test: fix test for literal folder: search
> test: make it possible to have several corpora
> test: add new corpus with folders
> test: add tests for the new boolean folder: and path: prefixes
> devel: add script to generate test databases
> test: add test database in format version 1
> test: add database upgrade test from format version 1 to 2
> man: update man pages for folder: and path: search terms
> man: try to clarify the folder: and path: vs. --output=files confusion
>
> devel/gen-testdb.sh| 124
> lib/database.cc| 45 -
> lib/message.cc | 249 ---
> lib/notmuch-private.h | 3 +
> man/man1/notmuch-search.1 | 10 +-
> man/man7/notmuch-search-terms.7| 28 ++-
> test/.gitignore| 2 +-
> test/Makefile.local| 2 +-
> test/T070-insert.sh| 10 +-
> test/T100-search-by-folder.sh | 24 ++-
> test/T101-search-by-folder-and-path.sh | 83
> test/T480-hex-escaping.sh | 4 +-
> test/T530-upgrade.sh | 103 ++
> test/corpus/{ => default}/cur/01:2,| 0
> test/corpus/{ => default}/cur/02:2,| 0
> test/corpus/{ => default}/cur/03:2,| 0
> test/corpus/{ => default}/cur/04:2,| 0
> test/corpus/{ => default}/cur/05:2,| 0
> test/corpus/{ => default}/cur/06:2,| 0
> test/corpus/{ => default}/cur/07:2,| 0
> test/corpus/{ => default}/cur/08:2,| 0
> test/corpus/{ => default}/cur/09:2,| 0
> test/corpus/{ => default}/cur/10:2,| 0
> test/corpus/{ => default}/cur/11:2,| 0
> test/corpus/{ => default}/cur/12:2,| 0
> test/corpus/{ => default}/cur/13:2,| 0
> test/corpus/{ => default}/cur/14:2,| 0
> test/corpus/{ => default}/cur/15:2,| 0
> test/corpus/{ => default}/cur/16:2,| 0
> test/corpus/{ => default}/cur/17:2,| 0
> test/corpus/{ => default}/cur/18:2,| 0
> test/corpus/{ => default}/cur/19:2,| 0
> test/corpus/{ => default}/cur/20:2,| 0
> test/corpus/{ => default}/cur/21:2,| 0
> test/corpus/{ => default}/cur/22:2,| 0
> test/corpus/{ => default}/cur/23:2,| 0
> test/corpus/{ => default}/cur/24:2,| 0
> test/corpus/{ => default}/cur/25:2,| 0
> test/corpus/{ => default}/cur/26:2,| 0
> test/corpus/{ => default}/cur/27:2,| 0
> test/corpus/{ => default}/cur/28:2,| 0
> test/corpus/{ => default}/cur/29:2,| 0
> test/corpus/{ => default}/cur/30:2,| 0
> test/corpus/{ => default}/cur/31:2,| 0
> test/corpus/{ => default}/cur/32:2,| 0
> test/corpus/{ => default}/cur/33:2,| 0
> test/corpus/{ => default}/cur/34:2,| 0
> test/corpus/{ => default}/cur/35:2,| 0
> test/corpus/{ => default}/cur/36:2,| 0
> test/corpus/{ => default}/cur/37:2,| 0
> test/corpus/{ =>
Re: [PATCH v2 00/13] literal folder: prefix, new path: prefix
I have read most of this series, tested it and run the tests and LGTM +1.
I read the C code fairly carefully, the tests rather less so but they
looked sane, and I didn't really look at patch 9 for building old
databases.
Best wishes
Mark
On Sat, 22 Feb 2014, Jani Nikula j...@nikula.org wrote:
Hi all, this is v2 of id:cover.1389304779.git.j...@nikula.org.
The new path: prefix is a literal boolean prefix matching the paths,
relative from the maildir root, of the message files. There's no
interpretation of the maildir special cur/new folders, but a recursive
match is provided with /** suffix. See the patch for details.
The folder: prefix becomes a literal boolean prefix, similar to path:,
except it matches the maildir cur/new folders in addition to the
specified path. There's no recursive version.
Patches 1-5 add the above.
Patches 6-8 change the test infrastructure to make it easier to add
multiple corpuses, and adds a new test for path: and folder:.
Patches 9-11 add support for testing the database upgrade.
Patches 12-13 update man pages.
I've dropped most of the content in patches 7 and 10 due to their
size. The patches (and the whole series) are available in the
boolean-folder-and-path-v2 branch at
git://gitorious.org/jani/notmuch.git. Web interface at
https://gitorious.org/jani/notmuch/commits/0b3dd2d1cc6c413ea07ea326883ac448499c0e79.
WARNING! The change requires a database format version bump, and a
database upgrade, which is automatically done on 'notmuch new'. The
upgrade is irreversible if you want to try this on your database! A
complete database rebuild is required for reverting the database format
version. Make sure your backups are in order!
BR,
Jani.
Jani Nikula (13):
lib: refactor folder term update after filename removal
lib: add support for path: prefix searches
test: make insert test use the path: prefix
lib: make folder: prefix literal
test: fix test for literal folder: search
test: make it possible to have several corpora
test: add new corpus with folders
test: add tests for the new boolean folder: and path: prefixes
devel: add script to generate test databases
test: add test database in format version 1
test: add database upgrade test from format version 1 to 2
man: update man pages for folder: and path: search terms
man: try to clarify the folder: and path: vs. --output=files confusion
devel/gen-testdb.sh| 124
lib/database.cc| 45 -
lib/message.cc | 249 ---
lib/notmuch-private.h | 3 +
man/man1/notmuch-search.1 | 10 +-
man/man7/notmuch-search-terms.7| 28 ++-
test/.gitignore| 2 +-
test/Makefile.local| 2 +-
test/T070-insert.sh| 10 +-
test/T100-search-by-folder.sh | 24 ++-
test/T101-search-by-folder-and-path.sh | 83
test/T480-hex-escaping.sh | 4 +-
test/T530-upgrade.sh | 103 ++
test/corpus/{ = default}/cur/01:2,| 0
test/corpus/{ = default}/cur/02:2,| 0
test/corpus/{ = default}/cur/03:2,| 0
test/corpus/{ = default}/cur/04:2,| 0
test/corpus/{ = default}/cur/05:2,| 0
test/corpus/{ = default}/cur/06:2,| 0
test/corpus/{ = default}/cur/07:2,| 0
test/corpus/{ = default}/cur/08:2,| 0
test/corpus/{ = default}/cur/09:2,| 0
test/corpus/{ = default}/cur/10:2,| 0
test/corpus/{ = default}/cur/11:2,| 0
test/corpus/{ = default}/cur/12:2,| 0
test/corpus/{ = default}/cur/13:2,| 0
test/corpus/{ = default}/cur/14:2,| 0
test/corpus/{ = default}/cur/15:2,| 0
test/corpus/{ = default}/cur/16:2,| 0
test/corpus/{ = default}/cur/17:2,| 0
test/corpus/{ = default}/cur/18:2,| 0
test/corpus/{ = default}/cur/19:2,| 0
test/corpus/{ = default}/cur/20:2,| 0
test/corpus/{ = default}/cur/21:2,| 0
test/corpus/{ = default}/cur/22:2,| 0
test/corpus/{ = default}/cur/23:2,| 0
test/corpus/{ = default}/cur/24:2,| 0
test/corpus/{ = default}/cur/25:2,| 0
test/corpus/{ = default}/cur/26:2,| 0
test/corpus/{ = default}/cur/27:2,| 0
test/corpus/{ = default}/cur/28:2,| 0
test/corpus/{ = default}/cur/29:2,| 0
test/corpus/{ = default}/cur/30:2,| 0
test/corpus/{ = default}/cur/31:2,| 0
test/corpus/{ = default}/cur/32:2,| 0
test/corpus/{ = default}/cur/33:2,| 0
test/corpus/{ = default}/cur/34:2,| 0
test/corpus/{ = default}/cur/35:2,| 0
test/corpus/{ = default}/cur/36:2,| 0
test/corpus/{ = default}/cur/37:2,| 0
test/corpus/{ = default}/cur/38:2,| 0
test/corpus/{ = default}/cur/39:2,| 0
test/corpus/{ = default}/cur/40:2,| 0
test/corpus/{ =
