v3 of boolean folder: patches

2014-03-10 Thread Jani Nikula 
On Mon, 10 Mar 2014, "W. Trevor King"  wrote:
> Since patches 06/15 and 07/15 never made it to the list [1] (or gmane
> [2]), I think we need to untag 06/15 in nmbug.  It was tagged in
> b3eb5c4 (2014-03-09), but without the messages in my local archive, I
> get:

I untagged it earlier today. Do you still see the problem?

BR,
Jani.

v3 of boolean folder: patches

2014-03-10 Thread W. Trevor King 
On Mon, Mar 10, 2014 at 08:24:50PM +0200, Jani Nikula wrote:
> On Mon, 10 Mar 2014, "W. Trevor King"  wrote:
> > Since patches 06/15 and 07/15 never made it to the list [1] (or gmane
> > [2]), I think we need to untag 06/15 in nmbug.  It was tagged in
> > b3eb5c4 (2014-03-09), but without the messages in my local archive, I
> > get:
> 
> I untagged it earlier today. Do you still see the problem?

Nope, all better :).  Sorry I didn't re-pull before posting :p.

Thanks,
Trevor

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy
-- next part --
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: OpenPGP digital signature
URL:

v3 of boolean folder: patches

2014-03-10 Thread W. Trevor King 
On Sat, Mar 08, 2014 at 05:40:45PM -0400, David Bremner wrote:
> David Bremner  writes:
> 
> >
> > I decided that the benefits of being to really apply and test the
> > patch series outweighed the fact that one of patchs is about
> > 150K. Apologies to those of you on GSM modems and the like.
> >
> 
> Oops. I didn't realize the "reorganize the corpus" patch was so big. So
> I guess if you want to test the series, you'll have to look at
> 
> git://pivot.cs.unb.ca/notmuch.git
> branch: fetch-databases

Since patches 06/15 and 07/15 never made it to the list [1] (or gmane
[2]), I think we need to untag 06/15 in nmbug.  It was tagged in
b3eb5c4 (2014-03-09), but without the messages in my local archive, I
get:

  $ nmbug status
  U   1394313585-28422-7-git-send-email-david at tethera.net obsolete

Cheers,
Trevor

[1]: http://notmuchmail.org/pipermail/notmuch/2014/thread.html
[2]: http://news.gmane.org/group/gmane.mail.notmuch.general/thread=17412

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy
-- next part --
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: OpenPGP digital signature
URL:

v3 of boolean folder: patches

2014-03-08 Thread David Bremner 
David Bremner  writes:

>
> I decided that the benefits of being to really apply and test the
> patch series outweighed the fact that one of patchs is about
> 150K. Apologies to those of you on GSM modems and the like.
>

Oops. I didn't realize the "reorganize the corpus" patch was so big. So
I guess if you want to test the series, you'll have to look at

git://pivot.cs.unb.ca/notmuch.git
branch: fetch-databases

v3 of boolean folder: patches

2014-03-08 Thread David Bremner 
This is a followup to 

 id:cover.1393105055.git.jani at nikula.org

The first 11 patches are very close to that series; in the last 4 I
have added some minimalist infrastructure to download and verify
pre-built test databases.

I decided that the benefits of being to really apply and test the
patch series outweighed the fact that one of patchs is about
150K. Apologies to those of you on GSM modems and the like.

Interdiff follows 

diff --git a/Makefile b/Makefile
index 0428160..97084b1 100644
--- a/Makefile
+++ b/Makefile
@@ -5,7 +5,8 @@ all:
 # List all subdirectories here. Each contains its own Makefile.local.
 # Use of '=', without '+=', seems to be required for out-of-tree
 # builds to work.
-subdirs = compat completion emacs lib man parse-time-string performance-test 
util test
+subdirs = compat completion emacs lib man parse-time-string performance-test 
util test \
+   test/test-databases

 # We make all targets depend on the Makefiles themselves.
 global_deps = Makefile Makefile.config Makefile.local \
diff --git a/devel/gen-testdb.sh b/devel/gen-testdb.sh
index c291dff..621b31e 100755
--- a/devel/gen-testdb.sh
+++ b/devel/gen-testdb.sh
@@ -81,6 +81,10 @@ shift `expr $OPTIND - 1`

 . ./test-lib.sh

+SHORT_CORPUS=$(basename ${CORPUS:-database})
+DBNAME=${SHORT_CORPUS}${SUFFIX}
+TARBALLNAME=${DBNAME}.tar.xz
+
 CORPUS=${CORPUS:-${TEST_DIRECTORY}/corpus}

 test_expect_code 0 "notmuch version specified on the command line" \
@@ -112,13 +116,16 @@ cp -a ${CORPUS} ${MAIL_DIR}
 test_expect_code 0 "index the corpus" \
 "notmuch new"

-# finally, wrap the resulting mail store and database in a tarball
-DBNAME=database${SUFFIX}
+# wrap the resulting mail store and database in a tarball
+
 cp -a ${MAIL_DIR} ${TMP_DIRECTORY}/${DBNAME}
-tar zcf ${TMP_DIRECTORY}/${DBNAME}.tar.gz -C ${TMP_DIRECTORY} ${DBNAME}
+tar Jcf ${TMP_DIRECTORY}/${TARBALLNAME} -C ${TMP_DIRECTORY} ${DBNAME}
 mkdir -p  ${TEST_DIRECTORY}/test-databases
-cp -a ${TMP_DIRECTORY}/${DBNAME}.tar.gz ${TEST_DIRECTORY}/test-databases
-test_expect_code 0 "create the output tarball ${DBNAME}.tar.gz" \
-"test -f ${TEST_DIRECTORY}/test-databases/${DBNAME}.tar.gz"
+cp -a ${TMP_DIRECTORY}/${TARBALLNAME} ${TEST_DIRECTORY}/test-databases
+test_expect_code 0 "create the output tarball ${TARBALLNAME}" \
+"test -f ${TEST_DIRECTORY}/test-databases/${TARBALLNAME}"

+# generate a checksum file
+test_expect_code 0 "compute checksum" \
+"(cd ${TEST_DIRECTORY}/test-databases/ && sha256sum ${TARBALLNAME} > 
${TARBALLNAME}.sha256)"
 test_done
diff --git a/test/Makefile.local b/test/Makefile.local
index 99324ba..bfabd15 100644
--- a/test/Makefile.local
+++ b/test/Makefile.local
@@ -2,6 +2,8 @@

 dir := test

+# save against changes in $(dir)
+test_src_dir := $(dir)
 extra_cflags += -I.

 smtp_dummy_srcs =  \
@@ -44,12 +46,12 @@ TEST_BINARIES=$(dir)/arg-test \

 test-binaries: $(TEST_BINARIES)

-test:  all test-binaries
-   @${dir}/notmuch-test $(OPTIONS)
+test:  all test-binaries fetch-test-databases
+   @${test_src_dir}/notmuch-test $(OPTIONS)

 check: test

 SRCS := $(SRCS) $(smtp_dummy_srcs)
 CLEAN += $(TEST_BINARIES) $(addsuffix .o,$(TEST_BINARIES)) \
 $(dir)/database-test.o \
-$(dir)/corpus.mail $(dir)/test-results $(dir)/tmp.*
+$(dir)/corpus.mail.* $(dir)/test-results $(dir)/tmp.*
diff --git a/test/README b/test/README
index 79a9b1b..81a1c82 100644
--- a/test/README
+++ b/test/README
@@ -64,6 +64,14 @@ The following command-line options are available when 
running tests:
Pointing this argument at a tmpfs filesystem can improve the
speed of the test suite for some users.

+Certain tests require precomputed databases to complete. You can fetch these
+databases with
+
+   make download-test-databases
+
+If you do not download the test databases, the relevant tests will be
+skipped.
+
 When invoking the test suite via "make test" any of the above options
 can be specified as follows:

diff --git a/test/T530-upgrade.sh b/test/T530-upgrade.sh
index cf9914e..e1e8ac5 100755
--- a/test/T530-upgrade.sh
+++ b/test/T530-upgrade.sh
@@ -3,7 +3,19 @@ test_description="database upgrade"

 . ./test-lib.sh

-tar zxf $TEST_DIRECTORY/test-databases/database-v1.tar.gz -C ${MAIL_DIR} 
--strip-components=1
+dbtarball=folders-v1.tar.xz
+
+# XXX: Accomplish the same with test lib helpers
+if [ ! -e ${TEST_DIRECTORY}/test-databases/${dbtarball} ]; then
+test_subtest_missing_external_prereq_["${dbtarball} - fetch with 'make 
download-test-databases'"]=t
+fi
+
+test_expect_success \
+'database checksum' \
+'( cd $TEST_DIRECTORY/test-databases &&
+   sha256sum --quiet --check --status ${dbtarball}.sha256 )'
+
+tar xf $TEST_DIRECTORY/test-databases/${dbtarball} -C ${MAIL_DIR} 
--strip-components=1

 test_begin_subtest "folder: search does not work with old database version"
 output=$(notmuch search folder:foo)
diff --git a/test/test-databases/.gitignore b/test/test-databases/.gitignore
new

v3 of boolean folder: patches

2014-03-08 Thread David Bremner 
This is a followup to 

 id:cover.1393105055.git.j...@nikula.org

The first 11 patches are very close to that series; in the last 4 I
have added some minimalist infrastructure to download and verify
pre-built test databases.

I decided that the benefits of being to really apply and test the
patch series outweighed the fact that one of patchs is about
150K. Apologies to those of you on GSM modems and the like.

Interdiff follows 

diff --git a/Makefile b/Makefile
index 0428160..97084b1 100644
--- a/Makefile
+++ b/Makefile
@@ -5,7 +5,8 @@ all:
 # List all subdirectories here. Each contains its own Makefile.local.
 # Use of '=', without '+=', seems to be required for out-of-tree
 # builds to work.
-subdirs = compat completion emacs lib man parse-time-string performance-test 
util test
+subdirs = compat completion emacs lib man parse-time-string performance-test 
util test \
+   test/test-databases
 
 # We make all targets depend on the Makefiles themselves.
 global_deps = Makefile Makefile.config Makefile.local \
diff --git a/devel/gen-testdb.sh b/devel/gen-testdb.sh
index c291dff..621b31e 100755
--- a/devel/gen-testdb.sh
+++ b/devel/gen-testdb.sh
@@ -81,6 +81,10 @@ shift `expr $OPTIND - 1`
 
 . ./test-lib.sh
 
+SHORT_CORPUS=$(basename ${CORPUS:-database})
+DBNAME=${SHORT_CORPUS}${SUFFIX}
+TARBALLNAME=${DBNAME}.tar.xz
+
 CORPUS=${CORPUS:-${TEST_DIRECTORY}/corpus}
 
 test_expect_code 0 notmuch version specified on the command line \
@@ -112,13 +116,16 @@ cp -a ${CORPUS} ${MAIL_DIR}
 test_expect_code 0 index the corpus \
 notmuch new
 
-# finally, wrap the resulting mail store and database in a tarball
-DBNAME=database${SUFFIX}
+# wrap the resulting mail store and database in a tarball
+
 cp -a ${MAIL_DIR} ${TMP_DIRECTORY}/${DBNAME}
-tar zcf ${TMP_DIRECTORY}/${DBNAME}.tar.gz -C ${TMP_DIRECTORY} ${DBNAME}
+tar Jcf ${TMP_DIRECTORY}/${TARBALLNAME} -C ${TMP_DIRECTORY} ${DBNAME}
 mkdir -p  ${TEST_DIRECTORY}/test-databases
-cp -a ${TMP_DIRECTORY}/${DBNAME}.tar.gz ${TEST_DIRECTORY}/test-databases
-test_expect_code 0 create the output tarball ${DBNAME}.tar.gz \
-test -f ${TEST_DIRECTORY}/test-databases/${DBNAME}.tar.gz
+cp -a ${TMP_DIRECTORY}/${TARBALLNAME} ${TEST_DIRECTORY}/test-databases
+test_expect_code 0 create the output tarball ${TARBALLNAME} \
+test -f ${TEST_DIRECTORY}/test-databases/${TARBALLNAME}
 
+# generate a checksum file
+test_expect_code 0 compute checksum \
+(cd ${TEST_DIRECTORY}/test-databases/  sha256sum ${TARBALLNAME}  
${TARBALLNAME}.sha256)
 test_done
diff --git a/test/Makefile.local b/test/Makefile.local
index 99324ba..bfabd15 100644
--- a/test/Makefile.local
+++ b/test/Makefile.local
@@ -2,6 +2,8 @@
 
 dir := test
 
+# save against changes in $(dir)
+test_src_dir := $(dir)
 extra_cflags += -I.
 
 smtp_dummy_srcs =  \
@@ -44,12 +46,12 @@ TEST_BINARIES=$(dir)/arg-test \
 
 test-binaries: $(TEST_BINARIES)
 
-test:  all test-binaries
-   @${dir}/notmuch-test $(OPTIONS)
+test:  all test-binaries fetch-test-databases
+   @${test_src_dir}/notmuch-test $(OPTIONS)
 
 check: test
 
 SRCS := $(SRCS) $(smtp_dummy_srcs)
 CLEAN += $(TEST_BINARIES) $(addsuffix .o,$(TEST_BINARIES)) \
 $(dir)/database-test.o \
-$(dir)/corpus.mail $(dir)/test-results $(dir)/tmp.*
+$(dir)/corpus.mail.* $(dir)/test-results $(dir)/tmp.*
diff --git a/test/README b/test/README
index 79a9b1b..81a1c82 100644
--- a/test/README
+++ b/test/README
@@ -64,6 +64,14 @@ The following command-line options are available when 
running tests:
Pointing this argument at a tmpfs filesystem can improve the
speed of the test suite for some users.
 
+Certain tests require precomputed databases to complete. You can fetch these
+databases with
+
+   make download-test-databases
+
+If you do not download the test databases, the relevant tests will be
+skipped.
+
 When invoking the test suite via make test any of the above options
 can be specified as follows:
 
diff --git a/test/T530-upgrade.sh b/test/T530-upgrade.sh
index cf9914e..e1e8ac5 100755
--- a/test/T530-upgrade.sh
+++ b/test/T530-upgrade.sh
@@ -3,7 +3,19 @@ test_description=database upgrade
 
 . ./test-lib.sh
 
-tar zxf $TEST_DIRECTORY/test-databases/database-v1.tar.gz -C ${MAIL_DIR} 
--strip-components=1
+dbtarball=folders-v1.tar.xz
+
+# XXX: Accomplish the same with test lib helpers
+if [ ! -e ${TEST_DIRECTORY}/test-databases/${dbtarball} ]; then
+test_subtest_missing_external_prereq_[${dbtarball} - fetch with 'make 
download-test-databases']=t
+fi
+
+test_expect_success \
+'database checksum' \
+'( cd $TEST_DIRECTORY/test-databases 
+   sha256sum --quiet --check --status ${dbtarball}.sha256 )'
+
+tar xf $TEST_DIRECTORY/test-databases/${dbtarball} -C ${MAIL_DIR} 
--strip-components=1
 
 test_begin_subtest folder: search does not work with old database version
 output=$(notmuch search folder:foo)
diff --git a/test/test-databases/.gitignore b/test/test-databases/.gitignore
new file mode 100644

Re: v3 of boolean folder: patches

2014-03-08 Thread David Bremner 
David Bremner da...@tethera.net writes:


 I decided that the benefits of being to really apply and test the
 patch series outweighed the fact that one of patchs is about
 150K. Apologies to those of you on GSM modems and the like.


Oops. I didn't realize the reorganize the corpus patch was so big. So
I guess if you want to test the series, you'll have to look at

git://pivot.cs.unb.ca/notmuch.git
branch: fetch-databases





___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch