v2 of valgrind based memory tests

2012-12-25 Thread David Bremner 
Austin Clements  writes:

> On Mon, 24 Dec 2012, david at tethera.net wrote:
>
> LGTM.  In the long run, I wonder if we want to separate the time and
> memory tests like this or if some tests would make sense in either mode.

pushed. At the moment, the split into time versus memory tests is a
simple way to separate out "heavy weight" tests, but this could be done
more explicitely.

d

v2 of valgrind based memory tests

2012-12-25 Thread Austin Clements 
On Mon, 24 Dec 2012, david at tethera.net wrote:
> These obsolete
>   
>   id:1355196820-29734-1-git-send-email-david at tethera.net
>
> I tried to follow the suggestions of 
>
>   id:20121216191121.GH6187 at mit.edu
>
> pretty closely.

LGTM.  In the long run, I wonder if we want to separate the time and
memory tests like this or if some tests would make sense in either mode.

Re: v2 of valgrind based memory tests

2012-12-25 Thread Austin Clements 
On Mon, 24 Dec 2012, da...@tethera.net wrote:
 These obsolete
   
   id:1355196820-29734-1-git-send-email-da...@tethera.net

 I tried to follow the suggestions of 

   id:20121216191121.gh6...@mit.edu

 pretty closely.

LGTM.  In the long run, I wonder if we want to separate the time and
memory tests like this or if some tests would make sense in either mode.
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: v2 of valgrind based memory tests

2012-12-25 Thread David Bremner 
Austin Clements amdra...@mit.edu writes:

 On Mon, 24 Dec 2012, da...@tethera.net wrote:

 LGTM.  In the long run, I wonder if we want to separate the time and
 memory tests like this or if some tests would make sense in either mode.

pushed. At the moment, the split into time versus memory tests is a
simple way to separate out heavy weight tests, but this could be done
more explicitely.

d
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

v2 of valgrind based memory tests

2012-12-24 Thread da...@tethera.net 
These obsolete

  id:1355196820-29734-1-git-send-email-david at tethera.net

I tried to follow the suggestions of 

  id:20121216191121.GH6187 at mit.edu

pretty closely.

diff --git a/performance-test/M00-new b/performance-test/M00-new
index 733e9b0..99c3f52 100755
--- a/performance-test/M00-new
+++ b/performance-test/M00-new
@@ -9,6 +9,7 @@ uncache_database

 memory_start

+# run 'notmuch new' a second time, to test different code paths
 memory_run "notmuch new" "notmuch new"

 memory_done
diff --git a/performance-test/Makefile.local b/performance-test/Makefile.local
index 357d800..73aa963 100644
--- a/performance-test/Makefile.local
+++ b/performance-test/Makefile.local
@@ -4,7 +4,6 @@ dir := performance-test

 include $(dir)/version.sh

-# these two are just make sure dir is expanded at the right time.
 TIME_TEST_SCRIPT := ${dir}/notmuch-time-test
 MEMORY_TEST_SCRIPT := ${dir}/notmuch-memory-test

@@ -17,11 +16,11 @@ perf-test: time-test memory-test

 time-test: setup-perf-test all
@echo
-   $(TIME_TEST_SCRIPT) $(TEST_OPTIONS)
+   $(TIME_TEST_SCRIPT) $(OPTIONS)

 memory-test: setup-perf-test all
@echo
-   $(MEMORY_TEST_SCRIPT) $(TEST_OPTIONS)
+   $(MEMORY_TEST_SCRIPT) $(OPTIONS)


 .PHONY: download-corpus setup-perf-test
diff --git a/performance-test/README b/performance-test/README
index 7eaf5f7..996724c 100644
--- a/performance-test/README
+++ b/performance-test/README
@@ -1,7 +1,7 @@
 Performance Tests
 -

-This directory contains two kinds of performance tests, time tests,
+This directory contains two kinds of performance tests: time tests,
 and memory tests. The former use gnu time, and the latter use
 valgrind.

@@ -12,7 +12,7 @@ In addition to having notmuch, you need:

 - gpg
 - gnu tar
-- gnu time (for the time tests).
+- gnu time (for the time tests)
 - xz. Some speedup can be gotten by installing "pixz", but this is
   probably only worthwhile if you are debugging the tests.
 - valgrind (for the memory tests)
@@ -59,13 +59,13 @@ supports the following arguments
temporary directories.

 When using the make targets, you can pass arguments to all test
-scripts by defining the make variable TEST_OPTIONS.
+scripts by defining the make variable OPTIONS.

 Writing tests
 -

-Have a look at "T01-dump-restore" for an example time test and and
-"M00-new" for an example memory tests. In both cases sourcing
+Have a look at "T01-dump-restore" for an example time test and
+"M00-new" for an example memory test. In both cases sourcing
 "perf-test-lib.sh" is mandatory.

 Basics:
diff --git a/performance-test/perf-test-lib.sh 
b/performance-test/perf-test-lib.sh
index 79eb2c5..10d05e0 100644
--- a/performance-test/perf-test-lib.sh
+++ b/performance-test/perf-test-lib.sh
@@ -89,11 +89,10 @@ add_email_corpus ()

 cp -lr $TAG_CORPUS $TMP_DIRECTORY/corpus.tags
 cp -lr $MAIL_CORPUS $MAIL_DIR
-
 }

-notmuch_new_with_cache () {
-
+notmuch_new_with_cache ()
+{
 if [ -d $DB_CACHE_DIR ]; then
cp -r $DB_CACHE_DIR ${MAIL_DIR}/.notmuch
 else
@@ -102,8 +101,8 @@ notmuch_new_with_cache () {
 fi
 }

-time_start () {
-
+time_start ()
+{
 add_email_corpus

 print_header
@@ -111,17 +110,19 @@ time_start () {
 notmuch_new_with_cache time_run
 }

-memory_start () {
-
+memory_start ()
+{
 add_email_corpus

-_timestamp=$(printf "%x" $(date +"%s"))
-log_dir=$(mktemp -d "${TEST_DIRECTORY}/log.$(basename 
$0)-$corpus_size-${_timestamp}-XX")
+local timestamp=$(date +%Y%m%dT%H%M%S)
+log_dir="${TEST_DIRECTORY}/log.$(basename $0)-$corpus_size-${timestamp}"
+mkdir -p ${log_dir}

 notmuch_new_with_cache memory_run
 }

-memory_run () {
+memory_run ()
+{
 test_count=$(($test_count+1))

 log_file=$log_dir/$test_count.log
@@ -134,11 +135,13 @@ memory_run () {
 echo
 }

-memory_done () {
+memory_done ()
+{
 time_done
 }

-cache_database () {
+cache_database ()
+{
 if [ -d $MAIL_DIR/.notmuch ]; then
cp -r $MAIL_DIR/.notmuch $DB_CACHE_DIR
 else
@@ -146,16 +149,18 @@ cache_database () {
 fi
 }

-uncache_database () {
+uncache_database ()
+{
 rm -rf $DB_CACHE_DIR
 }

-print_header () {
+print_header ()
+{
 printf "\t\t\tWall(s)\tUsr(s)\tSys(s)\tRes(K)\tIn/Out(512B)\n"
-
 }

-time_run () {
+time_run ()
+{
 printf "  %-22s" "$1"
 test_count=$(($test_count+1))
 if test "$verbose" != "t"; then exec 4>test.output 3>&4; fi
@@ -166,7 +171,8 @@ time_run () {
 return 0
 }

-time_done () {
+time_done ()
+{
 if [ "$test_failure" = "0" ]; then
rm -rf "$remove_tmp"
exit 0

v2 of valgrind based memory tests

2012-12-24 Thread david 
These obsolete
  
  id:1355196820-29734-1-git-send-email-da...@tethera.net

I tried to follow the suggestions of 

  id:20121216191121.gh6...@mit.edu

pretty closely.

diff --git a/performance-test/M00-new b/performance-test/M00-new
index 733e9b0..99c3f52 100755
--- a/performance-test/M00-new
+++ b/performance-test/M00-new
@@ -9,6 +9,7 @@ uncache_database
 
 memory_start
 
+# run 'notmuch new' a second time, to test different code paths
 memory_run notmuch new notmuch new
 
 memory_done
diff --git a/performance-test/Makefile.local b/performance-test/Makefile.local
index 357d800..73aa963 100644
--- a/performance-test/Makefile.local
+++ b/performance-test/Makefile.local
@@ -4,7 +4,6 @@ dir := performance-test
 
 include $(dir)/version.sh
 
-# these two are just make sure dir is expanded at the right time.
 TIME_TEST_SCRIPT := ${dir}/notmuch-time-test
 MEMORY_TEST_SCRIPT := ${dir}/notmuch-memory-test
 
@@ -17,11 +16,11 @@ perf-test: time-test memory-test
 
 time-test: setup-perf-test all
@echo
-   $(TIME_TEST_SCRIPT) $(TEST_OPTIONS)
+   $(TIME_TEST_SCRIPT) $(OPTIONS)
 
 memory-test: setup-perf-test all
@echo
-   $(MEMORY_TEST_SCRIPT) $(TEST_OPTIONS)
+   $(MEMORY_TEST_SCRIPT) $(OPTIONS)
 
 
 .PHONY: download-corpus setup-perf-test
diff --git a/performance-test/README b/performance-test/README
index 7eaf5f7..996724c 100644
--- a/performance-test/README
+++ b/performance-test/README
@@ -1,7 +1,7 @@
 Performance Tests
 -
 
-This directory contains two kinds of performance tests, time tests,
+This directory contains two kinds of performance tests: time tests,
 and memory tests. The former use gnu time, and the latter use
 valgrind.
 
@@ -12,7 +12,7 @@ In addition to having notmuch, you need:
 
 - gpg
 - gnu tar
-- gnu time (for the time tests).
+- gnu time (for the time tests)
 - xz. Some speedup can be gotten by installing pixz, but this is
   probably only worthwhile if you are debugging the tests.
 - valgrind (for the memory tests)
@@ -59,13 +59,13 @@ supports the following arguments
temporary directories.
 
 When using the make targets, you can pass arguments to all test
-scripts by defining the make variable TEST_OPTIONS.
+scripts by defining the make variable OPTIONS.
 
 Writing tests
 -
 
-Have a look at T01-dump-restore for an example time test and and
-M00-new for an example memory tests. In both cases sourcing
+Have a look at T01-dump-restore for an example time test and
+M00-new for an example memory test. In both cases sourcing
 perf-test-lib.sh is mandatory.
 
 Basics:
diff --git a/performance-test/perf-test-lib.sh 
b/performance-test/perf-test-lib.sh
index 79eb2c5..10d05e0 100644
--- a/performance-test/perf-test-lib.sh
+++ b/performance-test/perf-test-lib.sh
@@ -89,11 +89,10 @@ add_email_corpus ()
 
 cp -lr $TAG_CORPUS $TMP_DIRECTORY/corpus.tags
 cp -lr $MAIL_CORPUS $MAIL_DIR
-
 }
 
-notmuch_new_with_cache () {
-
+notmuch_new_with_cache ()
+{
 if [ -d $DB_CACHE_DIR ]; then
cp -r $DB_CACHE_DIR ${MAIL_DIR}/.notmuch
 else
@@ -102,8 +101,8 @@ notmuch_new_with_cache () {
 fi
 }
 
-time_start () {
-
+time_start ()
+{
 add_email_corpus
 
 print_header
@@ -111,17 +110,19 @@ time_start () {
 notmuch_new_with_cache time_run
 }
 
-memory_start () {
-
+memory_start ()
+{
 add_email_corpus
 
-_timestamp=$(printf %x $(date +%s))
-log_dir=$(mktemp -d ${TEST_DIRECTORY}/log.$(basename 
$0)-$corpus_size-${_timestamp}-XX)
+local timestamp=$(date +%Y%m%dT%H%M%S)
+log_dir=${TEST_DIRECTORY}/log.$(basename $0)-$corpus_size-${timestamp}
+mkdir -p ${log_dir}
 
 notmuch_new_with_cache memory_run
 }
 
-memory_run () {
+memory_run ()
+{
 test_count=$(($test_count+1))
 
 log_file=$log_dir/$test_count.log
@@ -134,11 +135,13 @@ memory_run () {
 echo
 }
 
-memory_done () {
+memory_done ()
+{
 time_done
 }
 
-cache_database () {
+cache_database ()
+{
 if [ -d $MAIL_DIR/.notmuch ]; then
cp -r $MAIL_DIR/.notmuch $DB_CACHE_DIR
 else
@@ -146,16 +149,18 @@ cache_database () {
 fi
 }
 
-uncache_database () {
+uncache_database ()
+{
 rm -rf $DB_CACHE_DIR
 }
 
-print_header () {
+print_header ()
+{
 printf \t\t\tWall(s)\tUsr(s)\tSys(s)\tRes(K)\tIn/Out(512B)\n
-
 }
 
-time_run () {
+time_run ()
+{
 printf   %-22s $1
 test_count=$(($test_count+1))
 if test $verbose != t; then exec 4test.output 34; fi
@@ -166,7 +171,8 @@ time_run () {
 return 0
 }
 
-time_done () {
+time_done ()
+{
 if [ $test_failure = 0 ]; then
rm -rf $remove_tmp
exit 0
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch