[PATCH] test: optionally print subtest number

[Tomi Ollila]

On Tue, 13 Dec 2011 15:56:47 -0400, David Bremner  wrote:
> From: David Bremner 
> 
> The idea is that $test_count could be used in tests to label
> intermediate files. The output enabled by this patch (and --debug)
> helps figure out which OUTPUT.nn file belongs to which test in case
> several subtests write to OUTPUT.$test_count
> ---
> 
> Is there something that depends on the test format? I find it pretty
> handy to have the subtest numbers, but I don't want to break some
> other tools. I followed the existing style of conditionally defining
> functions, but maybe someone with more bash-fu can improve that.
> 
>  test/test-lib.sh |   12 
>  1 files changed, 12 insertions(+), 0 deletions(-)
> 
> diff --git a/test/test-lib.sh b/test/test-lib.sh
> index 6be93fe..f5c 100644
> --- a/test/test-lib.sh
> +++ b/test/test-lib.sh
> @@ -116,6 +116,16 @@ do
>   esac
>  done
>  
> +if test -n "$debug"; then
> +print_subtest () {
> + printf "\t[%d]\t" $(($test_count - 1))

Like this, as iterated by Dmitry & me

printf " %-4s" "[$(($test_count - 1))]"
(or
printf " %-4s" "[$((test_count - 1))]"
)

I think this is very useful feature; When searching reasons why
test failed the output files can be easily found as the test
number is show on screen. Also, when debugging problems with 
users online in distant locations, passing around the test
number in any particular moment eases the interaction considerably.

Tomi

Re: [PATCH] test: optionally print subtest number

[Tomi Ollila]

On Tue, 13 Dec 2011 15:56:47 -0400, David Bremner da...@tethera.net wrote:
 From: David Bremner brem...@debian.org
 
 The idea is that $test_count could be used in tests to label
 intermediate files. The output enabled by this patch (and --debug)
 helps figure out which OUTPUT.nn file belongs to which test in case
 several subtests write to OUTPUT.$test_count
 ---
 
 Is there something that depends on the test format? I find it pretty
 handy to have the subtest numbers, but I don't want to break some
 other tools. I followed the existing style of conditionally defining
 functions, but maybe someone with more bash-fu can improve that.
 
  test/test-lib.sh |   12 
  1 files changed, 12 insertions(+), 0 deletions(-)
 
 diff --git a/test/test-lib.sh b/test/test-lib.sh
 index 6be93fe..f5c 100644
 --- a/test/test-lib.sh
 +++ b/test/test-lib.sh
 @@ -116,6 +116,16 @@ do
   esac
  done
  
 +if test -n $debug; then
 +print_subtest () {
 + printf \t[%d]\t $(($test_count - 1))

Like this, as iterated by Dmitry  me

printf  %-4s [$(($test_count - 1))]
(or
printf  %-4s [$((test_count - 1))]
)

I think this is very useful feature; When searching reasons why
test failed the output files can be easily found as the test
number is show on screen. Also, when debugging problems with 
users online in distant locations, passing around the test
number in any particular moment eases the interaction considerably.

Tomi
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH] test: optionally print subtest number

[Dmitry Kurochkin]

On Tue, 13 Dec 2011 15:35:53 -0800, Jameson Graef Rollins  wrote:
> On Wed, 14 Dec 2011 03:24:23 +0400, Dmitry Kurochkin  gmail.com> wrote:
> > BTW I have some plans to introduce optional explicit test ids that can
> > be used for inter-test dependencies.  E.g.:
> > 
> >   test_begin_subtest test-id-1 "A subtest"
> >   
> >   ;; in another test requre that test-id-1 passed
> >   test_require_subtest test-id-1
> 
> Would the required test need to be listed twice, both on the
> begin_subtest line *and* in the require_subtest line?
> 
> And again, why would the test id have to be any different that the
> existing test names?  The tests already have names, so I don't
> understand why we would want to introduce some other kind of
> identification.  Seems like it's just going to add extra confusion.
> 

What you listed in the other email are test scripts, each with many
subtests.  I was talking about dependencies between subtests, not test
scripts.

> And speaking of which, I sometimes worry that the test infrastructure
> itself is getting too complicated.  Pretty soon we're going to need
> tests for the tests.

We already have them :)  Though, pretty limited.

>  I don't necessarily see the need to all of these
> extra features in the test suite, so I worry that it's just making
> everything harder to debug.
> 

I hope we can keep balance here.

Without inter-subtest dependencies, we have unhealthy situation where
some tests may be skipped because of missing prerequisites, but test
that depend on them are failing.  The only alternative I see is to
rewrite these tests to remove the dependencies.  But that would
complicate test cases itself, so I believe inter-subtest dependencies is
a better option.

Regards,
  Dmitry

> jamie.

[PATCH] test: optionally print subtest number

[Dmitry Kurochkin]

On Tue, 13 Dec 2011 19:18:16 -0400, David Bremner  wrote:
> On Tue, 13 Dec 2011 14:22:21 -0800, Jameson Graef Rollins  finestructure.net> wrote:
> 
> > I've only been vaguely following this "test count" stuff, but I'm not
> > sure I understand what's the point of giving tests a number that is
> > ultimately mutable.  Why not just label things by the test name, instead
> > of the count?  That wouldn't require keeping track of number/name
> > mapping, which will change over time.
> 
> We don't actually have test names, at least not ones directly suitable
> for file names.  I guess we could encode them or something, is that what
> you mean?
> 

BTW I have some plans to introduce optional explicit test ids that can
be used for inter-test dependencies.  E.g.:

  test_begin_subtest test-id-1 "A subtest"
  
  ;; in another test requre that test-id-1 passed
  test_require_subtest test-id-1

Regards,
  Dmitry

> d
> ___
> notmuch mailing list
> notmuch at notmuchmail.org
> http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH] test: optionally print subtest number

[Dmitry Kurochkin]

On Tue, 13 Dec 2011 22:55:18 +0200, Tomi Ollila  wrote:
> On Wed, 14 Dec 2011 00:15:43 +0400, Dmitry Kurochkin  gmail.com> wrote:
> > On Tue, 13 Dec 2011 15:56:47 -0400, David Bremner  
> > wrote:
> > > From: David Bremner 
> > > 
> > > The idea is that $test_count could be used in tests to label
> > > intermediate files. The output enabled by this patch (and --debug)
> > > helps figure out which OUTPUT.nn file belongs to which test in case
> > > several subtests write to OUTPUT.$test_count
> > > ---
> > > 
> > > Is there something that depends on the test format? I find it pretty
> > > handy to have the subtest numbers, but I don't want to break some
> > > other tools. I followed the existing style of conditionally defining
> > > functions, but maybe someone with more bash-fu can improve that.
> > > 
> > 
> > Looks good to me.  Except for tabs taking too much space.  Perhaps the
> > following would be better?
> > 
> > printf "%-4s" "[$(($test_count - 1))]"
> 
> I attempted the same size reduction. Therefore I know that
> this should to be either
> 
>   printf " %-4s" "[$(($test_count - 1))]"
> or
>   printf "\t%-4s" "[$(($test_count - 1))]"
> 
> (or something similar) so that there is space betweem BROKEN and [num]
> 

Oh, apparently, not enough testing on my side.

I vote for the first version with a space.

Regards,
  Dmitry

> This takes 4 bytes out from width (and drops tab as field separator)
> (and only few lines goes over 80 char width (some goes even with this
> reduction). So ...
> 
> 
> > Regards,
> >   Dmitry
> 
> Tomi
> 

[PATCH] test: optionally print subtest number

[Dmitry Kurochkin]

On Tue, 13 Dec 2011 15:56:47 -0400, David Bremner  wrote:
> From: David Bremner 
> 
> The idea is that $test_count could be used in tests to label
> intermediate files. The output enabled by this patch (and --debug)
> helps figure out which OUTPUT.nn file belongs to which test in case
> several subtests write to OUTPUT.$test_count
> ---
> 
> Is there something that depends on the test format? I find it pretty
> handy to have the subtest numbers, but I don't want to break some
> other tools. I followed the existing style of conditionally defining
> functions, but maybe someone with more bash-fu can improve that.
> 

Looks good to me.  Except for tabs taking too much space.  Perhaps the
following would be better?

printf "%-4s" "[$(($test_count - 1))]"

Regards,
  Dmitry

>  test/test-lib.sh |   12 
>  1 files changed, 12 insertions(+), 0 deletions(-)
> 
> diff --git a/test/test-lib.sh b/test/test-lib.sh
> index 6be93fe..f5c 100644
> --- a/test/test-lib.sh
> +++ b/test/test-lib.sh
> @@ -116,6 +116,16 @@ do
>   esac
>  done
>  
> +if test -n "$debug"; then
> +print_subtest () {
> + printf "\t[%d]\t" $(($test_count - 1))
> +}
> +else
> +print_subtest () {
> + true
> +}
> +fi
> +
>  if test -n "$color"; then
>   say_color () {
>   (
> @@ -132,6 +142,7 @@ if test -n "$color"; then
>   printf " "
>  printf "$@"
>   tput sgr0
> + print_subtest
>   )
>   }
>  else
> @@ -140,6 +151,7 @@ else
>   shift
>   printf " "
>  printf "$@"
> + print_subtest
>   }
>  fi
>  
> -- 
> 1.7.5.4
> 
> ___
> notmuch mailing list
> notmuch at notmuchmail.org
> http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH] test: optionally print subtest number

[Tomi Ollila]

On Wed, 14 Dec 2011 00:58:35 +0400, Dmitry Kurochkin  wrote:
> On Tue, 13 Dec 2011 22:55:18 +0200, Tomi Ollila  wrote:
> > On Wed, 14 Dec 2011 00:15:43 +0400, Dmitry Kurochkin  > gmail.com> wrote:
> > > 
> > > Looks good to me.  Except for tabs taking too much space.  Perhaps the
> > > following would be better?
> > > 
> > >   printf "%-4s" "[$(($test_count - 1))]"
> > 
> > I attempted the same size reduction. Therefore I know that
> > this should to be either
> > 
> > printf " %-4s" "[$(($test_count - 1))]"
> > or
> > printf "\t%-4s" "[$(($test_count - 1))]"
> > 
> > (or something similar) so that there is space betweem BROKEN and [num]
> > 
> 
> Oh, apparently, not enough testing on my side.
> 
> I vote for the first version with a space.

Have to agree, Now that I checked tabs were not used in output lines
before this addition.

> 
> Regards,
>   Dmitry

Tomi


> 
> > This takes 4 bytes out from width (and drops tab as field separator)
> > (and only few lines goes over 80 char width (some goes even with this
> > reduction). So ...
> > 
> > 
> > > Regards,
> > >   Dmitry
> > 
> > Tomi

[PATCH] test: optionally print subtest number

[Tomi Ollila]

On Wed, 14 Dec 2011 00:15:43 +0400, Dmitry Kurochkin  wrote:
> On Tue, 13 Dec 2011 15:56:47 -0400, David Bremner  
> wrote:
> > From: David Bremner 
> > 
> > The idea is that $test_count could be used in tests to label
> > intermediate files. The output enabled by this patch (and --debug)
> > helps figure out which OUTPUT.nn file belongs to which test in case
> > several subtests write to OUTPUT.$test_count
> > ---
> > 
> > Is there something that depends on the test format? I find it pretty
> > handy to have the subtest numbers, but I don't want to break some
> > other tools. I followed the existing style of conditionally defining
> > functions, but maybe someone with more bash-fu can improve that.
> > 
> 
> Looks good to me.  Except for tabs taking too much space.  Perhaps the
> following would be better?
> 
>   printf "%-4s" "[$(($test_count - 1))]"

I attempted the same size reduction. Therefore I know that
this should to be either

printf " %-4s" "[$(($test_count - 1))]"
or
printf "\t%-4s" "[$(($test_count - 1))]"

(or something similar) so that there is space betweem BROKEN and [num]

This takes 4 bytes out from width (and drops tab as field separator)
(and only few lines goes over 80 char width (some goes even with this
reduction). So ...


> Regards,
>   Dmitry

Tomi

[PATCH] test: optionally print subtest number

[Tomi Ollila]

On Tue, 13 Dec 2011 15:56:47 -0400, David Bremner  wrote:
> From: David Bremner 
> 
> The idea is that $test_count could be used in tests to label
> intermediate files. The output enabled by this patch (and --debug)
> helps figure out which OUTPUT.nn file belongs to which test in case
> several subtests write to OUTPUT.$test_count
> ---
> 
> Is there something that depends on the test format? I find it pretty
> handy to have the subtest numbers, but I don't want to break some
> other tools. I followed the existing style of conditionally defining
> functions, but maybe someone with more bash-fu can improve that.

+1. I would also say '+1' even non-debug format (and update dependend code,
if there is any).

Tomi

> 
>  test/test-lib.sh |   12 
>  1 files changed, 12 insertions(+), 0 deletions(-)
> 
> diff --git a/test/test-lib.sh b/test/test-lib.sh
> index 6be93fe..f5c 100644
> --- a/test/test-lib.sh
> +++ b/test/test-lib.sh
> @@ -116,6 +116,16 @@ do
>   esac
>  done
>  
> +if test -n "$debug"; then
> +print_subtest () {
> + printf "\t[%d]\t" $(($test_count - 1))
> +}
> +else
> +print_subtest () {
> + true
> +}
> +fi
> +
>  if test -n "$color"; then
>   say_color () {
>   (
> @@ -132,6 +142,7 @@ if test -n "$color"; then
>   printf " "
>  printf "$@"
>   tput sgr0
> + print_subtest
>   )
>   }
>  else
> @@ -140,6 +151,7 @@ else
>   shift
>   printf " "
>  printf "$@"
> + print_subtest
>   }
>  fi
>  
> -- 
> 1.7.5.4
> 
> ___
> notmuch mailing list
> notmuch at notmuchmail.org
> http://notmuchmail.org/mailman/listinfo/notmuch
> 

-- 
Tomi Ollila !! tomi.ollila at nixu.fi !! +358 400 888 366
Nixu Oy !! http://www.nixu.fi/ !! Keilaranta 15, FI-02150 Espoo, Finland

[PATCH] test: optionally print subtest number

[David Bremner]

On Tue, 13 Dec 2011 15:25:14 -0800, Jameson Graef Rollins  wrote:
> On Tue, 13 Dec 2011 19:18:16 -0400, David Bremner  
> wrote:
> > We don't actually have test names, at least not ones directly suitable
> > for file names.  I guess we could encode them or something, is that what
> > you mean?
> 
> Don't we?  There is at least a word/phrase that's consistent between the
> actual test script names, what is listed in the TESTS variable (below),
> and what is output when the tests are run (ie. "basic:" or
> "search-folder-coherence:").  Are those not test names?

OK, we can call those test names. This thread/patch is about "subtest
numbers", so finer granularity. There are an average of 10 subtests per
test, those are are the things that don't have names.

d

[PATCH] test: optionally print subtest number

[David Bremner]

On Tue, 13 Dec 2011 14:22:21 -0800, Jameson Graef Rollins  wrote:

> I've only been vaguely following this "test count" stuff, but I'm not
> sure I understand what's the point of giving tests a number that is
> ultimately mutable.  Why not just label things by the test name, instead
> of the count?  That wouldn't require keeping track of number/name
> mapping, which will change over time.

We don't actually have test names, at least not ones directly suitable
for file names.  I guess we could encode them or something, is that what
you mean?

d

[PATCH] test: optionally print subtest number

[David Bremner]

From: David Bremner 

The idea is that $test_count could be used in tests to label
intermediate files. The output enabled by this patch (and --debug)
helps figure out which OUTPUT.nn file belongs to which test in case
several subtests write to OUTPUT.$test_count
---

Is there something that depends on the test format? I find it pretty
handy to have the subtest numbers, but I don't want to break some
other tools. I followed the existing style of conditionally defining
functions, but maybe someone with more bash-fu can improve that.

 test/test-lib.sh |   12 
 1 files changed, 12 insertions(+), 0 deletions(-)

diff --git a/test/test-lib.sh b/test/test-lib.sh
index 6be93fe..f5c 100644
--- a/test/test-lib.sh
+++ b/test/test-lib.sh
@@ -116,6 +116,16 @@ do
esac
 done

+if test -n "$debug"; then
+print_subtest () {
+   printf "\t[%d]\t" $(($test_count - 1))
+}
+else
+print_subtest () {
+   true
+}
+fi
+
 if test -n "$color"; then
say_color () {
(
@@ -132,6 +142,7 @@ if test -n "$color"; then
printf " "
 printf "$@"
tput sgr0
+   print_subtest
)
}
 else
@@ -140,6 +151,7 @@ else
shift
printf " "
 printf "$@"
+   print_subtest
}
 fi

-- 
1.7.5.4

[PATCH] test: optionally print subtest number

[Jameson Graef Rollins]

On Wed, 14 Dec 2011 03:24:23 +0400, Dmitry Kurochkin  wrote:
> BTW I have some plans to introduce optional explicit test ids that can
> be used for inter-test dependencies.  E.g.:
> 
>   test_begin_subtest test-id-1 "A subtest"
>   
>   ;; in another test requre that test-id-1 passed
>   test_require_subtest test-id-1

Would the required test need to be listed twice, both on the
begin_subtest line *and* in the require_subtest line?

And again, why would the test id have to be any different that the
existing test names?  The tests already have names, so I don't
understand why we would want to introduce some other kind of
identification.  Seems like it's just going to add extra confusion.

And speaking of which, I sometimes worry that the test infrastructure
itself is getting too complicated.  Pretty soon we're going to need
tests for the tests.  I don't necessarily see the need to all of these
extra features in the test suite, so I worry that it's just making
everything harder to debug.

jamie.
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL: 

[PATCH] test: optionally print subtest number

[Jameson Graef Rollins]

On Tue, 13 Dec 2011 19:18:16 -0400, David Bremner  wrote:
> We don't actually have test names, at least not ones directly suitable
> for file names.  I guess we could encode them or something, is that what
> you mean?

Don't we?  There is at least a word/phrase that's consistent between the
actual test script names, what is listed in the TESTS variable (below),
and what is output when the tests are run (ie. "basic:" or
"search-folder-coherence:").  Are those not test names?

jamie.


TESTS="
  basic
  help-test
  new
  count
  search
  search-output
  search-by-folder
  search-position-overlap-bug
  search-insufficient-from-quoting
  search-limiting
  tagging
  json
  multipart
  thread-naming
  raw
  reply
  dump-restore
  uuencode
  thread-order
  author-order
  from-guessing
  long-id
  encoding
  emacs
  emacs-large-search-buffer
  maildir-sync
  crypto
  symbol-hiding
  search-folder-coherence
  atomicity
  python
  hooks
  argument-parsing
"
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL: 

[PATCH] test: optionally print subtest number

[Jameson Graef Rollins]

On Tue, 13 Dec 2011 15:56:47 -0400, David Bremner  wrote:
> The idea is that $test_count could be used in tests to label
> intermediate files. The output enabled by this patch (and --debug)
> helps figure out which OUTPUT.nn file belongs to which test in case
> several subtests write to OUTPUT.$test_count

I've only been vaguely following this "test count" stuff, but I'm not
sure I understand what's the point of giving tests a number that is
ultimately mutable.  Why not just label things by the test name, instead
of the count?  That wouldn't require keeping track of number/name
mapping, which will change over time.

jamie.
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL: 

[PATCH] test: optionally print subtest number

[David Bremner]

From: David Bremner brem...@debian.org

The idea is that $test_count could be used in tests to label
intermediate files. The output enabled by this patch (and --debug)
helps figure out which OUTPUT.nn file belongs to which test in case
several subtests write to OUTPUT.$test_count
---

Is there something that depends on the test format? I find it pretty
handy to have the subtest numbers, but I don't want to break some
other tools. I followed the existing style of conditionally defining
functions, but maybe someone with more bash-fu can improve that.

 test/test-lib.sh |   12 
 1 files changed, 12 insertions(+), 0 deletions(-)

diff --git a/test/test-lib.sh b/test/test-lib.sh
index 6be93fe..f5c 100644
--- a/test/test-lib.sh
+++ b/test/test-lib.sh
@@ -116,6 +116,16 @@ do
esac
 done
 
+if test -n $debug; then
+print_subtest () {
+   printf \t[%d]\t $(($test_count - 1))
+}
+else
+print_subtest () {
+   true
+}
+fi
+
 if test -n $color; then
say_color () {
(
@@ -132,6 +142,7 @@ if test -n $color; then
printf  
 printf $@
tput sgr0
+   print_subtest
)
}
 else
@@ -140,6 +151,7 @@ else
shift
printf  
 printf $@
+   print_subtest
}
 fi
 
-- 
1.7.5.4

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH] test: optionally print subtest number

[Dmitry Kurochkin]

On Tue, 13 Dec 2011 15:56:47 -0400, David Bremner da...@tethera.net wrote:
 From: David Bremner brem...@debian.org
 
 The idea is that $test_count could be used in tests to label
 intermediate files. The output enabled by this patch (and --debug)
 helps figure out which OUTPUT.nn file belongs to which test in case
 several subtests write to OUTPUT.$test_count
 ---
 
 Is there something that depends on the test format? I find it pretty
 handy to have the subtest numbers, but I don't want to break some
 other tools. I followed the existing style of conditionally defining
 functions, but maybe someone with more bash-fu can improve that.
 

Looks good to me.  Except for tabs taking too much space.  Perhaps the
following would be better?

printf %-4s [$(($test_count - 1))]

Regards,
  Dmitry

  test/test-lib.sh |   12 
  1 files changed, 12 insertions(+), 0 deletions(-)
 
 diff --git a/test/test-lib.sh b/test/test-lib.sh
 index 6be93fe..f5c 100644
 --- a/test/test-lib.sh
 +++ b/test/test-lib.sh
 @@ -116,6 +116,16 @@ do
   esac
  done
  
 +if test -n $debug; then
 +print_subtest () {
 + printf \t[%d]\t $(($test_count - 1))
 +}
 +else
 +print_subtest () {
 + true
 +}
 +fi
 +
  if test -n $color; then
   say_color () {
   (
 @@ -132,6 +142,7 @@ if test -n $color; then
   printf  
  printf $@
   tput sgr0
 + print_subtest
   )
   }
  else
 @@ -140,6 +151,7 @@ else
   shift
   printf  
  printf $@
 + print_subtest
   }
  fi
  
 -- 
 1.7.5.4
 
 ___
 notmuch mailing list
 notmuch@notmuchmail.org
 http://notmuchmail.org/mailman/listinfo/notmuch
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH] test: optionally print subtest number

[Tomi Ollila]

On Wed, 14 Dec 2011 00:15:43 +0400, Dmitry Kurochkin 
dmitry.kuroch...@gmail.com wrote:
 On Tue, 13 Dec 2011 15:56:47 -0400, David Bremner da...@tethera.net wrote:
  From: David Bremner brem...@debian.org
  
  The idea is that $test_count could be used in tests to label
  intermediate files. The output enabled by this patch (and --debug)
  helps figure out which OUTPUT.nn file belongs to which test in case
  several subtests write to OUTPUT.$test_count
  ---
  
  Is there something that depends on the test format? I find it pretty
  handy to have the subtest numbers, but I don't want to break some
  other tools. I followed the existing style of conditionally defining
  functions, but maybe someone with more bash-fu can improve that.
  
 
 Looks good to me.  Except for tabs taking too much space.  Perhaps the
 following would be better?
 
   printf %-4s [$(($test_count - 1))]

I attempted the same size reduction. Therefore I know that
this should to be either

printf  %-4s [$(($test_count - 1))]
or
printf \t%-4s [$(($test_count - 1))]

(or something similar) so that there is space betweem BROKEN and [num]

This takes 4 bytes out from width (and drops tab as field separator)
(and only few lines goes over 80 char width (some goes even with this
reduction). So ...


 Regards,
   Dmitry

Tomi

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH] test: optionally print subtest number

[Dmitry Kurochkin]

On Tue, 13 Dec 2011 22:55:18 +0200, Tomi Ollila tomi.oll...@iki.fi wrote:
 On Wed, 14 Dec 2011 00:15:43 +0400, Dmitry Kurochkin 
 dmitry.kuroch...@gmail.com wrote:
  On Tue, 13 Dec 2011 15:56:47 -0400, David Bremner da...@tethera.net wrote:
   From: David Bremner brem...@debian.org
   
   The idea is that $test_count could be used in tests to label
   intermediate files. The output enabled by this patch (and --debug)
   helps figure out which OUTPUT.nn file belongs to which test in case
   several subtests write to OUTPUT.$test_count
   ---
   
   Is there something that depends on the test format? I find it pretty
   handy to have the subtest numbers, but I don't want to break some
   other tools. I followed the existing style of conditionally defining
   functions, but maybe someone with more bash-fu can improve that.
   
  
  Looks good to me.  Except for tabs taking too much space.  Perhaps the
  following would be better?
  
  printf %-4s [$(($test_count - 1))]
 
 I attempted the same size reduction. Therefore I know that
 this should to be either
 
   printf  %-4s [$(($test_count - 1))]
 or
   printf \t%-4s [$(($test_count - 1))]
 
 (or something similar) so that there is space betweem BROKEN and [num]
 

Oh, apparently, not enough testing on my side.

I vote for the first version with a space.

Regards,
  Dmitry

 This takes 4 bytes out from width (and drops tab as field separator)
 (and only few lines goes over 80 char width (some goes even with this
 reduction). So ...
 
 
  Regards,
Dmitry
 
 Tomi
 
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH] test: optionally print subtest number

[Tomi Ollila]

On Wed, 14 Dec 2011 00:58:35 +0400, Dmitry Kurochkin 
dmitry.kuroch...@gmail.com wrote:
 On Tue, 13 Dec 2011 22:55:18 +0200, Tomi Ollila tomi.oll...@iki.fi wrote:
  On Wed, 14 Dec 2011 00:15:43 +0400, Dmitry Kurochkin 
  dmitry.kuroch...@gmail.com wrote:
   
   Looks good to me.  Except for tabs taking too much space.  Perhaps the
   following would be better?
   
 printf %-4s [$(($test_count - 1))]
  
  I attempted the same size reduction. Therefore I know that
  this should to be either
  
  printf  %-4s [$(($test_count - 1))]
  or
  printf \t%-4s [$(($test_count - 1))]
  
  (or something similar) so that there is space betweem BROKEN and [num]
  
 
 Oh, apparently, not enough testing on my side.
 
 I vote for the first version with a space.

Have to agree, Now that I checked tabs were not used in output lines
before this addition.

 
 Regards,
   Dmitry

Tomi


 
  This takes 4 bytes out from width (and drops tab as field separator)
  (and only few lines goes over 80 char width (some goes even with this
  reduction). So ...
  
  
   Regards,
 Dmitry
  
  Tomi
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH] test: optionally print subtest number

[Jameson Graef Rollins]

On Tue, 13 Dec 2011 15:56:47 -0400, David Bremner da...@tethera.net wrote:
 The idea is that $test_count could be used in tests to label
 intermediate files. The output enabled by this patch (and --debug)
 helps figure out which OUTPUT.nn file belongs to which test in case
 several subtests write to OUTPUT.$test_count

I've only been vaguely following this test count stuff, but I'm not
sure I understand what's the point of giving tests a number that is
ultimately mutable.  Why not just label things by the test name, instead
of the count?  That wouldn't require keeping track of number/name
mapping, which will change over time.

jamie.


pgpnNvruW2Eyx.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH] test: optionally print subtest number

[David Bremner]

On Tue, 13 Dec 2011 14:22:21 -0800, Jameson Graef Rollins 
jroll...@finestructure.net wrote:

 I've only been vaguely following this test count stuff, but I'm not
 sure I understand what's the point of giving tests a number that is
 ultimately mutable.  Why not just label things by the test name, instead
 of the count?  That wouldn't require keeping track of number/name
 mapping, which will change over time.

We don't actually have test names, at least not ones directly suitable
for file names.  I guess we could encode them or something, is that what
you mean?

d
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH] test: optionally print subtest number

[Dmitry Kurochkin]

On Tue, 13 Dec 2011 19:18:16 -0400, David Bremner da...@tethera.net wrote:
 On Tue, 13 Dec 2011 14:22:21 -0800, Jameson Graef Rollins 
 jroll...@finestructure.net wrote:
 
  I've only been vaguely following this test count stuff, but I'm not
  sure I understand what's the point of giving tests a number that is
  ultimately mutable.  Why not just label things by the test name, instead
  of the count?  That wouldn't require keeping track of number/name
  mapping, which will change over time.
 
 We don't actually have test names, at least not ones directly suitable
 for file names.  I guess we could encode them or something, is that what
 you mean?
 

BTW I have some plans to introduce optional explicit test ids that can
be used for inter-test dependencies.  E.g.:

  test_begin_subtest test-id-1 A subtest
  
  ;; in another test requre that test-id-1 passed
  test_require_subtest test-id-1

Regards,
  Dmitry

 d
 ___
 notmuch mailing list
 notmuch@notmuchmail.org
 http://notmuchmail.org/mailman/listinfo/notmuch
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH] test: optionally print subtest number

[Jameson Graef Rollins]

On Wed, 14 Dec 2011 03:24:23 +0400, Dmitry Kurochkin 
dmitry.kuroch...@gmail.com wrote:
 BTW I have some plans to introduce optional explicit test ids that can
 be used for inter-test dependencies.  E.g.:
 
   test_begin_subtest test-id-1 A subtest
   
   ;; in another test requre that test-id-1 passed
   test_require_subtest test-id-1

Would the required test need to be listed twice, both on the
begin_subtest line *and* in the require_subtest line?

And again, why would the test id have to be any different that the
existing test names?  The tests already have names, so I don't
understand why we would want to introduce some other kind of
identification.  Seems like it's just going to add extra confusion.

And speaking of which, I sometimes worry that the test infrastructure
itself is getting too complicated.  Pretty soon we're going to need
tests for the tests.  I don't necessarily see the need to all of these
extra features in the test suite, so I worry that it's just making
everything harder to debug.

jamie.


pgp9TD61sKYHo.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH] test: optionally print subtest number

[Dmitry Kurochkin]

On Tue, 13 Dec 2011 15:35:53 -0800, Jameson Graef Rollins 
jroll...@finestructure.net wrote:
 On Wed, 14 Dec 2011 03:24:23 +0400, Dmitry Kurochkin 
 dmitry.kuroch...@gmail.com wrote:
  BTW I have some plans to introduce optional explicit test ids that can
  be used for inter-test dependencies.  E.g.:
  
test_begin_subtest test-id-1 A subtest

;; in another test requre that test-id-1 passed
test_require_subtest test-id-1
 
 Would the required test need to be listed twice, both on the
 begin_subtest line *and* in the require_subtest line?
 
 And again, why would the test id have to be any different that the
 existing test names?  The tests already have names, so I don't
 understand why we would want to introduce some other kind of
 identification.  Seems like it's just going to add extra confusion.
 

What you listed in the other email are test scripts, each with many
subtests.  I was talking about dependencies between subtests, not test
scripts.

 And speaking of which, I sometimes worry that the test infrastructure
 itself is getting too complicated.  Pretty soon we're going to need
 tests for the tests.

We already have them :)  Though, pretty limited.

  I don't necessarily see the need to all of these
 extra features in the test suite, so I worry that it's just making
 everything harder to debug.
 

I hope we can keep balance here.

Without inter-subtest dependencies, we have unhealthy situation where
some tests may be skipped because of missing prerequisites, but test
that depend on them are failing.  The only alternative I see is to
rewrite these tests to remove the dependencies.  But that would
complicate test cases itself, so I believe inter-subtest dependencies is
a better option.

Regards,
  Dmitry

 jamie.
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH] test: optionally print subtest number

[David Bremner]

On Tue, 13 Dec 2011 15:25:14 -0800, Jameson Graef Rollins 
jroll...@finestructure.net wrote:
 On Tue, 13 Dec 2011 19:18:16 -0400, David Bremner da...@tethera.net wrote:
  We don't actually have test names, at least not ones directly suitable
  for file names.  I guess we could encode them or something, is that what
  you mean?
 
 Don't we?  There is at least a word/phrase that's consistent between the
 actual test script names, what is listed in the TESTS variable (below),
 and what is output when the tests are run (ie. basic: or
 search-folder-coherence:).  Are those not test names?

OK, we can call those test names. This thread/patch is about subtest
numbers, so finer granularity. There are an average of 10 subtests per
test, those are are the things that don't have names.

d


___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch