Bug#839253: autopkgtest: --testname incompatible with Test-Command:

2016-12-01 Thread Martin Pitt 
Hello Barry, Ian,

Ian Jackson [2016-10-23 17:21 +0100]:
> I see in fact that is is still not compatible.  How about:
> 
>   Test-Command: tox -e qa
>   Depends: @builddeps@, git
>   Features: test-name=qa

That seems okay to me. It would be our first-ever supported
"Features:" item, too :-)

Barry, as you expressed interest in helping with autopkgtest, I think
this would make an excellent learning opportunity: implementing this
is reasonably easy, but it requires touching the testdesc.py unit
tests, writing an integration test, and updating the documentation, so
that you can get some familiarity with all these parts. Would you like
to look into this, and I'll guide you through the process? (We can do
that in person next week).

Thanks,

Martin
-- 
Martin Pitt| http://www.piware.de
Ubuntu Developer (www.ubuntu.com)  | Debian Developer  (www.debian.org)

Bug#839253: autopkgtest: --testname incompatible with Test-Command:

2016-10-23 Thread Ian Jackson 
Ian Jackson writes ("Re: Bug#839253: autopkgtest: --testname incompatible with 
Test-Command:"):
> Barry's suggestion of a Test-Name field is better.  This is because if
> a package provides
>   Test-Command-foobar:
> it won't be compatibile with software compatible with the previous
> DEP-8 spec.

I see in fact that is is still not compatible.  How about:

  Test-Command: tox -e qa
  Depends: @builddeps@, git
  Features: test-name=qa

?

-- 
Ian Jackson <ijack...@chiark.greenend.org.uk>   These opinions are my own.

If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.

Bug#839253: autopkgtest: --testname incompatible with Test-Command:

2016-10-23 Thread Ian Jackson 
Martin Pitt writes ("Bug#839253: autopkgtest: --testname incompatible with 
Test-Command:"):
> Hey Barry,
> > * Add a Test-Name: field solely for the purpose of matching
> >   --testname.  Of course if Test-Name doesn't exist, it would fall
> >   back to matching on Tests:
> 
> I'd slightly change that idea to be Test-Command-foobar:, to provide
> an optional name? Test-Command: would then continue to get
> autogenerated commandN names.

Barry's suggestion of a Test-Name field is better.  This is because if
a package provides
  Test-Command-foobar:
it won't be compatibile with software compatible with the previous
DEP-8 spec.

Ian.

-- 
Ian Jackson <ijack...@chiark.greenend.org.uk>   These opinions are my own.

If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.

Bug#839253: autopkgtest: --testname incompatible with Test-Command:

2016-09-30 Thread Barry Warsaw 
On Oct 01, 2016, at 12:16 AM, Martin Pitt wrote:

>You can actually -- they get named "command1", "command2", etc, same
>names that appear in the logs and in summary.

I must have missed that, but it's really nice.


pgpIUkqcn_THD.pgp
Description: OpenPGP digital signature

Bug#839253: autopkgtest: --testname incompatible with Test-Command:

2016-09-30 Thread Barry Warsaw 
On Oct 01, 2016, at 12:16 AM, Martin Pitt wrote:

>I'd slightly change that idea to be Test-Command-foobar:, to provide
>an optional name? Test-Command: would then continue to get
>autogenerated commandN names.

I rather like that.  One of my other minor problems is that in the summary, if
you have a lot of tests, it's a bit difficult to map 'test command6: ...' to
the actual test that ran.  You have to scroll back quite a bit to look those
up.

So if we had Test-Command-foobar and the summary output at the end said

'test foobar:'

then I think that would be a nice improvement!


pgpg749aRT5U1.pgp
Description: OpenPGP digital signature

Bug#839253: autopkgtest: --testname incompatible with Test-Command:

2016-09-30 Thread Martin Pitt 
Hey Barry,

Barry Warsaw [2016-09-30 14:17 -0400]:
> AFAICT, if you specify a test with Test-Command: there's no way to
> select it with `autopkgtest --testname`.  A couple of thoughts:

You can actually -- they get named "command1", "command2", etc, same
names that appear in the logs and in summary.

> * Allow Tests: and Test-Command: to coexist.  In such a case, the
>   Tests: field would be used for --testname matching while the actual
>   tests would run from Test-Command:

The interpretation of that would be a bit confusing as Tests: is
plural and can specify multiple tests.

> * Add a Test-Name: field solely for the purpose of matching
>   --testname.  Of course if Test-Name doesn't exist, it would fall
>   back to matching on Tests:

I'd slightly change that idea to be Test-Command-foobar:, to provide
an optional name? Test-Command: would then continue to get
autogenerated commandN names.

WDYT?

Martin

-- 
Martin Pitt| http://www.piware.de
Ubuntu Developer (www.ubuntu.com)  | Debian Developer  (www.debian.org)

Bug#839253: autopkgtest: --testname incompatible with Test-Command:

2016-09-30 Thread Barry Warsaw 
Package: autopkgtest
Version: 4.0.5
Severity: wishlist

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

Dear Maintainer,

AFAICT, if you specify a test with Test-Command: there's no way to
select it with `autopkgtest --testname`.  A couple of thoughts:

* Allow Tests: and Test-Command: to coexist.  In such a case, the
  Tests: field would be used for --testname matching while the actual
  tests would run from Test-Command:
  
* Add a Test-Name: field solely for the purpose of matching
  --testname.  Of course if Test-Name doesn't exist, it would fall
  back to matching on Tests:
  
* Allow --testname to match on patterns.  Thus if I had a stanza that
  looked like:
  
  Test-Command: tox -e qa
  Depends: @builddeps@, git
  
  I could run that with --testname=qa

Other thoughts of course welcome!  Or did I miss something?


- -- System Information:
Debian Release: stretch/sid
  APT prefers unstable
  APT policy: (500, 'unstable')
Architecture: amd64 (x86_64)

Kernel: Linux 4.7.0-1-amd64 (SMP w/8 CPU cores)
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)

Versions of packages autopkgtest depends on:
ii  apt-utils   1.3
ii  libdpkg-perl1.18.10
ii  procps  2:3.3.12-2
ii  python3 3.5.1-4
ii  python3-debian  0.1.29

Versions of packages autopkgtest recommends:
ii  autodep8  0.8

Versions of packages autopkgtest suggests:
pn  lxc  
pn  lxd-client   
pn  qemu-system  
pn  qemu-utils   
ii  schroot  1.6.10-2+b1

- -- no debconf information

-BEGIN PGP SIGNATURE-

iQIcBAEBCAAGBQJX7qwnAAoJEBJutWOnSwa/whsP/jbbENZ+nr2Gs2Kzy6ln/8Jw
FVyvifpxea1pwUWxF4wOC9YLEAJbLOlAXkD6VSxkIyicOndkcSwX4YG59H/YgyKF
3i6IVTVgcS6vN+FNSIOc8eU+9+V9ibv1+HS6u0MnV7Ab60PI7Z9dyuPm2gv3BZh8
i1+of8gM/YLRKK+sZoh4D2sPjk8qy9oyjwlJaiA8Vy2RvrdSLSGB6qf61ra2owfM
RxsjQFAAw93kkS7LdT/15PNI+xgYvSAwT5fGMJYfZB+qE4JEZfSjQZnNv1BDHj+m
b1PbWhuNIL41qliQNCBiaUq4KWRHDcz68YYqr23WQQCC+pzJospdHOYVXPrBJxtL
eMqRH0xl6V0iep1VA7r+l47G57IDXMlejgV2kZKURmhD9/tQDxq4SUF+NK0Q/Pz6
Cj621eF1LUxyfhai3Uw1sHlan5gLO7YkR1UEVEcV17c6pRewWOQgd+CiQBfaoD1W
ScwFBB/c/CyUaaYeo0qbwxCJQq4DVBf8QyyZprorZrG9WHOLmQQl5DxewpvyDd+O
XpaXokOGbJwTmprQ8tXNjIJ8fGafvX2chVJo5CriMspBfFL7Q4frvVzZtixIGS4E
aplNE1s/VbxwVReWPLKHMRit+eR5hlqW9Jp6/ulQOyqOcuLaYAilcJtF7sLb6i5b
pxDnKdFy2i70Mv/7d1fr
=Z/0G
-END PGP SIGNATURE-