Bug#823979: autodep8: Support for ELPA testsuites
control: tag -1 -moreinfo Hello, dh-elpa 1.0, which just hit unstable, has a new --autopkgtest flag which, as you suggested, disables non-test files in the source package (by changing their extension .el -> .disabled). I pushed an updated patch for autodep8 to the 'elpa' branch of the autodep8 collab-maint repo. Please consider merging it. -- Sean Whitton signature.asc Description: PGP signature
Bug#823979: autodep8: Support for ELPA testsuites
On Wed, May 11, 2016 at 08:08:48AM -0700, Sean Whitton wrote: > Dear Antonio, > > (CCing David to check my understanding on something below) > > On Wed, May 11, 2016 at 10:57:28AM -0300, Antonio Terceiro wrote: > > Even without having a clue about ELPA, or emacs in general, I can guess > > with a certain degree of confidence that those tests will be testing the > > source tree, not the installed packages. The purpose of > > autopkgtest/DEP-8 is testing the installed binary packages, not the > > source tree. > > I had forgotten that that was the purpose of DEP-8, so thanks for > reminding me. > > > Is it possible to run the tests against the installed packages, making > > sure that code from the source tree is not used (except of course the > > test code itself)? > > I do not think that there is a general way to ensure that an ELPA > testsuite uses the installed files rather than the source tree. > > Most ELPA test suites can be run without any helper code: dh_elpa_test > just loads and executes all the test definitions it can find. In that > case we could add some code to ensure it used the installed package when > it saw --autopkgtest. However, a lot of ELPA test suites use upstream's > helper code to do setup before the tests are run. This code generally > loads the package out of the source tree. There is no general way to > stop this from happening: each package would need its own patch to the > upstream helper code. And that makes it impossible for autodep8 to run > ELPA test suites. Well, that happens. The situation in Ruby is the following. By default Ruby source packages have their library code in lib/, and C extensions in ext/ (mostly). When passed the --autopkgtest switch, gem2deb-test-runner will first move those away, then run the tests, then restore them do their original locations. This make sure the package can't load code from the source package, even if it tries. The vast majority of upstream test suites rely on the testing frameworks setting the $LOAD_PATH (Ruby's list of directories to load code from) to include the source-local directories, and just use `require "foo"`; these work out of the box. Some upstream test suites do explicitly load stuff from the source package; these will pass during builds, and fail under autopkgtest. We will usually patch the test suite to not rely on a relative location, and load libraries assuming they are installed system-wide (which should also work during the build because gem2deb-test-runner does set the $LOAD_PATH correctly during builds); usualy that means changing something like `require test_directory + '/../lib/foo'` to something like `require "foo"` signature.asc Description: PGP signature
Bug#823979: autodep8: Support for ELPA testsuites
Dear Antonio, (CCing David to check my understanding on something below) On Wed, May 11, 2016 at 10:57:28AM -0300, Antonio Terceiro wrote: > Even without having a clue about ELPA, or emacs in general, I can guess > with a certain degree of confidence that those tests will be testing the > source tree, not the installed packages. The purpose of > autopkgtest/DEP-8 is testing the installed binary packages, not the > source tree. I had forgotten that that was the purpose of DEP-8, so thanks for reminding me. > Is it possible to run the tests against the installed packages, making > sure that code from the source tree is not used (except of course the > test code itself)? I do not think that there is a general way to ensure that an ELPA testsuite uses the installed files rather than the source tree. Most ELPA test suites can be run without any helper code: dh_elpa_test just loads and executes all the test definitions it can find. In that case we could add some code to ensure it used the installed package when it saw --autopkgtest. However, a lot of ELPA test suites use upstream's helper code to do setup before the tests are run. This code generally loads the package out of the source tree. There is no general way to stop this from happening: each package would need its own patch to the upstream helper code. And that makes it impossible for autodep8 to run ELPA test suites. If I'm right about this, I guess we have to close this as wontfix. (David: do you think I'm right about this?) > If it's possible, it would be useful if dh_elpa_test could get input > from outside of debian/rules so that tests can be run be used > independent of a build (e.g. dh_elpa_test --autopkgtest). For an > example on how that can be done, you can look at the "FILES" section > of dh_ruby(1) (package gem2deb), in special the description of > debian/ruby-test-files.yaml, debian/ruby-tests.rake, > debian/ruby-tests.rb, and gem2deb-test-runner(1) > > you could for example make dh_elpa_test read environment variables form > debian/dh_test_elpa.conf or something similar, and make it also have a > --autopkgtest switch that makes sure the tests will not load code from > the source tree, then use that switch here in autodep8: > > Test-Command: dh_test_elpa --autopkgtest If we can resolve the above issue, that's a good idea, although rather than introduce a new config file I'm inclined to try to parse the information out of debian/rules. -- Sean Whitton signature.asc Description: PGP signature
Bug#823979: autodep8: Support for ELPA testsuites
Control: tag -1 + moreinfo On Tue, May 10, 2016 at 02:53:07PM -0700, Sean Whitton wrote: > Package: autodep8 > Version: 0.5.1 > Severity: wishlist > Tags: patch > > Dear maintainers, > > Please consider applying the following patch which adds support for the > testsuites of Emacs Lisp addon packages. Hi, thanks for your patch. Comments below. > diff --git a/support/elpa/detect b/support/elpa/detect > new file mode 100755 > index 000..7fd6287 > --- /dev/null > +++ b/support/elpa/detect > @@ -0,0 +1,18 @@ > +#!/bin/sh > + > +# Maintainer: Sean Whitton > + > +# We check that: this package uses dh_elpa, debhelper compat level is > +# at least 10 (see dh_elpa_test(1)), and the user hasn't disabled > +# dh_elpa_test, and that there is actually a testsuite to run > + > +set -e > + > +# check that the file exists first, so that if it doesn't cat doesn't > +# produce stderr output which disrupts other test/*_test.sh > +test -e debian/compat > +test "$(cat debian/compat)" -ge 10 > +(! grep -q ".*DH_ELPA_TEST_DISABLE.*" debian/rules) > +grep-dctrl -q -F Build-Depends dh-elpa debian/control > +( grep -q -r ".*(ert-deftest.*" . || \ > +grep-dctrl -q -F Build-Depends elpa-buttercup debian/control ) > diff --git a/support/elpa/generate b/support/elpa/generate > new file mode 100755 > index 000..3d89bc7 > --- /dev/null > +++ b/support/elpa/generate > @@ -0,0 +1,19 @@ > +#!/bin/sh > + > +# Maintainer: Sean Whitton > + > +# Remarks: > + > +# 1. Many ELPA test suites write to the build tree, so we need that > +#restriction in the general case. > + > +# 2. We invoke `debian/rules build' rather than simply `dh_elpa_test' > +#because that ensures that any DH_ELPA_TEST_* environment > +#variables the user has specifies in debian/rules are exported > +#such that dh_elpa_test can see them. See dh_elpa_test(1). > + > +cat < +Test-Command: debian/rules build > +Depends: @, @builddeps@ > +Restrictions: rw-build-tree > +EOF Even without having a clue about ELPA, or emacs in general, I can guess with a certain degree of confidence that those tests will be testing the source tree, not the installed packages. The purpose of autopkgtest/DEP-8 is testing the installed binary packages, not the source tree. Is it possible to run the tests against the installed packages, making sure that code from the source tree is not used (except of course the test code itself)? If it's possible, it would be useful if dh_elpa_test could get input from outside of debian/rules so that tests can be run be used independent of a build (e.g. dh_elpa_test --autopkgtest). For an example on how that can be done, you can look at the "FILES" section of dh_ruby(1) (package gem2deb), in special the description of debian/ruby-test-files.yaml, debian/ruby-tests.rake, debian/ruby-tests.rb, and gem2deb-test-runner(1) you could for example make dh_elpa_test read environment variables form debian/dh_test_elpa.conf or something similar, and make it also have a --autopkgtest switch that makes sure the tests will not load code from the source tree, then use that switch here in autodep8: Test-Command: dh_test_elpa --autopkgtest signature.asc Description: PGP signature
Bug#823979: autodep8: Support for ELPA testsuites
Package: autodep8 Version: 0.5.1 Severity: wishlist Tags: patch Dear maintainers, Please consider applying the following patch which adds support for the testsuites of Emacs Lisp addon packages. -- System Information: Debian Release: stretch/sid APT prefers testing APT policy: (900, 'testing') Architecture: i386 (i686) Kernel: Linux 4.5.0-1-686-pae (SMP w/2 CPU cores) Locale: LANG=en_GB.utf8, LC_CTYPE=en_GB.utf8 (charmap=UTF-8) Shell: /bin/sh linked to /bin/dash Init: systemd (via /run/systemd/system) Versions of packages autodep8 depends on: ii dctrl-tools 2.24-2 ii python3 3.5.1-3 autodep8 recommends no packages. autodep8 suggests no packages. -- no debconf information -- Sean Whitton From 2bee7dc0e3f008312774acc04a94df167e804fa1 Mon Sep 17 00:00:00 2001 From: Sean Whitton Date: Tue, 10 May 2016 13:10:27 -0700 Subject: [PATCH] ELPA testsuite detection & update copyright years --- debian/changelog | 8 debian/control| 1 + debian/copyright | 2 +- support/elpa/detect | 18 + support/elpa/generate | 19 ++ test/elpa_test.sh | 55 +++ 6 files changed, 102 insertions(+), 1 deletion(-) create mode 100755 support/elpa/detect create mode 100755 support/elpa/generate create mode 100644 test/elpa_test.sh diff --git a/debian/changelog b/debian/changelog index 20fc757..faeb535 100644 --- a/debian/changelog +++ b/debian/changelog @@ -1,3 +1,11 @@ +autodep8 (0.6) UNRELEASED; urgency=medium + + * Add support for ELPA package testsuites run by dh_elpa_test. + * Add test suite for ELPA package testsuite detection. + * Bump copyright years in d/copyright. + + -- Sean Whitton Tue, 10 May 2016 11:51:14 -0700 + autodep8 (0.5.1) unstable; urgency=medium * test/helper.sh: include extra information on failed tests to help diff --git a/debian/control b/debian/control index 6a44ec8..9d0b81b 100644 --- a/debian/control +++ b/debian/control @@ -24,3 +24,4 @@ Description: DEP-8 test control file generator - NodeJS packages - Kernel module packages using DKMS - R packages + - Emacs Lisp ELPA packages diff --git a/debian/copyright b/debian/copyright index 99b3620..dabe791 100644 --- a/debian/copyright +++ b/debian/copyright @@ -3,7 +3,7 @@ Upstream-Name: autodep8 Source: http://git.debian.org/?p=collab-maint/autodep8.git;a=summary Files: * -Copyright: 2014 the contributing authors. See `git log` for details. +Copyright: 2014-2016 the contributing authors. See `git log` for details. License: GPL-3.0+ License: GPL-3.0+ diff --git a/support/elpa/detect b/support/elpa/detect new file mode 100755 index 000..7fd6287 --- /dev/null +++ b/support/elpa/detect @@ -0,0 +1,18 @@ +#!/bin/sh + +# Maintainer: Sean Whitton + +# We check that: this package uses dh_elpa, debhelper compat level is +# at least 10 (see dh_elpa_test(1)), and the user hasn't disabled +# dh_elpa_test, and that there is actually a testsuite to run + +set -e + +# check that the file exists first, so that if it doesn't cat doesn't +# produce stderr output which disrupts other test/*_test.sh +test -e debian/compat +test "$(cat debian/compat)" -ge 10 +(! grep -q ".*DH_ELPA_TEST_DISABLE.*" debian/rules) +grep-dctrl -q -F Build-Depends dh-elpa debian/control +( grep -q -r ".*(ert-deftest.*" . || \ +grep-dctrl -q -F Build-Depends elpa-buttercup debian/control ) diff --git a/support/elpa/generate b/support/elpa/generate new file mode 100755 index 000..3d89bc7 --- /dev/null +++ b/support/elpa/generate @@ -0,0 +1,19 @@ +#!/bin/sh + +# Maintainer: Sean Whitton + +# Remarks: + +# 1. Many ELPA test suites write to the build tree, so we need that +#restriction in the general case. + +# 2. We invoke `debian/rules build' rather than simply `dh_elpa_test' +#because that ensures that any DH_ELPA_TEST_* environment +#variables the user has specifies in debian/rules are exported +#such that dh_elpa_test can see them. See dh_elpa_test(1). + +cat < signature.asc Description: PGP signature
