Bug#941330: undocumented limitation/dependency in dh_elpa_test
Hello, On Sat 05 Oct 2019 at 02:01PM -04, Nicholas D Steeves wrote: > Sean Whitton writes: > >> control: tag -1 +pending >> >> Hello, >> >> On Sat 28 Sep 2019 at 04:50PM -04, Nicholas D Steeves wrote: >> >>> dh-elpa-test appears to have an undocumented hard dependency on >>> (ert-run-test-batch-and-exit), specifically "-and-exit", and this >>> one-function expression must appear as the final line of the defined >>> ert_helper. >> >> Per the existing documentation of ert_helper, it would not be correct to >> set ert_helper to run code that does some work, and then ends by calling >> (ert-run-test-batch-and-exit). That's what ert_eval is for. >> > > In this case it's more: run a second round of tests using a different > upstream provided method, and output how long each test took, plus time > for garbage collection, and of a course a PASS||FAIL. I think that the correct way to do this would be override_dh_elpa_test: dh_elpa_test other_upstream_thing -- Sean Whitton signature.asc Description: PGP signature
Bug#941330: undocumented limitation/dependency in dh_elpa_test
Sean Whitton writes: > control: tag -1 +pending > > Hello, > > On Sat 28 Sep 2019 at 04:50PM -04, Nicholas D Steeves wrote: > >> dh-elpa-test appears to have an undocumented hard dependency on >> (ert-run-test-batch-and-exit), specifically "-and-exit", and this >> one-function expression must appear as the final line of the defined >> ert_helper. > > Per the existing documentation of ert_helper, it would not be correct to > set ert_helper to run code that does some work, and then ends by calling > (ert-run-test-batch-and-exit). That's what ert_eval is for. > In this case it's more: run a second round of tests using a different upstream provided method, and output how long each test took, plus time for garbage collection, and of a course a PASS||FAIL. I believe some of the variables may be set different for this second. Upstream and I are finally hunting for whatever is causing those periodic failures on DebCI (3% failure rate) and his Travis instance (more than 3% failure rate) The script is scripts/elpy-test-benchmark.el I seem to remember that ert_eval didn't provide useful output in case of failure, where running it as an ert_helper did. > I believe you're correct that there is a requirement that the ert_helper > code kill Emacs with a correct exit code in order for dh_elpa_test to > work properly, so I'm adding documentation for that. Thanks! Nicholas signature.asc Description: PGP signature
Bug#941330: undocumented limitation/dependency in dh_elpa_test
control: tag -1 +pending Hello, On Sat 28 Sep 2019 at 04:50PM -04, Nicholas D Steeves wrote: > dh-elpa-test appears to have an undocumented hard dependency on > (ert-run-test-batch-and-exit), specifically "-and-exit", and this > one-function expression must appear as the final line of the defined > ert_helper. Per the existing documentation of ert_helper, it would not be correct to set ert_helper to run code that does some work, and then ends by calling (ert-run-test-batch-and-exit). That's what ert_eval is for. I believe you're correct that there is a requirement that the ert_helper code kill Emacs with a correct exit code in order for dh_elpa_test to work properly, so I'm adding documentation for that. -- Sean Whitton signature.asc Description: PGP signature
Bug#941330: undocumented limitation/dependency in dh_elpa_test
Package: dh-elpa Version: 2.0.2~bpo10+2 Severity: normal dh-elpa-test appears to have an undocumented hard dependency on (ert-run-test-batch-and-exit), specifically "-and-exit", and this one-function expression must appear as the final line of the defined ert_helper. Steps to reproduce: d/elpa-test: ert_helper = debian/ert-helper.el debian/ert-helper.el: (ert-run-tests-batch) Then run "dh_elpa_test" in a src:pkg with failing tests. Result: ERT test output failure, but there's no big block of "dh_elpa_test…exit code 1" in the shell, and the exit code is 0 (PASS). Expected result: Big block of "dh_elpa_test: emacs -batch…-l debian/ert-helper.el returned exit code 1" with an exit code 2 in the shell (FAIL). -- If this is a documentation bug, then the above requirement should be documented, plus it would be nice to have an example snippet of how to kill emacs from within ert-helper.el in such a way that status FAILED is correctly passed through. Regards, Nicholas -- System Information: Debian Release: 10.1 APT prefers stable-debug APT policy: (500, 'stable-debug'), (500, 'proposed-updates'), (500, 'stable') Architecture: amd64 (x86_64) Foreign Architectures: i386 Kernel: Linux 4.19.72-rt25 (SMP w/4 CPU cores; PREEMPT) Kernel taint flags: TAINT_USER <- I have to disable an i915 feature due to buggy BIOS Locale: LANG=en_CA.UTF-8, LC_CTYPE=en_CA.UTF-8 (charmap=UTF-8), LANGUAGE=en_CA:en (charmap=UTF-8) Shell: /bin/sh linked to /bin/dash Init: systemd (via /run/systemd/system) Versions of packages dh-elpa depends on: ii debhelper 12.1.1 ii dh-make-perl0.105 ii emacs 1:26.1+1-3.2+deb10u1 ii emacs-gtk [emacs] 1:26.1+1-3.2+deb10u1 ii libarray-utils-perl 0.5-1 ii libconfig-tiny-perl 2.23-1 ii libdebian-source-perl 0.105 ii libdpkg-perl1.19.7 ii libfile-find-rule-perl 0.34-1 ii libtext-glob-perl 0.10-1 ii perl5.28.1-6 dh-elpa recommends no packages. dh-elpa suggests no packages. -- no debconf information
