Bug#841146: diffoscope: Failure of test_superblock
Control: tag -1 pending On Sun, Oct 23, 2016 at 05:00:27PM -0400, Leo Famulari wrote: > When I format the code correctly, the test is skipped and we can build > diffoscope successfully: cool! > > >> Though I'm using pytest 3.0.3. > > >> That test is skipped by using pytest.mark.skip(), which I don't see in > > >> the docs of pytest for 2.7. > > >> The changelog of pytest tells me pytest.mark.skip() is recognized as a > > >> skipping marker starting from 2.9¹. Is there any chance you can instead > > >> upgrade pytest in your distribution? > > > > > > We are working on upgrading the core Python packages like pytest and > > > Setuptools but we can't do it overnight. > > > > > >> If so I'll add a versioned dependency on setup.py, otherwies I can > > >> always turn that pytest.mark.skip() into a pytest.mark.skipif(True), > > >> which is IMHO ugly but quick and effective for solving this bug, I > > >> think. Can you also try to convert that marking in > > >> tests/comparators/utils.py:49 to confirm? > > I don't think it's necessary for you to make that change in diffoscope; > we can skip the test in our packaging while we work on upgrading pytest. > But of course it's up to you. I don't see why we can't :) https://anonscm.debian.org/git/reproducible/diffoscope.git/commit/?id=fa07622 Really the only gain of pytest.mark.skip() over pytest.mark.skipif() is that the former is a bit shorter. I'm trying to finish up some things and pull up a release this week, if you want you could just wait for that instead of including a patch. -- regards, Mattia Rizzolo GPG Key: 66AE 2B4A FCCF 3F52 DA18 4D18 4B04 3FCD B944 4540 .''`. more about me: https://mapreri.org : :' : Launchpad user: https://launchpad.net/~mapreri `. `'` Debian QA page: https://qa.debian.org/developer.php?login=mattia `- signature.asc Description: PGP signature
Bug#841146: diffoscope: Failure of test_superblock
On Sat, Oct 22, 2016 at 08:39:00PM +, HW42 wrote:
> Leo Famulari:
> > def skip_unless_tool_is_older_than(tool, actual_ver, min_ver,
> > vcls=LooseVersion):
> > if tools_missing(tool):
> > -return pytest.mark.skip(reason="requires {}".format(tool))
> > +return pytest.mark.skipif(True))
>
> That's one closing paren to much.
Sorry for the bogus message :/
When I format the code correctly, the test is skipped and we can build
diffoscope successfully:
diff --git a/tests/comparators/utils.py b/tests/comparators/utils.py
index f8f6399..d2da9d6 100644
--- a/tests/comparators/utils.py
+++ b/tests/comparators/utils.py
@@ -46,7 +46,7 @@ def skip_unless_tools_exist(*required):
def skip_unless_tool_is_older_than(tool, actual_ver, min_ver,
vcls=LooseVersion):
if tools_missing(tool):
-return pytest.mark.skip(reason="requires {}".format(tool))
+return pytest.mark.skipif(True, reason="Requires pytest >= 2.9")
if callable(actual_ver):
actual_ver = actual_ver()
return pytest.mark.skipif(
> > On Fri, Oct 21, 2016 at 04:23:51PM +, Mattia Rizzolo wrote:
> >> Though I'm using pytest 3.0.3.
> >> That test is skipped by using pytest.mark.skip(), which I don't see in
> >> the docs of pytest for 2.7.
> >> The changelog of pytest tells me pytest.mark.skip() is recognized as a
> >> skipping marker starting from 2.9¹. Is there any chance you can instead
> >> upgrade pytest in your distribution?
> >
> > We are working on upgrading the core Python packages like pytest and
> > Setuptools but we can't do it overnight.
> >
> >> If so I'll add a versioned dependency on setup.py, otherwies I can
> >> always turn that pytest.mark.skip() into a pytest.mark.skipif(True),
> >> which is IMHO ugly but quick and effective for solving this bug, I
> >> think. Can you also try to convert that marking in
> >> tests/comparators/utils.py:49 to confirm?
I don't think it's necessary for you to make that change in diffoscope;
we can skip the test in our packaging while we work on upgrading pytest.
But of course it's up to you.
signature.asc
Description: PGP signature
Bug#841146: diffoscope: Failure of test_superblock
Leo Famulari:
> On Fri, Oct 21, 2016 at 04:23:51PM +, Mattia Rizzolo wrote:
>> Though I'm using pytest 3.0.3.
>> That test is skipped by using pytest.mark.skip(), which I don't see in
>> the docs of pytest for 2.7.
>> The changelog of pytest tells me pytest.mark.skip() is recognized as a
>> skipping marker starting from 2.9¹. Is there any chance you can instead
>> upgrade pytest in your distribution?
>
> We are working on upgrading the core Python packages like pytest and
> Setuptools but we can't do it overnight.
>
>> If so I'll add a versioned dependency on setup.py, otherwies I can
>> always turn that pytest.mark.skip() into a pytest.mark.skipif(True),
>> which is IMHO ugly but quick and effective for solving this bug, I
>> think. Can you also try to convert that marking in
>> tests/comparators/utils.py:49 to confirm?
>
> I tried making the following change:
>
> diff --git a/tests/comparators/utils.py b/tests/comparators/utils.py
> index f8f6399..acbdc9f 100644
> --- a/tests/comparators/utils.py
> +++ b/tests/comparators/utils.py
> @@ -46,7 +46,7 @@ def skip_unless_tools_exist(*required):
>
> def skip_unless_tool_is_older_than(tool, actual_ver, min_ver,
> vcls=LooseVersion):
> if tools_missing(tool):
> -return pytest.mark.skip(reason="requires {}".format(tool))
> +return pytest.mark.skipif(True))
That's one closing paren to much.
> if callable(actual_ver):
> actual_ver = actual_ver()
> return pytest.mark.skipif(
>
> But, that creates a bunch of invalid syntax. Here's one:
[...]
signature.asc
Description: OpenPGP digital signature
Bug#841146: diffoscope: Failure of test_superblock
On Fri, Oct 21, 2016 at 04:23:51PM +, Mattia Rizzolo wrote:
> Though I'm using pytest 3.0.3.
> That test is skipped by using pytest.mark.skip(), which I don't see in
> the docs of pytest for 2.7.
> The changelog of pytest tells me pytest.mark.skip() is recognized as a
> skipping marker starting from 2.9¹. Is there any chance you can instead
> upgrade pytest in your distribution?
We are working on upgrading the core Python packages like pytest and
Setuptools but we can't do it overnight.
> If so I'll add a versioned dependency on setup.py, otherwies I can
> always turn that pytest.mark.skip() into a pytest.mark.skipif(True),
> which is IMHO ugly but quick and effective for solving this bug, I
> think. Can you also try to convert that marking in
> tests/comparators/utils.py:49 to confirm?
I tried making the following change:
diff --git a/tests/comparators/utils.py b/tests/comparators/utils.py
index f8f6399..acbdc9f 100644
--- a/tests/comparators/utils.py
+++ b/tests/comparators/utils.py
@@ -46,7 +46,7 @@ def skip_unless_tools_exist(*required):
def skip_unless_tool_is_older_than(tool, actual_ver, min_ver,
vcls=LooseVersion):
if tools_missing(tool):
-return pytest.mark.skip(reason="requires {}".format(tool))
+return pytest.mark.skipif(True))
if callable(actual_ver):
actual_ver = actual_ver()
return pytest.mark.skipif(
But, that creates a bunch of invalid syntax. Here's one:
___ ERROR collecting tests/comparators/test_utils.py ___
/gnu/store/1n2h8244hw0xvldqdz10lspp60snqw2v-python-pytest-2.7.3/lib/python3.5/site-packages/pytest-2.7.3-py3.5.egg/_pytest/python.py:498:
in _importtestmodule
mod = self.fspath.pyimport(ensuresyspath=True)
/gnu/store/syf4ac6vw94d5qvaacmjq33wfhbgizcr-python-py-1.4.31/lib/python3.5/site-packages/py-1.4.31-py3.5.egg/py/_path/local.py:650:
in pyimport
__import__(modname)
:969: in _find_and_load
???
:958: in _find_and_load_unlocked
???
:664: in _load_unlocked
???
:634: in _load_backward_compatible
???
/gnu/store/1n2h8244hw0xvldqdz10lspp60snqw2v-python-pytest-2.7.3/lib/python3.5/site-packages/pytest-2.7.3-py3.5.egg/_pytest/assertion/rewrite.py:171:
in load_module
py.builtin.exec_(co, mod.__dict__)
tests/comparators/test_utils.py:27: in
from utils import tools_missing, skip_unless_tools_exist, data,
load_fixture, \
E File
"/tmp/guix-build-diffoscope-61.drv-0/diffoscope-61/tests/comparators/utils.py",
line 49
E return pytest.mark.skipif(True))
E ^
E SyntaxError: invalid syntax
I don't know Python well; it's possible I made the wrong change.
signature.asc
Description: PGP signature
Bug#841146: diffoscope: Failure of test_superblock
On Tue, Oct 18, 2016 at 03:33:14PM -0400, Leo Famulari wrote:
> > > === FAILURES
> > > ===
> > > ___ test_superblock
> > >
> > >
> > > differences = []
> > >
> > > @skip_unless_tool_is_older_than('unsquashfs', unsquashfs_version,
> > > '4.3')
> > > def test_superblock(differences):
> > > expected_diff =
> > > open(data('squashfs_superblock_expected_diff')).read()
> > > > assert differences[0].unified_diff == expected_diff
> > > E IndexError: list index out of range
I just can't reproduce this.
Though I'm using pytest 3.0.3.
That test is skipped by using pytest.mark.skip(), which I don't see in
the docs of pytest for 2.7.
The changelog of pytest tells me pytest.mark.skip() is recognized as a
skipping marker starting from 2.9¹. Is there any chance you can instead
upgrade pytest in your distribution? If so I'll add a versioned
dependency on setup.py, otherwies I can always turn that
pytest.mark.skip() into a pytest.mark.skipif(True), which is IMHO ugly
but quick and effective for solving this bug, I think. Can you also try
to convert that marking in tests/comparators/utils.py:49 to confirm?
¹ http://doc.pytest.org/en/latest/changelog.html#id95
--
regards,
Mattia Rizzolo
GPG Key: 66AE 2B4A FCCF 3F52 DA18 4D18 4B04 3FCD B944 4540 .''`.
more about me: https://mapreri.org : :' :
Launchpad user: https://launchpad.net/~mapreri `. `'`
Debian QA page: https://qa.debian.org/developer.php?login=mattia `-
signature.asc
Description: PGP signature
Bug#841146: diffoscope: Failure of test_superblock
On Tue, Oct 18, 2016 at 06:02:38PM +, Mattia Rizzolo wrote:
> > === FAILURES
> > ===
> > ___ test_superblock
> >
> >
> > differences = []
> >
> > @skip_unless_tool_is_older_than('unsquashfs', unsquashfs_version, '4.3')
> > def test_superblock(differences):
> > expected_diff =
> > open(data('squashfs_superblock_expected_diff')).read()
> > > assert differences[0].unified_diff == expected_diff
> > E IndexError: list index out of range
> >
> > tests/comparators/test_squashfs.py:59: IndexError
> > === short test summary info
> >
> ...
> > SKIP [1] tests/comparators/test_squashfs.py:73: requires unsquashfs
>
> so, it doesn't properly skip that test if unquashfs is not present,
> which is weird, since it's one thing we test in debci¹.
>
> you said you're trying to build the released v61, not a git snapshot
> which happens to be a few commits after v61, right?
Correct, I see this test failure when building the v61 release from
PyPi.
signature.asc
Description: PGP signature
Bug#841146: diffoscope: Failure of test_superblock
control: found -1 61
On Tue, Oct 18, 2016 at 10:28:33AM -0400, Leo Famulari wrote:
> ...
> tests/comparators/test_squashfs.py::test_identification PASSED
> tests/comparators/test_squashfs.py::test_no_differences PASSED
> tests/comparators/test_squashfs.py::test_no_warnings PASSED
> tests/comparators/test_squashfs.py::test_superblock FAILED
> tests/comparators/test_squashfs.py::test_symlink SKIPPED
> tests/comparators/test_squashfs.py::test_compressed_files SKIPPED
> tests/comparators/test_squashfs.py::test_compare_non_existing SKIPPED
...
> === FAILURES
> ===
> ___ test_superblock
>
>
> differences = []
>
> @skip_unless_tool_is_older_than('unsquashfs', unsquashfs_version, '4.3')
> def test_superblock(differences):
> expected_diff = open(data('squashfs_superblock_expected_diff')).read()
> > assert differences[0].unified_diff == expected_diff
> E IndexError: list index out of range
>
> tests/comparators/test_squashfs.py:59: IndexError
> === short test summary info
>
...
> SKIP [1] tests/comparators/test_squashfs.py:73: requires unsquashfs
so, it doesn't properly skip that test if unquashfs is not present,
which is weird, since it's one thing we test in debci¹.
you said you're trying to build the released v61, not a git snapshot
which happens to be a few commits after v61, right?
¹ https://ci.debian.net/packages/d/diffoscope/
--
regards,
Mattia Rizzolo
GPG Key: 66AE 2B4A FCCF 3F52 DA18 4D18 4B04 3FCD B944 4540 .''`.
more about me: https://mapreri.org : :' :
Launchpad user: https://launchpad.net/~mapreri `. `'`
Debian QA page: https://qa.debian.org/developer.php?login=mattia `-
signature.asc
Description: PGP signature
Bug#841146: diffoscope: Failure of test_superblock
On Tue, Oct 18, 2016 at 12:05:13PM +0100, Chris Lamb wrote: > Could you re-run the testsuite with: > > -vv -r sxX > > ... or similar? :) Many thanks. Using '-vvv -r sxX' with pytest 2.7.3, the output of the test suite is below. Please let me know if you need any more information. = test session starts == platform linux -- Python 3.5.2 -- py-1.4.31 -- pytest-2.7.3 -- /gnu/store/yrb0g5nd880b21x4g98mgxlzq9218pvy-python-wrapper-3.5.2/bin/python rootdir: /tmp/guix-build-diffoscope-61.drv-0/diffoscope-61, inifile: collecting ... collected 247 items tests/test_difference.py::test_too_much_input_for_diff PASSED tests/test_difference.py::test_too_long_diff_block_lines PASSED tests/test_difference.py::test_non_str_arguments_to_source1_source2 PASSED tests/test_main.py::test_non_existing_files PASSED tests/test_main.py::test_non_existing_left_with_new_file PASSED tests/test_main.py::test_non_existing_right_with_new_file PASSED tests/test_main.py::test_non_existing_files_with_new_file PASSED tests/test_main.py::test_remove_temp_files_on_sigterm PASSED tests/test_main.py::test_ctrl_c_handling PASSED tests/test_main.py::test_text_option_with_file PASSED tests/test_main.py::test_text_option_with_stdiout PASSED tests/test_main.py::test_no_report_option PASSED tests/test_main.py::test_html_option_with_file PASSED tests/test_main.py::test_html_option_with_stdout PASSED tests/test_main.py::test_no_differences PASSED tests/test_main.py::test_list_tools PASSED tests/comparators/test_binary.py::test_same_content PASSED tests/comparators/test_binary.py::test_not_same_content PASSED tests/comparators/test_binary.py::test_guess_file_type PASSED tests/comparators/test_binary.py::test_guess_encoding_binary PASSED tests/comparators/test_binary.py::test_guess_encoding_ascii PASSED tests/comparators/test_binary.py::test_guess_encoding_unicode PASSED tests/comparators/test_binary.py::test_guess_encoding_iso8859 PASSED tests/comparators/test_binary.py::test_no_differences_with_xxd PASSED tests/comparators/test_binary.py::test_compare_with_xxd SKIPPED tests/comparators/test_binary.py::test_compare_non_existing_with_xxd PASSED tests/comparators/test_binary.py::test_no_differences_without_xxd PASSED tests/comparators/test_binary.py::test_compare_without_xxd PASSED tests/comparators/test_binary.py::test_with_compare_details PASSED tests/comparators/test_binary.py::test_with_compare_details_and_fallback SKIPPED tests/comparators/test_binary.py::test_with_compare_details_and_no_actual_differences PASSED tests/comparators/test_binary.py::test_with_compare_details_and_failed_process SKIPPED tests/comparators/test_binary.py::test_with_compare_details_and_tool_not_found SKIPPED tests/comparators/test_binary.py::test_compare_two_nonexisting_files PASSED tests/comparators/test_binary.py::test_symlink_to_dir PASSED tests/comparators/test_bzip2.py::test_identification PASSED tests/comparators/test_bzip2.py::test_no_differences PASSED tests/comparators/test_bzip2.py::test_content_source PASSED tests/comparators/test_bzip2.py::test_content_source_without_extension PASSED tests/comparators/test_bzip2.py::test_content_diff PASSED tests/comparators/test_bzip2.py::test_compare_non_existing PASSED tests/comparators/test_cbfs.py::test_identification_using_offset SKIPPED tests/comparators/test_cbfs.py::test_identification_without_offset SKIPPED tests/comparators/test_cbfs.py::test_no_differences SKIPPED tests/comparators/test_cbfs.py::test_listing SKIPPED tests/comparators/test_cbfs.py::test_content SKIPPED tests/comparators/test_cbfs.py::test_compare_non_existing SKIPPED tests/comparators/test_cpio.py::test_identification PASSED tests/comparators/test_cpio.py::test_no_differences PASSED tests/comparators/test_cpio.py::test_listing SKIPPED tests/comparators/test_cpio.py::test_symlink SKIPPED tests/comparators/test_cpio.py::test_compressed_files SKIPPED tests/comparators/test_cpio.py::test_compare_non_existing SKIPPED tests/comparators/test_deb.py::test_identification PASSED tests/comparators/test_deb.py::test_no_differences PASSED tests/comparators/test_deb.py::test_metadata PASSED tests/comparators/test_deb.py::test_compressed_files PASSED tests/comparators/test_deb.py::test_identification_of_md5sums_outside_deb PASSED tests/comparators/test_deb.py::test_identification_of_md5sums_in_deb PASSED tests/comparators/test_deb.py::test_md5sums PASSED tests/comparators/test_deb.py::test_identical_files_in_md5sums PASSED tests/comparators/test_deb.py::test_identification_of_data_tar PASSED tests/comparators/test_deb.py::test_skip_comparison_of_known_identical_files PASSED tests/comparators/test_deb.py::test_compare_non_existing PASSED tests/comparators/test_debian.py::test_dot_changes_identification PASSED tests/comparators/test_debian.py::test_dot_changes_invalid PASSED tests/comparators/test_debian.py::test_dot_changes_no_differences PASSED
Bug#841146: diffoscope: Failure of test_superblock
Hi, > diffoscope: Failure of test_superblock Could you re-run the testsuite with: -vv -r sxX ... or similar? :) Many thanks. Regards, -- ,''`. : :' : Chris Lamb `. `'` la...@debian.org / chris-lamb.co.uk `-
Bug#841146: diffoscope: Failure of test_superblock
Source: diffoscope
Dear Maintainer,
While building diffoscope 61 on GNU Guix's "core-updates" branch
(roughly analogous to Debian's testing branch), the test test_superblock
fails like this:
=== FAILURES ===
___ test_superblock
differences = []
@skip_unless_tool_is_older_than('unsquashfs', unsquashfs_version, '4.3')
def test_superblock(differences):
expected_diff = open(data('squashfs_superblock_expected_diff')).read()
> assert differences[0].unified_diff == expected_diff
E IndexError: list index out of range
tests/comparators/test_squashfs.py:59: IndexError
== 1 failed, 173 passed, 73 skipped in 12.80 seconds ===
We are using Python 3.5.2 and pytest 2.7.3. Please let me know what
other information I can provide.
What do you think? Does this test failure indicate a serious problem or
is it safe to skip the test in our packaging?
