Bug#886963: Diffoscope: different readelf implementations cause version parsing errors on FreeBSD

[Guangyuan Yang]

2018-01-16 13:05 GMT+08:00 Guangyuan Yang <y...@freebsd.org>:
> Hi Chris,
>
> It still exists. Here is some investigation inside decorator
> `skip_if_tool_version_is`

For now, change the line

+ if m is None:
+ return 'unknown'

to

+ if m is None:
+ return '0.00'

in

https://anonscm.debian.org/git/reproducible/diffoscope.git/commit/?id=304660a54a433ff65ce0cef986b22b3d76793ab0

will solve the issue.

Bug#886963: Diffoscope: different readelf implementations cause version parsing errors on FreeBSD

[Guangyuan Yang]

2018-01-12 2:36 GMT-05:00 Chris Lamb :
> [Re-adding the bug to CC - please retain in future]
>
> Hi Guangyuan,
>
>> The error still exists
>
> This should fix it:
>
>   
> https://anonscm.debian.org/git/reproducible/diffoscope.git/commit/?id=304660a54a433ff65ce0cef986b22b3d76793ab0

Hi Chris,

It still exists. Here is some investigation inside decorator
`skip_if_tool_version_is`

platform freebsd11 -- Python 3.6.4, pytest-3.2.5, py-1.5.2,
pluggy-0.4.0 -- /usr/local/bin/python3
cachedir: .cache
rootdir: /usr/home/guangyuan/diffoscope, inifile:
collecting 162 items
> PDB set_trace (IO-capturing 
> turned off) 
> >>
(Pdb) s
> /usr/home/guangyuan/diffoscope/tests/utils/tools.py(47)skip_if_tool_version_is()
-> vcls(str(actual_ver)) == vcls(str(target_ver)),
(Pdb) actual_ver
'unknown'
(Pdb) target_ver
'2.29'
(Pdb) vcls(str(actual_ver))
LooseVersion ('unknown')
(Pdb) vcls(str(target_ver))
LooseVersion ('2.29')
(Pdb) vcls(str(actual_ver)) == vcls(str(target_ver))
*** TypeError: '<' not supported between instances of 'str' and 'int'

When 'unknown' and '2.29' passed in, LooseVersion will fail. Actually,
it seems that LooseVersion does not handle such situation:

➜  ~ python3
Python 3.6.3 (default, Oct  3 2017, 21:45:48)
[GCC 7.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from distutils.version import LooseVersion
>>> LooseVersion('1') == LooseVersion('a')
Traceback (most recent call last):
  File "", line 1, in 
  File "/usr/lib/python3.6/distutils/version.py", line 46, in __eq__
c = self._cmp(other)
  File "/usr/lib/python3.6/distutils/version.py", line 337, in _cmp
if self.version < other.version:
TypeError: '<' not supported between instances of 'int' and 'str'

Bug#886963: Diffoscope: different readelf implementations cause version parsing errors on FreeBSD

[Guangyuan Yang]

Source: diffoscope
Version: 90
Severity: normal

Test suite is failing on FreeBSD during the collecting phase:

___ ERROR collecting
tests/comparators/test_elf.py ___
tests/comparators/test_elf.py:64: in 
@skip_if_tool_version_is('readelf', readelf_version, '2.29')
tests/utils/tools.py:46: in skip_if_tool_version_is
vcls(str(actual_ver)) == vcls(str(target_ver)),
/usr/local/lib/python3.6/distutils/version.py:46: in __eq__
c = self._cmp(other)
/usr/local/lib/python3.6/distutils/version.py:337: in _cmp
if self.version < other.version:
E   TypeError: '<' not supported between instances of 'str' and 'int'
!! Interrupted: 1 errors during
collection !!!

The reason being the unexpected output format of `readelf --version` on
FreeBSD:

readelf --version

readelf (elftoolchain r3561M)

On Ubuntu for example, the result is:

readelf --version

GNU readelf (GNU Binutils for Ubuntu) 2.29.1
Copyright (C) 2017 Free Software Foundation, Inc.
This program is free software; you may redistribute it under the terms of
the GNU General Public License version 3 or (at your option) any later
version.
This program has absolutely no warranty.

Since the implementation of `skip_if_tool_version_is` is:

return out.decode('UTF-8').splitlines()[0].split()[-1].strip()

it will output the version as:

test_elf.py(46)readelf_version()->'r3561M)'

which causes problems in this case.

Bug#871029: BSD diff tool causes 5 errors in test suite

[Guangyuan Yang]

Source: diffoscope
Version: 84
Severity: normal

Some test cases failed when running diffoscope test suite on FreeBSD. Most of 
the issues are caused by the difference between BSD Diff (`diff` on FreeBSD) 
and GNU Diff (`gdiff` on FreeBSD).

Diffoscope should use `gdiff` when available. I will make sure that `gdiff` is 
a dependency of diffoscope in FreeBSD Ports. 

A proposed patch is attached. Not sure if this implementation is clean enough 
though.diff --git a/diffoscope/diff.py b/diffoscope/diff.py
index 17a5289..bf46ec6 100644
--- a/diffoscope/diff.py
+++ b/diffoscope/diff.py
@@ -31,11 +31,19 @@ from multiprocessing.dummy import Queue
 
 from diffoscope.tempfiles import get_temporary_directory
 
+from distutils.spawn import find_executable
+
 from .tools import tool_required
 from .config import Config
 
 DIFF_CHUNK = 4096
 
+DIFF_TOOL = 'diff'
+
+gdiff_path = find_executable('gdiff')
+if gdiff_path:
+DIFF_TOOL = 'gdiff'
+
 logger = logging.getLogger(__name__)
 re_diff_change = re.compile(r'^([+-@]).*', re.MULTILINE)
 
@@ -159,9 +167,9 @@ class DiffParser(object):
 return self.skip_block
 
 
-@tool_required('diff')
+@tool_required(DIFF_TOOL)
 def run_diff(fifo1, fifo2, end_nl_q1, end_nl_q2):
-cmd = ['diff', '-aU7', fifo1, fifo2]
+cmd = [DIFF_TOOL, '-aU7', fifo1, fifo2]
 
 logger.debug("Running %s", ' '.join(cmd))