Script 'mail_helper' called by obssrc Hello community, here is the log from the commit of package diffoscope for openSUSE:Factory checked in at 2025-09-12 21:10:04 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Comparing /work/SRC/openSUSE:Factory/diffoscope (Old) and /work/SRC/openSUSE:Factory/.diffoscope.new.1977 (New) ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Package is "diffoscope" Fri Sep 12 21:10:04 2025 rev:49 rq:1304154 version:306 Changes: -------- --- /work/SRC/openSUSE:Factory/diffoscope/diffoscope.changes 2025-03-24 13:31:49.018738091 +0100 +++ /work/SRC/openSUSE:Factory/.diffoscope.new.1977/diffoscope.changes 2025-09-12 21:10:23.677414504 +0200 @@ -1,0 +2,47 @@ +Thu Sep 11 20:18:49 UTC 2025 - Dirk Müller <[email protected]> + +- update to 306: + * Fix compatibility with RPM 6. + * Use regular 'open' calls instead of the deprecated + 'codecs.open'. + * Accept additional 'v' when calling 'fdtump --version'. + * Do not run jsondiff on files over 100KiB as the algorithm + runs in O(n^2) + * Fix test after the upload of systemd-ukify 258~rc3 (vs. + 258~rc2). + * Avoid a crash in the HTML presenter when page limit is None. + * Don't check for PyPDF version 3 specifically, check for >= 3. + * Ensure that Java class files are named .class on the + filesystem before passing them to javap(1). + * Mask stderr from the extract-vmlinux script. + * Make it much more explicit that we return 'success' from the + extract-vmlinux script instead of just falling through to + the bottom of the script. + * Use Difference.from_operation in an attempt to pipeline the + output of extract-vmlinux, potentially avoiding it all + residing in memory. This is an attempt to prevent out of + memory issues on try.diffoscope.org. + * Avoid spurious differences in h5dump output caused by + exposure of absolute internal extraction paths. + * Use our_check_output in the ODT comparator. + * Fix a regression and add a test so that diffoscope picks up + differences in metadata for identical files again. + * Handle RPM's HEADERSIGNATURES and HEADERIMMUTABLE specially + to avoid unncessarily large diffs. + * Add a LZMA comparator and tests. + * Don't rely on zipdetails' --walk functionality to be + available; only add that argument after testing for a new + enough versions. + * Use --walk over the potentially dangerous --scan argument of + zipdetails(1). + * Correct longstanding issue where many ">"-based version tests + used in conditional fixtures were broken due to the lack of a + __gt__ method. + * Address a long-hidden issue in the test_versions testsuite + where we weren't actually testing ">" as it was masked by the + tests for equality in the testsuite. + * Correct import masking issue. + * Ignore st_size entry for directories to avoid spurious diffs + as this value is essentially filesystem dependent. + +------------------------------------------------------------------- Old: ---- diffoscope-291.tar.bz2 diffoscope-291.tar.bz2.asc New: ---- diffoscope-306.tar.bz2 diffoscope-306.tar.bz2.asc ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Other differences: ------------------ ++++++ diffoscope.spec ++++++ --- /var/tmp/diff_new_pack.ZtZcmj/_old 2025-09-12 21:10:24.321441660 +0200 +++ /var/tmp/diff_new_pack.ZtZcmj/_new 2025-09-12 21:10:24.321441660 +0200 @@ -1,7 +1,7 @@ # # spec file for package diffoscope # -# Copyright (c) 2025 SUSE LLC +# Copyright (c) 2025 SUSE LLC and contributors # # All modifications and additions to the file contributed by third parties # remain the property of their copyright owners, unless otherwise agreed @@ -17,7 +17,7 @@ Name: diffoscope -Version: 291 +Version: 306 Release: 0 Summary: In-depth comparison of files, archives, and directories License: GPL-3.0-or-later ++++++ diffoscope-291.tar.bz2 -> diffoscope-306.tar.bz2 ++++++ diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/diffoscope-291/debian/changelog new/diffoscope-306/debian/changelog --- old/diffoscope-291/debian/changelog 2025-03-21 15:11:51.000000000 +0100 +++ new/diffoscope-306/debian/changelog 2025-09-08 00:31:34.000000000 +0200 @@ -1,3 +1,164 @@ +diffoscope (306) unstable; urgency=medium + + [ Zbigniew Jędrzejewski-Szmek ] + * Fix compatibility with RPM 6. + * Use regular 'open' calls instead of the deprecated 'codecs.open'. + * Accept additional 'v' when calling 'fdtump --version'. + + -- Chris Lamb <[email protected]> Sun, 07 Sep 2025 15:31:24 -0700 + +diffoscope (305) unstable; urgency=medium + + [ Chris Lamb ] + * Upload to unstable/sid after the release of trixie. + + -- Chris Lamb <[email protected]> Fri, 22 Aug 2025 10:40:18 -0700 + +diffoscope (304) experimental; urgency=medium + + [ Chris Lamb ] + * Do not run jsondiff on files over 100KiB as the algorithm runs in O(n^2) + time. (Closes: reproducible-builds/diffoscope#414) + * Fix test after the upload of systemd-ukify 258~rc3 (vs. 258~rc2). + * Move from a mono-utils dependency to versioned "mono-devel | mono-utils" + dependency, taking care to maintain the [!riscv64] architecture + restriction. (Closes: #1111742) + * Use sed -ne over awk -F= to to avoid mangling dependency lines containing + equals signs (=), for example version restrictions. + * Use sed backreferences when generating debian/tests/control to avoid DRY + violations. + * Update copyright years. + + [ Martin Joerg ] + * Avoid a crash in the HTML presenter when page limit is None. + + -- Chris Lamb <[email protected]> Fri, 22 Aug 2025 08:13:11 -0700 + +diffoscope (303) experimental; urgency=medium + + [ Chris Lamb ] + * Don't check for PyPDF version 3 specifically, check for >= 3. Thanks, + Vagrant, for the patch. (Closes: reproducible-builds/diffoscope#413) + * Ensure that Java class files are named .class on the filesystem before + passing them to javap(1). + * Update copyright years. + + -- Chris Lamb <[email protected]> Fri, 01 Aug 2025 11:50:45 -0700 + +diffoscope (302) experimental; urgency=medium + + [ Chris Lamb ] + * Mask stderr from the extract-vmlinux script. + * Make it much more explicit that we return 'success' from the + extract-vmlinux script instead of just falling through to the bottom of the + script. + * Use Difference.from_operation in an attempt to pipeline the output of + extract-vmlinux, potentially avoiding it all residing in memory. This is an + attempt to prevent out of memory issues on try.diffoscope.org. + + [ Siva Mahadevan ] + * Use --print-armap long option with nm in the "ar" comparator for wider + compatibility. + + -- Chris Lamb <[email protected]> Fri, 25 Jul 2025 09:44:26 -0700 + +diffoscope (301) experimental; urgency=medium + + [ Chris Lamb ] + * Avoid spurious differences in h5dump output caused by exposure of absolute + internal extraction paths. (Closes: #1108690) + * Use our_check_output in the ODT comparator. + * Memoize a number of calls to --version. Thanks, Jade! (Closes: #412) + * Update copyright years. + + -- Chris Lamb <[email protected]> Fri, 11 Jul 2025 08:52:31 -0700 + +diffoscope (300) experimental; urgency=medium + + [ "Alex" ] + * Fix a regression and add a test so that diffoscope picks up differences + in metadata for identical files again. (Closes: reproducible-builds/diffoscope#411) + + -- Chris Lamb <[email protected]> Fri, 27 Jun 2025 10:29:56 -0700 + +diffoscope (299) experimental; urgency=medium + + [ Chris Lamb ] + * Add python3-defusedxml to the Build-Depends in order to include it in the + Docker image. (Closes: #407) + + -- Chris Lamb <[email protected]> Fri, 20 Jun 2025 09:08:17 -0700 + +diffoscope (298) experimental; urgency=medium + + [ Chris Lamb ] + * Handle RPM's HEADERSIGNATURES and HEADERIMMUTABLE specially to avoid + unncessarily large diffs. Based almost entirely on code by Daniel Duan. + (Closes: reproducible-builds/diffoscope#410) + * Update copyright years. + + -- Chris Lamb <[email protected]> Fri, 13 Jun 2025 09:49:56 -0700 + +diffoscope (297) unstable; urgency=medium + + [ Will Hollywood ] + * Add a LZMA comparator and tests. + + -- Chris Lamb <[email protected]> Fri, 30 May 2025 09:34:34 -0700 + +diffoscope (296) unstable; urgency=medium + + [ Chris Lamb ] + * Don't rely on zipdetails' --walk functionality to be available; only add + that argument after testing for a new enough versions. + (Closes: reproducible-builds/diffoscope#408) + * Disable and then re-enable failing on stable-bpo. + * Update copyright years. + + [ Omair Majid ] + * Add NuGet package support. + + -- Chris Lamb <[email protected]> Fri, 16 May 2025 08:41:37 -0700 + +diffoscope (295) unstable; urgency=medium + + [ Chris Lamb ] + * Use --walk over the potentially dangerous --scan argument of zipdetails(1). + (Closes: reproducible-builds/diffoscope#406) + + -- Chris Lamb <[email protected]> Fri, 09 May 2025 09:13:39 -0700 + +diffoscope (294) unstable; urgency=medium + + [ Chris Lamb ] + * Correct longstanding issue where many ">"-based version tests used in + conditional fixtures were broken due to the lack of a __gt__ method. + Thanks, Colin Watson! (Closes: #1102658) + * Address a long-hidden issue in the test_versions testsuite where we weren't + actually testing ">" as it was masked by the tests for equality in the + testsuite. + * Update copyright years. + + -- Chris Lamb <[email protected]> Fri, 11 Apr 2025 12:22:46 -0700 + +diffoscope (293) unstable; urgency=medium + + [ Chris Lamb ] + * Correct import masking issue. + + -- Chris Lamb <[email protected]> Sat, 29 Mar 2025 18:05:20 +0000 + +diffoscope (292) unstable; urgency=medium + + [ Ivan Trubach ] + * Ignore st_size entry for directories to avoid spurious diffs as this value + is essentially filesystem dependent. + + [ Chris Lamb ] + * Update copyright years. + + -- Chris Lamb <[email protected]> Sat, 29 Mar 2025 17:19:49 +0000 + diffoscope (291) unstable; urgency=medium [ Chris Lamb ] diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/diffoscope-291/debian/control new/diffoscope-306/debian/control --- old/diffoscope-291/debian/control 2025-03-21 15:11:51.000000000 +0100 +++ new/diffoscope-306/debian/control 2025-09-08 00:31:34.000000000 +0200 @@ -55,7 +55,7 @@ llvm <!nocheck>, lz4 <!nocheck>, lzip <!nocheck>, - mono-utils <!nocheck>, + mono-devel (>= 6.14.1+ds-3) <!nocheck> | mono-utils (<< 6.14.1+ds-3) <!nocheck>, ocaml-nox <!nocheck>, odt2txt <!nocheck>, oggvideotools [!s390x] <!nocheck>, @@ -68,6 +68,7 @@ python3-argcomplete, python3-binwalk <!nocheck>, python3-debian <!nocheck>, + python3-defusedxml <!nocheck>, python3-distro <!nocheck>, python3-docutils, python3-guestfs <!nocheck>, diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/diffoscope-291/debian/tests/control new/diffoscope-306/debian/tests/control --- old/diffoscope-291/debian/tests/control 2025-03-21 15:11:51.000000000 +0100 +++ new/diffoscope-306/debian/tests/control 2025-09-08 00:31:34.000000000 +0200 @@ -7,7 +7,7 @@ # $ mv debian/tests/control.tmp debian/tests/control Tests: pytest-with-recommends -Depends: python3-all, diffoscope, black, python3-pytest, python3-h5py, file, linux-image-amd64 [amd64] | linux-image-generic [amd64], 7zip, abootimg, acl, apksigcopier, apksigner, apktool, binutils-multiarch, bzip2, caca-utils, colord, coreboot-utils [!risv64], db-util, default-jdk-headless | default-jdk | java-sdk, device-tree-compiler, docx2txt, e2fsprogs, enjarify, ffmpeg, fontforge-extras, fonttools, fp-utils [!riscv64 !s390x], genisoimage, gettext, ghc, ghostscript, giflib-tools, gnumeric, gnupg-utils, gpg, hdf5-tools, html2text, imagemagick, jsbeautifier, libarchive-tools, libxmlb-utils, llvm, lz4, lzip, mono-utils [!riscv64], ocaml-nox, odt2txt, oggvideotools [!s390x], openssh-client, openssl, perl, pgpdump, poppler-utils, procyon-decompiler, python3-pdfminer, r-base-core, rpm2cpio, sng, sqlite3, squashfs-tools, systemd-ukify, tcpdump, u-boot-tools, unzip, wabt, xmlbeans, xxd, xz-utils, zip, zstd, python3-argcomplete, python3-binwalk, python3-defusedxml, python3-distro, pytho n3-guestfs, python3-jsondiff, python3-progressbar, python3-pypdf, python3-debian, python3-pyxattr, python3-rpm, python3-tlsh +Depends: python3-all, diffoscope, black, python3-pytest, python3-h5py, file, linux-image-amd64 [amd64] | linux-image-generic [amd64], 7zip, abootimg, acl, apksigcopier, apksigner, apktool, binutils-multiarch, bzip2, caca-utils, colord, coreboot-utils [!risv64], db-util, default-jdk-headless | default-jdk | java-sdk, device-tree-compiler, docx2txt, e2fsprogs, enjarify, ffmpeg, fontforge-extras, fonttools, fp-utils [!riscv64 !s390x], genisoimage, gettext, ghc, ghostscript, giflib-tools, gnumeric, gnupg-utils, gpg, hdf5-tools, html2text, imagemagick, jsbeautifier, libarchive-tools, libxmlb-utils, llvm, lz4, lzip, mono-devel (>= 6.14.1+ds-3) [!riscv64] | mono-utils (<< 6.14.1+ds-3) [!riscv64], ocaml-nox, odt2txt, oggvideotools [!s390x], openssh-client, openssl, perl, pgpdump, poppler-utils, procyon-decompiler, python3-pdfminer, r-base-core, rpm2cpio, sng, sqlite3, squashfs-tools, systemd-ukify, tcpdump, u-boot-tools, unzip, wabt, xmlbeans, xxd, xz-utils, zip, zstd, python3-argcomplete, python3-binwalk, python3-defusedxml, python3-distro, python3-guestfs, python3-jsondiff, python3-progressbar, python3-pypdf, python3-debian, python3-pyxattr, python3-rpm, python3-tlsh Tests: pytest Depends: python3-all, diffoscope, python3-pytest, python3-h5py, file, python3-tlsh diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/diffoscope-291/debian/tests/control.sh new/diffoscope-306/debian/tests/control.sh --- old/diffoscope-291/debian/tests/control.sh 2025-03-21 15:11:51.000000000 +0100 +++ new/diffoscope-306/debian/tests/control.sh 2025-09-08 00:31:34.000000000 +0200 @@ -10,7 +10,7 @@ cat debian/tests/control.in >> debian/tests/control.tmp -sed -i "s#%RECOMMENDS%#$(bin/diffoscope --list-debian-substvars | awk -F= '/diffoscope:Recommends/ { print $2 }')#" debian/tests/control.tmp +sed -i "s#%RECOMMENDS%#$(bin/diffoscope --list-debian-substvars | sed -ne 's,^diffoscope:Recommends=,,p')#" debian/tests/control.tmp sed -i "s#%PYRECOMMENDS%#$(debian/tests/generate-recommends.py)#" debian/tests/control.tmp @@ -19,15 +19,17 @@ sed -i "s,python3-python-debian,python3-debian," debian/tests/control.tmp sed -i "s,python3-rpm-python,python3-rpm," debian/tests/control.tmp -sed -i "s,coreboot-utils,coreboot-utils [!risv64]," debian/tests/control.tmp -sed -i "s,fp-utils,fp-utils [!riscv64 !s390x]," debian/tests/control.tmp -sed -i "s,mono-utils,mono-utils [!riscv64]," debian/tests/control.tmp -sed -i "s,oggvideotools,oggvideotools [!s390x]," debian/tests/control.tmp + +sed -i "s,\(coreboot-utils\),\1 [!risv64]," debian/tests/control.tmp +sed -i "s,\(fp-utils\),\1 [!riscv64 !s390x]," debian/tests/control.tmp +sed -i "s,\(mono-devel (>= 6.14.1+ds-3)\),\1 [!riscv64]," debian/tests/control.tmp +sed -i "s,\(mono-utils (<< 6.14.1+ds-3)\),\1 [!riscv64]," debian/tests/control.tmp +sed -i "s,\(oggvideotools\),\1 [!s390x]," debian/tests/control.tmp #sed -i "s,python3-androguard,androguard," debian/tests/control.tmp -#sed -i "s,dexdump,dexdump [amd64 arm64 armhf i386]," debian/tests/control.tmp +#sed -i "s,\(dexdump\),\1 [amd64 arm64 armhf i386]," debian/tests/control.tmp # aapt removed due to not being in trixie atma - #1070416 # also remove androguard and dexdump for the same reason -#sed -i "s,aapt,aapt [amd64 arm64 armel armhf i386 mips64el mipsel]," debian/tests/control.tmp +#sed -i "s,\(aapt\),\1 [amd64 arm64 armel armhf i386 mips64el mipsel]," debian/tests/control.tmp sed -i "s#aapt, ##" debian/tests/control.tmp sed -i "s#dexdump, ##" debian/tests/control.tmp sed -i "s#python3-androguard, ##" debian/tests/control.tmp diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/diffoscope-291/diffoscope/__init__.py new/diffoscope-306/diffoscope/__init__.py --- old/diffoscope-291/diffoscope/__init__.py 2025-03-21 15:11:51.000000000 +0100 +++ new/diffoscope-306/diffoscope/__init__.py 2025-09-08 00:31:34.000000000 +0200 @@ -17,4 +17,4 @@ # You should have received a copy of the GNU General Public License # along with diffoscope. If not, see <https://www.gnu.org/licenses/>. -VERSION = "291" +VERSION = "306" diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/diffoscope-291/diffoscope/comparators/__init__.py new/diffoscope-306/diffoscope/comparators/__init__.py --- old/diffoscope-291/diffoscope/comparators/__init__.py 2025-03-21 15:11:51.000000000 +0100 +++ new/diffoscope-306/diffoscope/comparators/__init__.py 2025-09-08 00:31:34.000000000 +0200 @@ -82,6 +82,7 @@ ("java.ClassFile",), ("lz4.Lz4File",), ("lzip.LzipFile",), + ("lzma.LzmaFile",), ("mono.MonoExeFile",), ("pdf.PdfFile",), ("png.PngFile",), @@ -98,6 +99,7 @@ ("ocaml.OcamlInterfaceFile",), ("docx.DocxFile",), ("zip.MozillaZipFile",), + ("zip.NuGetPackageFile",), ("zip.JmodJavaModule",), ("zip.ZipFile",), ("image.JPEGImageFile",), diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/diffoscope-291/diffoscope/comparators/ar.py new/diffoscope-306/diffoscope/comparators/ar.py --- old/diffoscope-291/diffoscope/comparators/ar.py 2025-03-21 15:11:51.000000000 +0100 +++ new/diffoscope-306/diffoscope/comparators/ar.py 2025-09-08 00:31:34.000000000 +0200 @@ -55,7 +55,7 @@ class ArSymbolTableDumper(Command): @tool_required("nm") def cmdline(self): - return ["nm", "-s", self.path] + return ["nm", "--print-armap", self.path] class ArFile(File): diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/diffoscope-291/diffoscope/comparators/directory.py new/diffoscope-306/diffoscope/comparators/directory.py --- old/diffoscope-291/diffoscope/comparators/directory.py 2025-03-21 15:11:51.000000000 +0100 +++ new/diffoscope-306/diffoscope/comparators/directory.py 2025-09-08 00:31:34.000000000 +0200 @@ -2,7 +2,7 @@ # diffoscope: in-depth comparison of files, archives, and directories # # Copyright © 2015 Jérémy Bobbio <[email protected]> -# Copyright © 2015-2021 Chris Lamb <[email protected]> +# Copyright © 2015-2021, 2025 Chris Lamb <[email protected]> # # diffoscope is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -17,9 +17,10 @@ # You should have received a copy of the GNU General Public License # along with diffoscope. If not, see <https://www.gnu.org/licenses/>. +import logging import os import re -import logging +import stat import subprocess from diffoscope.exc import RequiredToolNotFound @@ -76,15 +77,19 @@ # compare only what matters def stat_results_same(stat1, stat2): - return all( - getattr(stat1, i) == getattr(stat2, i) - for i in [ - "st_mode", - "st_uid", - "st_gid", - "st_size", - "st_mtime", - ] + return stat_result_to_tuple(stat1) == stat_result_to_tuple(stat2) + + +def stat_result_to_tuple(s): + return ( + s.st_mode, + s.st_uid, + s.st_gid, + # Directory sizes are essentially arbitrary and filesystem dependent, + # and may even differ, depending on the history of the directory. + # (See MR !150) + s.st_size if not stat.S_ISDIR(s.st_mode) else None, + s.st_mtime, ) diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/diffoscope-291/diffoscope/comparators/hdf.py new/diffoscope-306/diffoscope/comparators/hdf.py --- old/diffoscope-291/diffoscope/comparators/hdf.py 2025-03-21 15:11:51.000000000 +0100 +++ new/diffoscope-306/diffoscope/comparators/hdf.py 2025-09-08 00:31:34.000000000 +0200 @@ -1,7 +1,7 @@ # # diffoscope: in-depth comparison of files, archives, and directories # -# Copyright © 2020 Chris Lamb <[email protected]> +# Copyright © 2020, 2025 Chris Lamb <[email protected]> # # diffoscope is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -30,6 +30,12 @@ def cmdline(self): return ["h5dump", self.path] + def filter(self, line): + # Strip absolute `HDF5 "/path/to/filename.db"` + return re.sub( + r"^(HDF5) \"[^\"]+\"", "\\1", line.decode("utf-8") + ).encode("utf-8") + class Hdf5File(File): DESCRIPTION = "Hierarchical Data Format database" diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/diffoscope-291/diffoscope/comparators/java.py new/diffoscope-306/diffoscope/comparators/java.py --- old/diffoscope-291/diffoscope/comparators/java.py 2025-03-21 15:11:51.000000000 +0100 +++ new/diffoscope-306/diffoscope/comparators/java.py 2025-09-08 00:31:34.000000000 +0200 @@ -3,7 +3,7 @@ # # Copyright © 2015 Reiner Herrmann <[email protected]> # Copyright © 2015 Jérémy Bobbio <[email protected]> -# Copyright © 2016, 2018-2020, 2024 Chris Lamb <[email protected]> +# Copyright © 2016, 2018-2020, 2024-2025 Chris Lamb <[email protected]> # # diffoscope is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -21,10 +21,12 @@ import re import os.path import logging +import shutil -from diffoscope.tools import tool_required from diffoscope.difference import Difference from diffoscope.exc import RequiredToolNotFound +from diffoscope.tempfiles import get_named_temporary_file +from diffoscope.tools import tool_required from .utils.file import File from .utils.command import Command, strip_ansi_escapes @@ -49,9 +51,19 @@ class Javap(Command): def __init__(self, path, *args, **kwargs): - super().__init__(path, *args, **kwargs) + # Ensure that class files are named .class on the filesystem as javap + # will not accept files with other extensions. + if not path.endswith(".class"): + self.tempfile = get_named_temporary_file(suffix=".class") + logger.debug(f"Copying {path} to {self.tempfile.name}") + shutil.copy(path, self.tempfile.name) + path = self.tempfile.name + + # Save the realpath for filter() self.real_path = os.path.realpath(path) + super().__init__(path, *args, **kwargs) + @tool_required("javap") def cmdline(self): return [ diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/diffoscope-291/diffoscope/comparators/json.py new/diffoscope-306/diffoscope/comparators/json.py --- old/diffoscope-291/diffoscope/comparators/json.py 2025-03-21 15:11:51.000000000 +0100 +++ new/diffoscope-306/diffoscope/comparators/json.py 2025-09-08 00:31:34.000000000 +0200 @@ -1,7 +1,7 @@ # # diffoscope: in-depth comparison of files, archives, and directories # -# Copyright © 2016-2022 Chris Lamb <[email protected]> +# Copyright © 2016-2022, 2025 Chris Lamb <[email protected]> # # diffoscope is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -18,6 +18,7 @@ import re import json +import os import pprint import logging import collections @@ -85,6 +86,15 @@ if jsondiff is None: return + # Ignore files over 100 KiB as jsondiff scales in O(n^2) time. + # <https://github.com/xlwings/jsondiff/issues/73> + limit = 100 * 1024 + if ( + os.stat(self.path).st_size > limit + or os.stat(other.path).st_size > limit + ): + return + logger.debug(f"Comparing using jsondiff {jsondiff.__version__}") a = getattr(self, "parsed", {}) diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/diffoscope-291/diffoscope/comparators/lzma.py new/diffoscope-306/diffoscope/comparators/lzma.py --- old/diffoscope-291/diffoscope/comparators/lzma.py 1970-01-01 01:00:00.000000000 +0100 +++ new/diffoscope-306/diffoscope/comparators/lzma.py 2025-09-08 00:31:34.000000000 +0200 @@ -0,0 +1,65 @@ +# +# diffoscope: in-depth comparison of files, archives, and directories +# +# Copyright © 2014-2015 Jérémy Bobbio <[email protected]> +# Copyright © 2015-2020, 2024-2025 Chris Lamb <[email protected]> +# +# diffoscope is free software: you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation, either version 3 of the License, or +# (at your option) any later version. +# +# diffoscope is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with diffoscope. If not, see <https://www.gnu.org/licenses/>. + +import re +import logging +import subprocess + +from diffoscope.tools import tool_required + + +from .utils.file import File +from .utils.archive import Archive + +logger = logging.getLogger(__name__) + + +class LzmaContainer(Archive): + def open_archive(self): + return self + + def close_archive(self): + pass + + def get_member_names(self): + return [self.get_compressed_content_name(".lzma")] + + @tool_required("xz") + def extract(self, member_name, dest_dir): + dest_path = self.get_path_name(dest_dir) + logger.debug("lzma extracting to %s", dest_path) + with open(dest_path, "wb") as fp: + subprocess.check_call( + [ + "xz", + "--decompress", + "--format=lzma", + "--stdout", + self.source.path, + ], + stdout=fp, + stderr=None, + ) + return dest_path + + +class LzmaFile(File): + DESCRIPTION = "LZMA compressed files" + CONTAINER_CLASSES = [LzmaContainer] + FILE_TYPE_RE = re.compile(r"^LZMA compressed data\b") diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/diffoscope-291/diffoscope/comparators/odt.py new/diffoscope-306/diffoscope/comparators/odt.py --- old/diffoscope-291/diffoscope/comparators/odt.py 2025-03-21 15:11:51.000000000 +0100 +++ new/diffoscope-306/diffoscope/comparators/odt.py 2025-09-08 00:31:34.000000000 +0200 @@ -1,7 +1,7 @@ # # diffoscope: in-depth comparison of files, archives, and directories # -# Copyright © 2017-2022 Chris Lamb <[email protected]> +# Copyright © 2017-2022, 2025 Chris Lamb <[email protected]> # # diffoscope is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -17,13 +17,14 @@ # along with diffoscope. If not, see <https://www.gnu.org/licenses/>. import re +import functools import subprocess from diffoscope.tools import tool_required from diffoscope.difference import Difference from .utils.file import File -from .utils.command import Command +from .utils.command import Command, our_check_output class Odt2txt(Command): @@ -37,9 +38,10 @@ return ("odt2txt", "--width=-1", self.path) @staticmethod + @functools.lru_cache def odt2txt_variant(): try: - out = subprocess.check_output(["odt2txt", "--version"]) + out = our_check_output(["odt2txt", "--version"]) except subprocess.CalledProcessError as e: out = e.output return out.decode("UTF-8").splitlines()[0].split()[0].strip() diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/diffoscope-291/diffoscope/comparators/rpm.py new/diffoscope-306/diffoscope/comparators/rpm.py --- old/diffoscope-291/diffoscope/comparators/rpm.py 2025-03-21 15:11:51.000000000 +0100 +++ new/diffoscope-306/diffoscope/comparators/rpm.py 2025-09-08 00:31:34.000000000 +0200 @@ -3,7 +3,7 @@ # # Copyright © 2015 Reiner Herrmann <[email protected]> # Copyright © 2015 Jérémy Bobbio <[email protected]> -# Copyright © 2015-2022 Chris Lamb <[email protected]> +# Copyright © 2015-2022, 2025 Chris Lamb <[email protected]> # # diffoscope is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -35,6 +35,9 @@ logger = logging.getLogger(__name__) +RPMTAG_HEADERSIGNATURES = 62 # Removed in rpm 5.99 +RPMTAG_HEADERIMMUTABLE = 63 + def convert_header_field(io, header): if isinstance(header, list): @@ -61,6 +64,7 @@ def get_rpm_header(path, ts): s = io.StringIO() + with open(path, "r") as f: try: hdr = ts.hdrFromFdno(f) @@ -70,9 +74,27 @@ for rpmtag in sorted(rpm.tagnames): if rpmtag not in hdr: continue + + if rpmtag in { + RPMTAG_HEADERSIGNATURES, + RPMTAG_HEADERIMMUTABLE, + }: + # Handle HEADERSIGNATURES and HEADERIMMUTABLE differently to + # avoid unnecessarily large diffs. (#410) + region = hdr[rpmtag] + # First 4 bytes are the index count + idx_count = int.from_bytes(region[:4], "big") + # ... the next 4 bytes are the data length + data_len = int.from_bytes(region[4:8], "big") + s.write( + f"{rpm.tagnames[rpmtag]}: [{idx_count} indexes, {data_len} bytes]\n" + ) + continue + s.write("{}: ".format(rpm.tagnames[rpmtag])) convert_header_field(s, hdr[rpmtag]) s.write("\n") + return s.getvalue() diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/diffoscope-291/diffoscope/comparators/text.py new/diffoscope-306/diffoscope/comparators/text.py --- old/diffoscope-291/diffoscope/comparators/text.py 2025-03-21 15:11:51.000000000 +0100 +++ new/diffoscope-306/diffoscope/comparators/text.py 2025-09-08 00:31:34.000000000 +0200 @@ -25,6 +25,18 @@ from .utils.file import File +def _open_with_codec(filename, encoding): + info = codecs.lookup(encoding) + # codecs.open added 'b' to the mode if encoding was specified + file = open(filename, "rb") + srw = codecs.StreamReaderWriter( + file, info.streamreader, info.streamwriter, errors="strict" + ) + # Add attributes to simplify introspection + srw.encoding = encoding + return srw + + class TextFile(File): DESCRIPTION = "text files" FILE_TYPE_RE = re.compile(r"\btext\b") @@ -39,10 +51,10 @@ my_encoding = self.encoding or "utf-8" other_encoding = other.encoding or "utf-8" try: - with codecs.open( - self.path, "r", encoding=my_encoding - ) as my_content, codecs.open( - other.path, "r", encoding=other_encoding + with _open_with_codec( + self.path, my_encoding + ) as my_content, _open_with_codec( + other.path, other_encoding ) as other_content: difference = Difference.from_text_readers( my_content, other_content, self.name, other.name, source diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/diffoscope-291/diffoscope/comparators/utils/compare.py new/diffoscope-306/diffoscope/comparators/utils/compare.py --- old/diffoscope-291/diffoscope/comparators/utils/compare.py 2025-03-21 15:11:51.000000000 +0100 +++ new/diffoscope-306/diffoscope/comparators/utils/compare.py 2025-09-08 00:31:34.000000000 +0200 @@ -68,10 +68,12 @@ file2 = FilesystemFile(path2, container=container2) with profile("has_same_content_as", file1): - if file1.has_same_content_as(file2): - return None + is_binary_identical = file1.has_same_content_as(file2) - difference = compare_files(file1, file2) + if is_binary_identical: + difference = None + else: + difference = compare_files(file1, file2) if Config().exclude_directory_metadata in ("no", "recursive"): meta = compare_meta(path1, path2) diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/diffoscope-291/diffoscope/comparators/vmlinuz.py new/diffoscope-306/diffoscope/comparators/vmlinuz.py --- old/diffoscope-291/diffoscope/comparators/vmlinuz.py 2025-03-21 15:11:51.000000000 +0100 +++ new/diffoscope-306/diffoscope/comparators/vmlinuz.py 2025-09-08 00:31:34.000000000 +0200 @@ -1,7 +1,7 @@ # # diffoscope: in-depth comparison of files, archives, and directories # -# Copyright © 2022 Chris Lamb <[email protected]> +# Copyright © 2022, 2025 Chris Lamb <[email protected]> # # diffoscope is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -17,49 +17,35 @@ # along with diffoscope. If not, see <https://www.gnu.org/licenses/>. import logging -import os.path import pathlib import re -import subprocess -from diffoscope.tools import tool_required +from diffoscope.difference import Difference from .utils.file import File -from .utils.archive import Archive +from .utils.command import Command -logger = logging.getLogger(__name__) +logger = logging.getLogger(__name__) -class VmlinuzContainer(Archive): - def open_archive(self): - return self - - def close_archive(self): - pass - - def get_member_names(self): - return [self.get_compressed_content_name(".vmlinuz")] - - @tool_required("readelf") - def extract(self, member_name, dest_dir): - dest_path = os.path.join(dest_dir, member_name) - logger.debug("extracting vmlinuz to %s", dest_path) +class ExtractVmlinux(Command): + def cmdline(self): # Locate extract-vmlinux script - script = pathlib.Path(__file__).parent.parent.joinpath( - "scripts", "extract-vmlinux" - ) - with open(dest_path, "wb") as f: - subprocess.check_call( - [script, self.source.path], - stdout=f, - stderr=None, + script = str( + pathlib.Path(__file__).parent.parent.joinpath( + "scripts", "extract-vmlinux" ) + ) - return dest_path + return [script, self.path] class VmlinuzFile(File): DESCRIPTION = "Linux kernel images" - CONTAINER_CLASSES = [VmlinuzContainer] FILE_TYPE_RE = re.compile(r"^Linux kernel\b") + + def compare_details(self, other, source=None): + return [ + Difference.from_operation(ExtractVmlinux, self.path, other.path) + ] diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/diffoscope-291/diffoscope/comparators/zip.py new/diffoscope-306/diffoscope/comparators/zip.py --- old/diffoscope-291/diffoscope/comparators/zip.py 2025-03-21 15:11:51.000000000 +0100 +++ new/diffoscope-306/diffoscope/comparators/zip.py 2025-09-08 00:31:34.000000000 +0200 @@ -17,6 +17,8 @@ # You should have received a copy of the GNU General Public License # along with diffoscope. If not, see <https://www.gnu.org/licenses/>. + +import functools import re import sys import shutil @@ -32,7 +34,12 @@ from .utils.file import File from .directory import Directory from .utils.archive import Archive, ArchiveMember -from .utils.command import Command +from .utils.command import Command, our_check_output + + [email protected]_cache() +def zipdetails_version(): + return our_check_output(["zipdetails", "--version"]).decode("UTF-8") class Zipinfo(Command): @@ -157,7 +164,19 @@ class Zipdetails(Command): @tool_required("zipdetails") def cmdline(self): - return ["zipdetails", "--redact", "--scan", "--utc", self.path] + # See <https://salsa.debian.org/reproducible-builds/diffoscope/-/issues/406> + # for discussion of zipdetails command line arguments. + # + # Older zipdetails does not support --walk; added in Debian + # 5.40.0~rc1-1, but "zipdetails --version" shipped in, say, perl + # 5.36.0-7+deb12u1 returns "2.104". + # + # See <https://salsa.debian.org/reproducible-builds/diffoscope/-/issues/408> + # + if float(zipdetails_version()) < 4.0: + return ["zipdetails", "--redact", "--utc", self.path] + + return ["zipdetails", "--redact", "--walk", "--utc", self.path] class ZipDirectory(Directory, ArchiveMember): @@ -340,6 +359,13 @@ return file.file_header[4:8] == b"PK\x01\x02" +class NuGetPackageFile(ZipFile): + DESCRIPTION = "NuGet packages" + CONTAINER_CLASSES = [ZipContainer] + FILE_TYPE_HEADER_PREFIX = b"PK\x03\x04" + FILE_EXTENSION_SUFFIX = {".nupkg"} + + class JmodJavaModule(ZipFile): DESCRIPTION = "Java .jmod modules" FILE_TYPE_RE = re.compile(r"^(Zip archive data|Java jmod module)") diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/diffoscope-291/diffoscope/external_tools.py new/diffoscope-306/diffoscope/external_tools.py --- old/diffoscope-291/diffoscope/external_tools.py 2025-03-21 15:11:51.000000000 +0100 +++ new/diffoscope-306/diffoscope/external_tools.py 2025-09-08 00:31:34.000000000 +0200 @@ -1,7 +1,7 @@ # # diffoscope: in-depth comparison of files, archives, and directories # -# Copyright © 2017-2024 Chris Lamb <[email protected]> +# Copyright © 2017-2025 Chris Lamb <[email protected]> # Copyright © 2021 Jean-Romain Garnier <[email protected]> # # diffoscope is free software: you can redistribute it and/or modify @@ -132,6 +132,7 @@ }, "lz4": {"debian": "lz4", "FreeBSD": "lz4", "guix": "lz4"}, "lzip": {"debian": "lzip", "guix": "lzip"}, + "lzma": {"debian": "xz-utils", "arch": "xz", "guix": "xz"}, "msgunfmt": { "debian": "gettext", "arch": "gettext", @@ -171,7 +172,7 @@ "guix": "poppler", }, "pedump": { - "debian": "mono-utils", + "debian": "mono-devel (>= 6.14.1+ds-3) | mono-utils (<< 6.14.1+ds-3)", "arch": "mono", "FreeBSD": "mono", }, diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/diffoscope-291/diffoscope/presenters/html/html.py new/diffoscope-306/diffoscope/presenters/html/html.py --- old/diffoscope-291/diffoscope/presenters/html/html.py 2025-03-21 15:11:51.000000000 +0100 +++ new/diffoscope-306/diffoscope/presenters/html/html.py 2025-09-08 00:31:34.000000000 +0200 @@ -31,7 +31,6 @@ # import base64 -import codecs import collections import contextlib import hashlib @@ -337,17 +336,13 @@ @contextlib.contextmanager def file_printer(directory, filename): - with codecs.open( - os.path.join(directory, filename), "w", encoding="utf-8" - ) as f: + with open(os.path.join(directory, filename), "w", encoding="utf-8") as f: yield f.write @contextlib.contextmanager def spl_file_printer(directory, filename, accum): - with codecs.open( - os.path.join(directory, filename), "w", encoding="utf-8" - ) as f: + with open(os.path.join(directory, filename), "w", encoding="utf-8") as f: print_func = f.write def recording_print_func(s): @@ -743,7 +738,7 @@ format(report_current, ","), format(self.report_limit, ","), format(page_current, ","), - format(page_limit, ","), + format(page_limit, ",") if page_limit else "-", want_to_add, ) if report_current + want_to_add > self.report_limit: diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/diffoscope-291/diffoscope/presenters/utils.py new/diffoscope-306/diffoscope/presenters/utils.py --- old/diffoscope-291/diffoscope/presenters/utils.py 2025-03-21 15:11:51.000000000 +0100 +++ new/diffoscope-306/diffoscope/presenters/utils.py 2025-09-08 00:31:34.000000000 +0200 @@ -17,7 +17,6 @@ # along with diffoscope. If not, see <https://www.gnu.org/licenses/>. import sys -import codecs import collections import contextlib import string @@ -93,7 +92,7 @@ output = sys.stdout if path != "-": - output = codecs.open(path, "w", encoding="utf-8") + output = open(path, "w", encoding="utf-8") def fn(*args, **kwargs): kwargs["file"] = output diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/diffoscope-291/diffoscope/scripts/extract-vmlinux new/diffoscope-306/diffoscope/scripts/extract-vmlinux --- old/diffoscope-291/diffoscope/scripts/extract-vmlinux 2025-03-21 15:11:51.000000000 +0100 +++ new/diffoscope-306/diffoscope/scripts/extract-vmlinux 2025-09-08 00:31:34.000000000 +0200 @@ -62,3 +62,5 @@ # Bail out: echo "$me: Cannot find vmlinux." >&2 + +return 0 diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/diffoscope-291/tests/comparators/test_directory.py new/diffoscope-306/tests/comparators/test_directory.py --- old/diffoscope-291/tests/comparators/test_directory.py 2025-03-21 15:11:51.000000000 +0100 +++ new/diffoscope-306/tests/comparators/test_directory.py 2025-09-08 00:31:34.000000000 +0200 @@ -19,12 +19,15 @@ import os import shutil +import time + import pytest from diffoscope.comparators.binary import FilesystemFile from diffoscope.comparators.directory import compare_directories from diffoscope.comparators.utils.compare import compare_root_paths from diffoscope.comparators.utils.specialize import specialize +from diffoscope.config import Config from ..utils.data import data, get_data, assert_diff @@ -125,3 +128,20 @@ assert_diff(compare_root_paths(a, b), "test_directory_a_b_diff") assert_diff(compare_root_paths(b, a), "test_directory_b_a_diff") + + +def test_identical_files_different_mtime(monkeypatch, tmpdir): + + monkeypatch.setattr(Config(), "exclude_directory_metadata", "no") + + file1_path = str(tmpdir.join("file1")) + file2_path = str(tmpdir.join("file2")) + + now = time.time() + for path, t in ((file1_path, now - 3600), (file2_path, now)): + with open(path, "w") as f: + f.write("identical content") + os.utime(path, (t, t)) + + difference = compare_root_paths(file1_path, file2_path) + assert difference is not None diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/diffoscope-291/tests/comparators/test_dtb.py new/diffoscope-306/tests/comparators/test_dtb.py --- old/diffoscope-291/tests/comparators/test_dtb.py 2025-03-21 15:11:51.000000000 +0100 +++ new/diffoscope-306/tests/comparators/test_dtb.py 2025-09-08 00:31:34.000000000 +0200 @@ -40,7 +40,7 @@ ("fdtdump", "--version"), stderr=subprocess.STDOUT ).decode("utf-8") - m = re.search(r"Version: DTC (?P<version>\d+\.\d+\.\d+)", out) + m = re.search(r"Version: DTC v?(?P<version>\d+\.\d+\.\d+)", out) if m is None: raise ValueError("Error parsing `fdtdump --version` output") diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/diffoscope-291/tests/comparators/test_gettext.py new/diffoscope-306/tests/comparators/test_gettext.py --- old/diffoscope-291/tests/comparators/test_gettext.py 2025-03-21 15:11:51.000000000 +0100 +++ new/diffoscope-306/tests/comparators/test_gettext.py 2025-09-08 00:31:34.000000000 +0200 @@ -17,7 +17,6 @@ # You should have received a copy of the GNU General Public License # along with diffoscope. If not, see <https://www.gnu.org/licenses/>. -import codecs import pytest from diffoscope.config import Config @@ -58,7 +57,7 @@ @skip_unless_tools_exist("msgunfmt") def test_charsets(mo_no_charset, mo_iso8859_1): difference = mo_no_charset.compare(mo_iso8859_1) - expected_diff = codecs.open( + expected_diff = open( data("mo_charsets_expected_diff"), encoding="utf-8" ).read() assert difference.details[0].unified_diff == expected_diff diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/diffoscope-291/tests/comparators/test_hdf.py new/diffoscope-306/tests/comparators/test_hdf.py --- old/diffoscope-291/tests/comparators/test_hdf.py 2025-03-21 15:11:51.000000000 +0100 +++ new/diffoscope-306/tests/comparators/test_hdf.py 2025-09-08 00:31:34.000000000 +0200 @@ -1,7 +1,7 @@ # # diffoscope: in-depth comparison of files, archives, and directories # -# Copyright © 2020 Chris Lamb <[email protected]> +# Copyright © 2020, 2025 Chris Lamb <[email protected]> # # diffoscope is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -16,22 +16,19 @@ # You should have received a copy of the GNU General Public License # along with diffoscope. If not, see <https://www.gnu.org/licenses/>. -import re import pytest from diffoscope.comparators.hdf import Hdf5File from diffoscope.comparators.binary import FilesystemFile from diffoscope.comparators.utils.specialize import specialize -from ..utils.data import load_fixture, get_data +from ..utils.data import load_fixture, assert_diff from ..utils.tools import skip_unless_tools_exist, skip_unless_module_exists from ..utils.nonexisting import assert_non_existing hdf5_1 = load_fixture("test1.hdf5") hdf5_2 = load_fixture("test2.hdf5") -re_normalise = re.compile(r'(HDF5 ")[^\"]+/([^\"]+")') - def hdf5_fixture(prefix): @pytest.fixture @@ -41,8 +38,8 @@ # Listed in debian/tests/control.in import h5py - with h5py.File(filename, "w"): - pass + with h5py.File(filename, "w") as f: + f.create_dataset(prefix, (100,), dtype="i") return specialize(FilesystemFile(filename)) @@ -72,12 +69,7 @@ @skip_unless_tools_exist("h5dump") @skip_unless_module_exists("h5py") def test_diff(differences): - expected_diff = get_data("hdf5_expected_diff") - # Remove absolute build path - normalised = re_normalise.sub( - lambda m: m.group(1) + m.group(2), differences[0].unified_diff - ) - assert normalised == expected_diff + assert_diff(differences[0], "hdf5_expected_diff") @skip_unless_tools_exist("h5dump") diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/diffoscope-291/tests/comparators/test_lzma.py new/diffoscope-306/tests/comparators/test_lzma.py --- old/diffoscope-291/tests/comparators/test_lzma.py 1970-01-01 01:00:00.000000000 +0100 +++ new/diffoscope-306/tests/comparators/test_lzma.py 2025-09-08 00:31:34.000000000 +0200 @@ -0,0 +1,75 @@ +# +# diffoscope: in-depth comparison of files, archives, and directories +# +# Copyright © 2015 Jérémy Bobbio <[email protected]> +# Copyright © 2016-2017, 2020, 2024-2025 Chris Lamb <[email protected]> +# +# diffoscope is free software: you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation, either version 3 of the License, or +# (at your option) any later version. +# +# diffoscope is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with diffoscope. If not, see <https://www.gnu.org/licenses/>. + +import shutil +import pytest + +from diffoscope.comparators.lzma import LzmaFile +from diffoscope.comparators.binary import FilesystemFile +from diffoscope.comparators.utils.specialize import specialize + +from ..utils.data import load_fixture, assert_diff +from ..utils.tools import skip_unless_tools_exist +from ..utils.nonexisting import assert_non_existing + +lzma1 = load_fixture("test1.lzma") +lzma2 = load_fixture("test2.lzma") + + +def test_identification(lzma1): + assert isinstance(lzma1, LzmaFile) + + +def test_no_differences(lzma1): + difference = lzma1.compare(lzma1) + assert difference is None + + [email protected] +def differences(lzma1, lzma2): + return lzma1.compare(lzma2).details + + +@skip_unless_tools_exist("xz") +def test_content_source(differences): + assert differences[0].source1 == "test1" + assert differences[0].source2 == "test2" + + +@skip_unless_tools_exist("xz") +def test_content_source_without_extension(tmpdir, lzma1, lzma2): + path1 = str(tmpdir.join("test1")) + path2 = str(tmpdir.join("test2")) + shutil.copy(lzma1.path, path1) + shutil.copy(lzma2.path, path2) + lzma1 = specialize(FilesystemFile(path1)) + lzma2 = specialize(FilesystemFile(path2)) + difference = lzma1.compare(lzma2).details + assert difference[0].source1 == "test1-content" + assert difference[0].source2 == "test2-content" + + +@skip_unless_tools_exist("xz") +def test_content_diff(differences): + assert_diff(differences[0], "text_ascii_expected_diff") + + +@skip_unless_tools_exist("xz") +def test_compare_non_existing(monkeypatch, lzma1): + assert_non_existing(monkeypatch, lzma1) diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/diffoscope-291/tests/comparators/test_pdf.py new/diffoscope-306/tests/comparators/test_pdf.py --- old/diffoscope-291/tests/comparators/test_pdf.py 2025-03-21 15:11:51.000000000 +0100 +++ new/diffoscope-306/tests/comparators/test_pdf.py 2025-09-08 00:31:34.000000000 +0200 @@ -2,7 +2,7 @@ # diffoscope: in-depth comparison of files, archives, and directories # # Copyright © 2015 Jérémy Bobbio <[email protected]> -# Copyright © 2015-2018, 2020-2021, 2023-2024 Chris Lamb <[email protected]> +# Copyright © 2015-2018, 2020-2021, 2023-2025 Chris Lamb <[email protected]> # # diffoscope is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -45,7 +45,7 @@ except (ImportError, RuntimeError): return True - return not pypdf.__version__.startswith("3.") + return not int(pypdf.__version__.split(".")[0]) >= 3 return skipif(fn(), reason="pypdf not installed or not version 3.x+") diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/diffoscope-291/tests/comparators/test_rlib.py new/diffoscope-306/tests/comparators/test_rlib.py --- old/diffoscope-291/tests/comparators/test_rlib.py 2025-03-21 15:11:51.000000000 +0100 +++ new/diffoscope-306/tests/comparators/test_rlib.py 2025-09-08 00:31:34.000000000 +0200 @@ -100,8 +100,8 @@ @skip_unless_tools_exist("nm") @skip_if_binutils_does_not_support_x86() def test_item0_armap(differences): - assert differences[0].source1 == "nm -s {}" - assert differences[0].source2 == "nm -s {}" + assert differences[0].source1 == "nm --print-armap {}" + assert differences[0].source2 == "nm --print-armap {}" expected_diff = get_data("rlib_armap_expected_diff") assert differences[0].unified_diff == expected_diff diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/diffoscope-291/tests/comparators/test_text.py new/diffoscope-306/tests/comparators/test_text.py --- old/diffoscope-291/tests/comparators/test_text.py 2025-03-21 15:11:51.000000000 +0100 +++ new/diffoscope-306/tests/comparators/test_text.py 2025-09-08 00:31:34.000000000 +0200 @@ -17,8 +17,6 @@ # You should have received a copy of the GNU General Public License # along with diffoscope. If not, see <https://www.gnu.org/licenses/>. -import codecs - from diffoscope.comparators.binary import FilesystemFile from diffoscope.comparators.text import TextFile from diffoscope.comparators.utils.specialize import specialize @@ -50,7 +48,7 @@ def test_difference_in_unicode(unicode1, unicode2): difference = unicode1.compare(unicode2) - expected_diff = codecs.open( + expected_diff = open( data("text_unicode_expected_diff"), encoding="utf-8" ).read() assert difference.unified_diff == expected_diff @@ -61,7 +59,7 @@ def test_difference_between_iso88591_and_unicode(iso8859, unicode1): difference = iso8859.compare(unicode1) - expected_diff = codecs.open( + expected_diff = open( data("text_iso8859_expected_diff"), encoding="utf-8" ).read() assert difference.unified_diff == expected_diff @@ -71,7 +69,7 @@ utf8_path = str(tmpdir.join("utf8")) with open(utf8_path, "wb") as f: f.write( - codecs.open(data("text_iso8859"), encoding="iso8859-1") + open(data("text_iso8859"), encoding="iso8859-1") .read() .encode("utf-8") ) diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/diffoscope-291/tests/comparators/test_uki.py new/diffoscope-306/tests/comparators/test_uki.py --- old/diffoscope-291/tests/comparators/test_uki.py 2025-03-21 15:11:51.000000000 +0100 +++ new/diffoscope-306/tests/comparators/test_uki.py 2025-09-08 00:31:34.000000000 +0200 @@ -2,6 +2,7 @@ # diffoscope: in-depth comparison of files, archives, and directories # # Copyright © 2024 Jelle van der Waa <[email protected]> +# Copyright © 2025 Chris Lamb <[email protected]> # # diffoscope is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -17,6 +18,8 @@ # along with diffoscope. If not, see <https://www.gnu.org/licenses/>. import pytest +import re +import subprocess from diffoscope.comparators.binary import FilesystemFile from diffoscope.comparators.uki import UKIFile @@ -24,11 +27,17 @@ from diffoscope.comparators.utils.specialize import specialize from ..utils.data import assert_diff, load_fixture -from ..utils.tools import skip_unless_tools_exist +from ..utils.tools import skip_unless_tools_exist, skip_unless_tool_is_at_least efi_stub = load_fixture("dummyx64.efi.stub") +def ukify_version(): + line = subprocess.check_output(("ukify", "--version")).decode("utf-8") + + return re.search(r"\((.*)\)", line).group(1) + + def uki_fixture(prefix, os_release, uname): @pytest.fixture def uki(tmpdir, efi_stub): @@ -70,6 +79,7 @@ @skip_unless_tools_exist("objdump") @skip_unless_tools_exist("ukify") +@skip_unless_tool_is_at_least("ukify", ukify_version, "258~rc3") def test_no_differences(uki1): difference = uki1.compare(uki1) assert difference is None @@ -77,5 +87,6 @@ @skip_unless_tools_exist("objdump") @skip_unless_tools_exist("ukify") +@skip_unless_tool_is_at_least("ukify", ukify_version, "258~rc3") def test_diff(differences): assert_diff(differences[0], "uki_expected_diff") diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/diffoscope-291/tests/comparators/test_utils.py new/diffoscope-306/tests/comparators/test_utils.py --- old/diffoscope-291/tests/comparators/test_utils.py 2025-03-21 15:11:51.000000000 +0100 +++ new/diffoscope-306/tests/comparators/test_utils.py 2025-09-08 00:31:34.000000000 +0200 @@ -17,7 +17,6 @@ # You should have received a copy of the GNU General Public License # along with diffoscope. If not, see <https://www.gnu.org/licenses/>. -import codecs import os import pytest import threading @@ -72,7 +71,7 @@ @skip_unless_module_exists("tlsh") def test_fuzzy_matching(fuzzy_tar1, fuzzy_tar2): differences = fuzzy_tar1.compare(fuzzy_tar2).details - expected_diff = codecs.open( + expected_diff = open( data("text_iso8859_expected_diff"), encoding="utf-8" ).read() assert differences[1].source1 == "./matching" diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/diffoscope-291/tests/comparators/test_zip.py new/diffoscope-306/tests/comparators/test_zip.py --- old/diffoscope-291/tests/comparators/test_zip.py 2025-03-21 15:11:51.000000000 +0100 +++ new/diffoscope-306/tests/comparators/test_zip.py 2025-09-08 00:31:34.000000000 +0200 @@ -2,7 +2,7 @@ # diffoscope: in-depth comparison of files, archives, and directories # # Copyright © 2015 Jérémy Bobbio <[email protected]> -# Copyright © 2015-2020, 2022, 2024 Chris Lamb <[email protected]> +# Copyright © 2015-2020, 2022, 2024-2025 Chris Lamb <[email protected]> # # diffoscope is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -20,7 +20,13 @@ import pytest import subprocess -from diffoscope.comparators.zip import ZipFile, MozillaZipFile, JmodJavaModule +from diffoscope.comparators.zip import ( + ZipFile, + MozillaZipFile, + NuGetPackageFile, + JmodJavaModule, + zipdetails_version, +) from ..utils.data import load_fixture, assert_diff from ..utils.tools import skip_unless_tools_exist, skip_unless_tool_is_at_least @@ -34,16 +40,14 @@ encrypted_zip2 = load_fixture("encrypted2.zip") mozzip1 = load_fixture("test1.mozzip") mozzip2 = load_fixture("test2.mozzip") +nupkg1 = load_fixture("test1.nupkg") +nupkg2 = load_fixture("test2.nupkg") jmod1 = load_fixture("test1.jmod") jmod2 = load_fixture("test2.jmod") test_comment1 = load_fixture("test_comment1.zip") test_comment2 = load_fixture("test_comment2.zip") -def zipdetails_version(): - return subprocess.check_output(["zipdetails", "--version"]).decode("UTF-8") - - def io_compress_zip_version(): try: return subprocess.check_output( @@ -135,6 +139,42 @@ assert_non_existing(monkeypatch, mozzip1) +def test_nupkg_identification(nupkg1): + assert isinstance(nupkg1, NuGetPackageFile) + + +def test_nupkg_no_differences(nupkg1): + difference = nupkg1.compare(nupkg1) + assert difference is None + + [email protected] +def nupkg_differences(nupkg1, nupkg2): + return nupkg1.compare(nupkg2).details + + +@skip_unless_tools_exist("zipinfo") +def test_nupkg_metadata(nupkg_differences, nupkg1, nupkg2): + assert_diff(nupkg_differences[0], "nupkg_expected_diff") + + +@skip_unless_tools_exist("zipinfo") +def test_nupkg_compressed_files(nupkg_differences): + assert ( + nupkg_differences[-1].source1 + == "package/services/metadata/core-properties/b44ebb537bbf4983b9527f9e3820fda6.psmdcp" + ) + assert ( + nupkg_differences[-1].source2 + == "package/services/metadata/core-properties/08f1f9d8789a4668a128f78560bd0107.psmdcp" + ) + + +@skip_unless_tools_exist("zipinfo") +def test_nupkg_compare_non_existing(monkeypatch, nupkg1): + assert_non_existing(monkeypatch, nupkg1) + + def test_jmod_identification(jmod1): assert isinstance(jmod1, JmodJavaModule) diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/diffoscope-291/tests/data/hdf5_expected_diff new/diffoscope-306/tests/data/hdf5_expected_diff --- old/diffoscope-291/tests/data/hdf5_expected_diff 2025-03-21 15:11:51.000000000 +0100 +++ new/diffoscope-306/tests/data/hdf5_expected_diff 2025-09-08 00:31:34.000000000 +0200 @@ -1,6 +1,12 @@ -@@ -1,4 +1,4 @@ --HDF5 "test1.db" { -+HDF5 "test2.db" { +@@ -1,10 +1,10 @@ + HDF5 { GROUP "/" { - } - } +- DATASET "test1" { ++ DATASET "test2" { + DATATYPE H5T_STD_I32LE + DATASPACE SIMPLE { ( 100 ) / ( 100 ) } + DATA { + (0): 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, + (22): 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, + (44): 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, + (66): 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/diffoscope-291/tests/data/nupkg_expected_diff new/diffoscope-306/tests/data/nupkg_expected_diff --- old/diffoscope-291/tests/data/nupkg_expected_diff 1970-01-01 01:00:00.000000000 +0100 +++ new/diffoscope-306/tests/data/nupkg_expected_diff 2025-09-08 00:31:34.000000000 +0200 @@ -0,0 +1,11 @@ +@@ -1,7 +1,7 @@ +-Zip file size: 3163 bytes, number of entries: 5 ++Zip file size: 3162 bytes, number of entries: 5 + -rw-r--r-- 2.0 unx 502 b- defN 25-Apr-14 18:14 _rels/.rels + -rw-r--r-- 2.0 unx 407 b- defN 25-Apr-14 18:14 ClassLibrary.nuspec + -rw-r--r-- 2.0 unx 3584 b- defN 25-Apr-14 22:14 lib/net8.0/ClassLibrary.dll + -rw-r--r-- 2.0 unx 459 b- defN 25-Apr-14 18:14 [Content_Types].xml +--rw-r--r-- 2.0 unx 625 b- defN 25-Apr-14 18:14 package/services/metadata/core-properties/b44ebb537bbf4983b9527f9e3820fda6.psmdcp +-5 files, 5577 bytes uncompressed, 2447 bytes compressed: 56.1% ++-rw-r--r-- 2.0 unx 625 b- defN 25-Apr-14 18:14 package/services/metadata/core-properties/08f1f9d8789a4668a128f78560bd0107.psmdcp ++5 files, 5577 bytes uncompressed, 2446 bytes compressed: 56.1% diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/diffoscope-291/tests/data/rpm_header_expected_diff new/diffoscope-306/tests/data/rpm_header_expected_diff --- old/diffoscope-291/tests/data/rpm_header_expected_diff 2025-03-21 15:11:51.000000000 +0100 +++ new/diffoscope-306/tests/data/rpm_header_expected_diff 2025-09-08 00:31:34.000000000 +0200 @@ -1,23 +1,5 @@ -@@ -10,62 +10,62 @@ - 00dc000000020000041a00000008000001170000000200000428000000060000012500000001000004470000000400000130 - 00000001000004480000000400000134000000010000044900000008000001380000000100000458000000040000013c0000 - 0002000004590000000800000144000000020000045c000000040000014c000000010000045d000000080000015000000001 - 0000045e00000008000001550000000100000462000000060000015b00000001000004640000000600000162000000010000 - 046500000006000001670000000100000466000000060000016c000000010000046c000000060000016e0000000100000474 - 0000000400000184000000010000047500000004000001880000000100000476000000080000018c00000001000004770000 - 0004000001980000000100000478000000040000019c00000001430074657374003000300054657374207061636b61676520 --666f722064656262696e646966660054657374207061636b61676500558aef0b6c6f617200000000000001be5075626c6963 --004170706c69636174696f6e732f53797374656d006c696e7578007838365f363400000001be81a40000558aef0637383733 --623630363432396137303238346436336630363734653637383762630000000000000000726f6f7400726f6f740074657374 -+666f722064656262696e646966660054657374207061636b61676500558aef316c6f6172000000000000029f5075626c6963 -+004170706c69636174696f6e732f53797374656d006c696e7578007838365f3634000000029f81a40000558aef2f31373836 -+363236326130633630643931366533636633323139373234623265370000000000000000726f6f7400726f6f740074657374 - 2d302d302e7372632e72706d0000ffffffff746573740074657374287838362d3634290000000100000a0100000a72706d6c - 696228436f6d7072657373656446696c654e616d6573290072706d6c6962285061796c6f616446696c657348617665507265 - 6669782900332e302e342d3100342e302d3100342e31322e302e310000000000000100000001000000000000000800000008 - 302d3000302d30000000000074657874002f6469722f002d4f32202d67006370696f00677a69700039007838365f36342d64 - 656269616e2d6c696e7578000000000000000000000041534349492074657874000000000000000000000000003f00000007 - fffffd0000000010 +@@ -1,47 +1,47 @@ + HEADERIMMUTABLE: [48 indexes, 432 bytes] HEADERI18NTABLE: - C -SIGSIZE: 1583 Binary files old/diffoscope-291/tests/data/test1.lzma and new/diffoscope-306/tests/data/test1.lzma differ Binary files old/diffoscope-291/tests/data/test1.nupkg and new/diffoscope-306/tests/data/test1.nupkg differ Binary files old/diffoscope-291/tests/data/test2.lzma and new/diffoscope-306/tests/data/test2.lzma differ Binary files old/diffoscope-291/tests/data/test2.nupkg and new/diffoscope-306/tests/data/test2.nupkg differ Binary files old/diffoscope-291/tests/data/uki_expected_diff and new/diffoscope-306/tests/data/uki_expected_diff differ diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/diffoscope-291/tests/test_versions.py new/diffoscope-306/tests/test_versions.py --- old/diffoscope-291/tests/test_versions.py 2025-03-21 15:11:51.000000000 +0100 +++ new/diffoscope-306/tests/test_versions.py 2025-09-08 00:31:34.000000000 +0200 @@ -2,7 +2,7 @@ # diffoscope: in-depth comparison of files, archives, and directories # # Copyright © 2021 Zbigniew Jędrzejewski-Szmek <[email protected]> -# Copyright © 2021-2023 Chris Lamb <[email protected]> +# Copyright © 2021-2023, 2025 Chris Lamb <[email protected]> # # diffoscope is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -151,6 +151,9 @@ ("0^", "0", +1), ("0^", "0^", 0), ("0^", "0^~", 1), + # OpenSSH (#1102658) + ("10.0p1", "9.7p1", 1), + ("9.7p1", "10.0p1", -1), ] @@ -172,6 +175,6 @@ @pytest.mark.parametrize("a,b", [c[:2] for c in cases if c[2] == 0]) -def test_version_gt(a, b): +def test_version_eq(a, b): assert Version(a) == b assert Version(a) == Version(b) diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/diffoscope-291/tests/utils/versions.py new/diffoscope-306/tests/utils/versions.py --- old/diffoscope-291/tests/utils/versions.py 2025-03-21 15:11:51.000000000 +0100 +++ new/diffoscope-306/tests/utils/versions.py 2025-09-08 00:31:34.000000000 +0200 @@ -2,7 +2,7 @@ # diffoscope: in-depth comparison of files, archives, and directories # # Copyright © 2021 Zbigniew Jędrzejewski-Szmek <[email protected]> -# Copyright © 2021-2023 Chris Lamb <[email protected]> +# Copyright © 2021-2023, 2025 Chris Lamb <[email protected]> # # diffoscope is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -101,3 +101,6 @@ def __lt__(self, other): return self._cmp(other) < 0 + + def __gt__(self, other): + return self._cmp(other) > 0
