Bug#774415: From srebuild sbuild-wrapper to debrebuild
Johannes Schauer: > Hi, > > Quoting Niko Tyni (2016-12-17 13:40:32) >> On Thu, Dec 15, 2016 at 02:21:49PM +0100, Johannes Schauer wrote: >>> Quoting Niko Tyni (2016-12-15 14:04:19) >>>> In general, I like the concept of sbuild/pbuilder accepting .buildinfo >>>> files >>>> as input. This makes the user interface simple. My expectation for this >>>> mode >>>> of operation would be for the builder to recreate the old build as closely >>>> as >>>> possible. >>> >>> I agree but would add that they should also have the ability to tell the >>> user >>> if the checksums of the re-compiled packages do or do not match the >>> information >>> in the supplied .buildinfo file. >> >> I suppose; please just make sure such a failure is easily distinguishable >> from a failing build. > > My plan would be to add it as a success/failure line next to the lintian or > autopkgtest status at the bottom of the build log. > >>> I don't care whether we have debrebuild as a wrapper to sbuild/pbuilder or >>> sbuild/pbuilder use a common tool to figure out the right lines for the >>> sources.list inside the chroot. I just want to have .buildinfo support for >>> sbuild in Stretch and if either solution is not in unstable soon, then my >>> plan is to just add .buildinfo support to sbuild myself so that it's still >>> included in the next Debian stable release. >> >> Having this just inside sbuild for stretch and refactoring it out later >> if necessary works for me, but I'm not sure if HW42 and/or Mattia have >> thoughts about the pbuilder side? > > Putting them back in CC. > > I am especially waiting for a response from HW42 who volunteered to "keep an > eye on it" but who didn't come back to my pings on IRC yet. > > HW42: I need to know what your plan is for Stretch so that I can decide what > to > include in the next sbuild release. Sorry about the late reply. I didn't had any plans sofar for stretch. Given that a) the .buildinfo format it self b) the services around .buildinfo c) the interface of debrebuild (or buildinfo-utils, or whatever) is not really clear/finished yet I would expect that one need anyway the backports version. If you think otherwise we can of course push to get the current version into stretch. >> I note that we're only getting started on working with .buildinfo files. It >> seems possible that we encounter enough common tasks that something like a >> 'buildinfo-utils' package will be warranted, in which case a 'buildinfo >> find-debs' command would fit in there. > > I'm all in for breaking out common functionality into tools used by multiple > parties. So you (at least josch and ntyi) seem to prefer to have the user facing part in sbuild/pbuilder and the common functionality in some kind of library. How should the "library" interface look like for sbuild? signature.asc Description: OpenPGP digital signature
Bug#847595: diffoscope: test_icc.py test failures
Ximin Luo: > Hello liblcms2-2 maintainer, just reassigning the bug described below. > You can reproduce it with $ cd-iccdump > > e.g. from the diffoscope source tree: > > $ cd-iccdump tests/data/test1.icc | grep 'en_US\|ne_SU' > ne_SU: sRGB [24 bytes] > [.. etc ..] > > Seems endian-related. > > Reiner Herrmann: >> On Fri, Dec 09, 2016 at 06:36:53PM +0100, Chris Lamb wrote: >>> E -ne_SU: sRGB [24 bytes] >>> E ? - - >>> E +en_US: sRGB [24 bytes] >>> E ?+ + >> >> Just found out that it is caused by liblcms2-2 (2.8-2). >> After downgrading it to the version in stretch (2.7-1), everything is >> printed normally. I think I found the cause. cmsMLUtranslationsCodes() now uses the new strFrom16() function. The problem is that strFrom16() does not work under little-endian systems. static cmsUInt16Number strTo16(const char str[3]) { cmsUInt16Number n = ((cmsUInt16Number) str[0] << 8) | str[1]; return n; // Always big endian in this case } static void strFrom16(char str[3], cmsUInt16Number n) { // Assiming this would be aligned union { cmsUInt16Number n; char str[2]; } c; c.n = n; // Always big endian in this case str[0] = c.str[0]; str[1] = c.str[1]; str[2] = 0; } On a little-endian system strFrom16() wrongly swaps the byte order (even though the comment says something different). You can easily test this with the attached minimal test case (see test.c). I think the easiest solution is the use a machine byte order independent calculation like in strTo16(). See attached patch. diff -u a/src/cmsnamed.c b/src/cmsnamed.c --- a/src/cmsnamed.c +++ b/src/cmsnamed.c @@ -192,18 +192,10 @@ static void strFrom16(char str[3], cmsUInt16Number n) { -// Assiming this would be aligned -union { - - cmsUInt16Number n; - char str[2]; - -} c; - -c.n = n; // Always big endian in this case - -str[0] = c.str[0]; str[1] = c.str[1]; str[2] = 0; - +// n is always big endian, see strTo16. +str[0] = (n >> 8) & 0xff; +str[1] = n & 0xff; +str[2] = 0; } // Add an ASCII entry. Do not add any \0 termination (ICC1v43_2010-12.pdf page 61) #include #include int main() { char s[] = "ab"; union { uint16_t n; char s[2]; } u; u.n = s[0] << 8 | s[1]; printf("%c%c\n", u.s[0], u.s[1]); } signature.asc Description: OpenPGP digital signature
Bug#774415: From srebuild sbuild-wrapper to debrebuild
Johannes Schauer: > Hi, > > On Thu, 10 Nov 2016 05:54:13 -0200 Johannes Schauer <jo...@debian.org> wrote: >> On Tue, 02 Aug 2016 22:49:00 +0200 Johannes Schauer <jo...@debian.org> wrote: >>> But then on IRC, HW42 suggested to approach this problem differently. >>> Instead of integrating the functionality of figuring out the right >>> repositories to reproduce the contents of a buildinfo file into sbuild, >>> write a tool that can drive any package builder (like pbuilder). > > there seems to be a conceptual problem with such an approach. > > For binNMUs, the full changelog entry has to be passed to sbuild or pbuilder. > How does one best pass such a multi-line value via command line options? Would > the best way to pass the changelog entry via the .buildinfo file? And if > pbuilder and sbuild then already are parsing the .buildinfo file, would it not > be better for the debrebuild machinery to be implemented by either in the > first > place? Since this is somewhat relevant to the discussion in the other part of this thread: I don't think this is a conceptual problem. Sure it could be nicer if we don't had binNMUs, but I see no real problem in passing it via cmd line option or via a plain file. I would anyway modify debrebuild to be able to call sbuild/pbuiler/etc. directly and then you are able to use a tempfile cleanly. signature.asc Description: OpenPGP digital signature
Bug#774415: From srebuild sbuild-wrapper to debrebuild
Johannes Schauer: > Hi all, > > On Tue, 02 Aug 2016 22:49:00 +0200 Johannes Schauer <jo...@debian.org> wrote: >> I was thinking about this issue again and thought that instead of creating a >> wrapper for sbuild which then uses a chroot-setup hook to install the >> dependencies, what I should instead do is to let sbuild itself accept >> .buildinfo files and then do the right thing like: >> >> - use snapshot.d.o to retrieve the right timestamps needed to gather all >>packages >> - mangle the build dependencies such that the source package now depends on >>the exact right package versions and let the resolver figure out the rest >>(thanks Benjamin for that idea) >> - check whether the generated binaries produce the same checksum as given in >>the supplied buildinfo file >> >> But then on IRC, HW42 suggested to approach this problem differently. Instead >> of integrating the functionality of figuring out the right repositories to >> reproduce the contents of a buildinfo file into sbuild, write a tool that can >> drive any package builder (like pbuilder). >> >> I now wrote such a script. > > now that libdpkg-perl comes with support for .buildinfo files, I improved the > script (new version attached) with the following changes: > > - don't use DateTime::Format::Strptime but Time::Piece instead (which is a >perl core module) > - don't use CTRL_INDEX_SRC but CTRL_FILE_BUILDINFO now that dpkg supports >.buildinfo files > - Dpkg::Compression::FileHandle as it is not needed > - the .dsc file name is no longer part of the .buildinfo file, so assemble > the >.dsc file name from the package name and version using > Dpkg::Source::Package > - use the information from the Environment field > - instead of splitting Installed-Build-Depends manually, use >Dpkg::Deps::deps_parse > - instead of using [trusted=yes], retrieve the gpg key of the reproducible >builds repository and verify its fingerprint > - set Binary::apt-get::Acquire::AllowInsecureRepositories to false so that >apt-get fails to update repositories it cannot authenticate > - use Dpkg::Vendor to retrieve the keyring filenames > > Thanks to Guillem Jover for the code review! After discussing this in the irc meeting yesterday I propose that: - we keep it as a separate tool. - put it in a git repo under https://anonscm.debian.org/git/reproducible/ - We have more than enough DDs who are willing to sponsor uploads, so having it in the Debian archive is no problem. - we mainly maintain this as a group. I will try to especially keep an eye on it. Since you have done all the work so far the final decision is obviously up to you. If the above is fine with you I will prepare packaging it during the next week (I also have a few improvements planed). signature.asc Description: OpenPGP digital signature
Bug#774415: From srebuild sbuild-wrapper to debrebuild
Johannes Schauer: > Hi, > > On Thu, 10 Nov 2016 05:54:13 -0200 Johannes Schauer <jo...@debian.org> wrote: >> On Tue, 02 Aug 2016 22:49:00 +0200 Johannes Schauer <jo...@debian.org> wrote: >>> But then on IRC, HW42 suggested to approach this problem differently. >>> Instead of integrating the functionality of figuring out the right >>> repositories to reproduce the contents of a buildinfo file into sbuild, >>> write a tool that can drive any package builder (like pbuilder). > > there seems to be a conceptual problem with such an approach. > > For binNMUs, the full changelog entry has to be passed to sbuild or pbuilder. > How does one best pass such a multi-line value via command line options? What's your problem with passing multi-line value via command line options? > Would the best way to pass the changelog entry via the .buildinfo > file? Not sure about that. If you dislike passing the value via a command line option, just use a plain file? > And if pbuilder and sbuild then already are parsing the .buildinfo > file, would it not be better for the debrebuild machinery to be > implemented by either in the first place? My point for an independent debrebuild was that a) Every builder needs nearly the same functionaly for this. b) It's security relevant since it parses semi-trusted (the .buildinfo) and untrusted (http response from snapshot.d.o) data. So I still think that having this separate of the builder is useful. If sbuild, pbuilder, etc. coordinate this, some kind of library might also be an option. signature.asc Description: OpenPGP digital signature
Bug#841146: diffoscope: Failure of test_superblock
Leo Famulari: > On Fri, Oct 21, 2016 at 04:23:51PM +, Mattia Rizzolo wrote: >> Though I'm using pytest 3.0.3. >> That test is skipped by using pytest.mark.skip(), which I don't see in >> the docs of pytest for 2.7. >> The changelog of pytest tells me pytest.mark.skip() is recognized as a >> skipping marker starting from 2.9¹. Is there any chance you can instead >> upgrade pytest in your distribution? > > We are working on upgrading the core Python packages like pytest and > Setuptools but we can't do it overnight. > >> If so I'll add a versioned dependency on setup.py, otherwies I can >> always turn that pytest.mark.skip() into a pytest.mark.skipif(True), >> which is IMHO ugly but quick and effective for solving this bug, I >> think. Can you also try to convert that marking in >> tests/comparators/utils.py:49 to confirm? > > I tried making the following change: > > diff --git a/tests/comparators/utils.py b/tests/comparators/utils.py > index f8f6399..acbdc9f 100644 > --- a/tests/comparators/utils.py > +++ b/tests/comparators/utils.py > @@ -46,7 +46,7 @@ def skip_unless_tools_exist(*required): > > def skip_unless_tool_is_older_than(tool, actual_ver, min_ver, > vcls=LooseVersion): > if tools_missing(tool): > -return pytest.mark.skip(reason="requires {}".format(tool)) > +return pytest.mark.skipif(True)) That's one closing paren to much. > if callable(actual_ver): > actual_ver = actual_ver() > return pytest.mark.skipif( > > But, that creates a bunch of invalid syntax. Here's one: [...] signature.asc Description: OpenPGP digital signature
Bug#841029: libfreetype6 wrongly detects (some) recursive glyphs as broken
Package: libfreetype6 Version: 2.6.3-3 Tags: upstream, fixed-upstream, patch Hi, FreeType wrongly detects a glyphs with multiple references to the same glyph, which has itself references, as broken. Somebody already reported it upstream [0] and it's already fixed upstream [1]. This affects for example DejaVu fonts: $ ftlint 10 /usr/share/fonts/truetype/dejavu/DejaVuSans.ttf /usr/share/fonts/truetype/dejavu/DejaVuSans.ttf: glyph 752: 0x0015 glyph 4703: 0x0015 glyph 4704: 0x0015 glyph 4731: 0x0015 glyph 4732: 0x0015 glyph 4753: 0x0015 glyph 4754: 0x0015 glyph 4759: 0x0015 glyph 4760: 0x0015 9 fails. $ As a consequence Scribus refuses to load DejaVu fonts at all and LibreOffice can't display U+033F. The upstream patch applies without problems (ignoring the changelog) and fixes the problem. Upgrading to an newer upstream release should of course also fix the problem. Thanks, HW42 [0]: https://lists.gnu.org/archive/html/freetype/2016-05/msg0.html [1]: http://git.savannah.gnu.org/cgit/freetype/freetype2.git/commit?id=a7d8bdbcfeb65b4859fe553df9d4922627837888 signature.asc Description: OpenPGP digital signature
Bug#839538: diffoscope: json: detect order-only differences
Daniel Shahaf: > It would be better to report "json files are equal up to order of > elements in an object (= hash, dictionary, associative array)", and to > print the difference in a more readable way than a hex dump. (For > example, a linewise diff of pretty-printed json.) While at it, it should also be considered that the formating can be different without changing the order. signature.asc Description: OpenPGP digital signature
Bug#837681: debug output (Re: diffoscope: 'ERROR guestfs can't be launched' when trying to compare to qubes ISOs)
Holger Levsen: > On Thu, Sep 15, 2016 at 04:38:00PM +0000, HW42 wrote: >>> 3.16.0-4-amd64 is the kernel (so jessie standard), and the host this is >>> running on is a profitbricks VM, so running on kvm. >> I assume you have this kernel only installed on the VM not the chroot. >> So try installing linux-image-amd64 _inside_ the chroot. > > the problem is, that this is a sid chroot, so linux-image would have > version 4.7.2, while the host has 3.16… I don't think this mismatch is a problem. signature.asc Description: OpenPGP digital signature
Bug#837681: debug output (Re: diffoscope: 'ERROR guestfs can't be launched' when trying to compare to qubes ISOs)
Holger Levsen: > On Thu, Sep 15, 2016 at 02:58:00PM +0000, HW42 wrote: >> It fails to find a kernel for the VM image it creates on the fly: > > ah, wow. > >> diffoscope-qubes-debug: >>> supermin: failed to find a suitable kernel (host_cpu=x86_64). >>> >>> I looked for kernels in /boot and modules in /lib/modules. >>> >>> If this is a Xen guest, and you only have Xen domU kernels >>> installed, try installing a fullvirt kernel (only for >>> supermin use, you shouldn't boot the Xen guest with it). >>> libguestfs: trace: launch = -1 (error) >>> 1473930826.096060ERROR guestfs can't be launched >> >> On what kind of VM is this run, and what kernels are installed? > > 3.16.0-4-amd64 is the kernel (so jessie standard), and the host this is > running on is a profitbricks VM, so running on kvm. I assume you have this kernel only installed on the VM not the chroot. So try installing linux-image-amd64 _inside_ the chroot. signature.asc Description: OpenPGP digital signature
Bug#837681: debug output (Re: diffoscope: 'ERROR guestfs can't be launched' when trying to compare to qubes ISOs)
Holger Levsen: > On Tue, Sep 13, 2016 at 03:35:58PM +0200, Holger Levsen wrote: >> so I build an Qubes ISO, twice and ran diffoscope against it: > > just now I enabled debugging like this: > >> To see full error messages you may need to enable debugging. >> Do: >> export LIBGUESTFS_DEBUG=1 LIBGUESTFS_TRACE=1 > > the result is attached. It fails to find a kernel for the VM image it creates on the fly: diffoscope-qubes-debug: > supermin: failed to find a suitable kernel (host_cpu=x86_64). > > I looked for kernels in /boot and modules in /lib/modules. > > If this is a Xen guest, and you only have Xen domU kernels > installed, try installing a fullvirt kernel (only for > supermin use, you shouldn't boot the Xen guest with it). > libguestfs: trace: launch = -1 (error) > 1473930826.096060ERROR guestfs can't be launched On what kind of VM is this run, and what kernels are installed? signature.asc Description: OpenPGP digital signature
Bug#837681: diffoscope: 'ERROR guestfs can't be launched' when trying to compare to qubes ISOs
Since guestfs works by running a modified kernel in an VM to parse the file system, I think it fails to start the VM (nested virt disabled, OOM, ...). So I think you should first try if guestfs works at all (without diffoscope) and/or enable the debug loggin like suggested in the error message. signature.asc Description: OpenPGP digital signature
Bug#836732: strip-nd: FTBFS when locale is not English
Chris Lamb: > tag 836732 + pending > thanks > >> Not sure how much related they are, but I guess it means something given >> that it's not in the English build. > > I don't really understand why that would be causing an error but we follow > your assumption that the "return" that is causing the FTBFS, then it was > removed and thus fixed in Git already: I think this has nothing to do with locale. The problem is that dash and bash are treating 'return' differently. $ bash -c 'true; return $?' && echo ok bash: line 0: return: can only `return' from a function or sourced script $ dash -c 'true; return $?' && echo ok ok $ So depending on what is used as /bin/sh the build fails or not. > > https://anonscm.debian.org/git/reproducible/strip-nondeterminism.git/commit/?id=72d70fcd7a088ca0f97d4a9f67e2e42c0c1505ff If it's not needed at all that's nice. Else you could replace 'return' with 'exit'. signature.asc Description: OpenPGP digital signature
Bug#827155: dpkg-buildflags: reproducible/fixdebugpath doesn't escape build path
Package: dpkg-dev Version: 1.18.7 User: reproducible-bui...@lists.alioth.debian.org Usertags: buildinfo X-Debbugs-Cc: reproducible-bui...@lists.alioth.debian.org, mat...@mapreri.org Hi, as Mattia noticed dpkg-buildflags doesn't escape the build path in the -fdebug-prefix-map CC argument when enabling the 'fixdebugpath' option. What assumptions does dpkg make about the build path? I think there are a lot of build scripts which anyway break if the build path contains a space. Unfortunately this isn't trivially fixed since the flags are concatenated as a string and later exported in different languages (shell, Makefile). So if we want to support spaces in buildpaths this need to be changed to an array. What do you think? HW42 signature.asc Description: OpenPGP digital signature
Bug#825643: [Reproducible-builds] Bug#825643: dash: please make the build reproducible
Hi, in addition to the sorting problem found by Reiner dash is also not reproducible since it indirectly captures the shell used as /bin/sh. The problem is that that dash sets argv[0] to the relative path if 'exec' is called with a relative path and bash sets argv[0] to the absolute path. When executing 'exec configure' in debian/rules argv[0] gets captured by the configure script as $srcdir which is later used in $DEFAULT_INCLUDES and passed to gcc. This leads to different build-ids depending on the choice of /bin/sh. The attached patch fix this by simply avoiding the exec call. Thanks, HW42 diff -urN a/debian/rules b/debian/rules --- a/debian/rules 2016-06-13 01:27:12.0 +0200 +++ b/debian/rules 2016-06-13 01:20:28.447939088 +0200 @@ -46,7 +46,7 @@ touch configure (cd build-tmp && CC='$(CC)' \ CFLAGS='$(CFLAGS)' CPPFLAGS='$(CPPFLAGS)' LDFLAGS='$(LDFLAGS)' \ - exec ../configure --enable-fnmatch --disable-lineno \ + ../configure --enable-fnmatch --disable-lineno \ --host='$(DEB_HOST_GNU_TYPE)') touch configure-stamp signature.asc Description: OpenPGP digital signature
Bug#827141: winbind: regression in 4.2.10: extremely slow user/group name lookup
Package: winbind Version: 2:4.2.10+dfsg-0+deb8u3 Severity: important Tags: patch Hi, with the update to the 4.2.10 based samba a regression has been introduced. Resolving of user/group names is extremely slow. An 'ls -l' which normally takes significantly less than a second now takes a minute or more. Filling as 'important' since this degenerates the usability considerably. I could identify the problem with a git-bisect. It's upstream bug #11852 [0]. And has been fixed with commit aec25b0 [1]. The upstream patch applies cleanly on top of 2:4.2.10+dfsg-0+deb8u3 (see attached patch). I have tested it and it resolves the problem in my use case. Thanks. [0]: https://bugzilla.samba.org/show_bug.cgi?id=11852 [1]: https://git.samba.org/?p=samba.git;a=commitdiff;h=aec25b0cc232286c3e4d85de7f00483c09f7c66e From aec25b0cc232286c3e4d85de7f00483c09f7c66e Mon Sep 17 00:00:00 2001 From: Uri SimchoniDate: Mon, 18 Apr 2016 23:08:38 +0300 Subject: [PATCH] libads: record session expiry for spnego sasl binds With the move to gensec-based spnego, record the session expiry in tgs_expire, so that libads users such as winbindd can use this info to determine how long to keep the connection. BUG: https://bugzilla.samba.org/show_bug.cgi?id=11852 Signed-off-by: Uri Simchoni Reviewed-by: Andrew Bartlett Autobuild-User(master): Uri Simchoni Autobuild-Date(master): Tue Apr 19 16:53:57 CEST 2016 on sn-devel-144 (cherry picked from commit 34482eb7cc3d74c8de510309332e8ab176d0f3c0) Autobuild-User(v4-2-test): Karolin Seeger Autobuild-Date(v4-2-test): Tue Apr 26 12:00:48 CEST 2016 on sn-devel-104 --- source3/libads/sasl.c | 9 + 1 file changed, 9 insertions(+) diff --git a/source3/libads/sasl.c b/source3/libads/sasl.c index 22aa9cf..b8d4527 100644 --- a/source3/libads/sasl.c +++ b/source3/libads/sasl.c @@ -134,6 +134,7 @@ static ADS_STATUS ads_sasl_spnego_gensec_bind(ADS_STRUCT *ads, struct auth_generic_state *auth_generic_state; bool use_spnego_principal = lp_client_use_spnego_principal(); const char *sasl_list[] = { sasl, NULL }; + NTTIME end_nt_time; nt_status = auth_generic_client_prepare(NULL, _generic_state); if (!NT_STATUS_IS_OK(nt_status)) { @@ -307,6 +308,14 @@ static ADS_STATUS ads_sasl_spnego_gensec_bind(ADS_STRUCT *ads, } } + ads->auth.tgs_expire = LONG_MAX; + end_nt_time = gensec_expire_time(auth_generic_state->gensec_security); + if (end_nt_time != GENSEC_EXPIRE_TIME_INFINITY) { + struct timeval tv; + nttime_to_timeval(, end_nt_time); + ads->auth.tgs_expire = tv.tv_sec; + } + if (ads->ldap.wrap_type > ADS_SASLWRAP_TYPE_PLAIN) { size_t max_wrapped = gensec_max_wrapped_size(auth_generic_state->gensec_security); ads->ldap.out.max_unwrapped = gensec_max_input_size(auth_generic_state->gensec_security); -- 1.9.1 signature.asc Description: OpenPGP digital signature
Bug#802294: apt: please add option to allow (automatic) downgrades
Package: apt Severity: wishlist AFAIK currently the only option to automatically (i.e. no user interaction) downgrade a package is to pass --force-yes to apt. But this has the undesired side effect of allowing unauthenticated packages. So it would be nice to have a separate options to allow downgrades. signature.asc Description: OpenPGP digital signature
Bug#774415: sbuild: please add the srebuild sbuild wrapper to reproduce builds
Hi, I just noted that the current implementation of srebuild [0] calls apt-get install with --force-yes which (as far as I remember) ignores signature verification errors. HW42 [0]: https://anonscm.debian.org/cgit/reproducible/sbuild.git/tree/bin/srebuild-hook?h=pu/reproducible_builds#n110 signature.asc Description: OpenPGP digital signature