Bug#849638: diffoscope 66 does binary diff on APK files

2016-12-29 Thread Hans-Christoph Steiner
Package: diffoscope Version: 66 Severity: important When running diffoscope on two APKs using version 66, it now just does a straight binary comparison of the direct file itself. Running diffoscope 64 generated a nice output of the individual files in the ZIP (an APK is a signed JAR with some ot

Bug#849638: downgrading back to 64 gives me the same

2016-12-29 Thread Hans-Christoph Steiner
So it seems that the issue is not in diffoscope per se, since now downgrading back to 64 from snapshot.debian.org generates the same output. I'm guessing then this is related to interactions with the dependencies, since I also did an `apt upgrade` at the same time. This is on a machine running st

Bug#849638: diffoscope 66 does binary diff on APK files

2016-12-29 Thread Hans-Christoph Steiner
Reiner Herrmann: > On Thu, Dec 29, 2016 at 12:41:16PM +0100, Hans-Christoph Steiner wrote: >> When running diffoscope on two APKs using version 66, it now just does a >> straight binary comparison of the direct file itself. Running >> diffoscope 64 generated a nice out

Bug#849403: androguard

2016-12-30 Thread Hans-Christoph Steiner
androguard can extract and convert the binary AndroidManifest.xml, its python2 and already in Debian. ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-b

Bug#849638: related bug

2017-01-03 Thread Hans-Christoph Steiner
FYI, I filed https://bugs.debian.org/849782 about APKs being inconsistently detected. ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds

Bug#850501: rerunning using diffoscope 67-21-gfe7ae15

2017-01-09 Thread Hans-Christoph Steiner
Thanks for your work on the APK diffing! I had to fix a typo to get it running that was introduced in diffoscope commit fe7ae15e1c177866acd478af4cc4a51bd5002017 at the bottom of it. It turned 'f_out' into a non-existent 'w'. With that change, diffoscope is now working for me again. I'm running

Bug#851147: --max-report-size does not apply to --text reports

2017-01-12 Thread Hans-Christoph Steiner
Package: diffoscope Version: 67 On https://verification.fdroid.org, diffoscope is run like this: diffoscope --max-report-size 12345678 --max-diff-block-lines 100 \ --html foo.html --text bar.txt The HTML reports are being size-limited, but there are still some giant text reports, including a

Bug#868486: diffoscope often fails to detect APKs

2017-07-15 Thread Hans-Christoph Steiner
Package: diffoscope Version: 83 APKs are basically a ZIP file with a JAR signature, but not necessarily the CAFEBABE byte sequence that marks a JAR. This means that comparing APKs with diffoscope often results in a straight binary diff, which is useless. Here's one example: https://verification

Bug#868486: updated URL

2017-07-17 Thread Hans-Christoph Steiner
I had to move this APK to here: https://verification.f-droid.org/logs/Zom-15.1.0-alpha-5-zomrelease-release-unsigned.apk ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/lis

Bug#868486: diffoscope often fails to detect APKs

2017-07-24 Thread Hans-Christoph Steiner
The APK format is a ZIP file that always includes the files AndroidManifest.xml and classes.dex. Then it also always has a JAR signature (i.e. META-INF/). It does not have the JAR magic number CAFEBABE in it. ___ Reproducible-builds mailing list Repro

Bug#875451: diffoscope crashes when using --max-diff-block-lines

2017-09-11 Thread Hans-Christoph Steiner
Package: diffoscope Version: 85~bpo9+1 `fdroid verify` calls diffoscope like this: diffoscope --max-report-size 12345678 \ --max-diff-block-lines 100 \ --html foo.html --text foo.txt \ foo.apk another_foo.apk And it has recently started to crash like this: Trac

Bug#875451: affects 86

2017-09-11 Thread Hans-Christoph Steiner
Just tried with diffoscope 86, same thing: Traceback (most recent call last): File "/usr/lib/python3/dist-packages/diffoscope/main.py", line 401, in main sys.exit(run_diffoscope(parsed_args)) File "/usr/lib/python3/dist-packages/diffoscope/main.py", line 360, in run_diffoscope Config

Bug#884095: flag to force file types

2017-12-11 Thread Hans-Christoph Steiner
Package: diffoscope Version: 88 The Janus bug for Android works by making a valid APK file that is also a valid DEX file. https://www.guardsquare.com/en/blog/new-android-vulnerability-allows-attackers-modify-apps-without-affecting-their-signatures Diffoscope sees these files as different file t

Bug#890904: diffoscope does not show classes.dex diff

2018-02-20 Thread Hans-Christoph Steiner
Package: diffoscope Version: 90~bpo9+1 Attached are two APKs that have different classes.dex files. They are the same size, but have different contents. diffoscope does not show a diff for them. When I extract the classes.dex files from the APK, diff and vbindiff do show the differences. Here

Bug#890904: diffoscope does not show classes.dex diff

2018-02-26 Thread Hans-Christoph Steiner
sorry, the server was down. Its back up so you should be able to access those links now. ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds

Bug#884095: flag to force file types

2018-03-21 Thread Hans-Christoph Steiner
Chris Lamb: > severity 884095 wishlist > thanks > > Hi hc, > >> Something like --force=apk would solve both. > > So, I'm a little nervous about introducing such a directive. > > This is primarily in terms that diffoscope should really just Do The > Right Thing by default in all cases and not ne

Bug#884095: flag to force file types

2018-03-22 Thread Hans-Christoph Steiner
Chris Lamb: > Hi Hans! > >>> Have we really exhausted the detection route for this? :) >> >> I think the detection route has been exhausted. It seems that no one >> wants to do what it takes to reliably detect APKs. > > I'm sorry you think so and, with the greatest of respect, I'm not > sure

Bug#884095: flag to force file types

2018-03-22 Thread Hans-Christoph Steiner
Chris Lamb: > Hi Hans, > >> It would be literally impossible to auto-detect since a Janus APK is >> both a valid DEX file (starting with the bytes "dex") and […] > > Oh dear, I got a little lost in the weeds of Janus/APK/ZIP here.. > > Could you excuse my pedanticness and ask for direct links t

Re: [Reproducible-builds] concrete steps for improving apt downloading security and privacy

2014-09-19 Thread Hans-Christoph Steiner
Daniel Kahn Gillmor wrote: > On 09/19/2014 12:34 AM, Paul Wise wrote: >> On Fri, Sep 19, 2014 at 9:30 AM, Hans-Christoph Steiner wrote: >> >>> Finally did this: >>> http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=762153 >> >> Please note that you

Re: [Reproducible-builds] concrete steps for improving apt downloading security and privacy

2014-09-19 Thread Hans-Christoph Steiner
Daniel Kahn Gillmor wrote: > On 09/19/2014 06:07 AM, Elmar Stellnberger wrote: >>Isn`t there really any way to include the signatures in the header of >> the .deb files? >> Why not simply add multiple signature files in the control.tar.gz of a >> .deb just next >> to the md5sums which should

[Reproducible-builds] put commit messages on a separate list?

2014-09-19 Thread Hans-Christoph Steiner
I fear that all these commit emails will drown out the communication between humans. Could the commit messages could be on a separate list, like using one of these official lists: https://packages.qa.debian.org/common/index.html Checkout git.debian.org:/git/collab-maint/setup-repository for an e

Re: [Reproducible-builds] concrete steps for improving apt downloading security and privacy

2014-09-19 Thread Hans-Christoph Steiner
Daniel Kahn Gillmor wrote: > Hi Hans-- > > I think we're in agreement here about most things actually, despite our > back-and-forth. hopefully this is a clarifying response: > >> Daniel Kahn Gillmor wrote: > In that case, the .deb that was installed on a sid system *is not* the > .deb that is

Re: [Reproducible-builds] concrete steps for improving apt downloading security and privacy

2014-09-19 Thread Hans-Christoph Steiner
Daniel Kahn Gillmor wrote: > Thanks for the discussion, Hans. > > On 09/19/2014 02:47 PM, Hans-Christoph Steiner wrote: >> Packages should not be accepted into any official repo, sid included, without >> some verification builds. A .deb should remain unchanged once it is

Re: [Reproducible-builds] concrete steps for improving apt downloading security and privacy

2014-09-19 Thread Hans-Christoph Steiner
Jérémy Bobbio wrote: > Hans-Christoph Steiner: >> I still strongly disagree. Very very few people care enough to learn a >> separate process. For security to be usable, it needs to be as transparent >> and automatic as possible. APKs and Android have demonstrated that

Re: [Reproducible-builds] concrete steps for improving apt downloading security and privacy

2014-09-20 Thread Hans-Christoph Steiner
Jérémy Bobbio wrote: > Hans-Christoph Steiner: >>> This makes `.deb` hard to use without a repository for anything >>> substantial. I would assume that's why Ubuntu developed the Click >>> package format. >> >> Check out apt-offline, it makes this p

Re: [Reproducible-builds] Bug#762397: libgpg-error: please do not capture the current time during the build process

2014-09-22 Thread Hans-Christoph Steiner
Another option is building with faketime. I've had good luck with a couple of C builds using faketime to freeze time entirely during the whole make process. But I think the reproducible work in Debian so far has avoided using faketime. signature.asc Description: OpenPGP digital signature

[Reproducible-builds] faketime by default?

2014-09-22 Thread Hans-Christoph Steiner
I've been using faketime in my work on reproducible builds on Android (both "native" C code and Java), and it has been working well. It seems to me that the current approach in Debian does not use faketime. Since so much of the little issues are due to timestamps, it could make a lot of sense if

Re: [Reproducible-builds] concrete steps for improving apt downloading security and privacy

2014-09-22 Thread Hans-Christoph Steiner
Elmar Stellnberger wrote: > Am 22.09.14 um 01:52 schrieb Paul Wise: >> On Mon, Sep 22, 2014 at 2:04 AM, Elmar Stellnberger wrote: >> >>> A package with some new signatures added is no more the old package. >> That is exactly what we do *not* want for reproducible builds. >> >>> It should have

Re: [Reproducible-builds] faketime by default?

2014-09-22 Thread Hans-Christoph Steiner
Jérémy Bobbio wrote: > Hans-Christoph Steiner: >> I've been using faketime in my work on reproducible builds on Android (both >> "native" C code and Java), and it has been working well. It seems to me that >> the current approach in Debian does not use

Re: [Reproducible-builds] concrete steps for improving apt downloading security and privacy

2014-09-22 Thread Hans-Christoph Steiner
Daniel Kahn Gillmor wrote: > On 09/22/2014 04:06 PM, Hans-Christoph Steiner wrote: >> I think we're starting to nail down the moving parts here, so I want to >> outline that so we can find out the parts where we agree and where we >> disagree. >> >> * I

Re: [Reproducible-builds] concrete steps for improving apt downloading security and privacy

2014-09-24 Thread Hans-Christoph Steiner
Daniel Kahn Gillmor wrote: > On 09/22/2014 04:06 PM, Hans-Christoph Steiner wrote: >> I think we're starting to nail down the moving parts here, so I want to >> outline that so we can find out the parts where we agree and where we >> disagree. >> >> * I

Re: [Reproducible-builds] concrete steps for improving apt downloading security and privacy

2014-09-29 Thread Hans-Christoph Steiner
Stefan Fritsch wrote: > On Sunday 21 September 2014 21:13:50, Richard van den Berg wrote: >> Package formats like apk and jar avoid this chicken and egg problem >> by hashing the files inside a package, and storing those hashes in >> a manifest file. Signatures only sign the manifest file. The >>

Re: [Reproducible-builds] [RFC] debbindiff

2014-09-30 Thread Hans-Christoph Steiner
Definitely use setup.py. It makes the packaging easy and standardized, and it is the standard way to build python. It also makes it easy to publish releases to pypi, the central package repository for python. I attached a quick untested one for you. .hc Jérémy Bobbio wrote: > Hi! > > I've be

Re: [Reproducible-builds] [RFC] debbindiff

2014-09-30 Thread Hans-Christoph Steiner
o, I updated the setup.py for two small things. I recommend using code checkers like pyflakes and pylint: $ pyflakes *.py debbindiff/*.py debbindiff/difference.py:20: 'difflib' imported but unused debbindiff/difference.py:41: redefinition of function 'comment' from line 37 hans@pala

Re: [Reproducible-builds] Wiki reorganization

2015-01-07 Thread Hans-Christoph Steiner
Looks drastically better! I think this wiki is really the central resource for anyone interested in making reproducible builds, Debian or not. So I'm glad to see it reorganized to look like a community resource rather than the giant notepad it was before. .hc Jérémy Bobbio: > Hi! > > While wa

Re: [Reproducible-builds] Preliminary review of dpkg-genbuildinfo

2015-02-13 Thread Hans-Christoph Steiner
Daniel Kahn Gillmor: > On Fri 2015-02-06 01:13:18 -0500, Guillem Jover wrote: >> Take the example I gave previously of a binary package detached from >> an archive, just a .deb package laying around, either from an old >> download or passed to you by someone. You have to *know* the origin of >> t

Re: [Reproducible-builds] Preliminary review of dpkg-genbuildinfo

2015-02-16 Thread Hans-Christoph Steiner
Daniel Kahn Gillmor: > On Fri 2015-02-13 03:36:20 -0500, Hans-Christoph Steiner wrote: >> I think it would be much simpler to just have the single package signature >> that is embedded in the package file itself, like Android APKs and Java JARs. >> Since the package is built

Re: [Reproducible-builds] Preliminary review of dpkg-genbuildinfo

2015-02-17 Thread Hans-Christoph Steiner
Daniel Kahn Gillmor: > On Mon 2015-02-16 04:39:32 -0500, Hans-Christoph Steiner wrote: >> I think this topic is far too vast with far too many dependencies to really >> have a useful discussion on without a full time, dedicated team. Since that >> seems highly unlikely i