Bug#655976: queue-viewer: support binary debdiffs
On Sat, 2016-10-15 at 22:53 +0100, Adam D. Barratt wrote: > - the output is current plain text, with no formatting That's now fixed - both HTML and plain text output can be generated. > - there's no filtering of "interesting" content; all architectures for > which there are packages have a corresponding binary debdiff link An initial version of support for this has been deployed. Currently, "Version:" fields and package dependency fields where the changes consist purely of switching "old source version" for "new source version" are automatically removed (with the caveat that only epochless versions work). As well as filtering more "boring" content, we also need some way of indicating that no "interesting" differences were found, in a way that queue-viewer can detect at runtime. Adam
Bug#655976: queue-viewer: support binary debdiffs
On Sat, 2016-10-15 at 22:53 +0100, Adam D. Barratt wrote: > - as a result of invoking debdiff once per architecture, passing a list > of "old" and "new" .deb files, the output contains a single copy of > sections such as "Files in second set of .debs but not in first", rather > than the information being attached to the binary package concerned I've fixed that by having the debdiff invocation use --show-moved. Regards, Adam
Bug#655976: queue-viewer: support binary debdiffs
On Sun, 2012-01-15 at 16:13 +, Adam D. Barratt wrote: > It would be useful if queue-viewer could run debdiff on binary uploads, > in order to catch things like missing or new files and permission > changes. If anything interesting is found this could then be included > in the queue output. As some may have noticed, initial support for this went live this evening. Some notes: - the output is current plain text, with no formatting - there's no filtering of "interesting" content; all architectures for which there are packages have a corresponding binary debdiff link - as a result of invoking debdiff once per architecture, passing a list of "old" and "new" .deb files, the output contains a single copy of sections such as "Files in second set of .debs but not in first", rather than the information being attached to the binary package concerned Regards, Adam
Bug#655976: queue-viewer: support binary debdiffs
On Sun, 2016-10-02 at 13:18 +0200, Julien Cristau wrote: > On Tue, Sep 27, 2016 at 20:35:43 +0100, Adam D. Barratt wrote: > > > - How should we invoke debdiff? We can avoid multiple debdiff > > invocations via "debdiff --show-moved --controlfiles=ALL --from > > base1.deb base2.deb --to new1.deb new2.deb" or, if we want to make the > > command line shorter, generate .changes files for the two sets of > > packages and then feed those to debdiff. In the latter case, we'd want > > to create symlink farms, as the .deb files would need to be in the same > > directory as the .changes files. > > > I don't think length of the command line is a big concern? I guess not, as we're not doing anything crazy like invoking a shell. > > - Is there a way we can make the wdiff output easier to review? We need > > to consider both on-screen display (presumably primarily via a web > > browser) and the possibility of mailing the result, as we do for source > > uploads. > > > If the html part could have some colors to show added/removed bits, > that'd be nice. Ack. > > - Should queue-viewer always display links to debdiffs for every > > architecture, or only those where those is an "interesting" difference? > > What would be a useful UI for that? Possibilities that come to mind: - a new "binary debdiffs" column - a "binary debdiffs" row in the upload information > > - What differences are we interested in? Some will always occur - > > version number changes from $old_source_ver to $new_source_ver (with > > adjustment for binNMUs); the addition of a "Source" header for a > > newly-binNMUed package; a change in Size / Installed-Size within a > > certain delta. > > Also adjusting for epoch, and binary packages with different versioning > schemes from the source. diffpackages doesn't quite deal with that, > yet... Yeah, a little intelligence would be needed to catch all of those. We can improve that as we go along of course. > I think it's ok to leave filtering to a later date though, if we > can get something it'll be better than the current "notice issues on the > point release Saturday". Indeed, I'm just wary of the queue-viewer output becoming too noisy and either difficult to use (in case of the online version) or ignored (for mails). >From an implementation perspective, I had wondered if it was worth trying to fit the binary methods in to the existing debrelease.debdiff class which handles source diffs, but I'm not sure if that would just make things more difficult. Adam
Bug#655976: queue-viewer: support binary debdiffs
Hi Adam, thanks for getting this rolling. On Tue, Sep 27, 2016 at 20:35:43 +0100, Adam D. Barratt wrote: > - How should we invoke debdiff? We can avoid multiple debdiff > invocations via "debdiff --show-moved --controlfiles=ALL --from > base1.deb base2.deb --to new1.deb new2.deb" or, if we want to make the > command line shorter, generate .changes files for the two sets of > packages and then feed those to debdiff. In the latter case, we'd want > to create symlink farms, as the .deb files would need to be in the same > directory as the .changes files. > I don't think length of the command line is a big concern? > - Is there a way we can make the wdiff output easier to review? We need > to consider both on-screen display (presumably primarily via a web > browser) and the possibility of mailing the result, as we do for source > uploads. > If the html part could have some colors to show added/removed bits, that'd be nice. > - Should queue-viewer always display links to debdiffs for every > architecture, or only those where those is an "interesting" difference? > What would be a useful UI for that? > > - What differences are we interested in? Some will always occur - > version number changes from $old_source_ver to $new_source_ver (with > adjustment for binNMUs); the addition of a "Source" header for a > newly-binNMUed package; a change in Size / Installed-Size within a > certain delta. Also adjusting for epoch, and binary packages with different versioning schemes from the source. diffpackages doesn't quite deal with that, yet... I think it's ok to leave filtering to a later date though, if we can get something it'll be better than the current "notice issues on the point release Saturday". > - What should happen if the list of binary packages is different between > the two versions? While still an exception, this happens often enough > with e.g. firefox updates, that we need to consider it. > I think big fat "Removed binary package(s)" and "Added binary package(s)" warnings together with the debdiff for common packages would be enough, at least initially. Otherwise you'd have to manually tell the tool which binary package was renamed to show the debdiff between old and new. Cheers, Julien
Bug#655976: queue-viewer: support binary debdiffs
On Wed, 2016-09-28 at 05:14 +, Niels Thykier wrote: > An additional idea (in the "nice to have" department"): >What about recursive diff/diffoscoping of (parts of) the packages? > Like including the diff of contents of the maintscripts/ctrl files. As > I recall, debdiff does not do that out of the box (except for control > itself). By default, no. It does, however, if you specify "--controlfiles=ALL", as suggested in one of the earlier parts of my mail. (Specifically, it runs "dpkg-deb -e" and then walks the result looking for the requested files; "ALL" returns the differences for all files found except md5sums.) Regards, Adam
Bug#655976: queue-viewer: support binary debdiffs
Adam D. Barratt: > On Sun, 2012-01-15 at 16:13 +, Adam D. Barratt wrote: >> It would be useful if queue-viewer could run debdiff on binary uploads, >> in order to catch things like missing or new files and permission >> changes. If anything interesting is found this could then be included >> in the queue output. > > As we never seem to get any further in that direction than saying "we > really should do that" after an issue, or a d-i binNMU, some thoughts / > debate points: > > [...] > > Regards, > > Adam > An additional idea (in the "nice to have" department"): What about recursive diff/diffoscoping of (parts of) the packages? Like including the diff of contents of the maintscripts/ctrl files. As I recall, debdiff does not do that out of the box (except for control itself). Thanks, ~Niels
Bug#655976: queue-viewer: support binary debdiffs
On Sun, 2012-01-15 at 16:13 +, Adam D. Barratt wrote: > It would be useful if queue-viewer could run debdiff on binary uploads, > in order to catch things like missing or new files and permission > changes. If anything interesting is found this could then be included > in the queue output. As we never seem to get any further in that direction than saying "we really should do that" after an issue, or a d-i binNMU, some thoughts / debate points: - How should we invoke debdiff? We can avoid multiple debdiff invocations via "debdiff --show-moved --controlfiles=ALL --from base1.deb base2.deb --to new1.deb new2.deb" or, if we want to make the command line shorter, generate .changes files for the two sets of packages and then feed those to debdiff. In the latter case, we'd want to create symlink farms, as the .deb files would need to be in the same directory as the .changes files. - Is there a way we can make the wdiff output easier to review? We need to consider both on-screen display (presumably primarily via a web browser) and the possibility of mailing the result, as we do for source uploads. - Should queue-viewer always display links to debdiffs for every architecture, or only those where those is an "interesting" difference? What would be a useful UI for that? - What differences are we interested in? Some will always occur - version number changes from $old_source_ver to $new_source_ver (with adjustment for binNMUs); the addition of a "Source" header for a newly-binNMUed package; a change in Size / Installed-Size within a certain delta. - What should happen if the list of binary packages is different between the two versions? While still an exception, this happens often enough with e.g. firefox updates, that we need to consider it. Regards, Adam
Bug#655976: queue-viewer: support binary debdiffs
Package: release.debian.org Severity: wishlist User: release.debian@packages.debian.org Usertags: tools It would be useful if queue-viewer could run debdiff on binary uploads, in order to catch things like missing or new files and permission changes. If anything interesting is found this could then be included in the queue output. Adam -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
