Re: [aur-general] Auto-generated Github tarballs format change (Was: TU Application: Daniel M. Capella)
On 11/15/18 10:52 AM, Baptiste Jonglez wrote: > On 15-11-18, Eli Schwartz via aur-general wrote: >> On 11/14/18 11:50 PM, Daniel M. Capella via aur-general wrote: >>> Quoting Levente Polyak via aur-general (2018-11-14 17:00:38) - tests are awesome <3 run them whenever possible! more is better! pulling sources from github is favorable when you get free tests and sometimes manpages/docs >>> >>> Will work with the upstreams to distribute these. I prefer to use published >>> offerings as they are what the authors intend to be used. GitHub >>> autogenerated >>> tarballs are also subject to change: >>> https://marc.info/?l=openbsd-ports=151973450514279=2 >> >> I've seen the occasional *claim* that this happens, but I've yet to see >> any actual case where this happens and it isn't because of upstream >> force-pushing a tag. > > See https://bugs.archlinux.org/task/60382 for an example. > > I still had the old archive around so I spent some time comparing it with > the new one: > > - I compared the checksum of each individual file in the archives, and > they were all identical > > - I compared the raw tar files after decompressing, and there were just a > few bytes that were moved around > > This really suggests a slight format change in the way the tarball was > generated (could be file ordering). > > If you want to double check, here they are: > > - old archive from May 2017: > https://files.polyno.me/arch/kashmir-20150805-20170525.tar.gz > > - new archive: https://files.polyno.me/arch/kashmir-20150805.tar.gz > > Baptiste > GitHub invalidating caches is not the problem here, they should be allowed to do it whenever they wish. The root of the issue is unreproduciblility as already pointed out here. The tarballs are stable per se if no weird magic applies via git export rules like dates being exported into files or no force pushes are done to the tree, they use git archive via tar which itself is reproducible. In fact, detatched pre-generated tarballs sometimes changes as well so blame upstream for any such happening (at least nowadays :P). Anyway, the differences we see here are just our digital legacy where the format was not reproducible yet. The example tarball indeed only contains metadata changes related to ordering of filenames inside the structure. This is definitively stable today. PS: You can simply use diffoscope for such analysis, it has been invented for this very purpose and is not only content but also meta-data aware. cheers, Levente signature.asc Description: OpenPGP digital signature
Re: [aur-general] Auto-generated Github tarballs format change (Was: TU Application: Daniel M. Capella)
Le 15/11/2018 à 10:52, Baptiste Jonglez a écrit : > On 15-11-18, Eli Schwartz via aur-general wrote: >> On 11/14/18 11:50 PM, Daniel M. Capella via aur-general wrote: >>> Quoting Levente Polyak via aur-general (2018-11-14 17:00:38) - tests are awesome <3 run them whenever possible! more is better! pulling sources from github is favorable when you get free tests and sometimes manpages/docs >>> Will work with the upstreams to distribute these. I prefer to use published >>> offerings as they are what the authors intend to be used. GitHub >>> autogenerated >>> tarballs are also subject to change: >>> https://marc.info/?l=openbsd-ports=151973450514279=2 >> I've seen the occasional *claim* that this happens, but I've yet to see >> any actual case where this happens and it isn't because of upstream >> force-pushing a tag. > See https://bugs.archlinux.org/task/60382 for an example. > > I still had the old archive around so I spent some time comparing it with > the new one: > > - I compared the checksum of each individual file in the archives, and > they were all identical > > - I compared the raw tar files after decompressing, and there were just a > few bytes that were moved around > > This really suggests a slight format change in the way the tarball was > generated (could be file ordering). > > If you want to double check, here they are: > > - old archive from May 2017: > https://files.polyno.me/arch/kashmir-20150805-20170525.tar.gz > > - new archive: https://files.polyno.me/arch/kashmir-20150805.tar.gz > > Baptiste But those are not tag tarballs though. That being said, yes, the tarball format changed once in the past, on purpose, so that it could actually be reproducible and allow things like the “alternative local workflow” of https://wiki.debian.org/Creating signed GitHub releases. I can’t remember when that happened, but per this page that was prior to April 2016. And AFAIK, it is not subject to change again for this exact reason. ;) Regards, Bruno signature.asc Description: OpenPGP digital signature
[aur-general] Auto-generated Github tarballs format change (Was: TU Application: Daniel M. Capella)
On 15-11-18, Eli Schwartz via aur-general wrote: > On 11/14/18 11:50 PM, Daniel M. Capella via aur-general wrote: > > Quoting Levente Polyak via aur-general (2018-11-14 17:00:38) > >> - tests are awesome <3 run them whenever possible! more is better! > >> pulling sources from github is favorable when you get free tests > >> and sometimes manpages/docs > > > > Will work with the upstreams to distribute these. I prefer to use published > > offerings as they are what the authors intend to be used. GitHub > > autogenerated > > tarballs are also subject to change: > > https://marc.info/?l=openbsd-ports=151973450514279=2 > > I've seen the occasional *claim* that this happens, but I've yet to see > any actual case where this happens and it isn't because of upstream > force-pushing a tag. See https://bugs.archlinux.org/task/60382 for an example. I still had the old archive around so I spent some time comparing it with the new one: - I compared the checksum of each individual file in the archives, and they were all identical - I compared the raw tar files after decompressing, and there were just a few bytes that were moved around This really suggests a slight format change in the way the tarball was generated (could be file ordering). If you want to double check, here they are: - old archive from May 2017: https://files.polyno.me/arch/kashmir-20150805-20170525.tar.gz - new archive: https://files.polyno.me/arch/kashmir-20150805.tar.gz Baptiste signature.asc Description: PGP signature
