Re: [Dev] FWD: [openbsd-ports] Porters, please read re GitHub auto-generated tarballs vs releases
On 03/02/2018 04:18 PM, Luke Shumaker wrote: > On 2018-02-27 at 12:28:07 > Stuart Henderson wrote: >> Many ports are using github's on-the-fly generated source-code tarballs >> via the GH_ variables in Makefiles. >> > Though I wonder if that's intentional/allowed, or if it's really just > a bug in GitHub. > >> : "It is not meant to be reliable or a way to distribute software >> : releases and nothing in the software stack is made to try to >> : produce consistent archives." > > I can't seem to find a source for that quote. > i would like to see that documentation also i dont know what those the GH_ variables in Makefiles actually do - but i can say from my experience that the github auto-generated "releases" that are based on git tags seem to be exactly what you get with the `git archive` command - i use a git commit hook that creates the tarball with `git archive` then signs it with GPG then downloads the auto-generated tarball from github and compares the local signature against the remote tarball before uploading the signature and i have not seen any in-consistency - maybe the "tagged" releases are more stable or maybe i have just been lucky i dunno signature.asc Description: OpenPGP digital signature ___ Dev mailing list Dev@lists.parabola.nu https://lists.parabola.nu/mailman/listinfo/dev
Re: [Dev] FWD: [openbsd-ports] Porters, please read re GitHub auto-generated tarballs vs releases
On 2018-02-27 at 12:28:07 Stuart Henderson wrote: > Many ports are using github's on-the-fly generated source-code tarballs > via the GH_ variables in Makefiles. > > These are *not* guaranteed to be stable, they can change as github > update software and caches expire (this has happened at some point over > the last few months so we have been seeing a number of hash failures > recently). Due to local caches at the github clusters, these files > can be different depending on which cluster you're connecting to, > so if you regenerate distinfo to match the file which you see > locally, it may cause breakage elsewhere in the world. That's a bummer. Around 2011, the auto-generated tarballs were really not stable; it would just about never give you the same file twice. Since then (2014-ish, maybe?), they had changed their implementation, and it it started consistently giving the same file, to different parts of the world (ie, probably hitting different clusters), for years. I had assumed that meant GitHub had committed to keeping them stable, and we started allowing them in Parabola. It's a bummer that in the last few months they've apparently started breaking that. Though I wonder if that's intentional/allowed, or if it's really just a bug in GitHub. > : "It is not meant to be reliable or a way to distribute software > : releases and nothing in the software stack is made to try to > : produce consistent archives." I can't seem to find a source for that quote. -- Happy hacking, ~ Luke Shumaker ___ Dev mailing list Dev@lists.parabola.nu https://lists.parabola.nu/mailman/listinfo/dev
[Dev] FWD: [openbsd-ports] Porters, please read re GitHub auto-generated tarballs vs releases
FYI. Many Parabola PKGBUILDs also use this. https://marc.info/?l=openbsd-ports=151973450514279 On 2018-02-27 at 12:28:07 Stuart Henderson wrote: > Many ports are using github's on-the-fly generated source-code tarballs > via the GH_ variables in Makefiles. > > These are *not* guaranteed to be stable, they can change as github > update software and caches expire (this has happened at some point over > the last few months so we have been seeing a number of hash failures > recently). Due to local caches at the github clusters, these files > can be different depending on which cluster you're connecting to, > so if you regenerate distinfo to match the file which you see > locally, it may cause breakage elsewhere in the world. > > : "It is not meant to be reliable or a way to distribute software > : releases and nothing in the software stack is made to try to > : produce consistent archives." > > Sometimes upstream *only* provides these auto generated files, but in > other cases they use github's releases infrastructure and upload "normal" > generated distfiles which are not then subject to change. > > To identify this, go to the project's "releases" page. > Using an example of a port I've just fixed: > > https://github.com/rdoeffinger/iec16022/releases/. > > Under "Assets" on this page, you might see files with a "box" icon > which are uploads. > > On the iec16022 page I'm using as an example, Assets has these: > > iec16022-0.3.0.exe<- uploaded windows binary > iec16022-0.3.0.tar.xz <- uploaded source tarball > iec16022-0.3.0.tar.xz.asc <- uploaded gpg sig > Source code (zip) <- auto generated > Source code (tar.gz) <- auto generated > > If there are only entries for "Source code (zip)" and "Source code > (tar.gz)" the only options are to use GH_TAGNAME or ask upstream to > upload a file. > > But in the case above, a proper distfile is available, so I've > switched the graphics/iec16022 port across to use it. > > This is done by avoiding the GH_* variables and reducing the amount > of magic in the ports Makefile: just use DISTNAME and MASTER_SITES > as standard, and set HOMEPAGE if needed (it's provided automatically > when GH_* are set). > > You will also often find that these uploaded tarballs have autoconf > scripts etc. generated, whereas they aren't present in the auto- > generated tarballs, so you may be able to remove make targets and > dependencies needed to generate these. > > If you are publishing your own software to github, please do use > the releases infrastructure and upload distfiles that you have > generated yourself! > > I've done a query of the github releases API for all ports that are > currently using GH_TAGNAME and looked at assets for these. I have skipped > the ones where the assets are clearly only for distributing binaries and > listed the others: it is quite likely that the following ports do have > source tarballs available which can be used instead of the auto-generated > ones. > > archivers/deutex > astro/stellarium > audio/audiality2 > audio/cantata > audio/mumble > audio/puddletag > devel/cpp-hocon > devel/leatherman > devel/lua-tz > devel/msgpack > devel/ocaml-extlib > devel/protobuf-c > fonts/fantasque-sans > fonts/overpass > games/fs2open > games/megaglest/base > games/megaglest/data > games/unknown-horizons > geo/osm-gps-map > graphics/ffmpegthumbnailer > graphics/glm > graphics/py-cairo > graphics/vigra > inputmethods/ibus > inputmethods/ibus-anthy > mail/mu > math/libtommath > misc/redshift > multimedia/streamlink > net/avahi > net/libproxy > net/libstrophe > net/lua-mmdb > net/lua-psl > net/nagios/check_rabbitmq > net/rabbitmq-c > net/rsnapshot > print/cups > print/system-config-printer > security/gopass > security/opensc > security/plaso > security/py-dfdatetime > security/py-dfvfs > security/py-dfwinreg > sysutils/consolekit > sysutils/exfat-fuse > sysutils/fleetctl > sysutils/opam > sysutils/riemann-c-client > sysutils/rofi > sysutils/tree > textproc/enchant > textproc/libical > www/kore > www/liferea > www/py-jinja2 > www/wp-cli > x11/compton > x11/xdotool > x11/xwallpaper ___ Dev mailing list Dev@lists.parabola.nu https://lists.parabola.nu/mailman/listinfo/dev
