Mark Phippard wrote on Mon, Mar 21, 2022 at 16:46:55 -0400: > On Mon, Mar 21, 2022 at 4:31 PM Stefan Sperling <s...@elego.de> wrote: > > On Mon, Mar 21, 2022 at 12:44:44PM -0400, Mark Phippard wrote: > > > Problem 1: Rolling the tarballs > > > > > > The process creates the tarballs but fails near the end. It looks GPG > > > related? > > > > > INFO:root:Building Unix tarballs > > > INFO:root:Moving artifacts and calculating checksums > > > Traceback (most recent call last): > > > File "trunk/tools/dist/release.py", line 1916, in <module> > > > main() > > > File "trunk/tools/dist/release.py", line 1912, in main > > > args.func(args) > > > File "trunk/tools/dist/release.py", line 983, in roll_tarballs > > > download_file(KEYS, filepath, None) > > > File "trunk/tools/dist/release.py", line 289, in download_file > > > response = urlopen(url) > > > File "/usr/lib/python2.7/urllib2.py", line 154, in urlopen > > > return opener.open(url, data, timeout) > > > File "/usr/lib/python2.7/urllib2.py", line 435, in open > > > response = meth(req, response) > > > File "/usr/lib/python2.7/urllib2.py", line 548, in http_response > > > 'http', request, response, code, msg, hdrs) > > > File "/usr/lib/python2.7/urllib2.py", line 473, in error > > > return self._call_chain(*args) > > > File "/usr/lib/python2.7/urllib2.py", line 407, in _call_chain > > > result = func(*args) > > > File "/usr/lib/python2.7/urllib2.py", line 556, in http_error_default > > > raise HTTPError(req.get_full_url(), code, msg, hdrs, fp) > > > urllib2.HTTPError: HTTP Error 404: Not Found > > > > It seems ASF have removed the KEYS file our script is trying to fetch. > > See http://people.apache.org/keys/ where it says "Project group files are > > no longer created." > > > > It looks like what the script wants to do here is obtain a copy of > > the Subversion project's KEYS file and store it along with release > > artifacts. If we want to keep doing this we will have to maintain > > our own KEYS file on the website, I suppose. Otherwise, we could > > decide to no longer provide such a file and remove relevant code > > from the script. Not sure what is better. > > Since I do not know what this was used for, maybe someone can help get > us to a decision and update the script? Daniel Shahaf, if you see > this, I think of you as the resident expert on this stuff. Any thoughts?
The design goals here are two: 1. When someone with commit access to N ASF projects updates their PGP key, they shouldn't have to do O(N) work to update N KEYS files. They should have to do either O(1) work or, ideally, nothing at all. 2. Releases should snapshot the keys that are current at the time they are generated, in order to remain verifiable in archive.a.o even if the keys in question are later removed from LDAP (by root@ as part of a manual password reset, or by the committer via id.a.o). The ASF-wide "generate group keys" scripts addressed #1. They used to generate two files, one with only the full committers' keys and one with both full and partial committers' keys. We used the former (to reduce attack surface) until it stopped getting generated. Copying the file to subversion-<version>.KEYS addressed #2. So, what to do? - We could talk to some ASF-wide list (comdev, infra, site-dev@) about resuming generation of group keys files. (If you do this, someone will probably ask you to comply with a markdown document that calls itself policy. I don't believe that document is Foundation policy, and even if it is, our practice has been +1ed by an Officer of the Foundation.) - We could roll our own automation, relying on the key fingerprints in id.a.o (LDAP) via <https://people.apache.org/keys/committer/>: for availid in $( perl -anE 'say $F[0] if (/^Blanket/../END ACTIVE FULL.*SCRIPTS LOOK FOR IT/ and /@/)' \ < /path/to/COMMITTERS ) ; do echo curl -sSfO https://people.apache.org/keys/committer/${availid}.asc || test $? -eq 22 || return 1; done cat *.asc > subversion-….keys - We could roll our own solution without relying even on <https://people.apache.org/keys/committer/>, since its API has already been broken twice. ---- As to the wider problem here: I've RM'd once and could probably produce a reference transcript of the actual rolling bit (the part that culminates in having ../subversion-1.14.2.tar.gz ready to upload). However, that's not at the top of my todo list.
