On Mon, Mar 21, 2022 at 12:44:44PM -0400, Mark Phippard wrote: > I have been toying with the idea of volunteering to be Release > Manager. I am not ready to do so yet as I have run into numerous > problems. > > The docs are decent but they are definitely written to an audience > that knows a lot of things that I do not and it feels like a lot of > knowledge is just assumed. Keep in mind I am a MacOS/Windows user with > no Python experience. My main fear is running into a problem during > the process where the script needs to be fixed and not being able to > do it. I do not have Mike/Paul on speed dial to help me any more.
This is good news, thank you Mark! I will try to help as best as I can. > It would probably be helpful if someone saved a start to finish log of > all of the commands they ran and from what directory when they were > RM. I do not have such a transcript. However, I have been following the docs almost to the letter for the last few releases. I have also tried to keep the docs updated if something was no longer working. > The release.py script implies you can/should run the process from > people.apache.org. I have no idea what that is, how I would access it, > or what the experience would be like once I was there. I imagine you > SSH into the box and I would assume you would do everything in your > home folder if you did this, but the docs use /opt/svnrm as the > example and it seems weird you would use a folder like this on a > shared system. I believe people.apache.org no longer offers shell access. > At the moment, I am not considering this option. Since I do not have a > Linux machine available, I am trying to use Docker to build a Debian > image to run the process. Using a virtual machine or container sounds like a good solution to me. > I am running everything as root. I could change it to a user account > if it would fix my problems. I was always running commands as my own user. There should be no need to use the root account for anything release.py is doing. > I should note up front that I do not have a GPG key so I would need to > create a new one and "do something" before I begin. The key I used to > use when signing releases was lost about 4 computers ago. I do note > that HACKING mentions if you use people.a.o you do not do the signing > from there. So I guess you use scp or something to pull the files down > to sign them locally and then push your signatures back up to > people.a.o after? > > Problem 1: Rolling the tarballs > > The process creates the tarballs but fails near the end. It looks GPG related? > INFO:root:Building Unix tarballs > INFO:root:Moving artifacts and calculating checksums > Traceback (most recent call last): > File "trunk/tools/dist/release.py", line 1916, in <module> > main() > File "trunk/tools/dist/release.py", line 1912, in main > args.func(args) > File "trunk/tools/dist/release.py", line 983, in roll_tarballs > download_file(KEYS, filepath, None) > File "trunk/tools/dist/release.py", line 289, in download_file > response = urlopen(url) > File "/usr/lib/python2.7/urllib2.py", line 154, in urlopen > return opener.open(url, data, timeout) > File "/usr/lib/python2.7/urllib2.py", line 435, in open > response = meth(req, response) > File "/usr/lib/python2.7/urllib2.py", line 548, in http_response > 'http', request, response, code, msg, hdrs) > File "/usr/lib/python2.7/urllib2.py", line 473, in error > return self._call_chain(*args) > File "/usr/lib/python2.7/urllib2.py", line 407, in _call_chain > result = func(*args) > File "/usr/lib/python2.7/urllib2.py", line 556, in http_error_default > raise HTTPError(req.get_full_url(), code, msg, hdrs, fp) > urllib2.HTTPError: HTTP Error 404: Not Found It seems ASF have removed the KEYS file our script is trying to fetch. See http://people.apache.org/keys/ where it says "Project group files are no longer created." It looks like what the script wants to do here is obtain a copy of the Subversion project's KEYS file and store it along with release artifacts. If we want to keep doing this we will have to maintain our own KEYS file on the website, I suppose. Otherwise, we could decide to no longer provide such a file and remove relevant code from the script. Not sure what is better. > Problem 2: Test failure > > One of the lock_tests consistently fails. > > I have confirmed I get this failure using the 1.14.1 tarball so I > assume it is related to running the tests inside Docker. If this ends > up being the only problem, I could probably just safely ignore it and > let the others that sign the release verify the tests all work for > them. The failing test has a comment on top which explains why :) /* XXX NOTE: This test will fail on most Unix-like systems when run as the root user, because flock() will ignore file permissions. */ > Problem 3: Python bindings tests fail > > I have tried this using Debian Buster and Python 2 and Debian Bullseye > and Python 3. The Perl and Ruby bindings are working OK. I am still > having some trouble with JavaHL but feel like I will figure that one > out. I have not tried the ctypes bindings yet. > > This fails using the 1.14.1 tarball as well as when I make a trial run > using release.py > > # make check-swig-py > if [ "LD_LIBRARY_PATH" = "DYLD_LIBRARY_PATH" ]; then for d in > /opt/svnrm/subversion-1.14.1/subversion/bindings/swig/python/libsvn_swig_py > /opt/svnrm/subversion-1.14.1/subversion/bindings/swig/python/../../../libsvn_*; > do if [ -n "$DYLD_LIBRARY_PATH" ]; then > LD_LIBRARY_PATH="$LD_LIBRARY_PATH:$d/.libs"; else > LD_LIBRARY_PATH="$d/.libs"; fi; done; export LD_LIBRARY_PATH; fi; \ > cd /opt/svnrm/subversion-1.14.1/subversion/bindings/swig/python; \ > /usr/bin/python > /opt/svnrm/subversion-1.14.1/subversion/bindings/swig/python/tests/run_all.py > Traceback (most recent call last): > File > "/opt/svnrm/subversion-1.14.1/subversion/bindings/swig/python/tests/run_all.py", > line 23, in <module> > import mergeinfo, core, client, delta, checksum, pool, fs, ra, wc, > repository, \ > File > "/opt/svnrm/subversion-1.14.1/subversion/bindings/swig/python/tests/mergeinfo.py", > line 22, in <module> > from svn import core, repos, fs > File > "/opt/svnrm/subversion-1.14.1/subversion/bindings/swig/python/svn/core.py", > line 26, in <module> > from libsvn.core import * > File > "/opt/svnrm/subversion-1.14.1/subversion/bindings/swig/python/libsvn/core.py", > line 144 > def apr_initialize() -> "apr_status_t": > ^ > SyntaxError: invalid syntax > make: *** [Makefile:944: check-swig-py] Error 1 This might be a swig problem? Perhaps the version of swig and the version of python don't work well together? Not sure. > Problem 4: Emails > > I have no idea how to send the emails from my Apache address Configure your mail client such that when you send mail from your @apache.org address, it uses SMTP server mail-relay.apache.org. ${user}_apache is the username (stsp_apache in my case). The password is the same as you are using to make commits. The server supports STARTTLS. > > Problem 5: Unknowns > > Since I have not been able to dry run the entire process I am still > unsure what else I will run into and how I would solve those problems. I believe you are already far enough down the road that any remaining issues will be ironed out. It is not unusual for an RM to deal with problems like you are seeing. Sometimes the environment changes and we have to adapt the script. Updates to the release process documentation usually happen when a release is made and the RM runs into issue like this. Otherwise, we mostly leave these docs alone. Provided you don't get stuck for too long and will be able to get help when you need it, I don't think there is anything you should worry about. Cheers, Stefan
