This is adapted from my blog post at http://asmeurersympy.wordpress.com/2013/07/07/automating-the-sympy-release-process/ and a comment there.
We have finally gotten a decently automated release process. At this point, it's no longer a question of automating the process, but rather of automating those parts of the process that aren't automated yet. For those of you who don't know, this started last year when I wrote a rant to the mailing list (https://groups.google.com/d/msg/sympy/UfNhyFv-oMg/PkwIz32K-lsJ) about how long it takes us to do releases. To give an updated status of that rant - The release process is mostly automated. See the release directory in the pull request https://github.com/sympy/sympy/pull/2256. - All our testing is done on Travis CI. The only thing that is tested locally are things specific to the release (like making sure that the tarballs are correct). We were in a good state of this when I wrote my rant, but we are even better now. There are still a few things that aren't tested in Travis (e.g., the slow tests, see https://github.com/sympy/sympy/pull/2225, and tests of external dependencies, see https://github.com/sympy/sympy/pull/2284 for a start). GitHub's new releases feature is very nice. The only issue with it is that there doesn't seem to be an API, so I have to upload the files to it manually. But even then, it's still better than PyPI, for which I still haven't figured out how to upload files to via their API (setup.py upload doesn't work unless you do setup.py sdist upload). And there are other issues too, like Google authentication not working on PyPI, and the inability to upload two files with the same md5 hash (which is required by the workaround of https://github.com/pypa/pip/issues/701). So now to get to the crux of the email, here is the current process for releasing: First, as always, we have to do these things: - Fix all blocking issues (this will always be the hardest part) - Write release notes (also not easy, but this is the one thing that I’ve generally been able to get other people to volunteer for) Once those are out of the way, we are in a state when we can release (OK, the second thing doesn’t have to happen until the release is done, but if we don’t do a release candidate, as we may not in the future, then it has to be done first). The remaining steps can be done in under an hour (depending on your internet connection). The steps are 1. Create a release branch 2. Change the version in master 3. Push the branch up and make a pull request out of it 4. Change the version in the branch (technically this has to go before 3, because you can’t make a pull request out of an empty branch, but that messes up my little goto logic here) 5. Run 'fab vagrant release' from the release directory 6. Fix any issues that it comes up with. In theory this will be none, because of our continuous testing, but in the current state of things, there will be issues with things like keeping AUTHORS and .mailmap up-to-date. 7. Copy the boilerplate text that it generates to the release notes (stuff like a table of md5 hashes, and a list of authors). 8. Upload everything to GitHub by hand (GitHub is not required, but I do feel a little better about having things at more than *just* PyPI) 9. If it’s a release candidate, wait a week, then goto 4. Fix bugs in the interim. Determine if another release candidate is needed. Otherwise, it’s a final release, so continue to 10. 10. Upload to PyPI by hand. Currently this is not automated, because of http://stackoverflow.com/q/17633633/161801, but I plan to make it so as soon as I figure it out. 11. Update the webpage and the docs (probably these can be at least partially automated) 12. Send a message to the mailing list (again, I think this can at least be copy-paste automated) 13. (optional) update the blog, post to google+ and twitter. As for updating other sites, like sympy live and sympy gamma, I plan to leave that to other people, or I’ll do it myself if I have the time. The same for sites we don’t own like Wikipedia. I don’t plan to do anything with Google Code anymore. Aside from bug fixing and the release notes, I anticipate the most annoying things to be: - Updating .mailmap and AUTHORS. We should probably test this on Travis. There is a new script, ./bin/mailmap_update.py, which currently only tells you what is missing, but in the future may do some work of fixing it for you too. Already this makes it about 1/10 as painful as it was before. - Updating the website and docs. - All the social media stuff - Uploading six files to GitHub by hand, one at a time. Some of this can be automated, and I will probably look into it at some point or another. Of course, these all pales in comparison to the real elephant in the room, which is writing the release notes. We should think about better ways to handle that. I have an idea. When you merge pull requests, there is an option to write a custom message. We could encourage people to write stuff in there whenever they merge important pull requests, and then write a tool that just pulls those things out into release notes. There would still need to be some way to check if something was missed, but I think this would be the dead-simplest way to encourage people to write release notes as features are implemented. Aaron Meurer -- You received this message because you are subscribed to the Google Groups "sympy" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/sympy. For more options, visit https://groups.google.com/groups/opt_out.
