Re: [sympy] Automated releases and the authors file
FYI -- the lastest sympy is always available through replit.com: when you start a python script and have `from sympy import *` it downloads the package. /c On Sunday, April 11, 2021 at 5:02:51 AM UTC-5 Oscar wrote: > On Sun, 11 Apr 2021 at 07:51, Aaron Meurer wrote: > > > > On Sat, Apr 10, 2021 at 3:27 AM Oscar Benjamin > > wrote: > > > > > > That script builds all of the release artifacts and then installs the > > > SymPy wheel in a bunch of Python versions and runs sympy.test() in > > > each. The next steps would be to add jobs to that workflow that will > > > upload the release files to GitHub and PyPI and update the docs repo. > > > There are already scripts for doing each of those steps but I just ran > > > those locally after downloading the release files from GitHub Actions > > > (they are stored as artifacts to the release workflow). > > > > > > I haven't automated the final push for the release because I'm not > > > entirely sure how to control access for it. A common pattern for > > > automated releases in other projects seems to be a release from CI > > > based on pushing a tag. It doesn't look like there is a way to control > > > who can push a tag in GitHub though and I don't want someone to > > > accidentally trigger an automated release. Maybe it's possible to > > > control who can push a tag to a particular branch. I've set access > > > restrictions so that only Aaron or me are listed as being able to push > > > to a branch named *.* (e.g. the 1.8 release branch). The settings > > > still say "Admins can always push. Users with the Maintain role can > > > push when required status checks pass" though so I'm not exactly sure > > > what that means. > > > > Another option would be to check if a certain gpg signature is on the > > tag (git tag --sign). I think we'd have to implement that manually. > > Yes, although I don't think it's such a big issue if this last part is > not fully automatic. There are other non-automatic things like > updating the release notes with the date of the release and sending an > email announcing the release. These are quick jobs though so what's > more important is just having clearly documented instructions (which > we don't yet have). > > > > Something that isn't a part of this process and that I haven't done is > > > updating the websites. I think I really need someone else to handle > > > the websites or otherwise to produce the scripts that will do it > > > automatically. Adding the new release to the list on sympy.org isn't > > > too difficult but updating live.sympy.org and gamma.sympy.org is more > > > involved. The SymPy Live website still runs SymPy 1.5.1 because it > > > hasn't been updated to Python 3 and so can't run newer SymPy versions. > > > SymPy Gamma has been fixed in that sense but still needs some process > > > for updating it with new SymPy versions (it currently runs 1.6.2). > > > > I think that's fine. Hopefully we can make Live and Gamma update > > automatically. Maybe Amit can comment. I think there's a github bot > > that automatically sends PRs for updates to requirements.txt. Maybe we > > can use that? > > > > Live is the most important one to update, because it confuses people > > when it doesn't match the docs. > > Live is also the hardest one to update right now. > > > > The most time consuming part of the last two releases has been working > > > on the release script. I think that most of what was needed there is > > > done now so that shouldn't be as much of an issue for future releases. > > > With that out of the way the other awkward part of each release is > > > updating the AUTHORS and .mailmap files. Before each release I have to > > > email a bunch of people who have added commits with their name > > > recorded as a pseudonym or an abbreviation like obenjamin. Mostly > > > these contributors actually want their full name to be listed in the > > > AUTHORS file and the release announcements but they haven't realised > > > that we get the name/email from their git commits. The other part is > > > updating the .mailmap file to list all the alternate names and email > > > addresses listed in different commits by the same authors. you can see > > > the PR where I did this for 1.8 here: > > > https://github.com/sympy/sympy/pull/21188 > > > > > > I would like to make a change that the bin/mailmap_update.py and > > > bin/authors_update.py scripts be run on each pull request. This would > > > mean that anyone who opens a PR with commits using a previously unseen > > > name/email will also need to run those scripts and update those files. > > > While this would be an additional barrier to new contributors I think > > > that it is the best way to make sure that the author information is > > > kept accurate and it also removes the main obstacle to automating the > > > release process. > > > > I'm +1 to this. If it is too involved we could also add something to > > the bot to automatically push something. I'd have to think about
Re: [sympy] Automated releases and the authors file
On Sun, 11 Apr 2021 at 07:51, Aaron Meurer wrote: > > On Sat, Apr 10, 2021 at 3:27 AM Oscar Benjamin > wrote: > > > > That script builds all of the release artifacts and then installs the > > SymPy wheel in a bunch of Python versions and runs sympy.test() in > > each. The next steps would be to add jobs to that workflow that will > > upload the release files to GitHub and PyPI and update the docs repo. > > There are already scripts for doing each of those steps but I just ran > > those locally after downloading the release files from GitHub Actions > > (they are stored as artifacts to the release workflow). > > > > I haven't automated the final push for the release because I'm not > > entirely sure how to control access for it. A common pattern for > > automated releases in other projects seems to be a release from CI > > based on pushing a tag. It doesn't look like there is a way to control > > who can push a tag in GitHub though and I don't want someone to > > accidentally trigger an automated release. Maybe it's possible to > > control who can push a tag to a particular branch. I've set access > > restrictions so that only Aaron or me are listed as being able to push > > to a branch named *.* (e.g. the 1.8 release branch). The settings > > still say "Admins can always push. Users with the Maintain role can > > push when required status checks pass" though so I'm not exactly sure > > what that means. > > Another option would be to check if a certain gpg signature is on the > tag (git tag --sign). I think we'd have to implement that manually. Yes, although I don't think it's such a big issue if this last part is not fully automatic. There are other non-automatic things like updating the release notes with the date of the release and sending an email announcing the release. These are quick jobs though so what's more important is just having clearly documented instructions (which we don't yet have). > > Something that isn't a part of this process and that I haven't done is > > updating the websites. I think I really need someone else to handle > > the websites or otherwise to produce the scripts that will do it > > automatically. Adding the new release to the list on sympy.org isn't > > too difficult but updating live.sympy.org and gamma.sympy.org is more > > involved. The SymPy Live website still runs SymPy 1.5.1 because it > > hasn't been updated to Python 3 and so can't run newer SymPy versions. > > SymPy Gamma has been fixed in that sense but still needs some process > > for updating it with new SymPy versions (it currently runs 1.6.2). > > I think that's fine. Hopefully we can make Live and Gamma update > automatically. Maybe Amit can comment. I think there's a github bot > that automatically sends PRs for updates to requirements.txt. Maybe we > can use that? > > Live is the most important one to update, because it confuses people > when it doesn't match the docs. Live is also the hardest one to update right now. > > The most time consuming part of the last two releases has been working > > on the release script. I think that most of what was needed there is > > done now so that shouldn't be as much of an issue for future releases. > > With that out of the way the other awkward part of each release is > > updating the AUTHORS and .mailmap files. Before each release I have to > > email a bunch of people who have added commits with their name > > recorded as a pseudonym or an abbreviation like obenjamin. Mostly > > these contributors actually want their full name to be listed in the > > AUTHORS file and the release announcements but they haven't realised > > that we get the name/email from their git commits. The other part is > > updating the .mailmap file to list all the alternate names and email > > addresses listed in different commits by the same authors. you can see > > the PR where I did this for 1.8 here: > > https://github.com/sympy/sympy/pull/21188 > > > > I would like to make a change that the bin/mailmap_update.py and > > bin/authors_update.py scripts be run on each pull request. This would > > mean that anyone who opens a PR with commits using a previously unseen > > name/email will also need to run those scripts and update those files. > > While this would be an additional barrier to new contributors I think > > that it is the best way to make sure that the author information is > > kept accurate and it also removes the main obstacle to automating the > > release process. > > I'm +1 to this. If it is too involved we could also add something to > the bot to automatically push something. I'd have to think about how > it could work. I don't think that the bot can fix this automatically except in the simple case of updating the AUTHORS file for someone who has a correct git config. Identifying that commits with different names are actually the same person or discovering the name they want to be attributed to when it isn't in the commits anywhere is not something that can be handled
Re: [sympy] Automated releases and the authors file
On Sat, Apr 10, 2021 at 3:27 AM Oscar Benjamin wrote: > > Hi all, > > Over the past couple of releases I've done a bunch of work to try and > automate the release processes. This is based on the release script > that Aaron originally wrote although I have rewritten it so that it is > no longer based on rever and is easier to debug locally. Instead of > being based on a Docker image it is based on a clean Ubuntu 20.04 > install which means it can easily be tested in a VM locally and > naturally runs in an Ubuntu workflow in GitHub Actions. Yesterday I > added a release workflow to GitHub Actions which you can see here: > https://github.com/sympy/sympy/blob/1.8/.github/workflows/release.yml Thanks for doing this. The releases are happening at a much faster pace now, which was always the goal of automating the release. > > That script builds all of the release artifacts and then installs the > SymPy wheel in a bunch of Python versions and runs sympy.test() in > each. The next steps would be to add jobs to that workflow that will > upload the release files to GitHub and PyPI and update the docs repo. > There are already scripts for doing each of those steps but I just ran > those locally after downloading the release files from GitHub Actions > (they are stored as artifacts to the release workflow). > > I haven't automated the final push for the release because I'm not > entirely sure how to control access for it. A common pattern for > automated releases in other projects seems to be a release from CI > based on pushing a tag. It doesn't look like there is a way to control > who can push a tag in GitHub though and I don't want someone to > accidentally trigger an automated release. Maybe it's possible to > control who can push a tag to a particular branch. I've set access > restrictions so that only Aaron or me are listed as being able to push > to a branch named *.* (e.g. the 1.8 release branch). The settings > still say "Admins can always push. Users with the Maintain role can > push when required status checks pass" though so I'm not exactly sure > what that means. Another option would be to check if a certain gpg signature is on the tag (git tag --sign). I think we'd have to implement that manually. > > Something that isn't a part of this process and that I haven't done is > updating the websites. I think I really need someone else to handle > the websites or otherwise to produce the scripts that will do it > automatically. Adding the new release to the list on sympy.org isn't > too difficult but updating live.sympy.org and gamma.sympy.org is more > involved. The SymPy Live website still runs SymPy 1.5.1 because it > hasn't been updated to Python 3 and so can't run newer SymPy versions. > SymPy Gamma has been fixed in that sense but still needs some process > for updating it with new SymPy versions (it currently runs 1.6.2). I think that's fine. Hopefully we can make Live and Gamma update automatically. Maybe Amit can comment. I think there's a github bot that automatically sends PRs for updates to requirements.txt. Maybe we can use that? Live is the most important one to update, because it confuses people when it doesn't match the docs. > > The most time consuming part of the last two releases has been working > on the release script. I think that most of what was needed there is > done now so that shouldn't be as much of an issue for future releases. > With that out of the way the other awkward part of each release is > updating the AUTHORS and .mailmap files. Before each release I have to > email a bunch of people who have added commits with their name > recorded as a pseudonym or an abbreviation like obenjamin. Mostly > these contributors actually want their full name to be listed in the > AUTHORS file and the release announcements but they haven't realised > that we get the name/email from their git commits. The other part is > updating the .mailmap file to list all the alternate names and email > addresses listed in different commits by the same authors. you can see > the PR where I did this for 1.8 here: > https://github.com/sympy/sympy/pull/21188 > > I would like to make a change that the bin/mailmap_update.py and > bin/authors_update.py scripts be run on each pull request. This would > mean that anyone who opens a PR with commits using a previously unseen > name/email will also need to run those scripts and update those files. > While this would be an additional barrier to new contributors I think > that it is the best way to make sure that the author information is > kept accurate and it also removes the main obstacle to automating the > release process. I'm +1 to this. If it is too involved we could also add something to the bot to automatically push something. I'd have to think about how it could work. Actually, we should run the whole release script (sans the pushing) on each PR. That way nothing breaks, and we can be confident that if we do push a tag, the release will actually
[sympy] Automated releases and the authors file
Hi all, Over the past couple of releases I've done a bunch of work to try and automate the release processes. This is based on the release script that Aaron originally wrote although I have rewritten it so that it is no longer based on rever and is easier to debug locally. Instead of being based on a Docker image it is based on a clean Ubuntu 20.04 install which means it can easily be tested in a VM locally and naturally runs in an Ubuntu workflow in GitHub Actions. Yesterday I added a release workflow to GitHub Actions which you can see here: https://github.com/sympy/sympy/blob/1.8/.github/workflows/release.yml That script builds all of the release artifacts and then installs the SymPy wheel in a bunch of Python versions and runs sympy.test() in each. The next steps would be to add jobs to that workflow that will upload the release files to GitHub and PyPI and update the docs repo. There are already scripts for doing each of those steps but I just ran those locally after downloading the release files from GitHub Actions (they are stored as artifacts to the release workflow). I haven't automated the final push for the release because I'm not entirely sure how to control access for it. A common pattern for automated releases in other projects seems to be a release from CI based on pushing a tag. It doesn't look like there is a way to control who can push a tag in GitHub though and I don't want someone to accidentally trigger an automated release. Maybe it's possible to control who can push a tag to a particular branch. I've set access restrictions so that only Aaron or me are listed as being able to push to a branch named *.* (e.g. the 1.8 release branch). The settings still say "Admins can always push. Users with the Maintain role can push when required status checks pass" though so I'm not exactly sure what that means. Something that isn't a part of this process and that I haven't done is updating the websites. I think I really need someone else to handle the websites or otherwise to produce the scripts that will do it automatically. Adding the new release to the list on sympy.org isn't too difficult but updating live.sympy.org and gamma.sympy.org is more involved. The SymPy Live website still runs SymPy 1.5.1 because it hasn't been updated to Python 3 and so can't run newer SymPy versions. SymPy Gamma has been fixed in that sense but still needs some process for updating it with new SymPy versions (it currently runs 1.6.2). The most time consuming part of the last two releases has been working on the release script. I think that most of what was needed there is done now so that shouldn't be as much of an issue for future releases. With that out of the way the other awkward part of each release is updating the AUTHORS and .mailmap files. Before each release I have to email a bunch of people who have added commits with their name recorded as a pseudonym or an abbreviation like obenjamin. Mostly these contributors actually want their full name to be listed in the AUTHORS file and the release announcements but they haven't realised that we get the name/email from their git commits. The other part is updating the .mailmap file to list all the alternate names and email addresses listed in different commits by the same authors. you can see the PR where I did this for 1.8 here: https://github.com/sympy/sympy/pull/21188 I would like to make a change that the bin/mailmap_update.py and bin/authors_update.py scripts be run on each pull request. This would mean that anyone who opens a PR with commits using a previously unseen name/email will also need to run those scripts and update those files. While this would be an additional barrier to new contributors I think that it is the best way to make sure that the author information is kept accurate and it also removes the main obstacle to automating the release process. Oscar -- You received this message because you are subscribed to the Google Groups "sympy" group. To unsubscribe from this group and stop receiving emails from it, send an email to sympy+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/sympy/CAHVvXxTtdyVyEhW7eCSMa-HxqY%2Bv%3DaOWCOU%2BXH-m3zkExDaLvQ%40mail.gmail.com.
