On Sun, 17 Nov 2019 at 05:01, Aaron Meurer <asmeu...@gmail.com> wrote: > > Thinking more about the automation, I think we need to have the > release be 100% automated. Right now it is 90% automated (not counting > the problem of fixing blocker issues). > > Oscar, it will be a good idea to make a note of anything in the > release process that isn't automated that should be.
I will make notes and rewrite the release/README afterwards. To be honest though the reason it has taken 5 days to get to the point of having a beta release is actually not because of the *lack* of automation. It has taken that long because I needed to figure out what was going on in the release script which is very slow to run and the only way to try and fix anything is to go through a PR with full Travis tests etc. I am in the process of getting the right permissions to be able to use docker at work which will speed things up but for now I am running this on a laptop at home with a relatively slow internet connection. The first time the release script runs it needs to get the docker containers. Each time the release script runs (and multiple times within each run) it needs to download a lot of files from conda. It also runs the whole sympy test suite in between doing the other tasks. For me right now it takes around 2 hours to run the release script. Once the script fails it takes about 5 minutes to write the code that fixes it but that needs to be sent back to github and it takes 1.5 hours to run the tests before they can be merged and the release script can start again. That means that any issue in the release script has a 4 hour turn around time to detect the problem and attempt something to fix it. What would make all of this faster is if the elements of the release script were split out as separate scripts that could be tested locally without going through a PR and docker etc. This is the case for the mailmap script but not for others like testing the tarball whitelist. > One thing I forgot about is the release notes. It looks like you already > added the > deprecations. We should figure out how to automate that. Maybe the bot > can do it. I've added deprecations that I knew about that weren't listed and I used git grep to find any deprecation warnings with the 1.5 version. That can be automated but that only catches things that are deprecated. The backward compatibility breaks that don't/can't have a deprecation warning are the more important ones to document here and I don't see any way to pick up on those automatically. Perhaps changing the PR template so that there is an explicit backward compatibility release notes section would help. > The highlights can't be automated because they are human > curated, so I think that means those should just be removed. Unless > there is some way we can do it with the bot as well (maybe by letting > people tag a PRs notes as a "highlighted note"?). > > The AUTHORS/.mailmap isn't automated either. I think they only way > around that would be to run the authors/mailmap scripts as part of the > Travis tests, so that we require any PR with a new new author or new > git metadata to update those files. The main problem with that is that > it puts an extra burden on new contributors, by definition. So I don't > know if there's a better way to do it, or if it's worth doing at all. > It isn't difficult to do and it also can be done at any time, not just > during a release. Maybe we could add it to the CI but as an allowed > failure, that way it isn't a requirement to merge, but it will alert > people when it gets out of date. Fixing the mailmap isn't a big problem. If we had more regular releases there wouldn't be so much to update there. It wold be good to split out the part of the release script that shows the authors though because that only happens at the end whereas the update_mailmap.py script runs at the beginning. > The only other thing I can remember that isn't automated is various > updates that you have to do to the release script, like updating the > tarball whitelist. But if we run the script on CI that will not be a > problem because it will be updated when it needs to be, instead of > when the release happens. This is a straight-forward thing to fix but it would be quicker and easier to do if it could be checked before running the full release script. Full automation is a possibility here but I also think that it can be made a lot more efficient as a mostly automated process. Oscar -- You received this message because you are subscribed to the Google Groups "sympy" group. To unsubscribe from this group and stop receiving emails from it, send an email to sympy+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/sympy/CAHVvXxS9XUS_unw844q4C-jC%2B-4EL6Hd9Gqnj%2BU8rBNUWkRK5A%40mail.gmail.com.
