I like this idea. Some thoughts: * Disable CI checks when a PR is first submitted. Trigger it only when the PR is flagged for that stage of review. * I have WIP PRs that waste CI resources because of the automatic trigger, but then I need CI to run on them sometimes. Changing the title could be a trigger too... Like removing [WIP] or "NO CI". Something like this that any contributor can do. The bot can tell you what to add or remove to change your pr state. * A variation of this is to have it only run sanity checks until the PR has moved to it's final phase. * Use GitHub API in CI to check the labels and/or the configs in your proposal to route the kind of checks that happen. * Labels could be the main functional operators so that committers can nudge things along without the config files and the bot being the only way. * I report a lot of broken tests. The bot could find and report these faster. If x number of PRs fail the same test there could be an alarm. * I'd like it if when something gets fixed, if CI was retriggered automatically. When I have a lot of PRs I have to spend a lot of time shepherding them through CI landmines of flaky tests. And I usually don't know if things are better or not. Nudge and pray is wasteful of time and resources. * At a bare minimum, I really like the ideas around helping people know what's next and moving things along. * Leaderboards are cool too. Sure, they can be manipulated and create their own problems, but I think we should start somewhere with incentives and recognition * The metrics that can come out this would be insightful
On Fri, Aug 23, 2019, 22:39 Marco de Abreu <marcoab...@apache.org> wrote: > Hello dev@, > > at the moment, the user experience when submitting a Pull Request has quite > a few issues and discourages people from further contributions. To improve > the situation, I made a design for a Pull Request review bot that will > introduce a process for PRs and also assist maintaining it. I'd appreciate > it if everyone could review my design [1] and give some feedback. > > Thanks in advance! > > Best regards, > Marco > > [1]: > https://cwiki.apache.org/confluence/display/MXNET/Pull+Request+review+bot >
