sbp commented on issue #851: URL: https://github.com/apache/tooling-trusted-releases/issues/851#issuecomment-4291235217
Suggestions and concerns require different responses from release managers. Suggestions don't really require a response at all, as long as the release manager is definitely aware of them, and are presumably not the focus of the present issue. There may be a level in the suggestion, e.g. the difference between "may" and "should", but in general as this is always the option of the release manager they can determine the level from the text of the suggestion. Concerns perhaps depend on the level of uncertainty. If ATR is 90% sure that something is a `blocker`, it can't block on it, but the release manager really should double check it. If ATR is only 10% sure, perhaps a rushed release manager could proceed to vote anyway and then rely on the binding voters to perform the research. Whether ATR is 90% sure or 10% sure, asking release managers to check an extra checkbox is almost certainly going to be unrelated to what the release manager _actually did or does_. Either they want to research the uncertain result further, or they don't, and if they want to do it, they'll almost certainly do it without us having to remind them. They are unlikely to forget that they had red errors on their compose page, and unlikely to be reminded by a checkbox. The strict checking mode that ATR had was difficult to use because it coerced all ATR uncertainties into certainties, no matter what the original confidence. Only 10% confident about an outcome? Well, it's 100% now no matter what. A more useful implementation of the strict mode would be to allow selected checks to be strict. It would be the opposite of check ignores: whereas check ignores allow a release manager to set _specific_ check results to 0% confident, i.e. not shown in the interface, this "selective strict mode" would allow them to set results to 100% confident, i.e. blocking. In summary, then, I propose that we change our check result outcomes from `success`, `error`, `blocker`, and `exception` (where the checker had an internal error) to `success`, `suggestion`, `concern`, `blocker`, and `exception`, or their close equivalents. It's usually not a good idea to start labels with the same prefix, so perhaps `success` should be renamed to something like `pass`. I also propose that we have a selective strict mode that works like check ignores but in reverse, turning `concerns` into the equivalent of `blockers` in the UI, whereas the existing "ignore mode" effectively turns `concerns` into the equivalent of `successes` (or `passes`). Third, I propose that we focus on how to help release managers to resolve `concerns`, perhaps by attaching advice to all such outcomes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
