justinmclean commented on PR #228: URL: https://github.com/apache/airflow-steward/pull/228#issuecomment-4527069319
Thanks for the thoughtful feedback. A few things worth clarifying: this skill is designed to be complementary to the contributor-nomination skill, not a standalone assessment tool. The intent is that a maintainer uses it as one input among several when considering a nomination, not as a scorecard a contributor optimizes toward. Keeping that distinction clear is one of the main guards against the Goodhart's Law dynamic you're describing, where the metric becomes the target. On the limitation you're pointing to, yes, it will only surface people with visible GitHub activity, and that's a deliberate acknowledgment of what's measurable rather than what's valuable. Other contribution types are genuinely hard to quantify. We could potentially look at mailing list traffic as an additional signal, but that immediately runs into name-mapping problems: email addresses don't reliably map to GitHub handles, and getting that wrong is worse than not having the data at all. I think we can address these concerns more directly in the skill output by strengthening the framing around scope. A committer is not expected to be active in all areas, and most contributors naturally focus on one or two streams rather than all. Someone who does deep, careful PR review but rarely files issues isn't a weaker candidate, they're a different kind of contributor. Making that explicit in the output should help resist the pull toward treating visible gaps as deficits to fill. Happy to add that language before this comes out of draft. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
