Hi everyone, Recently, several issues [1-3] have highlighted outage risks and developer inconveniences due to dependency management practices in Beam Python.
With dependabot and other tooling that we have integrated with Beam, one of the missing pieces seems to be having a clear guideline of how we should be specifying requirements for our dependencies and when and how we should be updating them to have a sustainable process. As a conversation starter, I put together a retrospective <https://docs.google.com/document/d/1gxQF8mciRYgACNpCy1wlR7TBa8zN-Tl6PebW-U8QvBk/edit?resourcekey=0-XcHRyFh4KRPkA0GsdUmU3g#>[4] covering a recent incident and would like to get community opinions on the open questions. In particular, if you have experience managing dependencies for other Python libraries with rich dependency chains, knowledge of available tooling or first hand experience dealing with other dependency issues in Beam, your input would be greatly appreciated. Thanks, Valentyn [1] https://github.com/apache/beam/issues/22218 [2] https://github.com/apache/beam/pull/22550#issuecomment-1217348455 [3] https://github.com/apache/beam/issues/22533 [4] https://docs.google.com/document/d/1gxQF8mciRYgACNpCy1wlR7TBa8zN-Tl6PebW-U8QvBk/edit
