damccorm commented on issue #29440: URL: https://github.com/apache/beam/issues/29440#issuecomment-1898641691
> Context managers are "the Pythonic way" because as users, we assume that __enter__() and __exit()__ are doing useful, hidden things that should be best practices. For instance, file or DB connection handles are gracefully and automatically closed, no matter what happens within the manager. I'd say that is one purpose, the other purpose is to clearly encapsulate usage of resources. It is generally expected that when you enter a `with Foo as bar:` block that you will stay in the block while `bar` is still an active reference to `Foo`'s resources. The `__exit()__` best practices you mention are usually making sure that you've actually released your references because you're no longer holding on to the resource. That is basically what this means for Beam > There is no documentation anywhere, at least nowhere that I could find, that explicitly states that one should not use the context manager when working with streaming jobs. +1 on adding docs, I agree this could be clearer. > I don't have evidence, but I would expect that a lot of Apache Beam users are using it for streaming purposes. This is definitely true :) > context managers should not be used for streaming This isn't quite right; context managers shouldn't be used for pipelines that will never terminate. It is possible for streaming pipelines to be launched and still eventually finish (either because they were written in a way where the watermark _can_ advance to the end of the global window or because a user cancels them). > if you would like me to work on documentation stating that context managers should not be used for streaming, I can do that. I would just like some promise / confidence that the PR I create would be merged in (assuming it's good). Sure, always happy to review a doc improvement -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
