potiuk commented on issue #10874: URL: https://github.com/apache/airflow/issues/10874#issuecomment-882084416
I read the issue and - unless I misunderstood it - I think we should close both the issue and PR. @baolsen Why would you like to run `execute()` method multiple times? The whole idea of `execute()` method is that it is run ONCE per task execution, in a separate process and the process is killed. This is pretty much how it always works, so the fact that it opens a new connection every time it is called is pretty much expected. Caching is hard, and if we do not have to do it, we should not do it. The right way of reusing a connection (even if you want to compose or subclass something) is to reuse HOOK not operator. You can easily create single Hook and pass it around and use it many times while keeping the connection open, but you should not try to do it with the operator - pretty much always when you want to write some behaviour where you want to reuse connection for multiple API calls etc, the hook reuse is right not the operator. You should always in this case write a custom operator. You can even subclass existing operator, if this operator has some reusable methods, but then you should never call super execute method multiple times. Execute is not designed to be called once. I am tempted to close it straight away, but this a very old issue @kaxil and @eladkal reacted to it - without pointing that out - maybe I do understand it wrongly ? Can someone shed some light on it :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
