pierrejeambrun commented on issue #43897: URL: https://github.com/apache/airflow/issues/43897#issuecomment-2491293544
I would say second approach is more straightforward to implement with an `asyncio.gather` on all the variables. The issue is for that to work we need the underlying methods, (called methods) to also be async and release the loop on blocking I/O. Which is not the case at the moment. We would need to async await the calls to the secret backend and also to the DB. So I would say that this is too much work and a deeper change. That leaves us with choice 1 only I believe. Or backup choice: no bulk endpoint, asyncio on the client side (CLI) to generate concurrent requests on the API to create all the different Variables at the same time using only 1 thread on the client. But I think that was already considered and discarded. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
