bugraoz93 commented on issue #43897: URL: https://github.com/apache/airflow/issues/43897#issuecomment-2492708098
Great findings! Thanks! My main concern was the Variable :) > Which is not the case at the moment. We would need to async await the calls to the secret backend and also to the DB. So I would say that this is too much work and a deeper change. I agree this would be too much effort. I would rather not touch those code pieces at all. > Or backup choice: no bulk endpoint, asyncio on the client side (CLI) to generate concurrent requests on the API to create all the different Variables at the same time using only 1 thread on the client. But I think that was already considered and discarded. My idea was to have a single/common approach for all the `file inserts`. Making these changes requires more effort than just handling this case on the CLI. Let's simplify this and aim to provide the features for `v3` rather than putting more effort which can have multiple side effects too. We may need to spend operational (bugfix/maintenance) time on top of it. We can include this as an improvement point to have a common approach for all `file inserts` as a next step maybe in `v3.1` and we would have more time to implement at ease. Let's take a step back and reject this ticket. :) What do you think? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
