michalmodras commented on PR #43446: URL: https://github.com/apache/airflow/pull/43446#issuecomment-2460692227
1) I agree that these operators should have place in the main Airflow repository in a provider. 2) These operators are different than most operators in the Google provider, which integrate with regular Cloud services - this is visible even in the implementation, e.g. the authentication pattern is different. 3) On splitting the provider - I don't think the main google provider will turn into a number of small ones. Instead, a few operators set for related services that are similar will be placed in dedicated providers - e.g. ads, workspace. The main google provider with hundreds of operators will remain as Google Cloud provider. https://github.com/apache/airflow/issues/15933 is a bit obsolete discussion from few years ago. I think the operators from this PR should be placed in a new, google-cloud-ai provider. I am not asking to contribute to the split - just create a separate, small provider for the operators you're contributing. >If the intention/new policy is that any additional services added to the google provider should be channelled down as an incremental process No - only operators for services that are very different (e.g. authentication point above, allowlist required), not useful to majority of users (because of required allowlisting / dedicated account vs. other cloud services available straightaway for cloud users), or introducing heavy dependency footprint (this is not the case with this PR), or other reason along these lines. 4) New provider should have a supporting CI dashboard. We can help with hosting it on the Google side. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
