potiuk commented on code in PR #49938: URL: https://github.com/apache/airflow/pull/49938#discussion_r2069036049
########## providers/common/messaging/src/airflow/providers/common/messaging/providers/kafka.py: ########## @@ -0,0 +1,94 @@ +# Licensed to the Apache Software Foundation (ASF) under one Review Comment: Just to explain what's currently problematic here. If you look at `pyproject.toml` - > both `kafka` and `sqs` are optional dependencies. But in fact - they are not. Currently in `__init__.py` we import **BOTH** Kafka and SQS Queue Provider - and transitively each of the Queue Provider implementions imports their respective provider. Which means that if someone wants to use just Kafka provider but not SQS provider, they will have to install amazon provider. That is very inconvenient for a generic `common.messaging` to have to install all providers that have "potential" implementations. Surely we can do some try/catches to catch import errors an do turn it into really optional provider features, but it seems brittle and introduces unnecessary "other providers" code in common.messaging that should be very generic. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
