potiuk commented on PR #30487: URL: https://github.com/apache/airflow/pull/30487#issuecomment-1498961017
> Thank you all, I misunderstood the point. > > I will leave the code that uses `mysql-connector-python`, but is it true that we have to exclude this package from dependencies anyway? If I get it right, leaving `mysql-connector-python` here will cause dependency conflicts. > > My suggestion is removing the `mysql-connector-python` from the dependencies, and throwing an exception with a readable message whenever the package is used. Those users, who will face this issue, are always able to install this package manually and resolve possible conflicts on their own, or transition to the recommended library. @eladkal , @potiuk WDYT? You can add optional dependencies (like mysql-connector-python in this case) via provider.yaml. We already have several examples of it. For example google provider has extra "[leveldb]" which brings in `plyvel` dependency. https://github.com/apache/airflow/blob/main/airflow/providers/google/provider.yaml#L1121 And you should move mysql-connector-python there from main dependencies - following the same pattern. There are a few more changes that will need to be added: * The tests that are using `mysql-connector-python` will start failing, and that's fine - we need to separate them to a separate module and run `pytest.importorskip("mysql.connector") for that file - this will skip them on our CI, but if you manually add "mysql-connector-python" and run them, we could potentially have a separate job to run it on CI separately (and I will likely add it later) but for now skipping it is fine. * In the original operator you should raise "AirflowOptionalProviderFeatureException" when import of mysql.connector fails (with some description that you need mysql-connector-python to be installed. That should remove mysql-connector-python from installed dependencies and turn it's usage into, effectively optional feature. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
