I would also like to see it deprecated. That said, I am not convinced there is anything that we cannot encode using URI though. I think the problem is just when one tries to use the same URI to mean two different things, e.g. both airflow connection URI and sqlalchemy URI. They are different. Hooks need to be defined such that the information they need can be properly encoded in a manner compatible with the airflow connection URI format. And I believe this should always be possible to do.
On Fri, Nov 17, 2023 at 2:45 PM Andrey Anshin <andrey.ans...@taragol.is> wrote: > Greetings! > > I want to propose to deprecate Airflow Connection URI representation and > remove it in Airflow 3 in favor of the already existing replacement - JSON > representation. > > In the past URI representation helped to add one of the awesome features - > Alternative Secrets Backends: Environment Variables, Files or Provider > specific Backends. > > However I have a feeling that nowadays it have some sort of limitations due > to the parser, see: > - https://github.com/apache/airflow/issues/33442 > > Or miss interpretation that Airflow connection it is something more than > just representation of Airflow Connection and it is a the replacement to > any other type of URIs such as: HTTP, Postgres URIs, SQLAlchemy URI, Spark > URI and etc > > And attempt to fix it > - https://github.com/apache/airflow/issues/10913 > - https://github.com/apache/airflow/pull/27796 > - https://github.com/apache/airflow/pull/35712 > > And attempt to use it in DbApiHooks, for detail see: > - https://lists.apache.org/thread/8rhmz3qh30hvkondct4sfmgk4vd07mn5 > > In addition, complicated Connection in the URI format is not human > readable, requires additional decoding, and it is hard to create it > manually. > > The only one beneficial part of URI representation is ability to create it > programatically > > https://airflow.apache.org/docs/apache-airflow/stable/howto/connection.html#generating-a-connection-uri > , there is no such of method for generate JSON, but I don't see the problem > to create such of method in Connection object which return string > representation of JSON connection. > > So my suggestion would be pretty simple: > 1. Deprecate accept URI as parameter to the Connection object > 2. Deprecate uri property > 3. Replace get_uri by get_json > 4. Replace all URI examples in Providers Connections by JSON alternatives > > > WDYT? Maybe I miss something and we should keep Airflow Connection as URI > in the future Airflow's major versions and support both ways to provide > connections as JSON and URI. > > ---- > Best Wishes > *Andrey Anshin* >
