+1 to Daniels's suggestion.
On Tue, Nov 28, 2023 at 5:25 AM Amogh Desai
wrote:
> Agree with @Daniel Standish on this.
> We should rename get_uri to reduce confusion.
>
> I am fine with renaming to to_uri_repr and then deprecating get_uri
>
>
> Thanks & Regards,
> Amogh Desai
>
> On Mon, Nov 27,
Agree with @Daniel Standish on this.
We should rename get_uri to reduce confusion.
I am fine with renaming to to_uri_repr and then deprecating get_uri
Thanks & Regards,
Amogh Desai
On Mon, Nov 27, 2023 at 8:56 PM Andrey Anshin
wrote:
> > Recently, a method was added `to_json_dict` which give
> Recently, a method was added `to_json_dict` which gives the json repr.
This method is internal method which use in
1. Connection Serialisation, and it is a replacement *to_dict* method which
was used before https://github.com/apache/airflow/pull/35723
2. Use in public method *as_json*
Personall
Just some history ... and a renaming suggestion...
When get_uri was added, generating the connection URI was a real pain, and
Airflow provided no way to generate one, and JSON repr was not yet
implemented. By adding get_uri, a user could define the connection object
we needed in python, and airf
Some context: I’ve been looking into putting the Connection URI to more use,
most importantly to consolidate it with OpenLineage’s URI format.
Skipping the details, one of the conclusions I had is get_uri on Connection is
not actually that useful, due to how different libraries (SQLAlchemy, Airf
A little late to the party here but convinced by what Daniel had to say.
I am ok with the idea of adding this into the deprecation bucket in the
coming time.
Implementing the get_uri in the hook seems quite reasonable to me
on the grounds that the Connection class' get_uri is meant to be general,
>
> I still think different people have different expectations of the Airflow
> URI format. My expectation is that this is only a SerDe mechanism around
> Airflow Connection. If we have a look at the codebase of Airflow we found
> that this is not always true and we try to use it as SA URI.
Yes,
Hey All,
Very detailed discussion, Andrey and others.
I am hearing everyone out here and I kinda liked the idea from Jarek about
prefixing with "airflowconnections+".
I share everyone's problem here because I have been trying to get Hive HA
connection type work which is of the format:
jdbc:hive2:
Hey guys! Thanks for all your feedback and suggestions.
I see here couple of examples with postgresql, however I would like to
remind that postgresql it is not a valid connection type which doesn't
register into the providers manager, and we have an exclusive hack to turn
this connection type into
I do think the URI format for airflow connections also serves the purpose
of conveying the intent, similar to how we URI in a browser to connect to a
website. We use Ariflow Connection URI to connect to one of the services
supported by Airflow. Due to this reason, I lean toward Jarek's proposal of
I would also be a -1 for deprecating URI connection representation, this is one
of the defacto standards across Python. I do agree with the suggestion from
Jarek that it could be less confusing if it follows SQLAlchemy style
formatting. That is a known format that users are likely going to be fa
So -1 on deprecating. I think we should do the opposite and go all in. The
suggestion by Jarek makes sense in this case, as it remains and is fully
standards compliant. The JSON representation isn't: it is our own crafted
format.
My 2 cents,
Bolke
On Sat, 18 Nov 2023 at 22:15, Jarek Potiuk wrot
I agree with Ash that it's much easier to define a lot of simpler
connections this way. But it is also very confusing the way now how you
can mix the "real" url with "airflow connection" URL.
And Daniel is very right about the magnitude of breaking change.
But possibly there is a way to eat cake
The thing that makes *me* hesitant to deprecate is the sheer magnitude of
breaking it would bring (even though we're only talking about a
hypothetical 3.0 release), balanced against the actual pain it causes.
I.e. it's confusing to use, and takes up space in docs (when, if removed,
we could just sh
-1 — for the simple case the URI format is perfect:
AIRFLOW_CONN_MY_DB=postgresql://user:pass@host
We can push JSON format for more complex cases, and/or limit URI to only the
simple cases as long as we don’t remove it entirely.
-ash
> On 17 Nov 2023, at 22:44, Andrey Anshin wrote:
>
> Greet
I would also like to see it deprecated.
That said, I am not convinced there is anything that we cannot encode using
URI though. I think the problem is just when one tries to use the same URI
to mean two different things, e.g. both airflow connection URI and
sqlalchemy URI. They are different. H
16 matches
Mail list logo
