potiuk commented on PR #52002: URL: https://github.com/apache/airflow/pull/52002#issuecomment-3096199766
> Looking at connection code, it seems to be originally intended mostly for DB connections (looking at all the comments in the class) and today the meaning changed, but not the class internals. Would it make sense to make it more generic and able to store more structured data for example "filesystem info" for ObjectStorage out of it? No. not really. It was supposed to be generic connection, and actually mostly it was started wiht "http" type of connections. I do not think we need to update it really there is a need to do more than just documenting possible patterns. Again - Connection is loosly coupled by design and should be as generic as possible. I do not see particular need to i introducing coupling instead of current "very little" duplication to be honest. It's much better if you can see the whole URL rather than joining it from several separate components, even if it means a little duplication. You could - for example - add addiontal validation, for example when you use different scheme in URL and different in connection, it could error out (but this validation should be possible to disable) if you want to have some "stronger" ties between scheme in connection and the one used in URL. And again - documenting the patterns that you described where you want to change schemes in prod/dev (either by changing connection id, or definition of connection) is a good idea. There is a good reason why Dags are written in Python because rather than declaratively describe and strongly couple such relations you can "loosely" couple them and add either automated deriving or validation in the Python code of yours - up to you. Same as Type Hinting in Python - you might want to use it for hinting, autocompletion etc, but you loose flexibility when you want to have it obligatory and erroring out like it is in other languages. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
