potiuk opened a new issue #12702: URL: https://github.com/apache/airflow/issues/12702
The discussion in https://github.com/apache/airflow/pull/12466#issuecomment-735347672 had shown that there is likely an expectation about making sure all the connections are 'registrable' in the UI of Airflow. This is not the case currently. Not all connections are visible in the UI (and possibly for a good reason) and currently connections are defined by Hook class variables. Only a subset of the existing hooks provides this information, first of all because many of the connections/hooks do not require any custom fields, behaviour, they do not use automated Hook creation by connection type, but also it is because many of the hooks share the same connection type (For example almost all Google Hooks, most Amazon hooks etc). Seems that we have `1 <> Many` relationships Connection/Hook rather than 1-1 as initially designed. Also, there is an existing "standard" introduced by the DBApi Hook where this metadata is stored in the Hook class as class-level field. Which we carried to those hooks that required to register themselves in the UI: For example: ``` class ImapHook(BaseHook): conn_name_attr = 'imap_conn_id' default_conn_name = 'imap_default' conn_type = 'imap' conn_name = 'IMAP' ``` We likely need to address that by defining clear rules for the hooks/providers: - whether we want to be able to map all hooks to connections - whether we still want to create hook class automatically based on connection type - especially where we have multiple hooks sharing one connection type - should we maybe have connection aliases to alias same connection type for different hooks - where to keep that information (in Hooks ? Elsewehere) - how to name the connection types - especially when there are siblings and multiple hooks sharing connection type. I think those questions need to be answered before we can think about starting any implementation. <!-- Welcome to Apache Airflow! For a smooth issue process, try to answer the following questions. Don't worry if they're not all applicable; just try to include what you can :-) If you need to include code snippets or logs, please put them in fenced code blocks. If they're super-long, please use the details tag like <details><summary>super-long log</summary> lots of stuff </details> Please delete these comment blocks before submitting the issue. --> **Description** <!-- A short description of your feature --> **Use case / motivation** <!-- What do you want to happen? Rather than telling us how you might implement this solution, try to take a step back and describe what you are trying to achieve. --> **Related Issues** <!-- Is there currently another issue associated with this? --> ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
