potiuk commented on issue #22248: URL: https://github.com/apache/airflow/issues/22248#issuecomment-1098309026
> I wonder if we should do this with the `Project-URL` package metadata. The provider package can add a URL like this (using `pyproject.toml` for example, there’s an equivalent for `setup.py`): We could - but I think we should do it if / when we separte providers to separate repositorry(ies). Also Providers are available for developers to change their sources in checked-out airflow code, so there has to be an alternative way of retrieving provider meta-data that does not use packages. Currently provider's metadata is kept in one place - provider.yaml - and the information is basically available as an entrypoint returning the provider-info.yaml -schema compliant dictionary (and for non-community providers it's just the dictionary tha shoudl be returned) This was really done this way to match "plugin" approach. And there was some reasoning behind it - because next step after the entrypoint is anyhow instantiations of some classes (Hooks mainly) to be able to add customisations to UI. It was done literally weeks before 2.0 release (actulally last changes were added in alphas and betas) as we decided that providers will also be split for 2.0 rather late in the process of preparation. I would rather stick with the current approach (we coudl simply add optional "pip-installation-url" attribute to the provider-info schema for now and work out the plan on moving to metadata-only approach for providers when we get rid of the connections customizations (at least in the form we have today when we literally execute Provider code in the UI to add the customizations. But this requires transitioning away fromt the FAB-Based Connections screen. The second thing that is done the same way are operator extra links and those might be harder to turn into declarative. So my proposal for now is to add it to provider.yaml, but - as soon as we will get rid of the connections screen we could also implement more "meta-data" driven approach (still we will have to figure out a way to enable people to edit the metadata in providers and see the result of without them having to build the provider packages and install them on top of airlfow. This might be connected with splitting-off the provider repos from core. But the last thing I would like with this split is to add extra pain for people who develop community providers. They should be able to add things like extra links or customisations to connections "on-the fly"- without having to go through the hoop of building and installing provider packages every time they change a single line of code. So no matter if we use entrypoint or metadata, we should have alternative "source driven" rather than "package driven" way of retrieving the metadata by Airflow. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
