potiuk commented on pull request #12383: URL: https://github.com/apache/airflow/pull/12383#issuecomment-728244324
Hey @ashb @kaxil @dimberman @turbaszek - we discussed it at the 2.0 call, I know there was my POC done differently and there were some ideas with pluggr, but I think using the metadata from `provider.yaml` that @mik-laj added is a super-simple and a very powerfulidea. I added the provider discovery mechanism - I made sure it works with implicit packages and supports multiple location of installation of provider packages (so you can have provider packages installed in several locations as well as locally and it should work). Also I am reading the provider.yaml using importlb.resources (from 3.7 but I added optional backport for 3.6). This means that this provider.yaml discovery should work regardless of the installation method of the packages (so it will even work if the packages are installed as .zip files, which is theoretically supported). This is very versatile and simple and it should allow us to do things like: * developing API to discover which providers are installed - including description/version/ even all the information that is now part of the documentation * displaying installed providers in the UI (including all the extra information) * CLI to inspect the providers installed as well * refreshing list of providers on signal or webserver reload or periodically (it is super-fast) Providing that the idea will be ok for others, I think I can have most of the discovery and registration done by tomorrow. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
