RamanDamayeu commented on PR #857: URL: https://github.com/apache/airflow-site/pull/857#issuecomment-1698886905
Thanks, very valid point! Of course, I left a link to the implementation of ODD integration with Airflow in a comment, but I agree that it will not be very easy for users to understand how exactly this platform is connected with Airflow. Please advise how best to proceed here. The bottom line is that the ODD application implies several components, the main of which is the so-called platform itself. Here is a [link](https://github.com/opendatadiscovery/odd-platform) to a repository with it. Also for various integrations with the platform (that is, in fact, the components that are mediators between external systems and the platform itself) are implemented as collectors (if we want to implement the pull approach, that is, the components themselves go to the systems to collect meta-information with some periodicity, for example, there are such collectors for a set of databases implemented [here](https://github.com/opendatadiscovery/odd-collector), or for collecting metadata from some of [AWS services]( https://github.com/opendatadiscovery/odd-collector-aws), there is own for [GCP](https://github.com/opendatadiscovery/odd-collector-gcp), for [Azure](https://github.com/opendatadiscovery/odd-collector-azure), etc. with their own repositories) or as adapters, if we want to expand the functionality of some external system and implement the push approach (here just like for the Airflow, this is a repository for versions up to [1.10.15](https://github.com/opendatadiscovery/odd -airflow-adapter) and for versions >= 2.5.1 here is a new [rep](https://github.com/opendatadiscovery/odd-airflow-2) (uses Listeners for integration), the same approach is used to integrate [Great Expectations](https://github.com/opendatadiscovery/odd-great-expectations), [dbt](https://github.com/opendatadiscovery/odd-dbt), [Spark](https://github.com/opendatadiscovery/odd-spark-adapter)). The push/pull approach is described [here](https://github.com/opendatadiscovery/opendatadiscovery-specification/blob/main/specification/specification.md#push-and-pull-models). An overview of the architecture we could find here: https://docs.opendatadiscovery.org/architecture Having said all this, as you say, taking into account that there is a whole set of repositories - and for integration with Airflow we need to use at least two: the platform itself and the adapter - as we need to leave a link on the page should I change it to the adapter for [Airflow versions >= 2.5.1](https://github.com/opendatadiscovery/odd-airflow-2), to [a platform](https://github.com/opendatadiscovery/odd-platform) or maybe to be generally simple to [the git organization of ODD](https://github.com/opendatadiscovery)? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
