RamanDamayeu commented on PR #857:
URL: https://github.com/apache/airflow-site/pull/857#issuecomment-1698886905

   Thanks, very valid point!
   
   Of course, I left a link to the implementation of ODD integration with 
Airflow in a comment, but I agree that it will not be very easy for users to 
understand how exactly this platform is connected with Airflow. 
   
   Please advise how best to proceed here. The bottom line is that the ODD 
application implies several components, the main of which is the so-called 
platform itself. Here is a 
[link](https://github.com/opendatadiscovery/odd-platform) to a repository with 
it. 
   
   Also for various integrations with the platform (that is, in fact, the 
components that are mediators between external systems and the platform itself) 
are implemented as collectors (if we want to implement the pull approach, that 
is, the components themselves go to the systems to collect meta-information 
with some periodicity, for example, there are such collectors for a set of 
databases implemented 
[here](https://github.com/opendatadiscovery/odd-collector), or for collecting 
metadata from some of [AWS services]( 
https://github.com/opendatadiscovery/odd-collector-aws), there is own for 
[GCP](https://github.com/opendatadiscovery/odd-collector-gcp), for 
[Azure](https://github.com/opendatadiscovery/odd-collector-azure), etc. with 
their own repositories) or as adapters, if we want to expand the functionality 
of some external system and implement the push approach (here just like for the 
Airflow, this is a repository for versions up to 
[1.10.15](https://github.com/opendatadiscovery/odd
 -airflow-adapter) and for versions >= 2.5.1 here is a new 
[rep](https://github.com/opendatadiscovery/odd-airflow-2) (uses Listeners for 
integration), the same approach is used to integrate [Great 
Expectations](https://github.com/opendatadiscovery/odd-great-expectations), 
[dbt](https://github.com/opendatadiscovery/odd-dbt), 
[Spark](https://github.com/opendatadiscovery/odd-spark-adapter)). The push/pull 
approach is described 
[here](https://github.com/opendatadiscovery/opendatadiscovery-specification/blob/main/specification/specification.md#push-and-pull-models).
 
   
   An overview of the architecture we could find here: 
https://docs.opendatadiscovery.org/architecture
   
   Having said all this, as you say, taking into account that there is a whole 
set of repositories - and for integration with Airflow we need to use at least 
two: the platform itself and the adapter - as we need to leave a link on the 
page should I change it to the adapter for [Airflow versions >= 
2.5.1](https://github.com/opendatadiscovery/odd-airflow-2), to [a 
platform](https://github.com/opendatadiscovery/odd-platform) or maybe to be 
generally simple to [the git organization of 
ODD](https://github.com/opendatadiscovery)?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to