I am not super experienced with python, please bear with me! Since I was not
able to find an existing MongoDB-Hook and Operator for airflow I decided to
take things into my own hand. I found out that I need to define a Hook which
inherits from the BaseHook and implements some functions. Then I would create
an operator that uses this hook within a DAG and provide the db, collection and
query to it in order to get the results back in my DAG.
When I put my custom hook into the "/home/airflow/airflow/plugins" folder the
operator is not able to recognize the MongoDbHook-Class, also not when I put it
into the “/usr/lib/python2.7/site-packages/airflow/hooks” folder and restart
all three airflow components.
I found this unanswered question, which is directly related to my question:
(1) How does the development flow look like when developing a custom hook?
(2) Where do I put what in order for airflow to find it?
(3) Is the MongoDB-Hook planned for the future? If so, can you estimate when?
GfK SE, Nuremberg, Germany, commercial register at the local court Amtsgericht
Nuremberg HRB 25014; Management Board: Peter Feld (CEO), Christian Bigatà
Joseph (CFO), David Krajicek, Alessandra Cama. Supervisory Board Chairman: Ralf
Klein-Bölting. This email and any attachments may contain confidential or
privileged information. Please note that unauthorized copying, disclosure or
distribution of the material in this email is not permitted.