Dear all, I am not super experienced with python, please bear with me! Since I was not able to find an existing MongoDB-Hook and Operator for airflow I decided to take things into my own hand. I found out that I need to define a Hook which inherits from the BaseHook and implements some functions. Then I would create an operator that uses this hook within a DAG and provide the db, collection and query to it in order to get the results back in my DAG.
When I put my custom hook into the "/home/airflow/airflow/plugins" folder the operator is not able to recognize the MongoDbHook-Class, also not when I put it into the “/usr/lib/python2.7/site-packages/airflow/hooks” folder and restart all three airflow components. I found this unanswered question, which is directly related to my question: https://groups.google.com/forum/#!msg/airbnb_airflow/O_FLCzWaZf8/qYuj0GzpDAAJ My questions: (1) How does the development flow look like when developing a custom hook? (2) Where do I put what in order for airflow to find it? (3) Is the MongoDB-Hook planned for the future? If so, can you estimate when? ________________________________ GfK SE, Nuremberg, Germany, commercial register at the local court Amtsgericht Nuremberg HRB 25014; Management Board: Peter Feld (CEO), Christian Bigatà Joseph (CFO), David Krajicek, Alessandra Cama. Supervisory Board Chairman: Ralf Klein-Bölting. This email and any attachments may contain confidential or privileged information. Please note that unauthorized copying, disclosure or distribution of the material in this email is not permitted.
