Yes. You can run any Python code and any system process. For Python, you can use PythonOperator or PythonVirtuaalenvOperator, For system process, you can use BashOperator See: https://airflow.readthedocs.io/en/latest/operators-and-hooks-ref.html#fundamentals
You can trigger a new DAG Run using REST API: https://airflow.readthedocs.io/en/stable/rest-api-ref.html#post--api-experimental-dags--DAG_ID--dag_run We are also working on major changes to the API for Airflow 2.0. But new DAG Runes can still be triggered. https://airflow.readthedocs.io/en/latest/stable-rest-api-ref.html#operation/post_dag_run To run tasks on a schedule, you must configure schedule_interval on DAG: https://airflow.readthedocs.io/en/latest/scheduler.html?highlight=schedule_interval#scheduler Best regards, Kamil On Sat, Oct 17, 2020 at 3:00 PM Giorgio Zoppi <[email protected]> wrote: > Hello Kamil, > looks like Airflow is a task executor, can I use it for executing any > shell script, any python code or java code. Am i wrong? > Well, i have found some medium post > https://towardsdatascience.com/selenium-on-airflow-automate-a-daily-online-task-60afc05afaae > So theoretically i can do whatever a python script can do, right. > Yes we are evaluating the right tool, but airflow looks promising. > We can have two user cases: > Use case 1. > 1. The CI/Jenkins activate airflow workflow (does aiflow exposes such > mechanism as API?) > 2. Once activated it starts the graph of tasks > Use 2. > 1. A cron job starts Nightly Airflow > 2. Once started it can do nightly stress test of our RStorage Platform. > I miss some dots, but could be interesting and more flexible than > robotframework for example. I try some experiments this weekend. > BR, > Giorgio. > > > > > > > > > >
