bhirsz commented on PR #26742: URL: https://github.com/apache/airflow/pull/26742#issuecomment-1262758926
> I know this was already merged by I have some questions: > > * Is there any risk to forcing the sensor to be always in Poke mode? (If it was determined by the original author to not be runnable in that mode) > * I'm curious what executor you folks use for your system tests, can you explain anything more about that :) 1) I think this page explains it the best: https://airflow.apache.org/docs/apache-airflow/stable/concepts/sensors.html in our particular case we force poke mode only for one sensor and typically it's up to user to chose the mode (I think so). However I'm not entirely sure if this sensor couldn't be implemented in a way that would allow reschedule mode. But it would require some extra analysis 2) well, you can take any example dag and run it any deployed Airflow like any other dag (granted you have copied test utils as well). In our case we deploy our Airflow image to test env and copy test Dags there. We also have monitoring dag which check whether other dags finished and when they do it save this information. Our test framework then generates dynamic test based on list of dags and each test polls for the information whether given dag completes (and with what status and logs). It can use any executor, for example KubernetesExecutor. Locally I'm testing either using breeze, or by deploying dag to airflow, or by using this CI based on cloud build: https://github.com/bhirsz/airflow-system-tests It will spawn 1 cloud build for each example dag and run it using pytest. It's not done but I hope to finish it in the close future - I'm trying to return to system testing since I was involved in other task recently. What do you use to run system tests? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
