GitHub user potiuk added a comment to the discussion: Failing DAGs out of nowhere
We generally do not provide "support" - people here help if they see they can help others - usually in their free time, but there is nothing like "support" - usually you get "support" when you pay for it - and here you get the software absolutely for free with no guarantees and promises of support (basicallly you get what you pay for). Generally, really hard to say what is wrong here - because it's hard to say what fails "dag fails" does not seem correct description (but I am guessing) it looks like your UI has a trouble with getting logs rather than your dag failing, which is a different problem So I'd start with more precise description of what exactly you do and what exactly fails. Adding screenshots might help - looking at k8s logs (not airflow logs - but kubernetes ones - in the same way you look for any other application you run - or learning how to do it if yoy do not use k8s is a good idea to - that might help you to identify failing components. It is likely your k8s is misconfigured. I am tempted to think that your DNS is a problem - the workers usually store the address of their pods as the URL where api server reads logs from. You can also look at "hostname_callable" parameter in Airflow Configuration and understand how it works - and configure it so tha it does not rely on your DNS. But this is wild guess only - more failure and warning logs that are suspicious from your system would be needed - but those are usually "deployment" logs - i.e. your k8s logs that you (as Deployment Manager who chose K8S as the deployment) should be able to browse, read and understand which are the "normal" logs and which indicate some erroneous behaviour. GitHub link: https://github.com/apache/airflow/discussions/59253#discussioncomment-15231367 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
