Are you using kubernetes on Google Cloud Platform? (GKE) You should be able to capture the logs from your nodes. In case you run GKE with logging automatically deployed then deamonsets with fluentd will ship logs from /var/log/containers on the nose to Google Cloud Logging.
Koen Op za 13 jan. 2018 om 01:18 schreef Anirudh Ramanathan <ramanath...@google.com.invalid> > > Any good way to debug this? > > One way might be reading the events from "kubectl get events". That should > reveal some information about the pod removal event. > This brings up another question - should errored pods be persisted for > debugging? > > On Fri, Jan 12, 2018 at 3:07 PM, jordan.zuc...@gmail.com < > jordan.zuc...@gmail.com> wrote: > > > I'm trying to use Airflow and Kubernetes and having trouble using git > sync > > to pull DAGs into workers. > > > > I use a git sync init container on the scheduler to pull in DAGs > initially > > and that works. But when worker pods are spawned, the workers terminate > > almost immediately because they cannot find the DAGs. But since the > workers > > terminate so quickly, I can't even inspect the file structure to see > where > > the DAGs ended up during the workers git sync init container. > > > > I noticed that the git sync init container for the workers is hard coded > > into /tmp/dags and there is a git_subpath config setting as well. But I > > can't understand how the git synced DAGs ever end up in > /root/airflow/dags > > > > I am successfully using a git sync init container for the scheduler, so I > > know my git credentials are valid. Any good way to debug this? Or an > > example of how to set this up correctly? > > > > > > -- > Anirudh Ramanathan > -- Kind regards, Met vriendelijke groet, *Koen Mevissen* Principal BI Developer *Travix Nederland B.V.* Piet Heinkade 55 1019 GM Amsterdam The Netherlands T. +31 (0)20 203 3241 E: kmevis...@travix.com www.travix.com *Brands: * CheapTickets | Vliegwinkel | Vayama | BudgetAir | Flugladen
