This is a different tack entirely, but how we do it is bake our DAGs etc
into the images we use for our deploy, which we push to Google Cloud
Registry. We use helm, an extra package on top of kubernetes to interpolate
the image name into the deployment files on release. We basically have a
`Make de
On 2018-01-12 16:17, Anirudh Ramanathan wrote:
> > Any good way to debug this?
>
> One way might be reading the events from "kubectl get events". That should
> reveal some information about the pod removal event.
> This brings up another question - should errored pods be persisted for
> debugg
On 2018-01-13 08:12, Daniel Imberman wrote:
> @jordan can you turn delete mode off and post the kubectl describe results
> for the workers?
Already had delete mode turned off. This was a really useful command. I can see
the basic logs in the k8s dashboard:
+ airflow run jordan_dag_3 run_this_1
@jordan can you turn delete mode off and post the kubectl describe results
for the workers?
On Sat, Jan 13, 2018, 3:20 AM Koen Mevissen wrote:
> Are you using kubernetes on Google Cloud Platform? (GKE)
>
> You should be able to capture the logs from your nodes. In case you run GKE
> with logging
Are you using kubernetes on Google Cloud Platform? (GKE)
You should be able to capture the logs from your nodes. In case you run GKE
with logging automatically deployed then deamonsets with fluentd will ship
logs from /var/log/containers on the nose to Google Cloud Logging.
Koen
Op za 13 ja
> Any good way to debug this?
One way might be reading the events from "kubectl get events". That should
reveal some information about the pod removal event.
This brings up another question - should errored pods be persisted for
debugging?
On Fri, Jan 12, 2018 at 3:07 PM, jordan.zuc...@gmail.com
I'm trying to use Airflow and Kubernetes and having trouble using git sync to
pull DAGs into workers.
I use a git sync init container on the scheduler to pull in DAGs initially and
that works. But when worker pods are spawned, the workers terminate almost
immediately because they cannot find th
