danccooper commented on a change in pull request #6377: [AIRFLOW-5589] monitor
pods by labels instead of names
URL: https://github.com/apache/airflow/pull/6377#discussion_r338022044
##########
File path: airflow/contrib/operators/kubernetes_pod_operator.py
##########
@@ -112,55 +113,54 @@ class KubernetesPodOperator(BaseOperator): # pylint:
disable=too-many-instance-
"""
template_fields = ('cmds', 'arguments', 'env_vars', 'config_file')
+ @staticmethod
+ def create_labels_for_pod(context):
+ """
+ Generate labels for the pod s.t. we can track it in case of Operator
crash
+ :param context:
+ :return:
+ """
+ labels = {
+ 'dag_id': context['dag'].dag_id,
+ 'task_id': context['task'].task_id,
+ 'exec_date': context['ts']
Review comment:
Hi guys,
I think we should consider the desired behaviour with this. If we're
including the try_number then it opens a situation where multiple tries could
be running alongside each other. I don't think that is ever desired & would
certainly cause problems with my jobs (back to the original cause of this one
😄).
If we want to avoid monitoring the previous try, then should we check for it
& kill it if exists before starting the next?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
