hussein-awala commented on code in PR #37447:
URL: https://github.com/apache/airflow/pull/37447#discussion_r1491561796
##########
airflow/providers/cncf/kubernetes/operators/spark_kubernetes.py:
##########
@@ -255,6 +258,8 @@ def custom_obj_api(self) -> CustomObjectsApi:
def execute(self, context: Context):
self.log.info("Creating sparkApplication.")
+ if self.override_existing:
+ self.on_kill()
Review Comment:
if you check the on_kill method, you find that it needs the launcher which
is initiated just after this line; so your PR will not work.
##########
airflow/providers/cncf/kubernetes/operators/spark_kubernetes.py:
##########
@@ -88,6 +89,7 @@ def __init__(
reattach_on_restart: bool = True,
delete_on_termination: bool = True,
kubernetes_conn_id: str = "kubernetes_default",
+ override_existing: bool = True,
Review Comment:
The default behavior should stay the same:
```suggestion
override_existing: bool = False,
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
