ashb commented on a change in pull request #9044:
URL: https://github.com/apache/airflow/pull/9044#discussion_r439069806
##########
File path: airflow/security/kerberos.py
##########
@@ -46,7 +46,7 @@
log = logging.getLogger(__name__)
-def renew_from_kt(principal: str, keytab: str):
+def renew_from_kt(principal: str, keytab: str, exit_on_fail: bool = False):
Review comment:
This should default to true to maintain previous behaviour.
##########
File path: airflow/providers/apache/spark/hooks/spark_submit.py
##########
@@ -617,15 +619,19 @@ def on_kill(self):
self._submit_sp.kill()
if self._yarn_application_id:
- self.log.info('Killing application %s on YARN',
self._yarn_application_id)
-
kill_cmd = "yarn application -kill {}" \
.format(self._yarn_application_id).split()
+ if self._keytab is not None and self._principal is not None:
+ renew_from_kt(self._principal, self._keytab, False)
+ krb5_env = "env KRB5CCNAME={}" \
+ .format(airflow_conf.get('kerberos', 'ccache')).split()
+ kill_cmd = krb5_env.extend(kill_cmd)
+
yarn_kill = subprocess.Popen(kill_cmd,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE)
Review comment:
Popen can take then environment dictionary directly:
```suggestion
env=None
if self._keytab is not None and self._principal is not None:
renew_from_kt(self._principal, self._keytab, False)
kenv = os.environ.copy()
env["KRB5CCNAME"] = conf.get('kerberos', 'ccache')
yarn_kill = subprocess.Popen(kill_cmd,
env=env,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE)
```
##########
File path: airflow/security/kerberos.py
##########
@@ -95,7 +98,12 @@ def renew_from_kt(principal: str, keytab: str):
# (From: HUE-640). Kerberos clock have seconds level granularity. Make
sure we
# renew the ticket after the initial valid time.
time.sleep(1.5)
- perform_krb181_workaround(principal)
+ ret = perform_krb181_workaround(principal)
+ if exit_on_fail and ret != 0:
+ sys.exit(exit_on_fail)
Review comment:
```suggestion
sys.exit(ret)
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
