Ravi Kotecha created AIRFLOW-2661:
-------------------------------------
Summary: Kubernetes Executor - dags_volume_subpath and
logs_volume_subpath don't do anything
Key: AIRFLOW-2661
URL: https://issues.apache.org/jira/browse/AIRFLOW-2661
Project: Apache Airflow
Issue Type: Bug
Components: executor
Affects Versions: Airflow 2.0
Environment: Kubernetes 1.10 (kops)
Reporter: Ravi Kotecha
Fix For: Airflow 2.0
It looks like the configuration options:
{code:java}
dags_volume_subpath =
logs_volume_subpath =
{code}
don't do anything when set. This is because the
{{_constructVolume()}}
function is called in
[https://github.com/apache/incubator-airflow/blob/ef3ead167be9d7dd72f75da9d1f4e00438b8ea6e/airflow/contrib/kubernetes/worker_configuration.py#L124].
It sets subPath on the {{volume}} rather than the mount.
This function should be changed to not take the optional parameter {{subPath}}
and we should use {{dags_volume_subpath}} if provided in the construction of
the {{volumeMount}}
If you look at
[https://kubernetes.io/docs/concepts/storage/volumes/#using-subpath] we can see
that it's a property of {{volumeMount}} not a {{Volume}}. We are correctly
handling this for the the {{configMap}} on line 167.
My colleague has also commented on the original PR that added this
functionality:
[https://github.com/apache/incubator-airflow/commit/e30a1f451aa5ec5aca4c886067ba8946a3d33395#r29462135]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
