amoghrajesh commented on code in PR #46860:
URL: https://github.com/apache/airflow/pull/46860#discussion_r1961778018
##########
providers/cncf/kubernetes/src/airflow/providers/cncf/kubernetes/pod_generator.py:
##########
@@ -341,18 +350,45 @@ def construct_pod(
run_id=run_id,
),
),
- spec=k8s.V1PodSpec(
- containers=[
- k8s.V1Container(
- name="base",
- args=args,
- image=image,
- env=[k8s.V1EnvVar(name="AIRFLOW_IS_K8S_EXECUTOR_POD",
value="True")],
- )
- ]
- ),
)
+ podspec = k8s.V1PodSpec(
+ containers=[main_container],
+ )
+
+ if content_json_for_volume:
+ input_file_path = "/tmp/execute/input.json"
+ execute_volume = V1Volume(
+ name="execute-volume",
+ empty_dir=V1EmptyDirVolumeSource(),
+ )
+
+ execute_volume_mount = V1VolumeMount(
+ name="execute-volume",
+ mount_path="/tmp/execute",
+ read_only=False,
+ )
+
+ init_container = k8s.V1Container(
+ name="init-container",
+ image="busybox",
+ command=["/bin/sh", "-c", f"echo '{content_json_for_volume}' >
{input_file_path}"],
+ volume_mounts=[execute_volume_mount],
+ )
Review Comment:
I do not love the idea of environment variables here as much as I do for
init containers. init container is a very common pattern used for such a
scenario (i am using a minimal busybox here to perform a copy). Why i dont like
usage of env variables here is cos:
1. IIRC, env variables are eventually stored in etcd as an object, so all
envs together cannot have more than 1MiB of data per pod/container. Creates
unnecessary issue on pod's limits.
2. If we store it in env variables, it is much more easy to see to the naked
eye -- for instance in the pod template on the UI.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
