vsoch commented on issue #7191: [AIRFLOW-4030] second attempt to add singularity to airflow URL: https://github.com/apache/airflow/pull/7191#issuecomment-579877224 okay so I'm tracing the kubernetes (runtime) as an example, and I have a quick question. In scripts/ci/in_container/entrypoint_ci.sh I see that given the kubernetes runtime, you are setting some variable for a container name to be the airflow name with suffix "-kubernetes." ```bash if [[ ${RUNTIME:=""} == "kubernetes" ]]; then unset KRB5_CONFIG unset KRB5_KTNAME export AIRFLOW_KUBERNETES_IMAGE=${AIRFLOW_CI_IMAGE}-kubernetes AIRFLOW_KUBERNETES_IMAGE_NAME=$(echo "${AIRFLOW_KUBERNETES_IMAGE}" | cut -f 1 -d ":") export AIRFLOW_KUBERNETES_IMAGE_NAME AIRFLOW_KUBERNETES_IMAGE_TAG=$(echo "${AIRFLOW_KUBERNETES_IMAGE}" | cut -f 2 -d ":") export AIRFLOW_KUBERNETES_IMAGE_TAG fi ``` What isn't clear is where this container is built, and then where it gets used again? At this point we are already sitting inside the airflow-testing container after a command like this: ``` else if [[ ${RUNTIME:=} == "singularity" ]]; then set +u # shellcheck disable=SC2016 docker-compose --log-level INFO \ -f "${MY_DIR}/docker-compose/base.yml" \ -f "${MY_DIR}/docker-compose/backend-${BACKEND}.yml" \ -f "${MY_DIR}/docker-compose/runtime-singularity.yml" \ "${INTEGRATIONS[@]}" \ "${DOCKER_COMPOSE_LOCAL[@]}" \ run airflow-testing \ '/opt/airflow/scripts/ci/in_container/entrypoint_ci.sh "${@}"' \ /opt/airflow/scripts/ci/in_container/entrypoint_ci.sh "${@}" # Note the command is there twice (!) because it is passed via bash -c # and bash -c starts passing parameters from $0. TODO: fixme set -u ``` and this would suggest the container is brought up inside of this container? Wouldn't it make more sense to have a docker-compose run that runs some equivalent of airflow-testing but with Singularity installed inside?
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
