potiuk commented on issue #7191: [AIRFLOW-4030] second attempt to add singularity to airflow URL: https://github.com/apache/airflow/pull/7191#issuecomment-579443300 > Could you give me some pointers about where to write the Dockerfile, what example from the current codebase (link) is what I should try to copy the logic for? I think the closest one is Kubernetes and I am leaning towards creating another RUNTIME (singularity). There are few of scripts to update. But it's not that hard at the end. I think a custom `--start-singularity` `--stop-singularity` switches should be added similar to `--start-kind-cluster` `--stop-kind-cluster`. It's best if you look for RUNTIME capital wherever it's used but here is the guide Breeze (for local development and testing your solution): - https://github.com/apache/airflow/blob/master/breeze-complete#L40 -> options to add runtime are added here amongst others - https://github.com/apache/airflow/blob/master/breeze#L203 -> Displaying chosen RUNTIME - https://github.com/apache/airflow/blob/master/breeze#L595 -> Flags of Breeze - https://github.com/apache/airflow/blob/master/breeze#L1007 -> that's where RUNTIME leads to adding extra docker compose - https://github.com/apache/airflow/blob/master/breeze#L391 -> help describing the options. Docker Compose: - https://github.com/apache/airflow/blob/master/scripts/ci/docker-compose/runtime-kubernetes.yml - similar should be created for singularity and used in Breeze. In container: - https://github.com/apache/airflow/blob/master/scripts/ci/in_container/entrypoint_ci.sh#L164 - entrypoint for the CI where Kubernetes is started after choosing the RUNTIME="kubernetes" here is where the setup for singularity can be done. It runs https://github.com/apache/airflow/blob/master/scripts/ci/in_container/kubernetes/setup_kind_cluster.sh under the hood. That's it for interactive "local" breeze session. T Then if you run tests the script goes further. It builds airflow kubernetes image and deploys it to kubernetes. In interactive session, the deployment should be run manually as described in https://github.com/apache/airflow/blob/master/TESTING.rst#running-tests-with-kubernetes but if you run tests in travis (RUN_TESTS="true") those steps are executed manually. https://github.com/apache/airflow/blob/master/scripts/ci/in_container/entrypoint_ci.sh#L218 you have the deployment of image to kubernetes: https://github.com/apache/airflow/blob/master/scripts/ci/in_container/deploy_airflow_to_kubernetes.sh and one of the scripts it runs https://github.com/apache/airflow/blob/master/scripts/ci/in_container/kubernetes/docker/rebuild_airflow_image.sh - that's where the Kubernetes image is built. I hope that's enough leads.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
