mik-laj commented on a change in pull request #6516: [AIRFLOW-5704] Improve Kind Kubernetes scripts for local testing [WIP] URL: https://github.com/apache/airflow/pull/6516#discussion_r363785446
########## File path: BREEZE.rst ########## @@ -528,6 +509,54 @@ As soon as you enter the Breeze environment, you can run Airflow unit tests via For supported CI test suites, types of unit tests, and other tests, see `TESTING.rst <TESTING.rst>`_. +Running Tests with Kubernetes in Breeze +======================================= + +In order to run Kubernetes in Breeze you can start Breeze with ``--start-kind-cluster`` switch. This will +automatically create a Kind Kubernetes cluster in a ``docker`` (docker-in-docker) container started as part +of Breeze environment. Setting up the Kubernetes cluster takes some time so the cluster continues running +until the environment is stopped with ``--stop-environment`` switch or until ``--recreate-kind-cluster`` +switch is used additionally to ``--start-kind-cluster``. + +The Kubernetes cluster is automatically initialized and starts Airflow webserver and scheduler +using latest sources. This is done in two steps. First Airflow images are build in the ``docker`` image, +then Airflow applications (``scheduler``/``webserver``) are prepared and deployed at the kind cluster using +the images prepared. + +Rebuilding images should be faster every next time we enter the environment but it is attempted every time +you restart breeze environment. Similarly applications are re-deployed every time the breeze environment +is restarted. + +Building Airflow Images +----------------------- + +This is done using ``scripts/ci/in_container/kubernetes/setup_kubernetes.sh`` script: + +1. Latest ``apache/airflow:master-pythonX.Y-ci`` images are pulled from DockerHub to the 'docker' container. +2. The image is rebuild to take into account the latest sources mounted to Breeze environment. +3. Scripts required for running the image as Kubernetes cluster are added and + ``apache/airflow:master-pythonX.Y-ci-kubernetes`` Review comment: ```diff - ``apache/airflow:master-pythonX.Y-ci-kubernetes`` + ``apache/airflow:master-pythonX.Y-ci-kubernetes`` image. ``` ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
