kaxil commented on a change in pull request #6516: [AIRFLOW-5704] Improve Kind Kubernetes scripts for local testing URL: https://github.com/apache/airflow/pull/6516#discussion_r365487681
########## File path: BREEZE.rst ########## @@ -528,6 +509,65 @@ As soon as you enter the Breeze environment, you can run Airflow unit tests via For supported CI test suites, types of unit tests, and other tests, see `TESTING.rst <TESTING.rst>`_. +Running Tests with Kubernetes in Breeze +======================================= + +To run Kubernetes in Breeze, start Breeze with the ``--start-kind-cluster`` switch. This will +automatically create a Kind Kubernetes cluster in the same ``docker`` engine that the user uses to run Breeze +Setting up the Kubernetes cluster takes some time, so the cluster continues running +until the user stops the cluster explicitly with the ``--stop-kind-cluster`` switch or until the user runs breeze with the ``--recreate-kind-cluster`` +switch (rather than ``--start-kind-cluster``). + +The cluster name follows the pattern ``airflow-python-X.Y.Z-vA.B.C`` where X.Y.Z is the Python version, +and A.B.C is kubernetes version. This allows the user to have multiple clusters running at the same +time for different python versions and different kubernetes versions. + +The Control Plane is available from inside the docker image via <CLUSTER_NAME>-control-plane:6443 +host:port, the worker of the kind cluster is available at <CLUSTER_NAME>-worker +and webserver port for the worker is 30809. + +The Kubernetes Cluster is started but in oder to deploy airflow to Kubernetes cluster you need to: + +1. Build the image. +2. Load it to Kubernetes cluster. +3. Deploy airflow application. + +It can be done with single script: ``./scripts/ci/in_container/kubernetes/deploy_airflow_to_kubernetes.sh`` + +You can however work separately on image in Kubernetes and the deploying the Airflow app in the cluster. + +Building Airflow Images and Loading them to Kubernetes cluster +-------------------------------------------------------------- + +This is done using ``./scripts/ci/in_container/kubernetes/docker/rebuild_airflow_image.sh`` script: + +1. Latest ``apache/airflow:master-pythonX.Y-ci`` images are rebuilt using latest sources. +3. New kubernetes image based on the ``apache/airflow:master-pythonX.Y-ci`` is build with Review comment: ```suggestion 2. New Kubernetes image based on the ``apache/airflow:master-pythonX.Y-ci`` is built with ``` ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
