This is an automated email from the ASF dual-hosted git repository.
potiuk pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git
The following commit(s) were added to refs/heads/main by this push:
new a5e5e1a74be Reduce K8s test resource usage with single-node Kind
clusters (#73998)
a5e5e1a74be is described below
commit a5e5e1a74be25f18629a346f85028fde27439b66
Author: Aaron Chen <[email protected]>
AuthorDate: Sun Oct 18 11:21:09 2026 -0700
Reduce K8s test resource usage with single-node Kind clusters (#73998)
* Reduce K8s test resource usage with single-node Kind clusters
* Add test for forwarded node port matching Kind config and NodePort service
* Refactor Kubernetes configuration and tests to use centralized port
rendering
---
.github/workflows/kustomize-overlays-tests.yml | 6 +-
contributing-docs/testing/k8s_tests.rst | 9 +--
dev/breeze/doc/05_test_commands.rst | 2 +-
.../src/airflow_breeze/utils/kubernetes_utils.py | 34 +++++++--
dev/breeze/tests/test_kubernetes_utils.py | 86 ++++++++++++++++++++++
scripts/ci/kubernetes/kind-cluster-conf.yaml | 5 +-
6 files changed, 123 insertions(+), 19 deletions(-)
diff --git a/.github/workflows/kustomize-overlays-tests.yml
b/.github/workflows/kustomize-overlays-tests.yml
index 94637194bfa..1cca2b85000 100644
--- a/.github/workflows/kustomize-overlays-tests.yml
+++ b/.github/workflows/kustomize-overlays-tests.yml
@@ -102,13 +102,13 @@ jobs:
# convention (sources come from the base
# image, not the workspace).
# 5. upload-k8s-image -> loads the now-built kubernetes-variant
- # image into every kind node, so kubelet
+ # image into the kind node, so kubelet
# can resolve the chart's image reference
# without hitting ghcr (which is private).
# 6. deploy-airflow
# 7. smoke-test-overlay -> renders the overlay, auto-preloads the
# overlay's own images into the kind
- # nodes (no per-overlay images list
+ # node (no per-overlay images list
# needed - discovery walks the rendered
# manifest), applies, walks the
# STATUS.yaml verify: block with
@@ -124,7 +124,7 @@ jobs:
run: breeze k8s configure-cluster
- name: "Build kubernetes-variant image on top of the restored PROD
image"
run: breeze k8s build-k8s-image --no-copy-local-sources
- - name: "Upload kubernetes-variant image to kind nodes"
+ - name: "Upload kubernetes-variant image to the kind node"
run: breeze k8s upload-k8s-image
- name: "Deploy Airflow chart"
run: breeze k8s deploy-airflow --executor CeleryExecutor
diff --git a/contributing-docs/testing/k8s_tests.rst
b/contributing-docs/testing/k8s_tests.rst
index f0b121bcee7..3a6afc11221 100644
--- a/contributing-docs/testing/k8s_tests.rst
+++ b/contributing-docs/testing/k8s_tests.rst
@@ -281,7 +281,6 @@ The typical session for tests with Kubernetes looks like
follows:
apiServerPort: 48366
nodes:
- role: control-plane
- - role: worker
extraPortMappings:
- containerPort: 30007
hostPort: 18150
@@ -290,12 +289,11 @@ The typical session for tests with Kubernetes looks like
follows:
Creating cluster "airflow-python-3.11-v1.24.2" ...
â Ensuring node image (kindest/node:v1.24.2) đŧ
- â Preparing nodes đĻ đĻ
+ â Preparing nodes đĻ
â Writing configuration đ
â Starting control-plane đšī¸
â Installing CNI đ
â Installing StorageClass đž
- â Joining worker nodes đ
Set kubectl context to "kind-airflow-python-3.11-v1.24.2"
You can now use your cluster with:
@@ -377,13 +375,11 @@ The typical session for tests with Kubernetes looks like
follows:
coredns-6d4b75cb6d-rwp9d 1/1
Running 0 71s
coredns-6d4b75cb6d-vqnrc 1/1
Running 0 71s
etcd-airflow-python-3.11-v1.24.2-control-plane 1/1
Running 0 84s
- kindnet-ckc8l 1/1
Running 0 69s
kindnet-qqt8k 1/1
Running 0 71s
kube-apiserver-airflow-python-3.11-v1.24.2-control-plane 1/1
Running 0 84s
kube-controller-manager-airflow-python-3.11-v1.24.2-control-plane 1/1
Running 0 84s
- kube-proxy-6g7hn 1/1
Running 0 69s
kube-proxy-dwfvp 1/1
Running 0 71s
- kube-scheduler-airflow-python-3.11-v1.24.2-control-plane 1/1
Running 0 84s
+ kube-scheduler-airflow-python-3.11-v1.24.2-control-plane 1/1
Running 0 84s
KinD Cluster API server URL: http://localhost:48366
Connecting to localhost:18150. Num try: 1
@@ -446,7 +442,6 @@ The typical session for tests with Kubernetes looks like
follows:
Good version of helm installed: 3.9.2 in
/Users/jarek/IdeaProjects/airflow/kubernetes-tests/.venv/bin
Stable repo is already added
Uploading Airflow image
ghcr.io/apache/airflow/main/prod/python3.11-kubernetes to cluster
airflow-python-3.11-v1.24.2
- Image: "ghcr.io/apache/airflow/main/prod/python3.11-kubernetes" with ID
"sha256:fb6195f7c2c2ad97788a563a3fe9420bf3576c85575378d642cd7985aff97412" not
yet present on node "airflow-python-3.11-v1.24.2-worker", loading...
Image: "ghcr.io/apache/airflow/main/prod/python3.11-kubernetes" with ID
"sha256:fb6195f7c2c2ad97788a563a3fe9420bf3576c85575378d642cd7985aff97412" not
yet present on node "airflow-python-3.11-v1.24.2-control-plane", loading...
NEXT STEP: You might now deploy Airflow by:
diff --git a/dev/breeze/doc/05_test_commands.rst
b/dev/breeze/doc/05_test_commands.rst
index 980caf471f0..84b1edcb590 100644
--- a/dev/breeze/doc/05_test_commands.rst
+++ b/dev/breeze/doc/05_test_commands.rst
@@ -703,7 +703,7 @@ The runner is overlay-agnostic. For every overlay it:
* renders the overlay and substitutes ``RELEASE-NAME`` / ``NAMESPACE``,
* **auto-preloads every ``image:`` referenced by the rendered manifest**
- into the kind nodes via ``docker pull`` (with retry on Docker Hub
+ into the kind node via ``docker pull`` (with retry on Docker Hub
rate limits) + ``kind load docker-image``, so the test does not flake
on registry availability,
* applies the overlay,
diff --git a/dev/breeze/src/airflow_breeze/utils/kubernetes_utils.py
b/dev/breeze/src/airflow_breeze/utils/kubernetes_utils.py
index 9b9eb299bdb..d6e9c343b9b 100644
--- a/dev/breeze/src/airflow_breeze/utils/kubernetes_utils.py
+++ b/dev/breeze/src/airflow_breeze/utils/kubernetes_utils.py
@@ -379,6 +379,20 @@ def _get_kind_cluster_config_content(python: str,
kubernetes_version: str) -> di
return yaml.safe_load(config_path.read_text())
+# Must match nodePort in scripts/ci/kubernetes/nodeport.yaml.
+FORWARDED_NODE_PORT = 30007
+
+
+def _render_kind_cluster_config(forwarded_port_number: int,
k8s_api_server_port: int) -> str:
+ return (
+ (SCRIPTS_CI_KUBERNETES_PATH / "kind-cluster-conf.yaml")
+ .read_text()
+ .replace("{{FORWARDED_NODE_PORT}}", str(FORWARDED_NODE_PORT))
+ .replace("{{FORWARDED_PORT_NUMBER}}", str(forwarded_port_number))
+ .replace("{{API_SERVER_PORT}}", str(k8s_api_server_port))
+ )
+
+
def set_random_cluster_ports(python: str, kubernetes_version: str, output:
Output | None) -> None:
"""
Creates cluster config file and returns sockets keeping the ports bound.
@@ -390,11 +404,8 @@ def set_random_cluster_ports(python: str,
kubernetes_version: str, output: Outpu
f"[info]Random ports: K8S API: {k8s_api_server_port}, API Server:
{forwarded_port_number}"
)
cluster_conf_path = get_kind_cluster_config_path(python=python,
kubernetes_version=kubernetes_version)
- config = (
- (AIRFLOW_ROOT_PATH / "scripts" / "ci" / "kubernetes" /
"kind-cluster-conf.yaml")
- .read_text()
- .replace("{{FORWARDED_PORT_NUMBER}}", str(forwarded_port_number))
- .replace("{{API_SERVER_PORT}}", str(k8s_api_server_port))
+ config = _render_kind_cluster_config(
+ forwarded_port_number=forwarded_port_number,
k8s_api_server_port=k8s_api_server_port
)
cluster_conf_path.write_text(config)
get_console(output=output).print(f"[info]Config created in
{cluster_conf_path}:\n")
@@ -402,12 +413,23 @@ def set_random_cluster_ports(python: str,
kubernetes_version: str, output: Outpu
get_console(output=output).print("\n")
+def _extract_forwarded_host_port(conf: dict[str, Any]) -> int:
+ for node in conf["nodes"]:
+ for port_mapping in node.get("extraPortMappings", []):
+ if port_mapping["containerPort"] == FORWARDED_NODE_PORT:
+ return port_mapping["hostPort"]
+ raise ValueError(
+ f"No node in the kind cluster config has an extraPortMappings entry
for "
+ f"containerPort {FORWARDED_NODE_PORT}. Delete and recreate the
cluster."
+ )
+
+
def get_kubernetes_port_numbers(python: str, kubernetes_version: str) ->
tuple[int, int]:
conf = _get_kind_cluster_config_content(python=python,
kubernetes_version=kubernetes_version)
if not conf:
return 0, 0
k8s_api_server_port = conf["networking"]["apiServerPort"]
- api_server_port = conf["nodes"][1]["extraPortMappings"][0]["hostPort"]
+ api_server_port = _extract_forwarded_host_port(conf)
return k8s_api_server_port, api_server_port
diff --git a/dev/breeze/tests/test_kubernetes_utils.py
b/dev/breeze/tests/test_kubernetes_utils.py
new file mode 100644
index 00000000000..88dee2d715f
--- /dev/null
+++ b/dev/breeze/tests/test_kubernetes_utils.py
@@ -0,0 +1,86 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+from __future__ import annotations
+
+from unittest.mock import patch
+
+import pytest
+import yaml
+
+from airflow_breeze.utils.kubernetes_utils import (
+ FORWARDED_NODE_PORT,
+ SCRIPTS_CI_KUBERNETES_PATH,
+ _render_kind_cluster_config,
+ get_kubernetes_port_numbers,
+)
+
+# Rendered config of a cluster created before the switch to single-node
clusters â
+# port extraction must keep working for clusters that already exist on disk.
+LEGACY_TWO_NODE_CONFIG = """\
+kind: Cluster
+apiVersion: kind.x-k8s.io/v1alpha4
+networking:
+ ipFamily: ipv4
+ apiServerAddress: "127.0.0.1"
+ apiServerPort: 48366
+nodes:
+ - role: control-plane
+ - role: worker
+ extraPortMappings:
+ - containerPort: 30007
+ hostPort: 18150
+ listenAddress: "127.0.0.1"
+ protocol: TCP
+"""
+
+NO_FORWARDED_PORT_CONFIG = """\
+kind: Cluster
+apiVersion: kind.x-k8s.io/v1alpha4
+networking:
+ ipFamily: ipv4
+ apiServerAddress: "127.0.0.1"
+ apiServerPort: 48366
+nodes:
+ - role: control-plane
+"""
+
+
[email protected](
+ "config",
+ [
+ _render_kind_cluster_config(forwarded_port_number=18150,
k8s_api_server_port=48366),
+ LEGACY_TWO_NODE_CONFIG,
+ ],
+ ids=["current-template", "legacy-two-node"],
+)
+@patch("airflow_breeze.utils.kubernetes_utils._get_kind_cluster_config_content",
autospec=True)
+def
test_get_kubernetes_port_numbers_extracts_ports_from_any_node(mock_get_content,
config):
+ mock_get_content.return_value = yaml.safe_load(config)
+ assert get_kubernetes_port_numbers(python="3.10",
kubernetes_version="v1.30.13") == (48366, 18150)
+
+
+@patch("airflow_breeze.utils.kubernetes_utils._get_kind_cluster_config_content",
autospec=True)
+def
test_get_kubernetes_port_numbers_raises_when_forwarded_port_mapping_is_missing(mock_get_content):
+ mock_get_content.return_value = yaml.safe_load(NO_FORWARDED_PORT_CONFIG)
+ with pytest.raises(ValueError, match="extraPortMappings"):
+ get_kubernetes_port_numbers(python="3.10",
kubernetes_version="v1.30.13")
+
+
+def test_nodeport_service_exposes_forwarded_node_port():
+ node_port_service = yaml.safe_load((SCRIPTS_CI_KUBERNETES_PATH /
"nodeport.yaml").read_text())
+ assert [port["nodePort"] for port in node_port_service["spec"]["ports"]]
== [FORWARDED_NODE_PORT]
diff --git a/scripts/ci/kubernetes/kind-cluster-conf.yaml
b/scripts/ci/kubernetes/kind-cluster-conf.yaml
index 4e891f80ddf..d9e93e9646b 100644
--- a/scripts/ci/kubernetes/kind-cluster-conf.yaml
+++ b/scripts/ci/kubernetes/kind-cluster-conf.yaml
@@ -22,10 +22,11 @@ networking:
apiServerAddress: "127.0.0.1"
apiServerPort: {{API_SERVER_PORT}}
nodes:
+ # A single node on purpose: kind drops the control-plane NoSchedule taint
only for single-node
+ # clusters, so the Airflow and KubernetesPodOperator pods are scheduled here.
- role: control-plane
- - role: worker
extraPortMappings:
- - containerPort: 30007
+ - containerPort: {{FORWARDED_NODE_PORT}}
hostPort: {{FORWARDED_PORT_NUMBER}}
listenAddress: "127.0.0.1"
protocol: TCP
