nzw921rx opened a new pull request, #12930: URL: https://github.com/apache/seatunnel/pull/12930
### Purpose of this pull request This contribution is to Apache SeaTunnel. This pull request adds an experimental Kubernetes-only Zeta Session Cluster: - Two masters run as a StatefulSet and elect the active master through Hazelcast. - The active master creates bare worker Pods; there is no worker Deployment or StatefulSet. - Workers are owned by a session root ConfigMap, so they survive active-master failover and are adopted by the next active master. - Jobs are submitted through the existing REST API or native client; existing job configuration files are unchanged. - Worker images, slots, CPU, memory, heap, and JVM options use one YAML worker template. - Fixed slots are required in this release. `session.worker.slots` is validated against the worker ConfigMap. - Dynamic provisioning is opt-in and calculates `ceil(missingSlots / slotsPerWorker)` workers, capped by `max-workers`. - `min-workers` defaults to two and is created when the active master initializes. - The session worker Pod uses the existing Kubernetes ConfigMap, Secret, and environment decorator chain. - A dedicated `seatunnel-session-cluster.sh` entrypoint is added without changing `seatunnel-cluster.sh`. The deployment path is manual `kubectl apply -k`; this PR does not add a lifecycle CLI. The static manifests intentionally use fixed resource names, so one session runs per namespace. Multi-session manifests for one namespace, automatic scale-down, manual scale/spec APIs, and YARN are out of scope for this change. ### Does this PR introduce _any_ user-facing change? Yes. It adds an experimental deployment mode, a new master entrypoint, Kubernetes manifests, REST submission instructions, and English/Chinese documentation. Existing application-mode job configuration and the existing `seatunnel-cluster.sh` are not changed. ### How was this patch tested? - `./mvnw -q -pl seatunnel-engine/seatunnel-engine-common -Dtest=SessionSpecificationTest -Dskip.spotless=true test` — 3 passed. - `./mvnw -q -pl seatunnel-engine/seatunnel-engine-server -Dtest=SessionResourceManagerTest -Dskip.spotless=true test` — 3 passed. - `./mvnw -q -pl seatunnel-resource-managers/kubernetes -Dtest=KubernetesSessionConfigTest,SessionWorkerPodFactoryTest,KubernetesResourceManagerDriverTest,KubernetesApplicationResourcesTest,KubernetesApplicationTest -Dskip.spotless=true test` — 90 passed total across the five classes. - `./mvnw -q -pl seatunnel-e2e/seatunnel-resource-managers-e2e -DskipTests -Dskip.spotless=true test-compile` — passed. - `bash -n seatunnel-core/seatunnel-starter/src/main/bin/seatunnel-session-cluster.sh` — passed. - `kubectl kustomize deploy/kubernetes/session` — rendered 12 Kubernetes objects successfully. - Earlier local Docker Desktop smoke test: two masters started, the active master created the two minimum workers, and a REST-submitted `FakeSource -> Console` job reached `FINISHED`. This smoke test used the pre-final example names and was not rerun after the rename. - The new K3s E2E covers session startup, a batch job, and a pending-job worker scale-up. It has been compiled but was not executed in this environment. - A full `seatunnel-engine-server` test run executed 618 tests and hit one unrelated timing failure in `SavePointTest`. The isolated rerun passed all 6 selected tests. ### Check list * [x] No new Jar binary package is added. * [x] Documentation is updated in `docs/en` and `docs/zh`. * [x] Incompatible changes are documented in the Session Cluster overview; existing modes and job configuration remain compatible. * [ ] Connector-specific checklist is not applicable to this Zeta deployment feature. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
