DanielLeens opened a new issue, #11663:
URL: https://github.com/apache/seatunnel/issues/11663
### Search before asking
- [x] I had searched in the feature issues and found no similar feature
requirement.
### Description
SeaTunnel Engine already has the core building blocks for resource-aware
scheduling:
- `slot-service.dynamic-slot` controls whether the engine uses dynamic slots.
- `slot-service.slot-allocate-strategy` supports `RANDOM`, `SLOT_RATIO`, and
`SYSTEM_LOAD`.
- `SYSTEM_LOAD` can use heartbeat-reported CPU and memory utilization to
prefer lower-load workers during slot allocation.
- Engine telemetry can expose Prometheus/OpenMetrics data from
`/hazelcast/rest/instance/metrics`.
- Job-level throughput metrics such as source/sink QPS and bytes-per-second
are available through REST API v2 `/job-info/:jobId`.
However, SeaTunnel does not currently provide a built-in autoscaling control
loop similar to Flink Autoscaler. In production, users who run Zeta on
Kubernetes can only build their own scaling policy through external HPA/custom
metrics, and users outside Kubernetes do not have a standard SeaTunnel-native
scaling mechanism.
It would be useful to add a first-class Zeta autoscaling design and
implementation that can evaluate cluster load, slot pressure, pending jobs, and
job throughput metrics, then recommend or trigger safe worker/slot capacity
changes.
Possible scope:
1. Add an autoscaler module for SeaTunnel Engine / Zeta.
2. Define autoscaler inputs, for example:
- worker CPU and memory utilization,
- slot usage and pending slot requests,
- pending job diagnostics,
- job-level source/sink QPS and BPS,
- queue/backpressure or throughput trend metrics when available.
3. Define scaling decisions:
- scale out when pending jobs or slot pressure remain high for a
configurable stabilization window,
- scale in only when workers stay underutilized and no running job would
be disrupted,
- avoid oscillation through cooldown, min/max workers, and hysteresis.
4. Expose autoscaler state and decisions through metrics and REST API, so
users can observe why a scaling action was or was not taken.
5. Provide a Kubernetes integration path. For example, SeaTunnel could
initially expose autoscaling recommendations/custom metrics that HPA or an
operator can consume, and later support a controller/operator-managed
implementation.
6. Keep backward compatibility: the feature should be disabled by default
and should not change current scheduling behavior unless explicitly enabled.
### Usage Scenario
A typical streaming cluster may have different workload patterns during the
day:
- During peak hours, source QPS/BPS increases, pending slot requests
accumulate, or sink throughput becomes the bottleneck.
- During low-traffic hours, workers remain idle for a long period.
Today, users must manually adjust worker replicas or maintain an external
HPA policy without a SeaTunnel-aware view of job topology, slots, and
throughput. A built-in autoscaler would make Zeta easier to run in production
because scaling decisions could be based on SeaTunnel's own resource and job
metrics instead of only generic CPU/memory usage.
Expected benefits:
- Better utilization of Zeta worker resources.
- Faster recovery from workload spikes or accumulated pending jobs.
- Lower operational cost during idle periods.
- A standard scaling model for Kubernetes and non-Kubernetes deployments.
- Clear observability of scaling decisions for operations teams.
### Related issues
No exact duplicate was found during the issue search. This is related to the
existing Zeta slot/resource management and telemetry areas, including dynamic
slots, slot allocation strategy, Prometheus/OpenMetrics telemetry, and REST API
v2 job information.
### Are you willing to submit a PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's Code of Conduct.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
