This is an automated email from the ASF dual-hosted git repository. gyfora pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/flink-kubernetes-operator.git
commit b1c40e3578120d5b892a6c80fe0dba8ebca58b6a Author: 1996fanrui <[email protected]> AuthorDate: Wed Sep 27 21:06:25 2023 +0800 [FLINK-33098][autoscaler] Removing the operator prefix for all autoscaler options --- docs/content/docs/custom-resource/autoscaler.md | 28 +++++++------- docs/content/docs/operations/configuration.md | 2 + .../generated/auto_scaler_configuration.html | 44 +++++++++++----------- e2e-tests/data/autoscaler.yaml | 8 ++-- examples/autoscaling/autoscaling-dynamic.yaml | 10 ++--- examples/autoscaling/autoscaling.yaml | 8 ++-- .../flink/autoscaler/config/AutoScalerOptions.java | 29 ++++++++++++-- 7 files changed, 77 insertions(+), 52 deletions(-) diff --git a/docs/content/docs/custom-resource/autoscaler.md b/docs/content/docs/custom-resource/autoscaler.md index 4140122a..7f67c1a3 100644 --- a/docs/content/docs/custom-resource/autoscaler.md +++ b/docs/content/docs/custom-resource/autoscaler.md @@ -123,7 +123,7 @@ The defaults might work reasonably well for many applications, but some tuning m The autoscaler also supports a passive/metrics-only mode where it only collects and evaluates scaling related performance metrics but does not trigger any job upgrades. This can be used to gain confidence in the module without any impact on the running applications. -To disable scaling actions, set: `kubernetes.operator.job.autoscaler.scaling.enabled: "false"` +To disable scaling actions, set: `job.autoscaler.scaling.enabled: "false"` {{< /hint >}} ### Job and per operator max parallelism @@ -140,30 +140,30 @@ It is also possible to set maxParallelism on a per operator level, which can be ### Stabilization and metrics collection intervals -The autoscaler always looks at average metrics in the collection time window defined by `kubernetes.operator.job.autoscaler.metrics.window`. +The autoscaler always looks at average metrics in the collection time window defined by `job.autoscaler.metrics.window`. The size of this window determines how small fluctuations will affect the autoscaler. The larger the window, the more smoothing and stability we get, but we may be slower to react to sudden load changes. We suggest you experiment with setting this anywhere between 3-60 minutes for best experience. -To allow jobs to stabilize after recovery users can configure a stabilization window by setting `kubernetes.operator.job.autoscaler.stabilization.interval`. +To allow jobs to stabilize after recovery users can configure a stabilization window by setting `job.autoscaler.stabilization.interval`. During this time period no metrics will be collected and no scaling actions will be taken. ### Target utilization and flexible boundaries -In order to provide stable job performance and some buffer for load fluctuations, the autoscaler allows users to set a target utilization level for the job (`kubernetes.operator.job.autoscaler.target.utilization`). +In order to provide stable job performance and some buffer for load fluctuations, the autoscaler allows users to set a target utilization level for the job (`job.autoscaler.target.utilization`). A target of `0.6` means we are targeting 60% utilization/load for the job vertexes. In general, it's not recommended to set target utilization close to 100% as performance usually degrades as we reach capacity limits in most real world systems. In addition to the utilization target we can set a utilization boundary, that serves as extra buffer to avoid immediate scaling on load fluctuations. -Setting `kubernetes.operator.job.autoscaler.target.utilization.boundary: "0.2"` means that we allow 20% deviation from the target utilization before triggering a scaling action. +Setting `job.autoscaler.target.utilization.boundary: "0.2"` means that we allow 20% deviation from the target utilization before triggering a scaling action. ### Target catch-up duration and restart time When taking scaling decisions the operator need to account for the extra capacity required to catch up the backlog created during scaling operations. The amount of extra capacity is determined automatically by the following 2 configs: - - `kubernetes.operator.job.autoscaler.restart.time` : Time it usually takes to restart the application - - `kubernetes.operator.job.autoscaler.catch-up.duration` : Time to job is expected to catch up after scaling + - `job.autoscaler.restart.time` : Time it usually takes to restart the application + - `job.autoscaler.catch-up.duration` : Time to job is expected to catch up after scaling In the future the autoscaler may be able to automatically determine the restart time, but the target catch-up duration depends on the users SLO. @@ -175,13 +175,13 @@ We suggest setting this based on your actual objective, such us 10,30,60 minutes ... flinkVersion: v1_17 flinkConfiguration: - kubernetes.operator.job.autoscaler.enabled: "true" - kubernetes.operator.job.autoscaler.stabilization.interval: 1m - kubernetes.operator.job.autoscaler.metrics.window: 5m - kubernetes.operator.job.autoscaler.target.utilization: "0.6" - kubernetes.operator.job.autoscaler.target.utilization.boundary: "0.2" - kubernetes.operator.job.autoscaler.restart.time: 2m - kubernetes.operator.job.autoscaler.catch-up.duration: 5m + job.autoscaler.enabled: "true" + job.autoscaler.stabilization.interval: 1m + job.autoscaler.metrics.window: 5m + job.autoscaler.target.utilization: "0.6" + job.autoscaler.target.utilization.boundary: "0.2" + job.autoscaler.restart.time: 2m + job.autoscaler.catch-up.duration: 5m pipeline.max-parallelism: "720" ``` diff --git a/docs/content/docs/operations/configuration.md b/docs/content/docs/operations/configuration.md index 48ad892b..7343faa8 100644 --- a/docs/content/docs/operations/configuration.md +++ b/docs/content/docs/operations/configuration.md @@ -130,6 +130,8 @@ These options can be configured on both an operator and a per-resource level. Wh Like other resource options these can be configured on both an operator and a per-resource level. When set under `spec.flinkConfiguration` for the Flink resources it will override the default value provided in the operator default configuration (`flink-conf.yaml`). +> Note: The option prefix `kubernetes.operator.` was removed in FLIP-334, because the autoscaler module was decoupled from flink-kubernetes-operator. + {{< generated/auto_scaler_configuration >}} ### System Metrics Configuration diff --git a/docs/layouts/shortcodes/generated/auto_scaler_configuration.html b/docs/layouts/shortcodes/generated/auto_scaler_configuration.html index bc35f57d..0b2d5b35 100644 --- a/docs/layouts/shortcodes/generated/auto_scaler_configuration.html +++ b/docs/layouts/shortcodes/generated/auto_scaler_configuration.html @@ -9,133 +9,133 @@ </thead> <tbody> <tr> - <td><h5>kubernetes.operator.job.autoscaler.backlog-processing.lag-threshold</h5></td> + <td><h5>job.autoscaler.backlog-processing.lag-threshold</h5></td> <td style="word-wrap: break-word;">5 min</td> <td>Duration</td> <td>Lag threshold which will prevent unnecessary scalings while removing the pending messages responsible for the lag.</td> </tr> <tr> - <td><h5>kubernetes.operator.job.autoscaler.catch-up.duration</h5></td> + <td><h5>job.autoscaler.catch-up.duration</h5></td> <td style="word-wrap: break-word;">15 min</td> <td>Duration</td> <td>The target duration for fully processing any backlog after a scaling operation. Set to 0 to disable backlog based scaling.</td> </tr> <tr> - <td><h5>kubernetes.operator.job.autoscaler.enabled</h5></td> + <td><h5>job.autoscaler.enabled</h5></td> <td style="word-wrap: break-word;">false</td> <td>Boolean</td> <td>Enable job autoscaler module.</td> </tr> <tr> - <td><h5>kubernetes.operator.job.autoscaler.flink.client.timeout</h5></td> + <td><h5>job.autoscaler.flink.rest-client.timeout</h5></td> <td style="word-wrap: break-word;">10 s</td> <td>Duration</td> <td>The timeout for waiting the flink rest client to return.</td> </tr> <tr> - <td><h5>kubernetes.operator.job.autoscaler.history.max.age</h5></td> + <td><h5>job.autoscaler.history.max.age</h5></td> <td style="word-wrap: break-word;">86400000 ms</td> <td>Duration</td> <td>Maximum age for past scaling decisions to retain.</td> </tr> <tr> - <td><h5>kubernetes.operator.job.autoscaler.history.max.count</h5></td> + <td><h5>job.autoscaler.history.max.count</h5></td> <td style="word-wrap: break-word;">3</td> <td>Integer</td> <td>Maximum number of past scaling decisions to retain per vertex.</td> </tr> <tr> - <td><h5>kubernetes.operator.job.autoscaler.metrics.busy-time.aggregator</h5></td> + <td><h5>job.autoscaler.metrics.busy-time.aggregator</h5></td> <td style="word-wrap: break-word;">MAX</td> <td><p>Enum</p></td> <td>Metric aggregator to use for busyTime metrics. This affects how true processing/output rate will be computed. Using max allows us to handle jobs with data skew more robustly, while avg may provide better stability when we know that the load distribution is even.<br /><br />Possible values:<ul><li>"AVG"</li><li>"MAX"</li><li>"MIN"</li></ul></td> </tr> <tr> - <td><h5>kubernetes.operator.job.autoscaler.metrics.window</h5></td> + <td><h5>job.autoscaler.metrics.window</h5></td> <td style="word-wrap: break-word;">10 min</td> <td>Duration</td> <td>Scaling metrics aggregation window size.</td> </tr> <tr> - <td><h5>kubernetes.operator.job.autoscaler.restart.time</h5></td> + <td><h5>job.autoscaler.restart.time</h5></td> <td style="word-wrap: break-word;">3 min</td> <td>Duration</td> <td>Expected restart time to be used until the operator can determine it reliably from history.</td> </tr> <tr> - <td><h5>kubernetes.operator.job.autoscaler.scale-down.max-factor</h5></td> + <td><h5>job.autoscaler.scale-down.max-factor</h5></td> <td style="word-wrap: break-word;">0.6</td> <td>Double</td> <td>Max scale down factor. 1 means no limit on scale down, 0.6 means job can only be scaled down with 60% of the original parallelism.</td> </tr> <tr> - <td><h5>kubernetes.operator.job.autoscaler.scale-up.grace-period</h5></td> + <td><h5>job.autoscaler.scale-up.grace-period</h5></td> <td style="word-wrap: break-word;">1 h</td> <td>Duration</td> <td>Duration in which no scale down of a vertex is allowed after it has been scaled up.</td> </tr> <tr> - <td><h5>kubernetes.operator.job.autoscaler.scale-up.max-factor</h5></td> + <td><h5>job.autoscaler.scale-up.max-factor</h5></td> <td style="word-wrap: break-word;">100000.0</td> <td>Double</td> <td>Max scale up factor. 2.0 means job can only be scaled up with 200% of the current parallelism.</td> </tr> <tr> - <td><h5>kubernetes.operator.job.autoscaler.scaling.effectiveness.detection.enabled</h5></td> + <td><h5>job.autoscaler.scaling.effectiveness.detection.enabled</h5></td> <td style="word-wrap: break-word;">false</td> <td>Boolean</td> <td>Whether to enable detection of ineffective scaling operations and allowing the autoscaler to block further scale ups.</td> </tr> <tr> - <td><h5>kubernetes.operator.job.autoscaler.scaling.effectiveness.threshold</h5></td> + <td><h5>job.autoscaler.scaling.effectiveness.threshold</h5></td> <td style="word-wrap: break-word;">0.1</td> <td>Double</td> <td>Processing rate increase threshold for detecting ineffective scaling threshold. 0.1 means if we do not accomplish at least 10% of the desired capacity increase with scaling, the action is marked ineffective.</td> </tr> <tr> - <td><h5>kubernetes.operator.job.autoscaler.scaling.enabled</h5></td> + <td><h5>job.autoscaler.scaling.enabled</h5></td> <td style="word-wrap: break-word;">true</td> <td>Boolean</td> <td>Enable vertex scaling execution by the autoscaler. If disabled, the autoscaler will only collect metrics and evaluate the suggested parallelism for each vertex but will not upgrade the jobs.</td> </tr> <tr> - <td><h5>kubernetes.operator.job.autoscaler.scaling.report.interval</h5></td> + <td><h5>job.autoscaler.scaling.report.interval</h5></td> <td style="word-wrap: break-word;">30 min</td> <td>Duration</td> <td>Time interval to resend the identical event</td> </tr> <tr> - <td><h5>kubernetes.operator.job.autoscaler.stabilization.interval</h5></td> + <td><h5>job.autoscaler.stabilization.interval</h5></td> <td style="word-wrap: break-word;">5 min</td> <td>Duration</td> <td>Stabilization period in which no new scaling will be executed</td> </tr> <tr> - <td><h5>kubernetes.operator.job.autoscaler.target.utilization</h5></td> + <td><h5>job.autoscaler.target.utilization</h5></td> <td style="word-wrap: break-word;">0.7</td> <td>Double</td> <td>Target vertex utilization</td> </tr> <tr> - <td><h5>kubernetes.operator.job.autoscaler.target.utilization.boundary</h5></td> + <td><h5>job.autoscaler.target.utilization.boundary</h5></td> <td style="word-wrap: break-word;">0.4</td> <td>Double</td> <td>Target vertex utilization boundary. Scaling won't be performed if the current processing rate is within [target_rate / (target_utilization - boundary), (target_rate / (target_utilization + boundary)]</td> </tr> <tr> - <td><h5>kubernetes.operator.job.autoscaler.vertex.exclude.ids</h5></td> + <td><h5>job.autoscaler.vertex.exclude.ids</h5></td> <td style="word-wrap: break-word;"></td> <td>List<String></td> <td>A (semicolon-separated) list of vertex ids in hexstring for which to disable scaling. Caution: For non-sink vertices this will still scale their downstream operators until https://issues.apache.org/jira/browse/FLINK-31215 is implemented.</td> </tr> <tr> - <td><h5>kubernetes.operator.job.autoscaler.vertex.max-parallelism</h5></td> + <td><h5>job.autoscaler.vertex.max-parallelism</h5></td> <td style="word-wrap: break-word;">200</td> <td>Integer</td> <td>The maximum parallelism the autoscaler can use. Note that this limit will be ignored if it is higher than the max parallelism configured in the Flink config or directly on each operator.</td> </tr> <tr> - <td><h5>kubernetes.operator.job.autoscaler.vertex.min-parallelism</h5></td> + <td><h5>job.autoscaler.vertex.min-parallelism</h5></td> <td style="word-wrap: break-word;">1</td> <td>Integer</td> <td>The minimum parallelism the autoscaler can use.</td> diff --git a/e2e-tests/data/autoscaler.yaml b/e2e-tests/data/autoscaler.yaml index ce41f325..6f53bbe9 100644 --- a/e2e-tests/data/autoscaler.yaml +++ b/e2e-tests/data/autoscaler.yaml @@ -36,10 +36,10 @@ spec: state.checkpoints.dir: file:///opt/flink/volume/flink-cp state.savepoints.dir: file:///opt/flink/volume/flink-sp - kubernetes.operator.job.autoscaler.enabled: "true" - kubernetes.operator.job.autoscaler.scaling.enabled: "true" - kubernetes.operator.job.autoscaler.stabilization.interval: "5s" - kubernetes.operator.job.autoscaler.metrics.window: "1m" + job.autoscaler.enabled: "true" + job.autoscaler.scaling.enabled: "true" + job.autoscaler.stabilization.interval: "5s" + job.autoscaler.metrics.window: "1m" jobmanager.scheduler: adaptive serviceAccount: flink diff --git a/examples/autoscaling/autoscaling-dynamic.yaml b/examples/autoscaling/autoscaling-dynamic.yaml index 92d2d159..b6bb3d04 100644 --- a/examples/autoscaling/autoscaling-dynamic.yaml +++ b/examples/autoscaling/autoscaling-dynamic.yaml @@ -24,11 +24,11 @@ spec: image: autoscaling-example flinkVersion: v1_18 flinkConfiguration: - kubernetes.operator.job.autoscaler.enabled: "true" - kubernetes.operator.job.autoscaler.stabilization.interval: "1m" - kubernetes.operator.job.autoscaler.metrics.window: "15m" - kubernetes.operator.job.autoscaler.target.utilization: "0.5" - kubernetes.operator.job.autoscaler.target.utilization.boundary: "0.3" + job.autoscaler.enabled: "true" + job.autoscaler.stabilization.interval: "1m" + job.autoscaler.metrics.window: "15m" + job.autoscaler.target.utilization: "0.5" + job.autoscaler.target.utilization.boundary: "0.3" pipeline.max-parallelism: "32" taskmanager.numberOfTaskSlots: "4" serviceAccount: flink diff --git a/examples/autoscaling/autoscaling.yaml b/examples/autoscaling/autoscaling.yaml index 26edaaad..a973ccd3 100644 --- a/examples/autoscaling/autoscaling.yaml +++ b/examples/autoscaling/autoscaling.yaml @@ -24,9 +24,9 @@ spec: image: autoscaling-example flinkVersion: v1_18 flinkConfiguration: - kubernetes.operator.job.autoscaler.enabled: "true" - kubernetes.operator.job.autoscaler.stabilization.interval: "1m" - kubernetes.operator.job.autoscaler.metrics.window: "3m" + job.autoscaler.enabled: "true" + job.autoscaler.stabilization.interval: "1m" + job.autoscaler.metrics.window: "3m" pipeline.max-parallelism: "24" taskmanager.numberOfTaskSlots: "4" state.savepoints.dir: file:///flink-data/savepoints @@ -35,7 +35,7 @@ spec: high-availability.storageDir: file:///flink-data/ha execution.checkpointing.interval: "1m" jobmanager.scheduler: adaptive - kubernetes.operator.job.autoscaler.target.utilization.boundary: "0.1" + job.autoscaler.target.utilization.boundary: "0.1" serviceAccount: flink jobManager: resource: diff --git a/flink-autoscaler/src/main/java/org/apache/flink/autoscaler/config/AutoScalerOptions.java b/flink-autoscaler/src/main/java/org/apache/flink/autoscaler/config/AutoScalerOptions.java index f22ea2f0..6c74b5d2 100644 --- a/flink-autoscaler/src/main/java/org/apache/flink/autoscaler/config/AutoScalerOptions.java +++ b/flink-autoscaler/src/main/java/org/apache/flink/autoscaler/config/AutoScalerOptions.java @@ -30,12 +30,12 @@ public class AutoScalerOptions { public static final String DEPRECATED_K8S_OP_CONF_PREFIX = "kubernetes.operator."; public static final String AUTOSCALER_CONF_PREFIX = "job.autoscaler."; - private static String operatorConfigKey(String key) { - return DEPRECATED_K8S_OP_CONF_PREFIX + key; + private static String deprecatedOperatorConfigKey(String key) { + return DEPRECATED_K8S_OP_CONF_PREFIX + AUTOSCALER_CONF_PREFIX + key; } private static String autoScalerConfigKey(String key) { - return operatorConfigKey(AUTOSCALER_CONF_PREFIX + key); + return AUTOSCALER_CONF_PREFIX + key; } private static ConfigOptions.OptionBuilder autoScalerConfig(String key) { @@ -46,12 +46,14 @@ public class AutoScalerOptions { autoScalerConfig("enabled") .booleanType() .defaultValue(false) + .withDeprecatedKeys(deprecatedOperatorConfigKey("enabled")) .withDescription("Enable job autoscaler module."); public static final ConfigOption<Boolean> SCALING_ENABLED = autoScalerConfig("scaling.enabled") .booleanType() .defaultValue(true) + .withDeprecatedKeys(deprecatedOperatorConfigKey("scaling.enabled")) .withDescription( "Enable vertex scaling execution by the autoscaler. If disabled, the autoscaler will only collect metrics and evaluate the suggested parallelism for each vertex but will not upgrade the jobs."); @@ -59,12 +61,14 @@ public class AutoScalerOptions { autoScalerConfig("metrics.window") .durationType() .defaultValue(Duration.ofMinutes(10)) + .withDeprecatedKeys(deprecatedOperatorConfigKey("metrics.window")) .withDescription("Scaling metrics aggregation window size."); public static final ConfigOption<Duration> STABILIZATION_INTERVAL = autoScalerConfig("stabilization.interval") .durationType() .defaultValue(Duration.ofMinutes(5)) + .withDeprecatedKeys(deprecatedOperatorConfigKey("stabilization.interval")) .withDescription( "Stabilization period in which no new scaling will be executed"); @@ -72,12 +76,14 @@ public class AutoScalerOptions { autoScalerConfig("target.utilization") .doubleType() .defaultValue(0.7) + .withDeprecatedKeys(deprecatedOperatorConfigKey("target.utilization")) .withDescription("Target vertex utilization"); public static final ConfigOption<Double> TARGET_UTILIZATION_BOUNDARY = autoScalerConfig("target.utilization.boundary") .doubleType() .defaultValue(0.4) + .withDeprecatedKeys(deprecatedOperatorConfigKey("target.utilization.boundary")) .withDescription( "Target vertex utilization boundary. Scaling won't be performed if the current processing rate is within [target_rate / (target_utilization - boundary), (target_rate / (target_utilization + boundary)]"); @@ -85,6 +91,7 @@ public class AutoScalerOptions { autoScalerConfig("scale-up.grace-period") .durationType() .defaultValue(Duration.ofHours(1)) + .withDeprecatedKeys(deprecatedOperatorConfigKey("scale-up.grace-period")) .withDescription( "Duration in which no scale down of a vertex is allowed after it has been scaled up."); @@ -92,12 +99,14 @@ public class AutoScalerOptions { autoScalerConfig("vertex.min-parallelism") .intType() .defaultValue(1) + .withDeprecatedKeys(deprecatedOperatorConfigKey("vertex.min-parallelism")) .withDescription("The minimum parallelism the autoscaler can use."); public static final ConfigOption<Integer> VERTEX_MAX_PARALLELISM = autoScalerConfig("vertex.max-parallelism") .intType() .defaultValue(200) + .withDeprecatedKeys(deprecatedOperatorConfigKey("vertex.max-parallelism")) .withDescription( "The maximum parallelism the autoscaler can use. Note that this limit will be ignored if it is higher than the max parallelism configured in the Flink config or directly on each operator."); @@ -105,6 +114,7 @@ public class AutoScalerOptions { autoScalerConfig("scale-down.max-factor") .doubleType() .defaultValue(0.6) + .withDeprecatedKeys(deprecatedOperatorConfigKey("scale-down.max-factor")) .withDescription( "Max scale down factor. 1 means no limit on scale down, 0.6 means job can only be scaled down with 60% of the original parallelism."); @@ -112,6 +122,7 @@ public class AutoScalerOptions { autoScalerConfig("scale-up.max-factor") .doubleType() .defaultValue(100000.) + .withDeprecatedKeys(deprecatedOperatorConfigKey("scale-up.max-factor")) .withDescription( "Max scale up factor. 2.0 means job can only be scaled up with 200% of the current parallelism."); @@ -119,6 +130,7 @@ public class AutoScalerOptions { autoScalerConfig("catch-up.duration") .durationType() .defaultValue(Duration.ofMinutes(15)) + .withDeprecatedKeys(deprecatedOperatorConfigKey("catch-up.duration")) .withDescription( "The target duration for fully processing any backlog after a scaling operation. Set to 0 to disable backlog based scaling."); @@ -126,6 +138,7 @@ public class AutoScalerOptions { autoScalerConfig("restart.time") .durationType() .defaultValue(Duration.ofMinutes(3)) + .withDeprecatedKeys(deprecatedOperatorConfigKey("restart.time")) .withDescription( "Expected restart time to be used until the operator can determine it reliably from history."); @@ -133,6 +146,8 @@ public class AutoScalerOptions { autoScalerConfig("backlog-processing.lag-threshold") .durationType() .defaultValue(Duration.ofMinutes(5)) + .withDeprecatedKeys( + deprecatedOperatorConfigKey("backlog-processing.lag-threshold")) .withDescription( "Lag threshold which will prevent unnecessary scalings while removing the pending messages responsible for the lag."); @@ -140,6 +155,8 @@ public class AutoScalerOptions { autoScalerConfig("scaling.effectiveness.detection.enabled") .booleanType() .defaultValue(false) + .withDeprecatedKeys( + deprecatedOperatorConfigKey("scaling.effectiveness.detection.enabled")) .withDescription( "Whether to enable detection of ineffective scaling operations and allowing the autoscaler to block further scale ups."); @@ -147,6 +164,8 @@ public class AutoScalerOptions { autoScalerConfig("scaling.effectiveness.threshold") .doubleType() .defaultValue(0.1) + .withDeprecatedKeys( + deprecatedOperatorConfigKey("scaling.effectiveness.threshold")) .withDescription( "Processing rate increase threshold for detecting ineffective scaling threshold. 0.1 means if we do not accomplish at least 10% of the desired capacity increase with scaling, the action is marked ineffective."); @@ -154,6 +173,7 @@ public class AutoScalerOptions { autoScalerConfig("history.max.count") .intType() .defaultValue(3) + .withDeprecatedKeys(deprecatedOperatorConfigKey("history.max.count")) .withDescription( "Maximum number of past scaling decisions to retain per vertex."); @@ -161,12 +181,14 @@ public class AutoScalerOptions { autoScalerConfig("history.max.age") .durationType() .defaultValue(Duration.ofHours(24)) + .withDeprecatedKeys(deprecatedOperatorConfigKey("history.max.age")) .withDescription("Maximum age for past scaling decisions to retain."); public static final ConfigOption<MetricAggregator> BUSY_TIME_AGGREGATOR = autoScalerConfig("metrics.busy-time.aggregator") .enumType(MetricAggregator.class) .defaultValue(MetricAggregator.MAX) + .withDeprecatedKeys(deprecatedOperatorConfigKey("metrics.busy-time.aggregator")) .withDescription( "Metric aggregator to use for busyTime metrics. This affects how true processing/output rate will be computed. Using max allows us to handle jobs with data skew more robustly, while avg may provide better stability when we know that the load distribution is even."); @@ -175,6 +197,7 @@ public class AutoScalerOptions { .stringType() .asList() .defaultValues() + .withDeprecatedKeys(deprecatedOperatorConfigKey("vertex.exclude.ids")) .withDescription( "A (semicolon-separated) list of vertex ids in hexstring for which to disable scaling. Caution: For non-sink vertices this will still scale their downstream operators until https://issues.apache.org/jira/browse/FLINK-31215 is implemented.");
