[
https://issues.apache.org/jira/browse/FLINK-31071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17691064#comment-17691064
]
xingbe commented on FLINK-31071:
--------------------------------
Hi [~wanglijie] ,
I have done the testing work according to the description and all test
results are as expected.
The verification steps are as follows:
# I create a simple batch job with a source and a sink, and rebalance them in
two different slotSharingGroups. The sink parallelism is automatically decided
when the config option `jobmanager.scheduler` is not set.
# All of configuration options with
`execution.batch.adaptive.auto-parallelism` prefix are verified to take effect.
And the sink parallelism is adjusted automatically by changing the source data
volume or the config option
`execution.batch.adaptive.auto-parallelism.avg-data-volume-per-task`, and also
make it reach the upper or lower bound configured.
The default parallelism set via parallelism.default and
StreamExecutionEnvironment#setParallelism() are also verified when the
`execution.batch.adaptive.auto-parallelism.max-parallelism` is not configured.
And I also found that if user configure the two configs together, the method of
StreamExecutionEnvironment#setParallelism() have higher priority. That doesn't
seem to be a problem.
# The data correctness has been verified in miniCluster mode, and all data
produced matches expectations.
> Release Testing: Verify FLIP-283 Use adaptive batch scheduler as default
> scheduler for batch jobs
> -------------------------------------------------------------------------------------------------
>
> Key: FLINK-31071
> URL: https://issues.apache.org/jira/browse/FLINK-31071
> Project: Flink
> Issue Type: Sub-task
> Components: Runtime / Coordination
> Reporter: Lijie Wang
> Assignee: xingbe
> Priority: Blocker
> Fix For: 1.17.0
>
>
> This task aims to verify FLIP-283 Use adaptive batch scheduler as default
> scheduler for batch jobs.
> The documentation of adaptive batch scheduler can be found
> [here|https://nightlies.apache.org/flink/flink-docs-master/docs/deployment/elastic_scaling/#adaptive-batch-scheduler]
> .
> Things to verify:
> 1. Verify the adaptive batch scheduler is the default scheduler of batch
> jobs. By default, Flink will automatically decide parallelism for operators
> of batch jobs if the scheduler is not specified.
> 2. Verify the configuration
> options({{{}execution.batch.adaptive.auto-parallelism.xxx{}}}) take effect.
> Besides, it is also necessary to verify that the default parallelism set via
> {{parallelism.default}} or {{StreamExecutionEnvironment#setParallelism()}}
> will be used as upper bound of allowed parallelism if the
> execution.batch.adaptive.auto-parallelism.max-parallelism is not configured.
> 3. Verify the final data produced are correct.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
