[ 
https://issues.apache.org/jira/browse/FLINK-13707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16907001#comment-16907001
 ] 

Jark Wu commented on FLINK-13707:
---------------------------------

Hi [~stukid], just want to make sure the logic of default max parallelism. I 
find the default max parallelism is computed [in this 
way|https://github.com/apache/flink/blob/master/flink-runtime/src/main/java/org/apache/flink/runtime/state/KeyGroupRangeAssignment.java#L127]:
 


{code:java}
Math.min(
                                Math.max(
                                                
MathUtils.roundUpToPowerOfTwo(operatorParallelism + (operatorParallelism / 2)),
                                                
DEFAULT_LOWER_BOUND_MAX_PARALLELISM),
                                UPPER_BOUND_MAX_PARALLELISM)
{code}


The max parallelism is {{roundUpToPowerOfTwo(1.5 * operatorParallelism)}}, so 
we have a safe range to increase the parallelism. 

Whatever,  I think it makes sense to have a global default max parallelism 
configuration.

> Make  max parallelism configurable
> ----------------------------------
>
>                 Key: FLINK-13707
>                 URL: https://issues.apache.org/jira/browse/FLINK-13707
>             Project: Flink
>          Issue Type: New Feature
>          Components: API / DataStream
>            Reporter: xuekang
>            Priority: Minor
>
> For now, if a user set parallelism larger than 128, and does not set max 
> parallelism explicitly, the system will compute a max parallelism, which is 
> 1.5 * parallelism.  When the job changes the parallelism and recover from a 
> savepoint, there may be some problem when restoring states from the 
> savepoint, as the number of key groups changed.
> To avoid this problem, and trying not to modify the code of existing jobs,  
> we want to configure the default max parallelism in flink-conf.yaml, but it 
> is not configurable now.
> Should we make it configurable? Any comments would be appreciated.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

Reply via email to