[jira] [Commented] (FLINK-8407) Setting the parallelism after a partitioning operation should be forbidden

[Xingcan Cui (JIRA)]

    [ 
https://issues.apache.org/jira/browse/FLINK-8407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16337561#comment-16337561
 ] 

Xingcan Cui commented on FLINK-8407:
------------------------------------

Yes, there's no {{setParallelism()}} method in {{DataStream.java}}. But 
{{DataStream.scala}} gets the following method 😅
{code:java}
def setParallelism(parallelism: Int): DataStream[T] = {
    stream match {
      case ds: SingleOutputStreamOperator[T] => ds.setParallelism(parallelism)
      case _ =>
        throw new UnsupportedOperationException(
          "Operator " + stream + " cannot set the parallelism.")
    }
    this
  }
{code}
 
I'd like to add a type check in 
{{SingleOutputStreamOperator.setParallelism()}}, such as 
{code:java}
if (transformation instanceof PartitionTransformation) 
    throw new UnsupportedOperationException("Can not set parallelism after 
partitioning.");
{code}
Do you think that's feasible?



> Setting the parallelism after a partitioning operation should be forbidden
> --------------------------------------------------------------------------
>
>                 Key: FLINK-8407
>                 URL: https://issues.apache.org/jira/browse/FLINK-8407
>             Project: Flink
>          Issue Type: Bug
>          Components: DataStream API
>            Reporter: Xingcan Cui
>            Assignee: Xingcan Cui
>            Priority: Major
>
> Partitioning operations ({{shuffle}}, {{rescale}}, etc.) for a {{DataStream}} 
> create new {{DataStreams}}, which allow the users to set parallelisms for 
> them. However, the {{PartitionTransformations}} in these returned 
> {{DataStreams}} will only add virtual nodes, whose parallelisms could not be 
> specified, in the execution graph. We should forbid users to set the 
> parallelism after a partitioning operation since they won't actually work. 
> Also the corresponding documents should be updated.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)