[ 
https://issues.apache.org/jira/browse/FLINK-9809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16540318#comment-16540318
 ] 

ASF GitHub Bot commented on FLINK-9809:
---------------------------------------

GitHub user StephanEwen opened a pull request:

    https://github.com/apache/flink/pull/6309

    [FLINK-9809] [DataSteam API] Allow setting co-location constraints on 
StreamTransformations

    ## What is the purpose of the change
    
    Flink supports co location constraints for operator placement during 
scheduling. This is used internally for iterations, for example, but is not 
exposed to users.
    
    This PR adds a way for expert users to set these constraints. As a first 
step, it adds them to the `StreamTransformation`, which is not part of the 
public user-facing classes, but a more internal class in the DataStream API. 
That way we make this initially a hidden feature and can gradually expose it 
more prominently when we agree that this would be a good idea.
    
    You can use them as follows:
    ```java
    DataStream<MyType> stream = ...
    stream.getTransformation().setCoLocationGroupKey("group2");
    ```
    
    ## Verifying this change
    
      - You can test setting the constraints as in the example above.
      - The unit test `StreamGraphCoLocationConstraintTest` adds further tests.
    
    ## Does this pull request potentially affect one of the following parts:
    
      - Dependencies (does it add or upgrade a dependency): (yes / **no**)
      - The public API, i.e., is any changed class annotated with 
`@Public(Evolving)`: (yes / **no**)
      - The serializers: (yes / **no** / don't know)
      - The runtime per-record code paths (performance sensitive): (yes / 
**no** / don't know)
      - Anything that affects deployment or recovery: JobManager (and its 
components), Checkpointing, Yarn/Mesos, ZooKeeper: (yes / **no** / don't know)
      - The S3 file system connector: (yes / **no** / don't know)
    
    ## Documentation
    
      - Does this pull request introduce a new feature? A *hidden* feature.
      - If yes, how is the feature documented? (not applicable / docs / 
JavaDocs / **not documented**)


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/StephanEwen/incubator-flink colocation

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/flink/pull/6309.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #6309
    
----
commit bd58f39290842e9088dfdb997b0704ada7bd79c8
Author: Stephan Ewen <sewen@...>
Date:   2018-07-11T15:48:10Z

    [FLINK-9809] [DataSteam API] Allow setting co-location constraints on 
StreamTransformations.
    
    This feature is currently only exposed on StreamTransformations (internal 
API) rather
    than in the public API, because it is a hidden expert feature.

----


> Support setting CoLocation constraints on the DataStream Transformations
> ------------------------------------------------------------------------
>
>                 Key: FLINK-9809
>                 URL: https://issues.apache.org/jira/browse/FLINK-9809
>             Project: Flink
>          Issue Type: Improvement
>          Components: Streaming
>            Reporter: Stephan Ewen
>            Assignee: Stephan Ewen
>            Priority: Major
>              Labels: pull-request-available
>             Fix For: 1.6.0
>
>
> Flink supports co location constraints for operator placement during 
> scheduling. This is used internally for iterations, for example, but is not 
> exposed to users.
> I propose to add a way for expert users to set these constraints. As a first 
> step, I would add them to the {{StreamTransformation}}, which is not part of 
> the public user-facing classes, but a more internal class in the DataStream 
> API. That way we make this initially a hidden feature and can gradually 
> expose it more prominently when we agree that this would be a good idea.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Reply via email to