GitHub user StephanEwen opened a pull request:
https://github.com/apache/flink/pull/6309
[FLINK-9809] [DataSteam API] Allow setting co-location constraints on
StreamTransformations
## What is the purpose of the change
Flink supports co location constraints for operator placement during
scheduling. This is used internally for iterations, for example, but is not
exposed to users.
This PR adds a way for expert users to set these constraints. As a first
step, it adds them to the `StreamTransformation`, which is not part of the
public user-facing classes, but a more internal class in the DataStream API.
That way we make this initially a hidden feature and can gradually expose it
more prominently when we agree that this would be a good idea.
You can use them as follows:
```java
DataStream<MyType> stream = ...
stream.getTransformation().setCoLocationGroupKey("group2");
```
## Verifying this change
- You can test setting the constraints as in the example above.
- The unit test `StreamGraphCoLocationConstraintTest` adds further tests.
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): (yes / **no**)
- The public API, i.e., is any changed class annotated with
`@Public(Evolving)`: (yes / **no**)
- The serializers: (yes / **no** / don't know)
- The runtime per-record code paths (performance sensitive): (yes /
**no** / don't know)
- Anything that affects deployment or recovery: JobManager (and its
components), Checkpointing, Yarn/Mesos, ZooKeeper: (yes / **no** / don't know)
- The S3 file system connector: (yes / **no** / don't know)
## Documentation
- Does this pull request introduce a new feature? A *hidden* feature.
- If yes, how is the feature documented? (not applicable / docs /
JavaDocs / **not documented**)
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/StephanEwen/incubator-flink colocation
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/6309.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #6309
----
commit bd58f39290842e9088dfdb997b0704ada7bd79c8
Author: Stephan Ewen <sewen@...>
Date: 2018-07-11T15:48:10Z
[FLINK-9809] [DataSteam API] Allow setting co-location constraints on
StreamTransformations.
This feature is currently only exposed on StreamTransformations (internal
API) rather
than in the public API, because it is a hidden expert feature.
----
---
