aljoscha opened a new pull request #8951: Dag bounded flag URL: https://github.com/apache/flink/pull/8951 ## What is the purpose of the change The new Blink-based Table Runner needs "streaming pipelines" to be executed with `ScheduleMode.LAZY_FROM_SOURCES` if all sources are bounded. The current Blink code base uses a global flag for this and configures the `StreamGraphGenerator` accordingly. We propose to add an `isBounded()` property to `Transformation` (formerly known as `StreamTransformation`). The property would only be explicitly settable on sources, other transformations inherit the property from their inputs. The `StreamGraphGenerator` must use `ScheduleMode.LAZY_FROM_SOURCES` if all sources are bounded, otherwise, it should use ScheduleMode.EAGER, as is the currently existing behaviour. ## Brief change log * ccf6f96 refactors `DataStreamSource` as preparation * 03aa30b adds the `isBounded()` property to `Transformation`, along with tests for the inheritance logic * 277c947 adds a new *public* method on `StreamExecutionEnvironment` for adding bounded sources * c37ffb9 changes `StreamGraphGenerator` to set the `ScheduleMode` based on the newly added boundedness property, along with tests ## Verifying this change This change added tests and can be verified as follows: * new tests in 03aa30b for the `isBounded()` property on `Transformation` * new tests in c37ffb9, i.e in `StreamGraphGeneratorTest` ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): no - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: yes - The serializers: no - The runtime per-record code paths (performance sensitive): no - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: no - The S3 file system connector: no ## Documentation - Does this pull request introduce a new feature? yes, there is a new public API method for adding bounded sources - If yes, how is the feature documented? documented in Javadocs, as for now this is more of an under-the-radar featre
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
