[GitHub] flink issue #5545: [FLINK-8718][DataStream] Set maxParallelism on non-parall...

2018-02-22 Thread StefanRRichter
Github user StefanRRichter commented on the issue: https://github.com/apache/flink/pull/5545 I agree with @aljoscha, there is a sanity check that `maxParallelism` of operators did not change when restoring. ---

[GitHub] flink issue #5545: [FLINK-8718][DataStream] Set maxParallelism on non-parall...

2018-02-22 Thread aljoscha
Github user aljoscha commented on the issue: https://github.com/apache/flink/pull/5545 Yes, no source can use keyed state. 😄 But the `StateAssignmentOperation` will still fail a restore if the `maxParallelism` of an operator changes. Maybe @StefanRRichter can chime in on that but

[GitHub] flink issue #5545: [FLINK-8718][DataStream] Set maxParallelism on non-parall...

2018-02-21 Thread tillrohrmann
Github user tillrohrmann commented on the issue: https://github.com/apache/flink/pull/5545 Shouldn't it always be possible to merge n key groups into a single key group? Maybe this could be the backwards compatibility path. Moreover, which sources use keyed state? Don't

[GitHub] flink issue #5545: [FLINK-8718][DataStream] Set maxParallelism on non-parall...

2018-02-21 Thread aljoscha
Github user aljoscha commented on the issue: https://github.com/apache/flink/pull/5545 I'm not sure we can do this change since it breaks compatibility for savepoints because previous versions of Flink hat some `maxParallelism` setting for sources which would be incompatible with the

[GitHub] flink issue #5545: [FLINK-8718][DataStream] Set maxParallelism on non-parall...

2018-02-21 Thread GJL
Github user GJL commented on the issue: https://github.com/apache/flink/pull/5545 The test relies on being able to set setParallelism on a non-parallel source: `DataStream input1 = env.fromElements(1, 2, 3, 4).setMaxParallelism(128);` ---

[GitHub] flink issue #5545: [FLINK-8718][DataStream] Set maxParallelism on non-parall...

2018-02-21 Thread tillrohrmann
Github user tillrohrmann commented on the issue: https://github.com/apache/flink/pull/5545 Could that be a faulty test? ---

[GitHub] flink issue #5545: [FLINK-8718][DataStream] Set maxParallelism on non-parall...

2018-02-21 Thread GJL
Github user GJL commented on the issue: https://github.com/apache/flink/pull/5545 I have test failures, e.g., ` StreamGraphGeneratorTest.testMaxParallelismWithConnectedKeyedStream` 😢 ---