Re: Why don't operations on KeyedStream return KeyedStream?

2018-08-29 Thread Fabian Hueske
Hi Elias, Your assumption is correct. An operation on a KeyedStream results in a regular DataStream because the operation might change the data type or the key field. Hence, it is not guaranteed that the same keys can be extracted from the output of the keyed operation. However, there is a way

Re: Why don't operations on KeyedStream return KeyedStream?

2018-08-28 Thread vino yang
Hi Elias, Can you express this matter more clearly? The reason the KeyedStream object exists is that it needs to provide some different transform methods than the DataStream object. These transform methods are limited to keyBy. Why do you need to execute keyBy twice to get a KeyedStream object?

Why don't operations on KeyedStream return KeyedStream?

2018-08-28 Thread Elias Levy
Operators on a KeyedStream don't return a new KeyedStream. Is there a reason for this? You need to perform `keyBy` again to get a KeyedStream. Presumably if you key by the same value there won't be any shuffled data, but the key may no longer be available within the stream record.