Github user fhueske commented on the issue:
https://github.com/apache/flink/pull/3590
Thanks for the update @rtudoran. I haven't looked at the changes yet. Just
a few general remarks to your comments:
1. thanks!
2. we do not use tests which are based on manual timing of operations.
These tests are typically very fragile and take a lot of time. The test harness
I was pointing to solves this issue by controlling the processing time service
of the ProcessFunction, i.e., we can control the time. See how
[WindowOperatorTest#testProcessingTimeTumblingWindows()](https://github.com/apache/flink/blob/master/flink-streaming-java/src/test/java/org/apache/flink/streaming/runtime/operators/windowing/WindowOperatorTest.java#L1080)
controls processing time using the [KeyedOneInputStreamOperatorTestHarness](
https://github.com/apache/flink/blob/master/flink-streaming-java/src/test/java/org/apache/flink/streaming/util/KeyedOneInputStreamOperatorTestHarness.java).
3. Using `ValueState` has the overhead of always serializing and
deserializing all data. The actual access inside the deserialized data is
probably much less of an issue than the de/serialization + object
instantiations itself. With `MapState` we can get ordered access to elements by
accessing only the `Long` keys. This will also help for the operations you
mentioned.
4. In order to use `MapState` we would need to use the
`NullByteKeyExtractor` approach, since non-keyed operators only support
`ListState`
Best, Fabian
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
