[
https://issues.apache.org/jira/browse/FLINK-19709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Flink Jira Bot updated FLINK-19709:
-----------------------------------
Labels: auto-deprioritized-major features stale-minor (was:
auto-deprioritized-major features)
I am the [Flink Jira Bot|https://github.com/apache/flink-jira-bot/] and I help
the community manage its development. I see this issues has been marked as
Minor but is unassigned and neither itself nor its Sub-Tasks have been updated
for 180 days. I have gone ahead and marked it "stale-minor". If this ticket is
still Minor, please either assign yourself or give an update. Afterwards,
please remove the label or in 7 days the issue will be deprioritized.
> Support KeyedSortedMapState in DataStream API
> ---------------------------------------------
>
> Key: FLINK-19709
> URL: https://issues.apache.org/jira/browse/FLINK-19709
> Project: Flink
> Issue Type: New Feature
> Components: API / DataStream, Runtime / State Backends
> Reporter: Sean Z
> Priority: Minor
> Labels: auto-deprioritized-major, features, stale-minor
>
> Current DataStream API doesn't have SortedMapState supported. There are lots
> of use cases based on sorted time-series data like range-query or
> higher/lower key fetch, and ordered data seems like a nature of time-series
> stream processing. Therefore, we propose to support the KeyedSortedMapState
> feature.
>
> There were some previous discussions FLINK-6219 about SortedMapState, and
> the thread was closed because blink code might cover this feature. However,
> the [blink
> code|https://github.com/apache/flink/blob/blink/flink-runtime/src/main/java/org/apache/flink/runtime/state/keyed/KeyedSortedMapState.java]
> wasn't merged into the master branch since then. The major concern is the
> inconsistent comparison between heap/off-heap state backends. In RocksDB, the
> comparison should be based on bytes, which makes generic key types support
> challenging, and in heap state backend, the comparison is more about
> Comparable interface.
>
> There are two possible solutions to this issue as [discussed in dev email
> list|http://mail-archives.apache.org/mod_mbox/flink-dev/202010.mbox/%3CCABKsfK6meYPPbxChC_jWzYn2knLTJv%2BpjFcBkvnMf3mRWC6LWw%40mail.gmail.com%3E],
>
> 1. There is a prototype feature called TemporalState which could limit the
> key type to Long type. It makes sense for most of the use cases are about
> timestamp as a key but it brings limitations to support generic key types. We
> also need to gracefully handle negative long.
>
> 2. We keep the different sorting behavior of different state backends and
> set it to bytes comparison for given serialization by default in off-heap
> state backends. Let users provide their own specific serializer if they want
> to sort some customized type on RocksDB. This solution could start with blink
> implementation, which has already supported serializing various key types
> (almost all the atomic types) into an ordered bytes.
> [code|https://github.com/apache/flink/tree/blink/flink-libraries/flink-table/src/main/java/org/apache/flink/table/typeutils/ordered]
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
