Hi David,
Could you share some explanations why SortedMapState cannot work in details? I
just cannot catch up what the statement below means:
This was rejected as being overly difficult to implement in a way that would
cleanly leverage RocksDB’s iterators.
Best
Yun Tang
________________________________
From: Aitozi <gjying1...@gmail.com>
Sent: Tuesday, April 12, 2022 15:00
To: dev@flink.apache.org <dev@flink.apache.org>
Subject: Re: [DISCUSS] FLIP-220: Temporal State
Hi David
I have look through the doc, I think it will be a good improvement to
this pattern usage, I'm interested in it. Do you have some POC work to
share for a closer look.
Besides, I have one question that can we support expose the namespace in
the different state type not limited to `TemporalState`. By this, user can
specify the namespace
and the TemporalState is one of the special case that it use timestamp as
the namespace. I think it will be more extendable.
What do you think about this ?
Best,
Aitozi.
David Anderson <dander...@apache.org> 于2022年4月11日周一 20:54写道：
> Greetings, Flink developers.
>
> I would like to open up a discussion of a proposal [1] to add a new kind of
> state to Flink.
>
> The goal here is to optimize a fairly common pattern, which is using
>
> MapState<Long, List<Event>>
>
> to store lists of events associated with timestamps. This pattern is used
> internally in quite a few operators that implement sorting and joins, and
> it also shows up in user code, for example, when implementing custom
> windowing in a KeyedProcessFunction.
>
> Nico Kruber, Seth Wiesman, and I have implemented a POC that achieves a
> more than 2x improvement in throughput when performing these operations on
> RocksDB by better leveraging the capabilities of the RocksDB state backend.
>
> See FLIP-220 [1] for details.
>
> Best,
> David
>
> [1] https://cwiki.apache.org/confluence/x/Xo_FD
>
