Hi Aaron We would usually compare MapState with Map in java language. And your request is not what typical Map interface but SortedMap defines.
I think it's reasonable If we introduce SortedMapState interface in Flink. However, there existed several challenges for RocksDB. * RocksDB only support byte-wise comparator by default, which means those non-fixed length serializer (e.g. StringSerializer) would generate bytes with different lexicon order compared with their original strings in java. Blink had ever tried to implement this feature named OrderedBytes [1], which still has many limitations and only applied in table API. * RocksDB actually support java comparator and we have also implemented it privately on RocksDB-4.2. However, the performance behaves badly. * RocksDB later support native comparatot for java users [2] and we are still looking at this future although this would be a bit complex for Flink users if they want their classes could be comparable as they want. Hope this could help you. [1] https://github.com/apache/flink/blob/blink/flink-libraries/flink-table/src/main/java/org/apache/flink/table/typeutils/ordered/OrderedBytes.java * <https://github.com/apache/flink/blob/blink/flink-libraries/flink-table/src/main/java/org/apache/flink/table/typeutils/ordered/OrderedBytes.java> [2] https://github.com/facebook/rocksdb/pull/3334 Best Yun Tang * ________________________________ From: Aaron Langford <aaron.langfor...@gmail.com> Sent: Tuesday, January 7, 2020 1:36 To: dev@flink.apache.org <dev@flink.apache.org> Subject: Why aren't Range Scans Supported in MapState? Hey Flink Community, RocksDB natively supports range scans for keys. Is there a good story why we don't have a way to ask for keys by range or prefix currently in MapState? Aaron
