[
https://issues.apache.org/jira/browse/FLINK-17302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17511070#comment-17511070
]
guxiang commented on FLINK-17302:
---------------------------------
Hi, I have written the following demo, do you have any suggestions?
in RocksDBMapState.java add a function
{code:java}
//代码占位符
public Map<UK, UV> multiGet(List<UK> userKey) throws Exception{
HashMap<UK, UV> ukUvHashMap = new HashMap<>();
List<byte[]> rawKeyBytesList = new LinkedList<>();
List<ColumnFamilyHandle> columnFamilyHandleList = new LinkedList<>();
Map<byte[], UK> ukBytesMap = new HashMap<>();
for (UK uk : userKey) {
byte[] rawKeyBytes =
serializeCurrentKeyWithGroupAndNamespacePlusUserKey(uk,
userKeySerializer);
rawKeyBytesList.add(rawKeyBytes);
columnFamilyHandleList.add(columnFamily);
ukBytesMap.put(rawKeyBytes, uk);
}
List<byte[]> resultBytes = backend.db.multiGetAsList(
columnFamilyHandleList,
rawKeyBytesList);
for (int i = 0; i < userKey.size(); i++) {
byte[] rawValueBytes = resultBytes.get(i);
UV tempEntry = rawValueBytes == null
? null
: deserializeUserValue(dataInputView, rawValueBytes,
userValueSerializer);
ukUvHashMap.put(userKey.get(i), tempEntry);
}
return ukUvHashMap;
} {code}
> Add multiget() interface for state of rocksdb statebackend
> -----------------------------------------------------------
>
> Key: FLINK-17302
> URL: https://issues.apache.org/jira/browse/FLINK-17302
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / State Backends
> Affects Versions: 1.10.0
> Reporter: Peidian Li
> Priority: Not a Priority
> Labels: auto-deprioritized-major, auto-deprioritized-minor
>
> Users that need to lookup many keys in order to process an application level
> request end up calling Get() in a loop to read the required KVs. Rocksdb
> provides multiget() interface to improve reading efficiency.Do we consider
> providing multiget() interface for rocksdb statebackend ？
> See also [https://github.com/facebook/rocksdb/wiki/MultiGet-Performance].
> For other statebackend, we can call the get () interface in a loop to
> implement multiget().
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
