[
https://issues.apache.org/jira/browse/HDDS-8195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17703979#comment-17703979
]
Devesh Kumar Singh commented on HDDS-8195:
------------------------------------------
As per analysis from code , provided the above call syntax, following possible
cases should be handled:
1: log size is huge 10GB crossing Heap size --> 1) fullsync, 2) analyze from
usages perspective if can reduce this.
2: commulative size is huge --> 1) sync with reduced limit to fit defined size
(1GB)
3: If system memory size is low or available is low, only action is full sync
--- requirement for availability, need analyze where issue is commig, only
action is "fulldb sync"
Currently in Ozone code, recon is the possible client within our code base, so
point #1 and point #2 can be handled in our code. However if any external
client use case, then point#1 can only be handled in such a way that exception
to be thrown back to client and recon as a client should handle for full sync
and if external client, then external client should handle.
> RDBStore.getUpdatesSince() throws RocksDBException: Requested array size
> exceeds VM limit
> -----------------------------------------------------------------------------------------
>
> Key: HDDS-8195
> URL: https://issues.apache.org/jira/browse/HDDS-8195
> Project: Apache Ozone
> Issue Type: Bug
> Affects Versions: 1.4.0
> Reporter: Wei-Chiu Chuang
> Assignee: Devesh Kumar Singh
> Priority: Major
>
> It appears that this operations attempted request an array larger than
> Integer.MAX_INT.
> {noformat}
> 2023-03-15 22:27:13,988 ERROR org.apache.hadoop.hdds.utils.db.RDBStore:
> Unable to get delta updates since sequenceNumber 1329458. This exception will
> not be thrown to
> the client
> org.rocksdb.RocksDBException: Requested array size exceeds VM limit
> at org.rocksdb.WriteBatch.data(Native Method)
> at org.rocksdb.WriteBatch.data(WriteBatch.java:75)
> at
> org.apache.hadoop.hdds.utils.db.RDBStore.getUpdatesSince(RDBStore.java:405)
> at
> org.apache.hadoop.ozone.om.OzoneManager.getDBUpdates(OzoneManager.java:3975)
> at
> org.apache.hadoop.ozone.protocolPB.OzoneManagerRequestHandler.getOMDBUpdates(OzoneManagerRequestHandler.java:354)
> at
> org.apache.hadoop.ozone.protocolPB.OzoneManagerRequestHandler.handleReadRequest(OzoneManagerRequestHandler.java:233)
> at
> org.apache.hadoop.ozone.protocolPB.OzoneManagerProtocolServerSideTranslatorPB.submitReadRequestToOM(OzoneManagerProtocolServerSideTranslatorPB.java:223)
> at
> org.apache.hadoop.ozone.protocolPB.OzoneManagerProtocolServerSideTranslatorPB.processRequest(OzoneManagerProtocolServerSideTranslatorPB.java:177)
> at
> org.apache.hadoop.hdds.server.OzoneProtocolMessageDispatcher.processRequest(OzoneProtocolMessageDispatcher.java:87)
> at
> org.apache.hadoop.ozone.protocolPB.OzoneManagerProtocolServerSideTranslatorPB.submitRequest(OzoneManagerProtocolServerSideTranslatorPB.java:147)
> at
> org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos$OzoneManagerService$2.callBlockingMethod(OzoneManagerProtocolProtos.java)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:533)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1070)
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:989)
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:917)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1898)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2894)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
