[
https://issues.apache.org/jira/browse/HDDS-3032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arpit Agarwal updated HDDS-3032:
--------------------------------
Target Version/s: 0.7.0 (was: 0.6.0)
> Cannot write 32MB chunks
> ------------------------
>
> Key: HDDS-3032
> URL: https://issues.apache.org/jira/browse/HDDS-3032
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
> Components: Ozone Datanode
> Reporter: Attila Doroszlai
> Assignee: Bharat Viswanadham
> Priority: Major
> Labels: TriagePending
>
> Writing 32MB chunks fails with various errors.
> {code:title=steps to reproduce}
> ozone freon dcg -t 1 -n 1 -s 33554432
> {code}
> 1. With Ratis 0.5.0-90cd474-SNAPSHOT (used by current Ozone master):
> {code}
> org.apache.ratis.thirdparty.io.grpc.StatusRuntimeException:
> RESOURCE_EXHAUSTED: gRPC message exceeds maximum size 33554432: 33554686
> at
> org.apache.ratis.thirdparty.io.grpc.Status.asRuntimeException(Status.java:524)
> at
> org.apache.ratis.thirdparty.io.grpc.internal.MessageDeframer.processHeader(MessageDeframer.java:387)
> {code}
> Which is strange, because [Datanode attempts to set max. message
> size|https://github.com/apache/hadoop-ozone/blob/4ba1932dab4692a9cc1bcfb8903ef650e32ec7ba/hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/transport/server/ratis/XceiverServerRatis.java#L204-L206]
> to 32MB + 16KB.
> 2. With Ratis built locally from current Ratis master (46f255cb):
> {code}
> Caused by: org.apache.ratis.protocol.StateMachineException:
> org.apache.ratis.server.raftlog.RaftLogIOException from Server
> ccb25fbf-9bd1-4094-a632-00f4168213bb@group-B1FA90A78F31: Log entry size
> 33554666 exceeds the max buffer limit of 33554432
> at org.apache.ratis.server.raftlog.RaftLog.appendImpl(RaftLog.java:178)
> at
> org.apache.ratis.server.raftlog.RaftLog.lambda$append$2(RaftLog.java:157)
> at
> org.apache.ratis.server.raftlog.RaftLogSequentialOps$Runner.runSequentially(RaftLogSequentialOps.java:68)
> at org.apache.ratis.server.raftlog.RaftLog.append(RaftLog.java:157)
> at
> org.apache.ratis.server.impl.ServerState.appendLog(ServerState.java:282)
> at
> org.apache.ratis.server.impl.RaftServerImpl.appendTransaction(RaftServerImpl.java:518)
> at
> org.apache.ratis.server.impl.RaftServerImpl.submitClientRequestAsync(RaftServerImpl.java:604)
> {code}
> With {{ozone.scm.chunk.size=32MB}} setting, {{ozone freon ockg -n 1 -t 1 -s
> 33554432}} also fails, but without apparent errors in the datanode log.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
