[
https://issues.apache.org/jira/browse/KUDU-2296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16362963#comment-16362963
]
Joe McDonnell commented on KUDU-2296:
-------------------------------------
Working on a patch
> Kudu RPC cannot deserialize messages larger than 64MB
> -----------------------------------------------------
>
> Key: KUDU-2296
> URL: https://issues.apache.org/jira/browse/KUDU-2296
> Project: Kudu
> Issue Type: Bug
> Components: rpc
> Affects Versions: 1.6.0
> Reporter: Joe McDonnell
> Assignee: Joe McDonnell
> Priority: Major
>
> Impala has been testing Kudu RPC with a larger value for
> rpc_max_message_size. I noticed that when the message size exceeds 64MB,
> rpc::serialization::ParseMessage() hits this condition:
> {code:java}
> if (PREDICT_FALSE(!in.Skip(main_msg_len))) {
> return Status::Corruption(
> StringPrintf("Invalid packet: data too short, expected %d byte
> main_msg", main_msg_len),
> KUDU_REDACT(buf.ToDebugString()));
> }
> {code}
> The actual buffer is the appropriate size. What is happening is that protobuf
> imposes a 64MB total byte limit by default. Once a message exceeds that, the
> Skip() call will return false when trying to go past the 64MB limit. The
> deserialization code can get around this by setting the total byte limit with
> CodedInputSTream::SetTotalByteLimit().
> This should not impact existing systems at the moment, because the default
> value for rpc_max_message_size is 50MB.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
