Joe McDonnell created KUDU-2296:
-----------------------------------
Summary: Kudu RPC cannot deserialize messages larger than 64MB
Key: KUDU-2296
URL: https://issues.apache.org/jira/browse/KUDU-2296
Project: Kudu
Issue Type: Bug
Components: rpc
Affects Versions: 1.6.0
Reporter: Joe McDonnell
Assignee: Joe McDonnell
Impala has been testing Kudu RPC with a larger value for rpc_max_message_size.
I noticed that when the message size exceeds 64MB,
rpc::serialization::ParseMessage() hits this condition:
{code:java}
if (PREDICT_FALSE(!in.Skip(main_msg_len))) {
return Status::Corruption(
StringPrintf("Invalid packet: data too short, expected %d byte main_msg",
main_msg_len),
KUDU_REDACT(buf.ToDebugString()));
}
{code}
The actual buffer is the appropriate size. What is happening is that protobuf
imposes a 64MB total byte limit by default. Once a message exceeds that, the
Skip() call will return false when trying to go past the 64MB limit. The
deserialization code can get around this by setting the total byte limit with
CodedInputSTream::SetTotalByteLimit().
This should not impact existing systems at the moment, because the default
value for rpc_max_message_size is 50MB.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
