Joe McDonnell commented on KUDU-2296:

Working on a patch

> Kudu RPC cannot deserialize messages larger than 64MB
> -----------------------------------------------------
>                 Key: KUDU-2296
>                 URL: https://issues.apache.org/jira/browse/KUDU-2296
>             Project: Kudu
>          Issue Type: Bug
>          Components: rpc
>    Affects Versions: 1.6.0
>            Reporter: Joe McDonnell
>            Assignee: Joe McDonnell
>            Priority: Major
> Impala has been testing Kudu RPC with a larger value for 
> rpc_max_message_size. I noticed that when the message size exceeds 64MB, 
> rpc::serialization::ParseMessage() hits this condition:
> {code:java}
> if (PREDICT_FALSE(!in.Skip(main_msg_len))) {
>   return Status::Corruption(
>       StringPrintf("Invalid packet: data too short, expected %d byte 
> main_msg", main_msg_len),
>       KUDU_REDACT(buf.ToDebugString()));
> }
> {code}
> The actual buffer is the appropriate size. What is happening is that protobuf 
> imposes a 64MB total byte limit by default. Once a message exceeds that, the 
> Skip() call will return false when trying to go past the 64MB limit. The 
> deserialization code can get around this by setting the total byte limit with 
> CodedInputSTream::SetTotalByteLimit().
> This should not impact existing systems at the moment, because the default 
> value for rpc_max_message_size is 50MB.

This message was sent by Atlassian JIRA

Reply via email to