[
https://issues.apache.org/jira/browse/KUDU-2305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16393584#comment-16393584
]
Michael Ho edited comment on KUDU-2305 at 3/9/18 11:19 PM:
-----------------------------------------------------------
It appears that there are still few places in the code which may overflow if
the combined size of the entire message exceeds 2GB.
{{addition_size}} in {{SerializeMessage()}} is a 32-bit signed integer.
However, the caller in {{OutboundCall::SetRequestPayload()}} may be passing a
value larger than UINT_MAX. In which case, we will be truncating the value.,
leading to incorrect {{total_size}} being computed in {{SerializeMessage()}}
{noformat}
void SerializeMessage(const MessageLite& message, faststring* param_buf, int
additional_size, bool use_cached_size) {
{noformat}
Along the same line, even when the {{total_size}} is larger than
{{FLAGS_max_message_size}}, it appears that the client may keep sending the
payload anyway and the receiver will reject it in
{{InboundTransfer::ReceiveBuffer(}} path. It's possible that the sender may try
to send the entire message before the receiver gets a chance to reject it and
close the connection. In that sense, it's possible for {{total_size}} of a
message to be larger than INT_MAX and some local variables in
{{OutboundTransfer::SendBuffer}}, {{TlsSocket::Writev()}}, {{Socket::Writev()}}
may overflow too. So, we may need to fix those places too.
was (Author: kwho):
It appears that there are still few places in the code which may overflow if
the combined size of the entire message exceeds 2GB.
{{addition_size}} in {{SerializeMessage()}} is a 32-bit signed integer.
However, the caller in {{OutboundCall::SetRequestPayload()}} may be passing a
value larger than INT_MAX. In which case, we will be truncating the value.,
leading to incorrect {{total_size}} being computed in {{SerializeMessage()}}
{noformat}
void SerializeMessage(const MessageLite& message, faststring* param_buf, int
additional_size, bool use_cached_size) {
{noformat}
Along the same line, even when the {{total_size}} is larger than
{{FLAGS_max_message_size}}, it appears that the client may keep sending the
payload anyway and the receiver will reject it in
{{InboundTransfer::ReceiveBuffer(}} path. It's possible that the sender may try
to send the entire message before the receiver gets a chance to reject it and
close the connection. In that sense, it's possible for {{total_size}} of a
message to be larger than INT_MAX and some local variables in
{{OutboundTransfer::SendBuffer}}, {{TlsSocket::Writev()}}, {{Socket::Writev()}}
may overflow too. So, we may need to fix those places too.
> Local variables can overflow when serializing a 2GB message
> -----------------------------------------------------------
>
> Key: KUDU-2305
> URL: https://issues.apache.org/jira/browse/KUDU-2305
> Project: Kudu
> Issue Type: Bug
> Components: rpc
> Affects Versions: 1.6.0
> Reporter: Joe McDonnell
> Assignee: Joe McDonnell
> Priority: Major
> Fix For: 1.7.0
>
>
> When rpc_max_message_size is set to its maximum of INT_MAX (2147483647),
> certain local variables in SerializeMessage can overflow as messages approach
> this size. Specifically, recorded_size, size_with_delim, and total_size are 4
> byte signed integers and could overflow when additional_size becomes large.
> Since INT_MAX is the largest allowable value for rpc_max_message_size (a 4
> byte signed integer), these variables will not overflow if changed to 4 byte
> unsigned integers. This would eliminate the potential problem for
> serialization.
> A similar problem exists in the InboundTransfer::ReceiveBuffer() and similar
> codepaths. Changing those variables to unsigned integers should resolve the
> issue.
> This does not impact existing systems, because the default value of
> rpc_max_message_size is 50MB.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
