[
https://issues.apache.org/jira/browse/IMPALA-4875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tim Armstrong resolved IMPALA-4875.
-----------------------------------
Resolution: Won't Do
> Bound the max size of statestore UpdateState() RPCs
> ---------------------------------------------------
>
> Key: IMPALA-4875
> URL: https://issues.apache.org/jira/browse/IMPALA-4875
> Project: IMPALA
> Issue Type: Sub-task
> Components: Distributed Exec
> Affects Versions: Impala 2.9.0
> Reporter: Henry Robinson
> Priority: Major
>
> Although IMPALA-4874 will help mitigate the effects of large messages, the
> Statestore can still theoretically send very large messages when making
> {{UpdateState()}} RPCs.
> We can bound the size of those payloads by only including a few topic
> entries, up to a desired message size. This works well if those entries are
> split up by different topic versions - we just send up to version K <=
> topic_version on each UpdateState() call.
> Situations where a single topic version are larger than the max size are
> trickier. We can still stream them to clients, but the clients have to buffer
> them until they get the complete update, at which point they can deliver the
> full update.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
