Joe McDonnell created IMPALA-8749:
-------------------------------------
Summary: Split TExecQueryFInstancesSidecar into query-wide and
backend specific segments
Key: IMPALA-8749
URL: https://issues.apache.org/jira/browse/IMPALA-8749
Project: IMPALA
Issue Type: Improvement
Components: Backend
Affects Versions: Impala 3.3.0
Reporter: Joe McDonnell
Currently, the TExecQueryFInstancesSidecar contains both a TQueryCtx (which is
the same across all backends) and TPlanFragmentCtx/TPlanFragmentInstanceCtx
lists (which are distinct across different backends). Right now, for each
message to a backend, the coordinator needs to construct this sidecar and
serialize it. This is wasteful, because it requires copying and serializing the
TQueryCtx multiple times even though it is the same across backends.
If we split up the message into query-wide and backend-specific parts, then the
main coordinator thread can serialize the TQueryCtx once and avoid serializing
it for each backend. Additionally, serializing Thrift structures reduces the
memory size, because unused optional fields are omitted. The copy of a
serialized structure is also easier, because it is a single block of memory,
rather than a large number of objects.
This query-wide information can be passed as a separate sidecar.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
