Hello Thomas Tauber-Marshall, Wenzhe Zhou, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/16848
to look at the new patch set (#3).
Change subject: IMPALA-10343: increase control_service_queue_mem_limit
......................................................................
IMPALA-10343: increase control_service_queue_mem_limit
--control_service_queue_mem_limit is set to 1% by default
to increase the maximum size of the queue in typical
production deployments. E.g. an Impala daemon with
a 50GB memory limit will have a limit of 512MB on
the control service queue.
Add --control_service_queue_mem_limit_floor_bytes so
that this does not have the unintended effect of
reducing the memory given to the control service queue.
I.e. the default behaviour does not change for
impala daemons with a daemon mem limit of <= 5000MB,
but does increase the control service queue memory
limit for impala daemons with mem limits > 5000MB.
The default process memory limit in the mocked backend
test ExecEnv is changed to be 8GB. Previously it
was unlimited, so we couldn't calculate 1% of it.
It cannot be unlimited in an actual impalad since
IMPALA-5653 was fixed.
Testing:
This had been previously problematic on a 64 node TPC-DS
workload with mt_dop=12 where impalads had ~100GB of memory.
Status report RPCs would fail and have to be retried.
We tested this new value on the same workload and the retries
were avoided.
Change-Id: Ic7fe93b5ce7eb6b63e48293ac287d98cc1d9e3fa
---
M be/src/runtime/test-env.h
M be/src/service/control-service.cc
M tests/custom_cluster/test_rpc_timeout.py
3 files changed, 12 insertions(+), 3 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/48/16848/3
--
To view, visit http://gerrit.cloudera.org:8080/16848
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ic7fe93b5ce7eb6b63e48293ac287d98cc1d9e3fa
Gerrit-Change-Number: 16848
Gerrit-PatchSet: 3
Gerrit-Owner: Tim Armstrong <[email protected]>
Gerrit-Reviewer: Impala Public Jenkins <[email protected]>
Gerrit-Reviewer: Thomas Tauber-Marshall <[email protected]>
Gerrit-Reviewer: Tim Armstrong <[email protected]>
Gerrit-Reviewer: Wenzhe Zhou <[email protected]>
