Tim Armstrong has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/16848 )
Change subject: IMPALA-10343: increase control_service_queue_mem_limit ...................................................................... IMPALA-10343: increase control_service_queue_mem_limit --control_service_queue_mem_limit is set to 1% by default to increase the maximum size of the queue in typical production deployments. E.g. an Impala daemon with a 50GB memory limit will have a limit of 512MB on the control service queue. Add --control_service_queue_mem_limit_floor_bytes so that this does not have the unintended effect of reducing the memory given to the control service queue. I.e. the default behaviour does not change for impala daemons with a daemon mem limit of <= 5000MB, but does increase the control service queue memory limit for impala daemons with mem limits > 5000MB. The default process memory limit in the mocked backend test ExecEnv is changed to be 8GB. Previously it was unlimited, so we couldn't calculate 1% of it. It cannot be unlimited in an actual impalad since IMPALA-5653 was fixed. Testing: This had been previously problematic on a 64 node TPC-DS workload with mt_dop=12 where impalads had ~100GB of memory. Status report RPCs would fail and have to be retried. We tested this new value on the same workload and the retries were avoided. Change-Id: Ic7fe93b5ce7eb6b63e48293ac287d98cc1d9e3fa Reviewed-on: http://gerrit.cloudera.org:8080/16848 Tested-by: Impala Public Jenkins <[email protected]> Reviewed-by: Thomas Tauber-Marshall <[email protected]> --- M be/src/runtime/test-env.h M be/src/service/control-service.cc M tests/custom_cluster/test_rpc_timeout.py 3 files changed, 12 insertions(+), 3 deletions(-) Approvals: Impala Public Jenkins: Verified Thomas Tauber-Marshall: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/16848 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Ic7fe93b5ce7eb6b63e48293ac287d98cc1d9e3fa Gerrit-Change-Number: 16848 Gerrit-PatchSet: 4 Gerrit-Owner: Tim Armstrong <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Thomas Tauber-Marshall <[email protected]> Gerrit-Reviewer: Tim Armstrong <[email protected]> Gerrit-Reviewer: Wenzhe Zhou <[email protected]>
