Impala Public Jenkins has submitted this change and it was merged. Change subject: IMPALA-4703: reservation denial debug action ......................................................................
IMPALA-4703: reservation denial debug action Add debug action to deny reservation increases with some probability. This allows us to test various scenarios, particularly: * The case when the node only gets its initial reservation and must run to completion without increasing its reservation. * The case when there is some memory pressure and the node sometimes gets a reservation increase and sometimes doesn't. E.g. to deny all reservation requests after an ExecNode has opened: set debug_action=-1:OPEN:SET_DENY_RESERVATION_PROBABILITY@1.0 This was applied to test_spilling. It caught a bug in the PAGG with spilling string aggregations. This required some minor extensions to the debug actions. * Allow debug actions that apply to all ExecNodes if node_id is -1. * Allow passing parameters to debug actions. The current grammar of the actions is not well-oriented towards extension, so I resorted to using @ as a new delimiter. I also optimised ExecDebugAction() so that it is much faster in the common case and extended --disable_mem_pools to prevent the buffer pool from holding onto unused buffers. Change-Id: Ied39bb091b12156e5dc61b528c6c0cd8de3fe657 Reviewed-on: http://gerrit.cloudera.org:8080/7022 Reviewed-by: Tim Armstrong <tarmstr...@cloudera.com> Tested-by: Impala Public Jenkins --- M be/src/exec/exec-node.cc M be/src/exec/exec-node.h M be/src/runtime/bufferpool/buffer-allocator.cc M be/src/runtime/bufferpool/buffer-pool.cc M be/src/runtime/bufferpool/buffer-pool.h M be/src/runtime/bufferpool/reservation-tracker.cc M be/src/runtime/bufferpool/reservation-tracker.h M be/src/runtime/coordinator-backend-state.cc M be/src/runtime/debug-options.cc M be/src/runtime/debug-options.h M common/thrift/ImpalaInternalService.thrift M common/thrift/PlanNodes.thrift M tests/query_test/test_spilling.py 13 files changed, 140 insertions(+), 37 deletions(-) Approvals: Impala Public Jenkins: Verified Tim Armstrong: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/7022 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: merged Gerrit-Change-Id: Ied39bb091b12156e5dc61b528c6c0cd8de3fe657 Gerrit-PatchSet: 13 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Tim Armstrong <tarmstr...@cloudera.com> Gerrit-Reviewer: Dan Hecht <dhe...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong <tarmstr...@cloudera.com>