Tim Armstrong has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/15715 )
Change subject: IMPALA-9643: fix runtime filter race for mt_dop ...................................................................... IMPALA-9643: fix runtime filter race for mt_dop This patch avoids the race with registration of a consumer filter by registering all filters upfront when the filter bank is constructed. Then registration of producers and consumers hands out references to the pre-constructed filters. A nice bonus of this change is that RegisterConsumer() and RegisterProducer() don't mutate anything and we can avoid lock acquisitions. Also adds test infrastructure and fixes TestRuntimeRowFilters to work with mt_dop=4 (it was accidentally not enabled before). That mostly involved modifying the tests to use aggregates of counters instead of picking out lines with regexes. Testing: Added a regression test that reliably failed before this fix. This relies on extending debug actions to allow longer delays, plus a minor extension to the RUNTIME_PROFILE .test file parser to handle spaces in counter names. Ran exhaustive tests. Change-Id: I194c0d2515b6a0e5474e1c0c8647f0e54dc94397 Reviewed-on: http://gerrit.cloudera.org:8080/15715 Reviewed-by: Tim Armstrong <[email protected]> Tested-by: Impala Public Jenkins <[email protected]> --- M be/src/exec/exec-node.cc M be/src/exec/partitioned-hash-join-builder.cc M be/src/exec/scan-node.cc M be/src/exec/scan-node.h M be/src/runtime/query-state.cc M be/src/runtime/runtime-filter-bank.cc M be/src/runtime/runtime-filter-bank.h M common/thrift/PlanNodes.thrift A testdata/workloads/functional-query/queries/QueryTest/runtime_row_filter_reservations.test M testdata/workloads/functional-query/queries/QueryTest/runtime_row_filters.test M tests/common/test_result_verifier.py M tests/query_test/test_runtime_filters.py 12 files changed, 252 insertions(+), 158 deletions(-) Approvals: Tim Armstrong: Looks good to me, approved Impala Public Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/15715 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I194c0d2515b6a0e5474e1c0c8647f0e54dc94397 Gerrit-Change-Number: 15715 Gerrit-PatchSet: 9 Gerrit-Owner: Tim Armstrong <[email protected]> Gerrit-Reviewer: Bikramjeet Vig <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Thomas Tauber-Marshall <[email protected]> Gerrit-Reviewer: Tim Armstrong <[email protected]>
