Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/18824 )
Change subject: IMPALA-11477: Adding Codegen to sorted-run-merger ...................................................................... IMPALA-11477: Adding Codegen to sorted-run-merger SortedRunMerger is used to merge multiple, already sorted runs. It is used for external merge in the sorter (SortNode), and in KRPC data stream receiver (ExchangeNode). SortedRunMerger builds and maintains a min heap of the sorted input runs. Rewrote SortedRunMerger::Heapify from recursive to iterative and moved to a separate new source file: sorted-run-merger-ir.cc. Added a static Codegen() to SortedRunMerger and call it from the corresponding ExecNodes: SortNode and ExchangeNode. This change lets the merger use the codegened version of TupleRowComparator instead of the interpreted one, which can increase the speed, especially in case of complex comparison expressions. This change also serves as a base for further codegen-related optimizations in the merger. Testing: - run existing E2E sort tests (test-sort.py) - manual testing: run queries that instantiate sort nodes and merging exchange nodes Benchmarking: - did not cause regression on TPCH query set - made merge-intensive queries and IMPALA-4530 (in-memory merge of quicksorted small runs) faster Change-Id: Ic35c7460bdbd54b8ec5872a83680e2f41ceae9fd Reviewed-on: http://gerrit.cloudera.org:8080/18824 Reviewed-by: Impala Public Jenkins <[email protected]> Tested-by: Impala Public Jenkins <[email protected]> --- M be/src/codegen/gen_ir_descriptions.py M be/src/codegen/impala-ir.cc M be/src/exec/exchange-node.cc M be/src/exec/exchange-node.h M be/src/exec/sort-node.cc M be/src/exec/sort-node.h M be/src/runtime/CMakeLists.txt M be/src/runtime/data-stream-test.cc M be/src/runtime/krpc-data-stream-recvr.cc M be/src/runtime/krpc-data-stream-recvr.h A be/src/runtime/sorted-run-merger-ir.cc M be/src/runtime/sorted-run-merger.cc M be/src/runtime/sorted-run-merger.h M be/src/runtime/sorter.cc M be/src/runtime/sorter.h 15 files changed, 254 insertions(+), 106 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/18824 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Ic35c7460bdbd54b8ec5872a83680e2f41ceae9fd Gerrit-Change-Number: 18824 Gerrit-PatchSet: 13 Gerrit-Owner: Noemi Pap-Takacs <[email protected]> Gerrit-Reviewer: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Daniel Becker <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Noemi Pap-Takacs <[email protected]>
