Tim Armstrong has posted comments on this change. Change subject: IMPALA-3567 Part 2, IMPALA-3899: factor out PHJ builder ......................................................................
Patch Set 4: I put together a spreadsheet of results for memory usage before and after this patch: https://docs.google.com/spreadsheets/d/17jyrLpDVUut_TKHrIhwROAa3BHh4ibVu9MqbLZxCGzE/edit?usp=sharing Overall there are lots of small improvements. There are a couple of bad regressions (q8 and q9), which are a bit surprising because on paper this should use a lot less memory to spill when there are many join nodes in the plan. I dug into those regressions. Q8: the difference is that scan memory usage was much higher for the scan node at the bottom of the join pipeline (~407M v 126M), so the problem was IMPALA-3209. Q9: I couldn't reproduce the lower memory usage before the patch -- To view, visit http://gerrit.cloudera.org:8080/3873 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I1e02ea9c7a7d1a0f373b11aa06c3237e1c7bd4cb Gerrit-PatchSet: 4 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Tim Armstrong <[email protected]> Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Tim Armstrong <[email protected]> Gerrit-HasComments: No
