Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/10550 )
Change subject: PREVIEW: IMPALA-7078: improve memory consumption of wide Avro scans ...................................................................... Patch Set 2: (1 comment) http://gerrit.cloudera.org:8080/#/c/10550/2//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/10550/2//COMMIT_MSG@18 PS2, Line 18: Cap the maximum row batch queue size at 5 * the number of active : scanner threads. > I want to get a tighter upper bound on the amount of memory in the queue an I did some benchmarks and the dynamic queue size caused performance changes because the queue became full sooner and fewer scanner threads were started. I removed the dynamic queue sizing. I think it's worth capping the queue based on num_scanner_threads still so that setting num_scanner_threads has more teeth in limiting memory consumption. I can still see the argument for separating that out from this patch though if you prefer. -- To view, visit http://gerrit.cloudera.org:8080/10550 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iebd2600b4784fd19696c9b92eefb7d7e9db0c80b Gerrit-Change-Number: 10550 Gerrit-PatchSet: 2 Gerrit-Owner: Tim Armstrong <[email protected]> Gerrit-Reviewer: Dan Hecht <[email protected]> Gerrit-Reviewer: Tim Armstrong <[email protected]> Gerrit-Comment-Date: Thu, 31 May 2018 15:55:25 +0000 Gerrit-HasComments: Yes
