Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/14129 )
Change subject: IMPALA-8819: BufferedPlanRootSink should handle non-default fetch sizes ...................................................................... Patch Set 2: (2 comments) http://gerrit.cloudera.org:8080/#/c/14129/2/be/src/exec/buffered-plan-root-sink.cc File be/src/exec/buffered-plan-root-sink.cc: http://gerrit.cloudera.org:8080/#/c/14129/2/be/src/exec/buffered-plan-root-sink.cc@143 PS2, Line 143: // If 'num_results' <= 0 then by default fetch BATCH_SIZE rows. I think it might be better to return > 1024 rows at a time by default - e.g. if you consider a client fetching a largish result set over a connection with high latency but reasonable bandwidth, then throughput is mostly going to be related to the # of RPCs. I haven't looked at the behaviour of different clients. Actually I think generally if the server is running faster than the client (very common), returning larger batches is good. There's a trade-off between minimising round-trips and also responsiveness, but having a timeout on fetches might help if the client is running faster than the server, so that if the server is returning rows slower than the client can fetch them, you get some output back. This actually depends a lot on the behaviour of clients - I don't know how many depend on the default fetch size. http://gerrit.cloudera.org:8080/#/c/14129/2/be/src/exec/buffered-plan-root-sink.cc@152 PS2, Line 152: rows_available_.Wait(l); Don't need to solve in this PS, but I think we could solve IMPALA-7312 with a timed wait here -- To view, visit http://gerrit.cloudera.org:8080/14129 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8dd4b397ab6457a4f85e635f239b2c67130fcce4 Gerrit-Change-Number: 14129 Gerrit-PatchSet: 2 Gerrit-Owner: Sahil Takiar <stak...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Michael Ho <k...@cloudera.com> Gerrit-Reviewer: Sahil Takiar <stak...@cloudera.com> Gerrit-Reviewer: Tim Armstrong <tarmstr...@cloudera.com> Gerrit-Comment-Date: Fri, 23 Aug 2019 17:32:41 +0000 Gerrit-HasComments: Yes
