Sahil Takiar has uploaded this change for review. ( http://gerrit.cloudera.org:8080/14129
Change subject: IMPALA-8819: BufferedPlanRootSink should handle non-default fetch sizes ...................................................................... IMPALA-8819: BufferedPlanRootSink should handle non-default fetch sizes Adds support for non-default fetch sizes when result spooling is enabled (the default is to return BATCH_SIZE rows for each fetch request). When result spooling is disabled, Impala can only return up to BATCH_SIZE rows because it only buffers a single RowBatch at a time. When result spooling is enabled, each fetch request returns exactly the number of rows requested assuming there are that many rows left in the result set. There is also an upper limit on the fetch size to prevent the resulting QueryResultSet from getting too big. Unlike the behavior when result spooling is disabled, fetches do not break on RowBatch boundaries. For example, when result spooling is disabled, if the fetch size is 10 and the batch size is 15, the second fetch will return 5 rows. However, when result spooling is enabled the second fetch will return 10 rows (assuming there is another RowBatch to read). Testing: * Ran core tests * Added new tests to test_result_spooling.py Change-Id: I8dd4b397ab6457a4f85e635f239b2c67130fcce4 --- M be/src/exec/buffered-plan-root-sink.cc M be/src/exec/buffered-plan-root-sink.h M tests/query_test/test_result_spooling.py 3 files changed, 191 insertions(+), 42 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/29/14129/1 -- To view, visit http://gerrit.cloudera.org:8080/14129 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I8dd4b397ab6457a4f85e635f239b2c67130fcce4 Gerrit-Change-Number: 14129 Gerrit-PatchSet: 1 Gerrit-Owner: Sahil Takiar <[email protected]>
