[
https://issues.apache.org/jira/browse/PHOENIX-539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gabriel Reid updated PHOENIX-539:
---------------------------------
Attachment: PHOENIX-539.patch
Here's a patch that does chunked reading for scans -- it still using the
SpoolingResultIterator, but each chunk is (potentially) spooled instead of
spooling the full scan.
The integration tests run successfully (including when the chunk size is set to
1), and in some testing on a cluster there is a huge difference in
responsiveness when doing a "select *" in sqlline (as expected).
One thing I'm not happy about with it is the specific reference to HashJoinInfo
in ChunkedResultIterator.newResultIterator, but I wasn't able to find another
way to get around this -- any suggestions on how to improve this would be
appreciated.
> Implement parallel scanner that does not spool to disk
> ------------------------------------------------------
>
> Key: PHOENIX-539
> URL: https://issues.apache.org/jira/browse/PHOENIX-539
> Project: Phoenix
> Issue Type: Task
> Reporter: James Taylor
> Assignee: larsh
> Attachments: PHOENIX-539.patch
>
>
> In scenarios where a LIMIT is not present on a non aggregate query that will
> return a lot of results, Phoenix spools the results to disk. This is less
> than ideal in these situations. @larsh has created a very good and relatively
> simple implementation that is queue based to replace this.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
