[ 
https://issues.apache.org/jira/browse/PHOENIX-539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14054522#comment-14054522
 ] 

Gabriel Reid commented on PHOENIX-539:
--------------------------------------

Thanks for taking a look [~jamestaylor]. 

I would definitely be interesting if [~lhofhansl] could take a look and give 
more feedback on his original idea. The one thing that I would possibly worry 
about with leaving the scanners open on the server is running into scanner 
lease timeouts -- I actually assumed that this was the original reason why the 
SpoolingResultIterator was used in the first place.

I'll upload a patch with the points you noted (doc on why chunking is disabled 
when an order by is present or a join is being done) and a separate 
HashJoinInfo.isHashJoin(Scan) method. 

For reference (without having to look through the new patch), the reason that 
the ChunkedResultIterator isn't used for when a join or order by are done is 
because the scans used for those operations don't work properly with restarting 
at the last-encountered row. As far as I understand, it's probably better to 
not do chunking there anyhow, as the scan results for those operations are only 
useful in their entirety, so chunking would probably only slow things down even 
if it did work there.



> Implement parallel scanner that does not spool to disk
> ------------------------------------------------------
>
>                 Key: PHOENIX-539
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-539
>             Project: Phoenix
>          Issue Type: Task
>            Reporter: James Taylor
>            Assignee: larsh
>         Attachments: PHOENIX-539.1.patch, PHOENIX-539.patch
>
>
> In scenarios where a LIMIT is not present on a non aggregate query that will 
> return a lot of results, Phoenix spools the results to disk. This is less 
> than ideal in these situations. @larsh has created a very good and relatively 
> simple implementation that is queue based to replace this.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Reply via email to