[
https://issues.apache.org/jira/browse/PHOENIX-6436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17314632#comment-17314632
]
Lars Hofhansl commented on PHOENIX-6436:
----------------------------------------
Yet another option is to limit the maximum amount of RAM estimated per scan to
some fraction of the region size (it can't use more memory than the region
size).
Something like {{min((limit + offset) * estimatedEntrySize,
MAX_REGION_SIZE/10)}}, this does not have to be perfect just mostly right.
Perhaps one actually determine the actual region size and limit that way.
> OrderedResultIterator does bad size estimation
> ----------------------------------------------
>
> Key: PHOENIX-6436
> URL: https://issues.apache.org/jira/browse/PHOENIX-6436
> Project: Phoenix
> Issue Type: Wish
> Reporter: Lars Hofhansl
> Priority: Major
>
> Just came across this.
> The size estimation is: {{(limit + offset) * estimatedEntrySize}}
> with just the passed limit and offset, and this estimate is applied for each
> single scan.
> This is way too pessimistic when a large limit is passed as just a safety
> measure.
> Assuming you pass 10.000.000. That is the overall limit, but Phoenix will
> apply it to every scan (at least one per involved region) and take that much
> memory of the pool.
> Not sure what a better estimate would be. Ideally we'd divide by the number
> of involved regions with some fuss, or use a size estimate of the region.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
