[
https://issues.apache.org/jira/browse/HBASE-11544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14377104#comment-14377104
]
stack commented on HBASE-11544:
-------------------------------
bq ...or is the chance of very large rows so severe that pushing this all the
way down into the scanner is absolutely necessary
Up for discussion any time but to echo Jonathan, we need the accounting at a
lowest levels so we trigger the crossing of the size threshold closer to where
it actually happens. Ain't sure how else we'd do it. All ears though. Multiple
benefits when trigger happens the sooner: avoid the subjects' OOME if the cells
that make up a row are beyond heap capacity, but also A. in Lars's A+B list
above.
Any tests you lot think I should run? No problem. Just say. I do like the
zeroing in on object creation. No harm undoing this if we can
([~jonathan.lawlor] and I will give a looke see)....
Thanks.
> [Ergonomics] hbase.client.scanner.caching is dogged and will try to return
> batch even if it means OOME
> ------------------------------------------------------------------------------------------------------
>
> Key: HBASE-11544
> URL: https://issues.apache.org/jira/browse/HBASE-11544
> Project: HBase
> Issue Type: Bug
> Reporter: stack
> Assignee: Jonathan Lawlor
> Priority: Critical
> Fix For: 2.0.0, 1.1.0
>
> Attachments: HBASE-11544-branch_1_0-v1.patch,
> HBASE-11544-branch_1_0-v2.patch, HBASE-11544-v1.patch, HBASE-11544-v2.patch,
> HBASE-11544-v3.patch, HBASE-11544-v4.patch, HBASE-11544-v5.patch,
> HBASE-11544-v6.patch, HBASE-11544-v6.patch, HBASE-11544-v6.patch,
> HBASE-11544-v7.patch, HBASE-11544-v8-branch-1.patch, HBASE-11544-v8.patch,
> gc.j.png, hits.j.png, mean.png, net.j.png
>
>
> Running some tests, I set hbase.client.scanner.caching=1000. Dataset has
> large cells. I kept OOME'ing.
> Serverside, we should measure how much we've accumulated and return to the
> client whatever we've gathered once we pass out a certain size threshold
> rather than keep accumulating till we OOME.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
