[
https://issues.apache.org/jira/browse/HBASE-23158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17489351#comment-17489351
]
Yutong Xiao edited comment on HBASE-23158 at 2/10/22, 6:00 AM:
---------------------------------------------------------------
Encountered this problem also. For this problem, could we just use the number
of cells times a predefined blocksize to approximate the total block size (one
cell, one block)?
For cells in memstore, they will be flushed in short time, so that we can just
regard them as cells read from HFiles I think. This may solve this annoying
exception.
The problem is the approximation is rough (different table can have different
blocksize).
was (Author: xytss123):
Encountered this problem also. For this problem, could we just use the number
of cells times a predefined blocksize to approximate the total block size (one
cell, one block)?
For cells in memstore, they will be flushed in short time, so that we can just
regard them as cells read from HFiles I think. This may solve this annoying
exception.
The problem is the approximation is rough granularity (different table can have
different blocksize).
What do you think?
> If KVs are in memstore, small batch get can come across
> MultiActionResultTooLarge
> ----------------------------------------------------------------------------------
>
> Key: HBASE-23158
> URL: https://issues.apache.org/jira/browse/HBASE-23158
> Project: HBase
> Issue Type: Bug
> Components: regionserver, rpc
> Environment: [^TestMultiRespectsLimitsMemstore.patch]
> Reporter: junfei liang
> Priority: Minor
> Attachments: TestMultiRespectsLimitsMemstore.patch
>
>
> to protect against big scan, we set hbase.server.scanner.max.result.size =
> 10MB in our customer hbase cluster, however our clients can meet
> MultiActionResultTooLarge even in small batch get (for ex. 15 batch get, and
> row size is about 5KB ) .
> after [HBASE-14978|https://issues.apache.org/jira/browse/HBASE-14978] hbase
> take the data block reference into consideration, but the block size is 64KB
> (the default value ), even if all cells are from different block , the block
> size retained is less than 1MB, so what's the problem ?
> finally i found that HBASE-14978 also consider the cell in memstore, as
> MSLAB is enabled default, so if the cell is from memstore, cell backend array
> can be large (2MB as default), so even if a small batch can meet this error,
> is this reasonable ?
> plus:
> when throw MultiActionResultTooLarge exception, hbase client should retry
> ignore rpc retry num, however if set retry num to zero, client will fail
> without retry in this case.
>
> see attachment TestMultiRespectsLimitsMemstore for details.
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
