[
https://issues.apache.org/jira/browse/HBASE-17288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15738352#comment-15738352
]
Andrew Purtell edited comment on HBASE-17288 at 12/10/16 7:30 PM:
------------------------------------------------------------------
The logging improvement is ok, but this patch changes whitespace unnecessarily
and includes this:
{code}
@@ -591,16 +607,11 @@ public class StoreScanner extends
NonReversedNonLazyKeyValueScanner
// Update local tracking information
count++;
- totalBytesRead += CellUtil.estimatedSerializedSizeOf(cell);
+ checkKvSize(cell);
// Update the progress of the scanner context
scannerContext.incrementSizeProgress(CellUtil.estimatedHeapSizeOf(cell));
scannerContext.incrementBatchProgress(1);
-
- if (matcher.isUserScan() && totalBytesRead > maxRowSize) {
- throw new RowTooBigException(
- "Max row size allowed: " + maxRowSize + ", but the row is
bigger than that.");
- }
}
{code}
which is a unrelated/unwanted? behavioral change because it moves the throw of
RowTooBigException up into checkKvSize() ahead of size and batch progress
accounting in the scanner context, and no longer updates totalBytesRead
was (Author: apurtell):
The logging improvement is ok, but this patch changes whitespace unnecessarily
and includes this:
{code}
@@ -591,16 +607,11 @@ public class StoreScanner extends
NonReversedNonLazyKeyValueScanner
// Update local tracking information
count++;
- totalBytesRead += CellUtil.estimatedSerializedSizeOf(cell);
+ checkKvSize(cell);
// Update the progress of the scanner context
scannerContext.incrementSizeProgress(CellUtil.estimatedHeapSizeOf(cell));
scannerContext.incrementBatchProgress(1);
-
- if (matcher.isUserScan() && totalBytesRead > maxRowSize) {
- throw new RowTooBigException(
- "Max row size allowed: " + maxRowSize + ", but the row is
bigger than that.");
- }
}
{code}
which is a unrelated/unwanted? behavioral change because it moves the throw of
RowTooBigException up into checkKvSize() ahead of size and batch progress
accounting in the scanner context.
> Add warn log for huge keyvalue and huge row
> -------------------------------------------
>
> Key: HBASE-17288
> URL: https://issues.apache.org/jira/browse/HBASE-17288
> Project: HBase
> Issue Type: Improvement
> Components: scan
> Reporter: Guanghao Zhang
> Assignee: Guanghao Zhang
> Priority: Minor
> Attachments: HBASE-17288.patch
>
>
> Some log examples from our production cluster.
> {code}
> 2016-12-10,17:08:11,478 WARN
> org.apache.hadoop.hbase.regionserver.StoreScanner: adding a HUGE KV into
> result list, kv size:1253360,
> kv:105671140000001-1-c/R:r1/1481360887152/Put/vlen=1253245/ts=923099, from
> table XXXXX
> 2016-12-10,17:08:16,724 WARN
> org.apache.hadoop.hbase.regionserver.StoreScanner: adding a HUGE KV into
> result list, kv size:1048680,
> kv:0220459/I:i_0/1481360889551/Put/vlen=1048576/ts=13642, from table XXXXXX
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
