[
https://issues.apache.org/jira/browse/HBASE-3421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13040082#comment-13040082
]
Todd Lipcon commented on HBASE-3421:
------------------------------------
I think I'm seeing this on one workload as well - I added a "stats" command for
the HFile tool, then ran it on a problematic 1.5GB gzip-compressed HFile that
won't compact due to OOME.
Key length: count: 272358037 min: 59 max: 2259 mean: 59.620136732003246
Val length: count: 272358037 min: 0 max: 0 mean: 0.0
Row size (bytes): count: 1287745 min: 67 max: 15615127736 mean:
14301.657317248368
Row size (columns): count: 1287745 min: 1 max: 233061608 mean:
211.49997631518661
so in this case there's actually a row with 233 million columns which takes up
15GB uncompressed!
> Very wide rows -- 30M plus -- cause us OOME
> -------------------------------------------
>
> Key: HBASE-3421
> URL: https://issues.apache.org/jira/browse/HBASE-3421
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.0
> Reporter: stack
>
> From the list, see 'jvm oom' in
> http://mail-archives.apache.org/mod_mbox/hbase-user/201101.mbox/browser, it
> looks like wide rows -- 30M or so -- causes OOME during compaction. We
> should check it out. Can the scanner used during compactions use the 'limit'
> when nexting? If so, this should save our OOME'ing (or, we need to add to
> the next a max size rather than count of KVs).
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
