[
https://issues.apache.org/jira/browse/HBASE-3323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12971136#action_12971136
]
Todd Lipcon commented on HBASE-3323:
------------------------------------
bq. Is stuff protected rather than private for the subclassers - the
transactional hbasers?
Yep, I made it protected since inheritors will need to access this stuff.
Hopefully we can get rid of the subclasses in favor of log coprocessors some
day.
bq. Minor.... the below javadoc is now stale..
bq. ...Its javadoc of params no longer present on this method.
bq. Mistype '+ "An HLogSplitter instance may only be used one");'
bq. Extremely minor comment, the below formatting will be destroyed when
rendered by javadoc:
Fixed.
bq. So, it looks like we keep the order in which edits were written across the
split process as best as I can tell. We just append to the Entry List in
RegionEntryBuffer. That looks right.
Added verification to the testThreading test that makes sure the edits come in
the right order
bq. You iterate logWriters, a synchronized Map, a couple of times. Is this safe
at the time of iteration?
It was safe in the current usage, but I added synchronization on this map for
getOutputCounts just in case someone starts to call it in a different context.
bq. You keep the old format for naming edit files? Naming them for the
sequenceid of their first edit, it seems (you use getRegionSplitEditsPath - not
in the patch).
Yep, left that as-is.
> OOME in master splitting logs
> -----------------------------
>
> Key: HBASE-3323
> URL: https://issues.apache.org/jira/browse/HBASE-3323
> Project: HBase
> Issue Type: Bug
> Components: master
> Affects Versions: 0.90.0
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Priority: Blocker
> Fix For: 0.90.0
>
> Attachments: hbase-3323.4.txt, hbase-3323.txt, hbase-3323.txt,
> hbase-3323.txt, sizes.png
>
>
> In testing a RS failure under heavy increment workload I ran into an OOME
> when the master was splitting the logs.
> In this test case, I have exactly 136 bytes per log entry in all the logs,
> and the logs are all around 66-74MB). With a batch size of 3 logs, this means
> the master is loading about 500K-600K edits per log file. Each edit ends up
> creating 3 byte[] objects, the references for which are each 8 bytes of RAM,
> so we have 160 (136+8*3) bytes per edit used by the byte[]. For each edit we
> also allocate a bunch of other objects: one HLog$Entry, one WALEdit, one
> ArrayList, one LinkedList$Entry, one HLogKey, and one KeyValue. Overall this
> works out to 400 bytes of overhead per edit. So, with the default settings on
> this fairly average workload, the 1.5M log entries takes about 770MB of RAM.
> Since I had a few log files that were a bit larger (around 90MB) it exceeded
> 1GB of RAM and I got an OOME.
> For one, the 400 bytes per edit overhead is pretty bad, and we could probably
> be a lot more efficient. For two, we should actually account this rather than
> simply having a configurable "batch size" in the master.
> I think this is a blocker because I'm running with fairly default configs
> here and just killing one RS made the cluster fall over due to master OOME.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
