[
https://issues.apache.org/jira/browse/HBASE-17018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15747317#comment-15747317
]
stack commented on HBASE-17018:
-------------------------------
bq, I don't want those attributes to be transmitted to HBase, that would be a
waste.
The bit where you'd have to peel-apart-a-submission to mark all Mutations kills
my suggestion I'd say.
bq. Is that a question or statement? If the HBase community is interested to
have this part of HBase, that would be great and I'll continue the code in
place. If not, then I'll move this to yarn.
Question. IMO, it is an exotic feature. Lets see if anyone else interested. If
not, probably better have it live w/ TimelineV2.
bq. Reformatting javadoc wasn't intended. I'll remove that. In fact, I'll file
a separate sub-jira to just add the clone method to the BufferedMutatorParams
to separate out that concern.
Good. Lets get that in.
Patch looks great so far. Is there anything point in AP that could be exposed
that might help simplify the implementation at all? Your approach of treating
BMI as black box is probably best way to go... justifies all the rigging you
have about... rigging that you'd probably end up building anyway because AP
signaling would likely be mixed, not-clean.
> Spooling BufferedMutator
> ------------------------
>
> Key: HBASE-17018
> URL: https://issues.apache.org/jira/browse/HBASE-17018
> Project: HBase
> Issue Type: New Feature
> Reporter: Joep Rottinghuis
> Attachments: HBASE-17018.master.001.patch,
> HBASE-17018SpoolingBufferedMutatorDesign-v1.pdf, YARN-4061 HBase requirements
> for fault tolerant writer.pdf
>
>
> For Yarn Timeline Service v2 we use HBase as a backing store.
> A big concern we would like to address is what to do if HBase is
> (temporarily) down, for example in case of an HBase upgrade.
> Most of the high volume writes will be mostly on a best-effort basis, but
> occasionally we do a flush. Mainly during application lifecycle events,
> clients will call a flush on the timeline service API. In order to handle the
> volume of writes we use a BufferedMutator. When flush gets called on our API,
> we in turn call flush on the BufferedMutator.
> We would like our interface to HBase be able to spool the mutations to a
> filesystems in case of HBase errors. If we use the Hadoop filesystem
> interface, this can then be HDFS, gcs, s3, or any other distributed storage.
> The mutations can then later be re-played, for example through a MapReduce
> job.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
