[
https://issues.apache.org/jira/browse/YARN-4061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15755117#comment-15755117
]
Joep Rottinghuis commented on YARN-4061:
----------------------------------------
For the next version of the patch I'll modify the code so that even if HBase is
down we can create a SpoolingBufferedMutatorImpl. I'll check a little whether
we need to also push down the connection creation into the code. Part of the
issue there is that we now will be creating multiple connections rather than
having one shared one. Keep in mind that we have a BufferedMutator per table.
It would be unfortunate to create 5 connections.
wrt. "at least once" here is my reply from the Google doc:
bq. Yes, we guarantee at least once, if one considers spooling the mutation a
delivery. The code to replay those mutations is still missing. I'm not sure if
that should belong here, or if that should be a separate thing.
bq. From the Yarn timeline perspective we set the timestamp on each put
explicitly exactly for the reason of delayed mutations. We want to know when
they happened, and not when they were submitted. For our usecase, that makes
the puts idempotent. Other use-cases may not need this requirement, but they do
need to deal with duplicate puts.
> [Fault tolerance] Fault tolerant writer for timeline v2
> -------------------------------------------------------
>
> Key: YARN-4061
> URL: https://issues.apache.org/jira/browse/YARN-4061
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
> Reporter: Li Lu
> Assignee: Joep Rottinghuis
> Labels: YARN-5355
> Attachments: FaulttolerantwriterforTimelinev2.pdf
>
>
> We need to build a timeline writer that can be resistant to backend storage
> down time and timeline collector failures.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
