[
https://issues.apache.org/jira/browse/YARN-5269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15905481#comment-15905481
]
Haibo Chen commented on YARN-5269:
----------------------------------
Having putEntities call putEntitiesAsync and writer.flush() will make sure data
is flushed to HBase when putEntities() returns. It however, may have issues for
TimelineClients under certain circumstances (if these circumstances are
possible). The scenario I am think of is, before a client sends some entities
through the synchronous putEntities() call, there are already some data in the
BufferredMutator buffer, so when writer.flush() is called, I'd assume that the
client's data as well as the data that is already in the buffer will be flushed
to HBase together. If somehow HBase backend returns an error because of the
data that is already buffer (i.e., not because of this specific client's data),
how are we going to handle this? Does HBase allow us to tell if the error is
caused by client's data or not? One reason I can think of why clients want to
use the synchronous putEntities() call is for critical data. If clients retry
if they see an error/exception is returned just to ensure data is persisted in
HBase, all of a sudden multiple copies of the same data in HBase. My apologies
for my imaginary cases if they are unrealistic.
It seems to me that with all data going though BufferedMutator,
putEntitiesAsync + writer.flush() is probably the best we can provide for
putEntitties(). Regardless of whether SpoolingBufferedMutator can do better or
not, I think we should add some javadoc to describe what we can provide.
> Bubble exceptions and errors all the way up the calls, including to clients.
> ----------------------------------------------------------------------------
>
> Key: YARN-5269
> URL: https://issues.apache.org/jira/browse/YARN-5269
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
> Affects Versions: YARN-2928
> Reporter: Joep Rottinghuis
> Assignee: Haibo Chen
> Labels: YARN-5355, yarn-5355-merge-blocker
>
> Currently we ignore (swallow) exception from the HBase side in many cases
> (reads and writes).
> Also, on the client side, neither TimelineClient#putEntities (the v2 flavor)
> nor the #putEntitiesAsync method return any value.
> For the second drop we may want to consider how we properly bubble up
> exceptions throughout the write and reader call paths and if we want to
> return a response in putEntities and some future kind of result for
> putEntitiesAsync.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
