[ https://issues.apache.org/jira/browse/YARN-5269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15905481#comment-15905481 ]
Haibo Chen commented on YARN-5269: ---------------------------------- Having putEntities call putEntitiesAsync and writer.flush() will make sure data is flushed to HBase when putEntities() returns. It however, may have issues for TimelineClients under certain circumstances (if these circumstances are possible). The scenario I am think of is, before a client sends some entities through the synchronous putEntities() call, there are already some data in the BufferredMutator buffer, so when writer.flush() is called, I'd assume that the client's data as well as the data that is already in the buffer will be flushed to HBase together. If somehow HBase backend returns an error because of the data that is already buffer (i.e., not because of this specific client's data), how are we going to handle this? Does HBase allow us to tell if the error is caused by client's data or not? One reason I can think of why clients want to use the synchronous putEntities() call is for critical data. If clients retry if they see an error/exception is returned just to ensure data is persisted in HBase, all of a sudden multiple copies of the same data in HBase. My apologies for my imaginary cases if they are unrealistic. It seems to me that with all data going though BufferedMutator, putEntitiesAsync + writer.flush() is probably the best we can provide for putEntitties(). Regardless of whether SpoolingBufferedMutator can do better or not, I think we should add some javadoc to describe what we can provide. > Bubble exceptions and errors all the way up the calls, including to clients. > ---------------------------------------------------------------------------- > > Key: YARN-5269 > URL: https://issues.apache.org/jira/browse/YARN-5269 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver > Affects Versions: YARN-2928 > Reporter: Joep Rottinghuis > Assignee: Haibo Chen > Labels: YARN-5355, yarn-5355-merge-blocker > > Currently we ignore (swallow) exception from the HBase side in many cases > (reads and writes). > Also, on the client side, neither TimelineClient#putEntities (the v2 flavor) > nor the #putEntitiesAsync method return any value. > For the second drop we may want to consider how we properly bubble up > exceptions throughout the write and reader call paths and if we want to > return a response in putEntities and some future kind of result for > putEntitiesAsync. -- This message was sent by Atlassian JIRA (v6.3.15#6346) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org