[
https://issues.apache.org/jira/browse/YARN-3949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14640974#comment-14640974
]
Sangjin Lee commented on YARN-3949:
-----------------------------------
For the hbase writer, {{flush()}} and {{close()}} (or any other method for that
matter) are synchronized. So they will not execute concurrently.
I think the only difference between {{shutdown()}} and {{shutdownNow()}} in
this case is whether we would want to interrupt the running {{flush()}} and go
straight to {{close()}} (for {{shutdownNow()}}), or let {{flush()}} finish and
go to {{close()}} (for {{shutdown()}}). Again, that's only for hbase.
Since {{shutdown()}} is more graceful and will let {{flush()}} finish, I'm bit
more inclined to do that now. I'd still need to introduce a call to
{{awaitTermination()}} to be correct in all cases then although the hbase
writer doesn't really need it. Let me see what I can do, along with those
comments for the configuration.
I'm stepping out for now, but will post a new patch by EOD.
> ensure timely flush of timeline writes
> --------------------------------------
>
> Key: YARN-3949
> URL: https://issues.apache.org/jira/browse/YARN-3949
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
> Affects Versions: YARN-2928
> Reporter: Sangjin Lee
> Assignee: Sangjin Lee
> Attachments: YARN-3949-YARN-2928.001.patch,
> YARN-3949-YARN-2928.002.patch, YARN-3949-YARN-2928.002.patch
>
>
> Currently flushing of timeline writes is not really handled. For example,
> {{HBaseTimelineWriterImpl}} relies on HBase's {{BufferedMutator}} to batch
> and write puts asynchronously. However, {{BufferedMutator}} may not flush
> them to HBase unless the internal buffer fills up.
> We do need a flush functionality first to ensure that data are written in a
> reasonably timely manner, and to be able to ensure some critical writes are
> done synchronously (e.g. key lifecycle events).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
