[
https://issues.apache.org/jira/browse/YARN-3949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15369778#comment-15369778
]
Hudson commented on YARN-3949:
------------------------------
SUCCESS: Integrated in Hadoop-trunk-Commit #10074 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/10074/])
YARN-3949. Ensure timely flush of timeline writes. Contributed by (sjlee: rev
8603736ef2bb34605ea10f516dbe4e73bed3f309)
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice/src/main/java/org/apache/hadoop/yarn/server/timelineservice/storage/TimelineWriter.java
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice/src/test/java/org/apache/hadoop/yarn/server/timelineservice/collector/TestNMTimelineCollectorManager.java
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice/src/main/java/org/apache/hadoop/yarn/server/timelineservice/storage/HBaseTimelineWriterImpl.java
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice/src/main/java/org/apache/hadoop/yarn/server/timelineservice/collector/TimelineCollectorManager.java
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice/src/main/java/org/apache/hadoop/yarn/server/timelineservice/storage/FileSystemTimelineWriterImpl.java
*
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice/src/main/java/org/apache/hadoop/yarn/server/timelineservice/storage/PhoenixTimelineWriterImpl.java
> ensure timely flush of timeline writes
> --------------------------------------
>
> Key: YARN-3949
> URL: https://issues.apache.org/jira/browse/YARN-3949
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
> Affects Versions: YARN-2928
> Reporter: Sangjin Lee
> Assignee: Sangjin Lee
> Fix For: YARN-2928
>
> Attachments: YARN-3949-YARN-2928.001.patch,
> YARN-3949-YARN-2928.002.patch, YARN-3949-YARN-2928.002.patch,
> YARN-3949-YARN-2928.003.patch, YARN-3949-YARN-2928.004.patch,
> YARN-3949-YARN-2928.004.patch
>
>
> Currently flushing of timeline writes is not really handled. For example,
> {{HBaseTimelineWriterImpl}} relies on HBase's {{BufferedMutator}} to batch
> and write puts asynchronously. However, {{BufferedMutator}} may not flush
> them to HBase unless the internal buffer fills up.
> We do need a flush functionality first to ensure that data are written in a
> reasonably timely manner, and to be able to ensure some critical writes are
> done synchronously (e.g. key lifecycle events).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
