[
https://issues.apache.org/jira/browse/YARN-4696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15183986#comment-15183986
]
Li Lu commented on YARN-4696:
-----------------------------
Thanks [[email protected]] for the work! Mostly LGTM, only a few nits:
FileSystemTimelineWriter.java
- TIMELINE_SERVICE_ENTITYFILE_FS_SUPPORT_APPEND move to YarnConfiguration?
- Why LogFDsCache#flush was changed into synchronized? I believe we're doing
fine-grained locking here (with each of the FDs), and only flush in LogFDsCache
is marked as synchronized? What am I missing here? cc/[~xgong]
TimelineWriter.java
- Not sure if "Direct timeline writer" is clear enough to indicate where the
data goes to and which pattern the writer is following? By saying "direct"
here, do we mean we're using a write-through strategy?
EntityGroupFSTimelineStore.java
- In scanActiveLogs, the new variable "scanned" looks like a little bit
confusing: when we return the variable scanned, the actual scanning jobs are
not guaranteed to be done. So it looks like something "to be scanned" when we
return? My only concern is this naming may give people false indication that by
the time this method returns, there are a number of logs that are already
scanned. This also applies to EntityLogScanner.
> EntityGroupFSTimelineStore to work in the absence of an RM
> ----------------------------------------------------------
>
> Key: YARN-4696
> URL: https://issues.apache.org/jira/browse/YARN-4696
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
> Affects Versions: 2.8.0
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Attachments: YARN-4696-001.patch, YARN-4696-002.patch,
> YARN-4696-003.patch, YARN-4696-005.patch, YARN-4696-006.patch,
> YARN-4696-007.patch, YARN-4696-008.patch, YARN-4696-009.patch,
> YARN-4696-010.patch, YARN-4696-012.patch
>
>
> {{EntityGroupFSTimelineStore}} now depends on an RM being up and running; the
> configuration pointing to it. This is a new change, and impacts testing where
> you have historically been able to test without an RM running.
> The sole purpose of the probe is to automatically determine if an app is
> running; it falls back to "unknown" if not. If the RM connection was
> optional, the "unknown" codepath could be called directly, relying on age of
> file as a metric of completion
> Options
> # add a flag to disable RM connect
> # skip automatically if RM not defined/set to 0.0.0.0
> # disable retries on yarn client IPC; if it fails, tag app as unknown.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
