[
https://issues.apache.org/jira/browse/YARN-3505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14539808#comment-14539808
]
Junping Du commented on YARN-3505:
----------------------------------
Thanks [~xgong] for updating the patch! Latest patch get much closer.
{code}
+ private static final String MAX_LOG_AGGREGATION_DIAGNOSTICS_IN_MEMORY =
+ YarnConfiguration.RM_PREFIX +
"max-log-aggregation-diagnostics-in-memory";
+ private static int DEFAULT_MAX_LOG_AGGREGATION_DIAGNOSTICS_IN_MEMORY = 10;
{code}
Do we want to expose this configuration to user? If so, we should put it in
yarn-default.xml with properly description on it.
{code}
+ // If the log aggregation status got from latest nm heartbeat
+ // is Running, and current log aggregation status is TimeOut,
+ // based on whether there are any failure messages for this NM,
+ // we will reset the log aggregation status as RUNNING or
+ // RUNNING_WITH_FAILURE
+ if (logAggregationFailureMessagesForNMs.get(nodeId) != null
+ &&
!logAggregationFailureMessagesForNMs.get(nodeId).isEmpty()) {
+ curReport
+
.setLogAggregationStatus(LogAggregationStatus.RUNNING_WITH_FAILURE);
+ }
+ }
+ curReport.setLogAggregationStatus(report
+ .getLogAggregationStatus());
{code}
"curReport.setLogAggregationStatus(LogAggregationStatus.RUNNING_WITH_FAILURE);
" should be replaced by "report.setLogAggregationStatus(..." ? Or it could be
override later by RUNNING.
Other looks fine to me.
> Node's Log Aggregation Report with SUCCEED should not cached in RMApps
> ----------------------------------------------------------------------
>
> Key: YARN-3505
> URL: https://issues.apache.org/jira/browse/YARN-3505
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: log-aggregation
> Affects Versions: 2.8.0
> Reporter: Junping Du
> Assignee: Xuan Gong
> Priority: Critical
> Attachments: YARN-3505.1.patch, YARN-3505.2.patch,
> YARN-3505.2.rebase.patch, YARN-3505.3.patch, YARN-3505.4.patch
>
>
> Per discussions in YARN-1402, we shouldn't cache all node's log aggregation
> reports in RMApps for always, especially for those finished with SUCCEED.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
