[
https://issues.apache.org/jira/browse/YARN-6339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15935235#comment-15935235
]
Wangda Tan edited comment on YARN-6339 at 3/21/17 8:18 PM:
-----------------------------------------------------------
[~zhaoyunjiong], thanks for explanation, generally make sense to me.
Few more suggestions:
1) Can we make use of {{isLogAggregationFinished}} in
{{createAndGetApplicationReport}} to avoid following checks:
{code}
if (LogAggregationStatus.FAILED == logAggregationStatusApp ||
LogAggregationStatus.SUCCEEDED == logAggregationStatusApp ||
LogAggregationStatus.TIME_OUT == logAggregationStatusApp) {
try {
this.writeLock.lock();
this.logAggregationStatusForAppReport = logAggregationStatusApp;
} finally {
this.writeLock.unlock();
}
}
{code}
2) Can we make logAggregationStatusForAppReport to be volatile to avoid the
additional writelock of {{createAndGetApplicationReport}}?
was (Author: leftnoteasy):
[~zhaoyunjiong], thanks for explanation, generally make sense to me.
Few more suggestions:
1) Can we make use of {{isLogAggregationFinished}} in
{{createAndGetApplicationReport}} to avoid following checks:
{code}
if (LogAggregationStatus.FAILED == logAggregationStatusApp ||
LogAggregationStatus.SUCCEEDED == logAggregationStatusApp ||
LogAggregationStatus.TIME_OUT == logAggregationStatusApp) {
try {
this.writeLock.lock();
this.logAggregationStatusForAppReport = logAggregationStatusApp;
} finally {
this.writeLock.unlock();
}
}
{code}
2) Can we make logAggregationStatusForAppReport to be volatile to avoid the
additional writelock?
> Improve performance for createAndGetApplicationReport
> -----------------------------------------------------
>
> Key: YARN-6339
> URL: https://issues.apache.org/jira/browse/YARN-6339
> Project: Hadoop YARN
> Issue Type: Improvement
> Reporter: yunjiong zhao
> Assignee: yunjiong zhao
> Attachments: YARN-6339.001.patch, YARN-6339.002.patch
>
>
> There are two performance issue when calling createAndGetApplicationReport:
> One is inside ProtoUtils.convertFromProtoFormat, replace is too slow for
> clusters which have more than 3000 nodes. Use substring is much better:
> https://issues.apache.org/jira/browse/YARN-6285?focusedCommentId=15923241&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15923241
> Another one is inside getLogAggregationReportsForApp, if some application's
> LogAggregationStatus is TIME_OUT, every time it was called it will create an
> HashMap which will produce lots of garbage.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
