Xuan Gong commented on YARN-2468:
bq. Also ContainerLogAggregator.uploadedFileMeta is also not needed to be a
I think ContainerLogAggregator.uploadedFileMeta is needed to be a class member.
It is used to keep track of all previous uploaded log files for each container.
We will use this information to decide whether this log can be aggregated.
New patch addressed other comments
> Log handling for LRS
> Key: YARN-2468
> URL: https://issues.apache.org/jira/browse/YARN-2468
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: log-aggregation, nodemanager, resourcemanager
> Reporter: Xuan Gong
> Assignee: Xuan Gong
> Attachments: YARN-2468.1.patch, YARN-2468.10.patch,
> YARN-2468.11.patch, YARN-2468.2.patch, YARN-2468.3.patch,
> YARN-2468.3.rebase.2.patch, YARN-2468.3.rebase.patch, YARN-2468.4.1.patch,
> YARN-2468.4.patch, YARN-2468.5.1.patch, YARN-2468.5.1.patch,
> YARN-2468.5.2.patch, YARN-2468.5.3.patch, YARN-2468.5.4.patch,
> YARN-2468.5.patch, YARN-2468.6.1.patch, YARN-2468.6.patch,
> YARN-2468.7.1.patch, YARN-2468.7.patch, YARN-2468.8.patch,
> YARN-2468.9.1.patch, YARN-2468.9.patch
> Currently, when application is finished, NM will start to do the log
> aggregation. But for Long running service applications, this is not ideal.
> The problems we have are:
> 1) LRS applications are expected to run for a long time (weeks, months).
> 2) Currently, all the container logs (from one NM) will be written into a
> single file. The files could become larger and larger.
This message was sent by Atlassian JIRA