[
https://issues.apache.org/jira/browse/YARN-10648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
D M Murali Krishna Reddy updated YARN-10648:
--------------------------------------------
Description:
YARN-8273 has induced the following issues.
# The {color:#000000}delService.delete(deletionTask){color} has been removed
from the for loop, and added at the end in finally block. Inside the for loop
we are creating FileDeletionTask for each container, but not storing it, due to
this, only the last container log files will be present in the deletionTask and
only those files will be removed. Ideally all the container log files which are
uploaded must be deleted.
# The LogAggregationDFSException is caught in the closeswriter, but when we
configure LogAggregationTFileController as logAggregationFileController,
this.logAggregationFileController.closeWriter() itself calls closeWriter,
which throws LogAggregationDFSException if any, and the exception is not saved.
Again when we try to do closeWriter we dont get any exception and, we are not
throwing the LogAggregationDFSException in this scenario.
was:
YARN-8273 has induced the following issues.
# The {color:#000000}delService.delete(deletionTask){color} has been removed
from the for loop, and added at the end in finally block. Inside the for loop
we are creating FileDeletionTask for each container, but not storing it, due to
this, only the last container log files will be present in the deletionTask and
only those files will be removed. Ideally all the container log files which are
uploaded must be deleted.
# The LogAggregationDFSException is caught in the closeswriter, but when we
configure LogAggregationTFileController as logAggregationFileController,
this.logAggregationFileController.closeWriter() itself calls closeWriter,
which throws LogAggregationDFSException if any, and the exception is not saved.
Again when we try to do closeWriter we dont get any exception and, we are not
throwing the LogAggregationDFSException in this scenario.
> NM local logs are not cleared after uploading to hdfs
> -----------------------------------------------------
>
> Key: YARN-10648
> URL: https://issues.apache.org/jira/browse/YARN-10648
> Project: Hadoop YARN
> Issue Type: Bug
> Components: log-aggregation
> Affects Versions: 3.2.0
> Reporter: D M Murali Krishna Reddy
> Assignee: D M Murali Krishna Reddy
> Priority: Major
>
> YARN-8273 has induced the following issues.
> # The {color:#000000}delService.delete(deletionTask){color} has been removed
> from the for loop, and added at the end in finally block. Inside the for loop
> we are creating FileDeletionTask for each container, but not storing it, due
> to this, only the last container log files will be present in the
> deletionTask and only those files will be removed. Ideally all the container
> log files which are uploaded must be deleted.
> # The LogAggregationDFSException is caught in the closeswriter, but when we
> configure LogAggregationTFileController as logAggregationFileController,
> this.logAggregationFileController.closeWriter() itself calls closeWriter,
> which throws LogAggregationDFSException if any, and the exception is not
> saved. Again when we try to do closeWriter we dont get any exception and, we
> are not throwing the LogAggregationDFSException in this scenario.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
