Tarun Parimi created YARN-8627:
----------------------------------
Summary: EntityGroupFSTimelineStore hdfs done directory keeps on
accumulating
Key: YARN-8627
URL: https://issues.apache.org/jira/browse/YARN-8627
Project: Hadoop YARN
Issue Type: Bug
Components: timelineserver
Affects Versions: 2.8.0
Reporter: Tarun Parimi
Assignee: Tarun Parimi
The EntityLogCleaner threads exits with the following ERROR every time it runs.
{code:java}
2018-07-18 19:59:39,837 INFO timeline.EntityGroupFSTimelineStore
(EntityGroupFSTimelineStore.java:cleanLogs(462)) - Deleting
hdfs://namenode/ats/done/1499684568068/0000/018/application_1499684568068_18268
2018-07-18 19:59:39,844 INFO timeline.EntityGroupFSTimelineStore
(EntityGroupFSTimelineStore.java:cleanLogs(462)) - Deleting
hdfs://namenode/ats/done/1499684568068/0000/018/application_1499684568068_18270
2018-07-18 19:59:39,848 ERROR timeline.EntityGroupFSTimelineStore
(EntityGroupFSTimelineStore.java:run(899)) - Error cleaning files
java.io.FileNotFoundException: File
hdfs://namenode/ats/done/1499684568068/0000/018/application_1499684568068_18270
does not exist. at
org.apache.hadoop.hdfs.DistributedFileSystem$DirListingIterator.<init>(DistributedFileSystem.java:1062)
at
org.apache.hadoop.hdfs.DistributedFileSystem$DirListingIterator.<init>(DistributedFileSystem.java:1069)
at
org.apache.hadoop.hdfs.DistributedFileSystem$DirListingIterator.<init>(DistributedFileSystem.java:1040)
at
org.apache.hadoop.hdfs.DistributedFileSystem$23.doCall(DistributedFileSystem.java:1019)
at
org.apache.hadoop.hdfs.DistributedFileSystem$23.doCall(DistributedFileSystem.java:1015)
at
org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at
org.apache.hadoop.hdfs.DistributedFileSystem.listStatusIterator(DistributedFileSystem.java:1015)
at
org.apache.hadoop.yarn.server.timeline.EntityGroupFSTimelineStore.shouldCleanAppLogDir(EntityGroupFSTimelineStore.java:480)
{code}
Each time the thread gets scheduled, it is a different folder encountering the
error. As a result, the thread is not able to clean all the old done
directories, since it stops after this error.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
