[
https://issues.apache.org/jira/browse/YARN-11585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
hansonhe updated YARN-11585:
----------------------------
Component/s: resourcemanager
timelineservice
yarn
Affects Version/s: 3.1.2
(was: 3.1.3)
Description:
In my hadoop cluster :
set yarn.timeline-service.generic-application-history.enabled=true;
set
yarn.timeline-service.generic-application-history.store-class=org.apache.hadoop.yarn.server.applicationhistoryservice.FileSystemApplicationHistoryStore;
set
yarn.timeline-service.generic-application-history.fs-history-store.uri=/user/yarn/timeline/generic_history;
hdfs dfs -count /user/yarn/timeline/generic_history/ApplicationHistoryDataRoot
1 3150832 82436180812
the number of files in it will increase to the limit of
dfs.namenode.fs-limits.max-directory-items=3200000 in less than one month and
increase too fast.Now the files were cleaned manually. Does have any ttl
suggestions to clean automatically ？what should i do?
was:
In my cluster hdfs path:
hdfs dfs -count /user/yarn/timeline/generic_history/ApplicationHistoryDataRoot
1 3150832 82436180812
the number of files in it will increase to the limit of
dfs.namenode.fs-limits.max-directory-items=3200000 in less than one month and
increase too fast.Now the files were cleaned manually. Does have any ttl
suggestions to clean automatically ？what should i do?
> Yarn Timeline generic-application-history.fs-history-store application log
> files increase too much,How to set ttl?
> ------------------------------------------------------------------------------------------------------------------
>
> Key: YARN-11585
> URL: https://issues.apache.org/jira/browse/YARN-11585
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: resourcemanager, timelineservice, yarn
> Affects Versions: 3.1.2
> Reporter: hansonhe
> Priority: Major
>
> In my hadoop cluster :
> set yarn.timeline-service.generic-application-history.enabled=true;
> set
> yarn.timeline-service.generic-application-history.store-class=org.apache.hadoop.yarn.server.applicationhistoryservice.FileSystemApplicationHistoryStore;
> set
> yarn.timeline-service.generic-application-history.fs-history-store.uri=/user/yarn/timeline/generic_history;
> hdfs dfs -count /user/yarn/timeline/generic_history/ApplicationHistoryDataRoot
> 1 3150832 82436180812
> the number of files in it will increase to the limit of
> dfs.namenode.fs-limits.max-directory-items=3200000 in less than one month and
> increase too fast.Now the files were cleaned manually. Does have any ttl
> suggestions to clean automatically ？what should i do?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
