Modifying /etc/security/limits.conf won't take immediate effect. Either restart the system or check out the ulimit -n command and try again.
Thanks, Jon On Tue, May 31, 2016 at 11:48 PM, Mungeol Heo <[email protected]> wrote: > Hi, > > After starting app timeline server, > '/var/log/hadoop-yarn/yarn/yarn-yarn-timelineserver-hostname.net.log' > gives many message like below. > > ... > 2016-06-01 12:36:28,641 INFO timeline.RollingLevelDB > (RollingLevelDB.java:initRollingLevelDB(266)) - Added rolling leveldb > instance 2016-05-23-08 to indexes-ldb > 2016-06-01 12:36:28,641 INFO timeline.RollingLevelDB > (RollingLevelDB.java:initRollingLevelDB(258)) - Initializing rolling > leveldb instance > :file:/data/hadoop/yarn/timeline/leveldb-timeline-store/indexes-ldb.2016-05-09-14 > for start time: 1462802400000 > ... > > Then, it gives 'too mnay open files' error. > > ... > 2016-06-01 12:36:28,715 INFO service.AbstractService > (AbstractService.java:noteFailure(272)) - Service > org.apache.hadoop.yarn.server.timeline.RollingLevelDBTimelineStore > failed in state INITED; cause: > org.fusesource.leveldbjni.internal.NativeDB$DBException: IO error: > /data/hadoop/yarn/timeline/leveldb-timeline-store/starttime-ldb/LOCK: > Too many open files > org.fusesource.leveldbjni.internal.NativeDB$DBException: IO error: > /data/hadoop/yarn/timeline/leveldb-timeline-store/starttime-ldb/LOCK: > Too many open files > at > org.fusesource.leveldbjni.internal.NativeDB.checkStatus(NativeDB.java:200) > at org.fusesource.leveldbjni.internal.NativeDB.open(NativeDB.java:218) > at org.fusesource.leveldbjni.JniDBFactory.open(JniDBFactory.java:168) > at > org.apache.hadoop.yarn.server.timeline.RollingLevelDBTimelineStore.serviceInit(RollingLevelDBTimelineStore.java:324) > at > org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) > at > org.apache.hadoop.yarn.server.timeline.EntityGroupFSTimelineStore.serviceInit(EntityGroupFSTimelineStore.java:151) > at > org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) > at > org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107) > at > org.apache.hadoop.yarn.server.applicationhistoryservice.ApplicationHistoryServer.serviceInit(ApplicationHistoryServer.java:104) > at > org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) > at > org.apache.hadoop.yarn.server.applicationhistoryservice.ApplicationHistoryServer.launchAppHistoryServer(ApplicationHistoryServer.java:168) > at > org.apache.hadoop.yarn.server.applicationhistoryservice.ApplicationHistoryServer.main(ApplicationHistoryServer.java:178) > ... > > I've tired to increase nofile from 65000 to 650000 at > '/etc/security/limits.conf'. > And also increased 'yarn_user_nofile_limit', which is a yarn config, > value to 65536. > However, nothing works. > > Any help will be grate. > Thank you. > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
