https://askubuntu.com/questions/181215/too-many-open-files-how-to-find-the-culprit
You can do some investigation to see which process open so many files, please refer to above link. I did encounter such a problem several years ago but didn't see it for some time. Best regards, Shaofeng Shi 史少锋 Apache Kylin PMC Email: [email protected] Apache Kylin FAQ: https://kylin.apache.org/docs/gettingstarted/faq.html Join Kylin user mail group: [email protected] Join Kylin dev mail group: [email protected] 恩爸 <[email protected]> 于2020年7月9日周四 下午10:31写道： > Hi, you can use command 'ulimit -a' to see the limitation of opening > files, and google how to modify this limitation. > > ------------------------------ > > > Best regards, > Zhichao Zhang > > > > > ------------------ 原始邮件 ------------------ > *发件人:* "user" <[email protected]>; > *发送时间:* 2020年7月9日(星期四) 下午5:43 > *收件人:* "user"<[email protected]>; > *抄送:* "crowgns"<[email protected]>; > *主题:* 回复：Re: 执行cube在第4步报错，提示“打开的文件过多” 是怎么回事？ > > 不好意思，对于kylin我了解还不是很深刻，也不知道句柄是啥意思。请问要改哪个的句柄数？具体操作是怎么样的呢？ > > > ------------------ 原始邮件 ------------------ > *发件人:* "初晓"<[email protected]>; > *发送时间:* 2020年7月9日(星期四) 下午3:53 > *收件人:* "user"<[email protected]>; > *主题:* Re:Re: 执行cube在第4步报错，提示“打开的文件过多” 是怎么回事？ > > 句柄用光了吧，改一下句柄数 > > > > > > > At 2020-07-09 10:39:53, "Yaqian Zhang" <[email protected]> wrote: > > Hi: > > If it's useful to restart kylin or rebuild the job? > > 在 2020年7月8日，11:16，梅秋莹 <[email protected]> 写道： > > > > > ------------------ 原始邮件 ------------------ > *发件人:* "梅秋莹"<[email protected]>; > *发送时间:* 2020年7月3日(星期五) 下午4:55 > *收件人:* "user"<[email protected]>; > *主题:* When build cube in kylin, it failed at the 4th step "Build > Dimension Dictionary" > > Hello everyone: > > Recently, we have encountered a new problem. That is, when build > cube in kylin, it failed at the 4th step "Build Dimension Dictionary". > > <[email protected]> > and the error message is like below. Before tha， we have never met such > quetion, and our model and cube have functioned properly for at least one > month. After reading some articles and blogs, someone think the errors > could be related to the cube which has been set some measures like, "TOP-N" > and "COUNT_DISTINCT". Indeed，I have set the measures like "TOP-N" and > "COUNT_DISTINCT" when create cube. What should we do to solve the proble? > Please tell me if you know the reason. Thank you very much. > > our kylin is 2.6.3. > > org.apache.kylin.engine.mr.exception.HadoopShellException: > java.lang.RuntimeException: java.io.FileNotFoundException: > /etc/hadoop/3.1.0.0-78/0/core-site.xml (打开的文件过多) > at > org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:3000) > at > org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2926) > at > org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2806) > at org.apache.hadoop.conf.Configuration.get(Configuration.java:1200) > at > org.apache.kylin.common.util.HadoopUtil.healSickConfig(HadoopUtil.java:69) > at > org.apache.kylin.common.util.HadoopUtil.getCurrentConfiguration(HadoopUtil.java:59) > at > org.apache.kylin.common.util.HadoopUtil.getFileSystem(HadoopUtil.java:103) > at > org.apache.kylin.common.util.HadoopUtil.getFileSystem(HadoopUtil.java:95) > at > org.apache.kylin.common.util.HadoopUtil.getWorkingFileSystem(HadoopUtil.java:82) > at > org.apache.kylin.engine.mr.steps.CreateDictionaryJob$2.getDictionary(CreateDictionaryJob.java:92) > at > org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:95) > at > org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:69) > at > org.apache.kylin.engine.mr.steps.CreateDictionaryJob.run(CreateDictionaryJob.java:73) > at org.apache.kylin.engine.mr.MRUtil.runMRJob(MRUtil.java:93) > at > org.apache.kylin.engine.mr.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:63) > at > org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:167) > at > org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:71) > at > org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:167) > at > org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:114) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) > Caused by: java.io.FileNotFoundException: > /etc/hadoop/3.1.0.0-78/0/core-site.xml (打开的文件过多) > at java.io.FileInputStream.open0(Native Method) > at java.io.FileInputStream.open(FileInputStream.java:195) > at java.io.FileInputStream.<init>(FileInputStream.java:138) > at java.io.FileInputStream.<init>(FileInputStream.java:93) > at > sun.net.www.protocol.file.FileURLConnection.connect(FileURLConnection.java:90) > at > sun.net.www.protocol.file.FileURLConnection.getInputStream(FileURLConnection.java:188) > at org.apache.hadoop.conf.Configuration.parse(Configuration.java:2901) > at > org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2953) > ... 21 more > > result code:2 > at > org.apache.kylin.engine.mr.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:73) > at > org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:167) > at > org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:71) > at > org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:167) > at > org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:114) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > > at java.lang.Thread.run(Thread.java:748) > > <[email protected]> > > <[email protected]><[email protected]> > <kylin(1).log> > > > > > >
