Hi,
I encountered following FileNotFoundException resulting from "too many
open files" error when i tried to run a job. The job had been run for
several times before without problem. I am confused by the exception
because my code closes all the files and even it doesn't, the job only
have only 10-20 small input/output files. The limit on the open file
on my box is 1024. Besides, the error seemed to happen even before
the task was executed, I am using 0.17 version. I'd appreciate if
somebody can shed some light on this issue. BTW, the job ran ok after i
restarted hadoop. Yes, the hadoop-site.xml did exist in that directory.
java.lang.RuntimeException: java.io.FileNotFoundException:
/home/y/conf/hadoop/hadoop-site.xml (Too many open files)
at
org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:901)
at
org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:804)
at
org.apache.hadoop.conf.Configuration.getProps(Configuration.java:772)
at org.apache.hadoop.conf.Configuration.get(Configuration.java:272)
at
org.apache.hadoop.conf.Configuration.getBoolean(Configuration.java:414)
at
org.apache.hadoop.mapred.JobConf.getKeepFailedTaskFiles(JobConf.java:306)
at
org.apache.hadoop.mapred.TaskTracker$TaskInProgress.setJobConf(TaskTracker.java:1487)
at
org.apache.hadoop.mapred.TaskTracker.launchTaskForJob(TaskTracker.java:722)
at
org.apache.hadoop.mapred.TaskTracker.localizeJob(TaskTracker.java:716)
at
org.apache.hadoop.mapred.TaskTracker.startNewTask(TaskTracker.java:1274)
at
org.apache.hadoop.mapred.TaskTracker.offerService(TaskTracker.java:915)
at org.apache.hadoop.mapred.TaskTracker.run(TaskTracker.java:1310)
at org.apache.hadoop.mapred.TaskTracker.main(TaskTracker.java:2251)
Caused by: java.io.FileNotFoundException:
/home/y/conf/hadoop/hadoop-site.xml (Too many open files)
at java.io.FileInputStream.open(Native Method)
at java.io.FileInputStream.<init>(FileInputStream.java:106)
at java.io.FileInputStream.<init>(FileInputStream.java:66)
at
sun.net.www.protocol.file.FileURLConnection.connect(FileURLConnection.java:70)
at
sun.net.www.protocol.file.FileURLConnection.getInputStream(FileURLConnection.java:161)
at
com.sun.org.apache.xerces.internal.impl.XMLEntityManager.setupCurrentEntity(XMLEntityManager.java:653)
at
com.sun.org.apache.xerces.internal.impl.XMLVersionDetector.determineDocVersion(XMLVersionDetector.java:186)
at
com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:771)
at
com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:737)
at
com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:107)
at
com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:225)
at
com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:283)
at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:180)
at
org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:832)
... 12 more
Sometimes it gave me this message:
java.io.IOException: Cannot run program "bash": java.io.IOException:
error=24, Too many open files
at java.lang.ProcessBuilder.start(ProcessBuilder.java:459)
at org.apache.hadoop.util.Shell.runCommand(Shell.java:149)
at org.apache.hadoop.util.Shell.run(Shell.java:134)
at org.apache.hadoop.fs.DF.getAvailable(DF.java:73)
at
org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathForWrite(LocalDirAllocator.java:296)
at
org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirAllocator.java:124)
at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:133)
Caused by: java.io.IOException: java.io.IOException: error=24, Too many
open files
at java.lang.UNIXProcess.<init>(UNIXProcess.java:148)
at java.lang.ProcessImpl.start(ProcessImpl.java:65)
at java.lang.ProcessBuilder.start(ProcessBuilder.java:452)
... 6 more
--
Eric Zhang
408-349-2466
Vespa Content team
