Is /tmp present in a partition that doesn't have enough space? Does it have enough space left when this error occurs? Nutch often needs GBs of space for /tmp. If there isn't enough space on the partition having /tmp, then you can add the following property in 'conf/hadoop-site.xml' to make it use a different directory for writing the temporary files.
<property> <name>hadoop.tmp.dir</name> <value>/opt/tmp</value> <description>Base for Nutch Temporary Directories</description> </property> Regards, Susam Pal On Nov 21, 2007 8:54 AM, Josh Attenberg <[EMAIL PROTECTED]> wrote: > I had this error when fetching with nutch 0.8.1 there is ~450GB left on the > disk where the crawl db and segments folder. Are there any other settings i > need to make? I know there isnt much space in my home directory, if it was > trying to write there, but there is at least 500M. what are the possible > culprits/fixes? > > > Exception in thread "main" org.apache.hadoop.fs.FSError: java.io.IOException: > No space left on device > at > org.apache.hadoop.fs.LocalFileSystem$LocalFSFileOutputStream.write( > LocalFileSystem.java:150) > at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.write( > FSDataOutputStream.java:112) > at java.io.BufferedOutputStream.flushBuffer( > BufferedOutputStream.java:65) > at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:123) > at java.io.DataOutputStream.flush(DataOutputStream.java:106) > at java.io.FilterOutputStream.close(FilterOutputStream.java:140) > at org.apache.hadoop.fs.FSDataOutputStream$Summer.close( > FSDataOutputStream.java:96) > at java.io.FilterOutputStream.close(FilterOutputStream.java:143) > at java.io.FilterOutputStream.close(FilterOutputStream.java:143) > at java.io.FilterOutputStream.close(FilterOutputStream.java:143) > at org.apache.hadoop.fs.FileUtil.copyContent(FileUtil.java:154) > at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:74) > at org.apache.hadoop.fs.LocalFileSystem.copyFromLocalFile( > LocalFileSystem.java:311) > at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:254) > at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:327) > at org.apache.nutch.fetcher.Fetcher.fetch(Fetcher.java:443) > at org.apache.nutch.fetcher.Fetcher.main(Fetcher.java:477) > Caused by: java.io.IOException: No space left on device > at java.io.FileOutputStream.writeBytes(Native Method) > at java.io.FileOutputStream.write(FileOutputStream.java:260) > at > org.apache.hadoop.fs.LocalFileSystem$LocalFSFileOutputStream.write( > LocalFileSystem.java:148) > ... 16 more >
