i did as you say, and moved the files to a new directory on a big drive, but now have some additional errors. are there any other pointers i need to update?
On Nov 20, 2007 11:33 PM, Susam Pal <[EMAIL PROTECTED]> wrote: > Is /tmp present in a partition that doesn't have enough space? Does it > have enough space left when this error occurs? Nutch often needs GBs > of space for /tmp. If there isn't enough space on the partition having > /tmp, then you can add the following property in > 'conf/hadoop-site.xml' to make it use a different directory for > writing the temporary files. > > <property> > <name>hadoop.tmp.dir</name> > <value>/opt/tmp</value> > <description>Base for Nutch Temporary Directories</description> > </property> > > Regards, > Susam Pal > > On Nov 21, 2007 8:54 AM, Josh Attenberg <[EMAIL PROTECTED]> wrote: > > I had this error when fetching with nutch 0.8.1 there is ~450GB left on > the > > disk where the crawl db and segments folder. Are there any other > settings i > > need to make? I know there isnt much space in my home directory, if it > was > > trying to write there, but there is at least 500M. what are the possible > > culprits/fixes? > > > > > > Exception in thread "main" org.apache.hadoop.fs.FSError: > java.io.IOException: > > No space left on device > > at > > org.apache.hadoop.fs.LocalFileSystem$LocalFSFileOutputStream.write( > > LocalFileSystem.java:150) > > at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.write( > > FSDataOutputStream.java:112) > > at java.io.BufferedOutputStream.flushBuffer( > > BufferedOutputStream.java:65) > > at java.io.BufferedOutputStream.flush(BufferedOutputStream.java > :123) > > at java.io.DataOutputStream.flush(DataOutputStream.java:106) > > at java.io.FilterOutputStream.close(FilterOutputStream.java:140) > > at org.apache.hadoop.fs.FSDataOutputStream$Summer.close( > > FSDataOutputStream.java:96) > > at java.io.FilterOutputStream.close(FilterOutputStream.java:143) > > at java.io.FilterOutputStream.close(FilterOutputStream.java:143) > > at java.io.FilterOutputStream.close(FilterOutputStream.java:143) > > at org.apache.hadoop.fs.FileUtil.copyContent(FileUtil.java:154) > > at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:74) > > at org.apache.hadoop.fs.LocalFileSystem.copyFromLocalFile( > > LocalFileSystem.java:311) > > at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java > :254) > > at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:327) > > at org.apache.nutch.fetcher.Fetcher.fetch(Fetcher.java:443) > > at org.apache.nutch.fetcher.Fetcher.main(Fetcher.java:477) > > Caused by: java.io.IOException: No space left on device > > at java.io.FileOutputStream.writeBytes(Native Method) > > at java.io.FileOutputStream.write(FileOutputStream.java:260) > > at > > org.apache.hadoop.fs.LocalFileSystem$LocalFSFileOutputStream.write( > > LocalFileSystem.java:148) > > ... 16 more > > >
