Increased the "ulimit" to 64000 ... same problem stop/start-all ... same problem but on a different block which of course present, so it looks like there is nothing wrong with actual data in the hdfs.
I use the Nutch default hadoop 0.19.x anything related ? 2010/1/30 Ken Goodhope <[email protected]> > "Could not obtain block" errors are often caused by running out of > available > file handles. You can confirm this by going to the shell and entering > "ulimit -n". If it says 1024, the default, then you will want to increase > it to about 64,000. > > On Fri, Jan 29, 2010 at 4:06 PM, MilleBii <[email protected]> wrote: > > > X-POST with Nutch mailing list. > > > > HEEELP !!! > > > > Kind of get stuck on this one. > > I backed-up my hdfs data, reformated the hdfs, put data back, try to > merge > > my segments together and it explodes again. > > > > Exception in thread "Lucene Merge Thread #0" > > org.apache.lucene.index.MergePolicy$MergeException: java.io.IOException: > > Could not obtain block: blk_4670839132945043210_1585 > > file=/user/nutch/crawl/indexed-segments/20100113003609/part-00000/_ym.frq > > at > > > > > org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:309) > > > > If I go into the hfds/data directory I DO find the faulty block ???? > > Could it be a synchro problem on the segment merger code ? > > > > 2010/1/29 MilleBii <[email protected]> > > > > > I'm looking for some help. I'm Nutch user, everything was working fine, > > but > > > now I get the following error when indexing. > > > I have a single note pseudo distributed set up. > > > Some people on the Nutch list indicated to me that I could full, so I > > > remove many things and hdfs is far from full. > > > This file & directory was perfectly OK the day before. > > > I did a "hadoop fsck"... report says healthy. > > > > > > What can I do ? > > > > > > Is is safe to do a Linux FSCK just in case ? > > > > > > Caused by: java.io.IOException: Could not obtain block: > > > blk_8851198258748412820_9031 > > > > > > file=/user/nutch/crawl/indexed-segments/20100111233601/part-00000/_103.frq > > > > > > > > > -- > > > -MilleBii- > > > > > > > > > > > -- > > -MilleBii- > > > > > > -- > Ken Goodhope > Cell: 425-750-5616 > > 362 Bellevue Way NE Apt N415 > Bellevue WA, 98004 > -- -MilleBii-
