Hi This is a really strange problem. I'm using the Nutch 0.9 release. Apparently this error is logged in NUTCH-503 and requires me to patch this problem and build Nutch again. I think someone should fix this issue and then provide people with a release build or just direct them to a nightly build instead.
I found out why I was facing this problem. I had too few URLs in my seed list. When I put in a bigger list of URLs, this problem disappeared. I still can't figure out how to get the search servlet working. I'm getting really strange UTF-8 exceptions. Can any one tell me what's wrong? On Tue, 2008-05-20 at 13:04 +0530, Abhijit Bera wrote: > I changed the log4j.properties file. > > I ran nutch. It didn't give me any errors. > > But the crawl gets over REALLY quick and I guess it doesn't do anything > at all. I'm trying to crawl lucene.apache.org > > [EMAIL PROTECTED]:/usr/local/nutch$ bin/nutch crawl urls/ -dir > crawl_lucene_20 -depth 3 > crawl started in: crawl_lucene_20 > rootUrlDir = urls > threads = 10 > depth = 3 > Injector: starting > Injector: crawlDb: crawl_lucene_20/crawldb > Injector: urlDir: urls > Injector: Converting injected urls to crawl db entries. > Injector: Merging injected urls into crawl db. > Injector: done > Generator: Selecting best-scoring urls due for fetch. > Generator: starting > Generator: segment: crawl_lucene_20/segments/20080520130107 > Generator: filtering: false > Generator: topN: 2147483647 > Generator: 0 records selected for fetching, exiting ... > Stopping at depth=0 - no more URLs to fetch. > No URLs to fetch - check your seed list and URL filters. > crawl finished: crawl_lucene_20 > > It doesn't go to the other phases. I ran it off a single machine and it > seemed to have executed correctly. When I copied the search results into > a local directory and tried running the search servlet, it gave me > exceptions. > > I can't figure out what's wrong. > > > On Mon, 2008-05-19 at 09:25 -0700, Sathyam Y wrote: > > I encountered a similar logging error while running distributed crawl - It > > was related to https://issues.apache.org/jira/browse/NUTCH-508 > > > > You can follow the workaround suggested in the bug report and it resolved > > my logging issues. > > > > - Sathyam > > > > [EMAIL PROTECTED] wrote: > > It looks like there are several errors there. Let's take care of the > > first one: > > > > java.io.FileNotFoundException: /usr/local/nutch/logs (Is a directory) > > > > The error message is pretty clear. Double check the log4j.properties file > > and make sure you are pointing logging appender(s) to files, not > > directories. > > > > Otis > > -- > > Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch > > > > > > ----- Original Message ---- > > > From: Abhijit Bera > > > To: [email protected] > > > Sent: Monday, May 19, 2008 6:43:14 AM > > > Subject: problem running Nutch 0.9 > > > > > > Hi > > > > > > I'm running Nutch 0.9 on 4 Ubuntu 7.04 nodes. Hadoop is setup correctly > > > because the word count example works perfectly. Unfortunately, Nutch is > > > giving me the following exceptions: > > > > > > [EMAIL PROTECTED]:/usr/local/nutch$ bin/nutch crawl urls/ -dir crawl02 > > > -depth > > > 3 > > > > > > crawl started in: crawl02 > > > rootUrlDir = urls > > > threads = 10 > > > depth = 3 > > > Injector: starting > > > Injector: crawlDb: crawl02/crawldb > > > Injector: urlDir: urls > > > Injector: Converting injected urls to crawl db entries. > > > task_0010_m_000006_0: log4j:ERROR setFile(null,true) call failed. > > > task_0010_m_000006_0: > > > java.io.FileNotFoundException: /usr/local/nutch/logs (Is a directory) > > > ..... > > > ..... > > > ..... > > > ......and some more errors which keep repeating in each of the > > > stages..... > > > > > > task_0010_m_000006_0: log4j:ERROR Either File or DatePattern options are > > > not set for appender [DRFA]. > > > > > > task_0010_m_000006_0: Exception in thread "main" > > > java.net.NoRouteToHostException: No route to host > > > > > > I checked my network settings. I'm not behind any firewall. > > > > > > I have placed the urls in a file called seed under the urls directory > > > and also copied it to the DFS. > > > > > > Any help in sorting this out will be appreciated. > > > > > > Thanks > > > > > > -- > > > Abhijit Bera > > > > > > Associate Software Engineer - Web Enterprise Division > > > > > > Geodesic Information Systems Ltd. > > > > > > Please show concern for the environment. Print this e-mail only if > > > required. > > > > > > I use Ubuntu Linux. > > > > > > --Disclaimer-- > > > > > > This email and any files transmitted with it are confidential and > > > intended solely for the use of the entity to which they are addressed. > > > If you have received this email in error please notify the sender > > > immediately. Please note that any views presented in the email are > > > solely those of the author and do not necessarily represent those > > > of Geodesic. > > > > > > While all care has been taken to avoid viruses the recipient is advised > > > to check this email and attachments for presence of viruses. Geodesic > > > accepts no liability on this account. Mails may be stored for monitoring > > > and review > > > > > > > > -- Abhijit Bera Associate Software Engineer - Web Enterprise Division Geodesic Information Systems Ltd. Please show concern for the environment. Print this e-mail only if required. I use Ubuntu Linux. --Disclaimer-- This email and any files transmitted with it are confidential and intended solely for the use of the entity to which they are addressed. If you have received this email in error please notify the sender immediately. Please note that any views presented in the email are solely those of the author and do not necessarily represent those of Geodesic. While all care has been taken to avoid viruses the recipient is advised to check this email and attachments for presence of viruses. Geodesic accepts no liability on this account. Mails may be stored for monitoring and review
