I changed the log4j.properties file. I ran nutch. It didn't give me any errors.
But the crawl gets over REALLY quick and I guess it doesn't do anything at all. I'm trying to crawl lucene.apache.org [EMAIL PROTECTED]:/usr/local/nutch$ bin/nutch crawl urls/ -dir crawl_lucene_20 -depth 3 crawl started in: crawl_lucene_20 rootUrlDir = urls threads = 10 depth = 3 Injector: starting Injector: crawlDb: crawl_lucene_20/crawldb Injector: urlDir: urls Injector: Converting injected urls to crawl db entries. Injector: Merging injected urls into crawl db. Injector: done Generator: Selecting best-scoring urls due for fetch. Generator: starting Generator: segment: crawl_lucene_20/segments/20080520130107 Generator: filtering: false Generator: topN: 2147483647 Generator: 0 records selected for fetching, exiting ... Stopping at depth=0 - no more URLs to fetch. No URLs to fetch - check your seed list and URL filters. crawl finished: crawl_lucene_20 It doesn't go to the other phases. I ran it off a single machine and it seemed to have executed correctly. When I copied the search results into a local directory and tried running the search servlet, it gave me exceptions. I can't figure out what's wrong. On Mon, 2008-05-19 at 09:25 -0700, Sathyam Y wrote: > I encountered a similar logging error while running distributed crawl - It > was related to https://issues.apache.org/jira/browse/NUTCH-508 > > You can follow the workaround suggested in the bug report and it resolved > my logging issues. > > - Sathyam > > [EMAIL PROTECTED] wrote: > It looks like there are several errors there. Let's take care of the first > one: > > java.io.FileNotFoundException: /usr/local/nutch/logs (Is a directory) > > The error message is pretty clear. Double check the log4j.properties file and > make sure you are pointing logging appender(s) to files, not directories. > > Otis > -- > Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch > > > ----- Original Message ---- > > From: Abhijit Bera > > To: [email protected] > > Sent: Monday, May 19, 2008 6:43:14 AM > > Subject: problem running Nutch 0.9 > > > > Hi > > > > I'm running Nutch 0.9 on 4 Ubuntu 7.04 nodes. Hadoop is setup correctly > > because the word count example works perfectly. Unfortunately, Nutch is > > giving me the following exceptions: > > > > [EMAIL PROTECTED]:/usr/local/nutch$ bin/nutch crawl urls/ -dir crawl02 > > -depth > > 3 > > > > crawl started in: crawl02 > > rootUrlDir = urls > > threads = 10 > > depth = 3 > > Injector: starting > > Injector: crawlDb: crawl02/crawldb > > Injector: urlDir: urls > > Injector: Converting injected urls to crawl db entries. > > task_0010_m_000006_0: log4j:ERROR setFile(null,true) call failed. > > task_0010_m_000006_0: > > java.io.FileNotFoundException: /usr/local/nutch/logs (Is a directory) > > ..... > > ..... > > ..... > > ......and some more errors which keep repeating in each of the > > stages..... > > > > task_0010_m_000006_0: log4j:ERROR Either File or DatePattern options are > > not set for appender [DRFA]. > > > > task_0010_m_000006_0: Exception in thread "main" > > java.net.NoRouteToHostException: No route to host > > > > I checked my network settings. I'm not behind any firewall. > > > > I have placed the urls in a file called seed under the urls directory > > and also copied it to the DFS. > > > > Any help in sorting this out will be appreciated. > > > > Thanks > > > > -- > > Abhijit Bera > > > > Associate Software Engineer - Web Enterprise Division > > > > Geodesic Information Systems Ltd. > > > > Please show concern for the environment. Print this e-mail only if > > required. > > > > I use Ubuntu Linux. > > > > --Disclaimer-- > > > > This email and any files transmitted with it are confidential and > > intended solely for the use of the entity to which they are addressed. > > If you have received this email in error please notify the sender > > immediately. Please note that any views presented in the email are > > solely those of the author and do not necessarily represent those > > of Geodesic. > > > > While all care has been taken to avoid viruses the recipient is advised > > to check this email and attachments for presence of viruses. Geodesic > > accepts no liability on this account. Mails may be stored for monitoring > > and review > > > > -- Abhijit Bera Associate Software Engineer - Web Enterprise Division Geodesic Information Systems Ltd. Please show concern for the environment. Print this e-mail only if required. I use Ubuntu Linux. --Disclaimer-- This email and any files transmitted with it are confidential and intended solely for the use of the entity to which they are addressed. If you have received this email in error please notify the sender immediately. Please note that any views presented in the email are solely those of the author and do not necessarily represent those of Geodesic. While all care has been taken to avoid viruses the recipient is advised to check this email and attachments for presence of viruses. Geodesic accepts no liability on this account. Mails may be stored for monitoring and review
