The most basic shell script would be putting the commands of the tutorial in order and have a cron job execute that script once every interval. In case of large crawls you need to add some locking mechanism to avoid overlap.
http://wiki.apache.org/nutch/NutchTutorial On Monday 14 November 2011 01:55:15 xander wrote: > Hi, > I wan to write a shell scirpt which will crawl data and update the database > for me every 2 hours . Can you help me write a shell script for it. I am > new to this and would appreciate any sort of help. You can divert me to a > useful link too. > > thanks > > -- > View this message in context: > http://lucene.472066.n3.nabble.com/Continuous-crawling-tp3497615p3505600.h > tml Sent from the Nutch - User mailing list archive at Nabble.com. -- Markus Jelsma - CTO - Openindex http://www.linkedin.com/in/markus17 050-8536620 / 06-50258350
