Hadoop Script

Has anyone written a script for whole web crawling using Hadoop? The script for nutch doesn't work since the data is inside the HDFS (tail - f wont work with this). Thanks, Eric

Re: Hadoop Script

Sorry Ryan, I should have clarified that I am using Nutch as my crawler. There is a script for Nutch to do Whole web crawling, but it is not compatible with Hadoop. Eric Osgood - Cal Poly - Computer Engineering Moon Valley Software