Indeed, Kiran. Sorry that my prior email and yours crossed with each other. Thanks for the help!
On Wed, Jan 30, 2013 at 12:02 PM, kiran chitturi <[email protected]> wrote: > The steps occur in this order > > 1) Inject > 2) Generate > 3) Fetcher > 4) Parse > 5) dbUpdate > > I would suggest you to clean the database and start again. Please let me > know your results. > > > > On Wed, Jan 30, 2013 at 2:52 PM, Weilei Zhang <[email protected]> wrote: > >> Hi >> I am trying to use Nutch 2.x and have one question regarding Generator >> and Injector: >> Basically, I only have link as root to crawl and I see (by >> instrumenting the code) that this one link was written to Context in >> the last step of InjectorJob and that is the only link written to >> Context from GeneratorJob. However, I saw multiple links sent to map >> function in the first steps of GeneratorJob ( I instrumented setup >> function). Those links seem to include all URLs referenced from the >> original link. My question is where does fetch/parse happen? From the >> Crawler code, it is straightforward to me that Injector is immediately >> followed by Generator; I tried to scrub the code down to do the job >> but failed. >> >> I ran crawl in the following way: >> >/nutch crawl urlsDir >> >> There is only one link under a file in urlsDir. >> >cat urlsDir/* >> http://www.bmw.com >> >> The following is excerpt from the Generator map function >> instrumentation output. Those are reversedURL. >> al.com.bmw.www:http/ >> al.com.bmw.www:http/al/en >> am.bmw.www:http/ >> am.bmw.www:http/am/en >> ao.co.bmw:http/ >> ao.co.bmw:http/ao/pt >> ar.com.bmw.www:http/ >> ar.com.bmw.www:http/ar/es/ >> at.bmw.www:http/ >> at.bmw.www:http/at/de/general/configurations_center/configure.html >> at.bmw.www:http/de/index.html >> >> at.bmw.www:http/de/topics/services-angebote/connecteddrivedienste/connecteddrive-antrag/ueberblick.html >> au.com.bmw.www:http/ >> >> au.com.bmw.www:http/com/en/newvehicles/1series/5door/2011/showroom/compare.html >> >> au.com.bmw.www:http/com/en/newvehicles/1series/5door/2011/showroom/configurator.html >> >> au.com.bmw.www:http/com/en/newvehicles/1series/5door/2011/showroom/driveawayprice.html >> >> au.com.bmw.www:http/com/en/newvehicles/1series/5door/2011/showroom/financecalculator.html >> >> au.com.bmw.www:http/com/en/newvehicles/1series/5door/2011/showroom/highlights/ >> >> au.com.bmw.www:http/com/en/newvehicles/1series/5door/2011/showroom/introduction.html >> >> au.com.bmw.www:http/com/en/newvehicles/1series/5door/2011/showroom/requestebrochure.html >> >> au.com.bmw.www:http/com/en/newvehicles/1series/5door/2011/showroom/requesttestdrive.html >> >> au.com.bmw.www:http/com/en/newvehicles/1series/convertible/2011/showroom/compare.html >> >> au.com.bmw.www:http/com/en/newvehicles/1series/convertible/2011/showroom/configurator.html >> >> au.com.bmw.www:http/com/en/newvehicles/1series/convertible/2011/showroom/driveawayprice.html >> >> au.com.bmw.www:http/com/en/newvehicles/1series/convertible/2011/showroom/financecalculator.html >> >> au.com.bmw.www:http/com/en/newvehicles/1series/convertible/2011/showroom/highlights/ >> >> >> Thanks for any hints! >> -- >> Best Regards >> -Weilei >> > > > > -- > Kiran Chitturi -- Best Regards -Weilei
