Will that work with other conf files? On Dec 3, 2013 12:00 AM, "Talat UYARER" <[email protected]> wrote:
> Hi, > > You can use hadoop job xml configuration. When submitting your job use > -conf parameter. For example > > bin/nutch -conf ~/nutch-site.xml generate -crawlId news > > Talat > > 03-12-2013 09:49 tarihinde, mesenthil1 yazdı: > >> Hi, >> We will have to crawl around 10 web sites with different configurations in >> parallel. In our current set up(5 node hadoop), we need to have 10 >> separate >> .job file with different configurations. Is there anyway we can use single >> nutch job file with different configurations in parallel in hadoop >> cluster. >> >> It looks NUTCH_CONF_DIR is effective only in local mode as per the post >> http://www.mail-archive.com/[email protected]/msg01647.html. >> >> Has anyone tried this already? Please let us know. >> >> Senthil >> >> >> >> -- >> View this message in context: http://lucene.472066.n3. >> nabble.com/Nutch-2-1-Having-multiple-different-configurations-for-single- >> Nutch-job-in-deploy-distributed-mode-tp4104604.html >> Sent from the Nutch - User mailing list archive at Nabble.com. >> >> >
