It takes job configuration. You can write what you want. Every property
will be effected your job.
03-12-2013 10:37 tarihinde, Nicholas Roberts yazdı:
Will that work with other conf files?
On Dec 3, 2013 12:00 AM, "Talat UYARER" <[email protected]> wrote:
Hi,
You can use hadoop job xml configuration. When submitting your job use
-conf parameter. For example
bin/nutch -conf ~/nutch-site.xml generate -crawlId news
Talat
03-12-2013 09:49 tarihinde, mesenthil1 yazdı:
Hi,
We will have to crawl around 10 web sites with different configurations in
parallel. In our current set up(5 node hadoop), we need to have 10
separate
.job file with different configurations. Is there anyway we can use single
nutch job file with different configurations in parallel in hadoop
cluster.
It looks NUTCH_CONF_DIR is effective only in local mode as per the post
http://www.mail-archive.com/[email protected]/msg01647.html.
Has anyone tried this already? Please let us know.
Senthil
--
View this message in context: http://lucene.472066.n3.
nabble.com/Nutch-2-1-Having-multiple-different-configurations-for-single-
Nutch-job-in-deploy-distributed-mode-tp4104604.html
Sent from the Nutch - User mailing list archive at Nabble.com.
