you can either:
1. run on hadoop
2. not run multiple concurrent jobs on a local machine
3. set a hadoop.tmp.dir per job
4. merge all crawls to a single crawl
On Thu, 1 Mar 2012 16:26:00 -0500, Jeremy Villalobos
<jeremyvillalo...@gmail.com> wrote:
Hello:
I am running multiple small crawls on one machine. I notice that
they are
conflicting because they all access
/tmp/hadoop-username/mapred
How do I change the location of this folder ?
Do I have use hadoop to run multiple crawlers each specific to a site
?
thanks
Jeremy
--
Markus Jelsma - CTO - Openindex
http://www.linkedin.com/in/markus17
050-8536600 / 06-50258350
