Hello Alex,
We have tried the setup with HDFS and worked fine. The
shared filesstem talked in here
is a Lustre parallel filesystem and is mounted on all the compute
nodes(tasktracker).
The problem as it seems to me is not about different nodes messing up but
temp data written
by one tasktracker on one node and being accessed by another. The dir :
/tmp/hadoop-nazgul/mapred/temp/inject-temp-1557478199/_temporary does exists
on the second node.
On Mon, Jan 17, 2011 at 1:59 PM, Alex McLintock <[email protected]>wrote:
> I'm not sure if you can do this (I would recommend HDFS instead of a shared
> area) but can you insert the hostname of the node into the temp dir? That
> might stop separate nodes from messing up each others temp areas.
>
> (However I am guessing here)
>
>
> On 17 January 2011 08:21, rishi pathak <[email protected]> wrote:
>
> > # Error ########
> >
> > java.io.IOException: The temporary job-output directory
> > file:/tmp/hadoop-nazgul/mapred/temp/inject-temp-1557478199/_temporary
> > doesn't exist!
> >
> >
>
--
---
Rishi Pathak
National PARAM Supercomputing Facility
C-DAC, Pune, India
