On Fri, Jul 24, 2009 at 11:00 AM, Aaron Kimball <[email protected]> wrote:
> Amandeep,
>
> Does the job fail after that happens? Are there any WARN or ERROR lines in
> the log nearby, or any exceptions?
>
These occur in the JT logs:
2009-07-24 00:19:21,598 WARN org.apache.hadoop.mapred.JobInProgress:
Running cache for maps missing!! Job details are missing
.
2009-07-24 00:19:21,598 WARN org.apache.hadoop.mapred.JobInProgress:
Non-running cache for maps missing!! Job details are mis
sing.
There arent any WARN on ERROR messages in the TT logs. The exact message I'm
getting is:
2009-07-24 10:50:32,519 INFO org.apache.hadoop.mapred.TaskTracker:
org.apache.hadoop.util.DiskChecker$DiskErrorException: Cou
ld not find
taskTracker/jobcache/job_200907221738_0103/attempt_200907221738_0103_m_000015_0/output/file.out
in any of the con
figured local directories
> Three possibilities I can think of:
>
> You may have configured Hadoop to run under /tmp, and tmpwatch or another
> cleanup utility like that decided to throw away a bunch of files in the
> temp
> space while your job was running. In this case, you should consider moving
> hadoop.tmp.dir and mapred.local.dir out from under the default /tmp.
>
<name>hadoop.tmp.dir</name>
<value>/hadoop/tmp/${user.name}</value>
and...
<name>mapred.local.dir</name>
<value>${hadoop.tmp.dir}/mapred/local</value>
Its creating the jobcache directory in mapred.local.dir/local/taskTracker/
when the job is run...
> You might be out of disk space?
>
Plenty available.
>
> mapred.local.dir or hadoop.tmp.dir might be set to paths that Hadoop
> doesn't
> have the privileges to write to?
>
Nope.. They are set to paths that the user running hadoop can write to.
My hdfs goes to a directory: /hadoop/hdfs
Temp goes to: /hadoop/tmp
I deleted the /hadoop/tmp directory and ran the jobs again. It created the
directory again on its own.
Cant figure out whats wrong. Any other pointers?
>
> - A
>
>
> On Thu, Jul 23, 2009 at 2:06 AM, Amandeep Khurana <[email protected]>
> wrote:
>
> > Hi
> >
> > I get these messages in the TT log while running a job:
> >
> > 2009-07-23 02:03:59,091 INFO org.apache.hadoop.mapred.TaskTracker:
> > org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find
> >
> >
> taskTracker/jobcache/job_200907221738_0020/attempt_200907221738_0020_r_000000_0/output/file.out
> > in any of the configured local directories
> >
> > Whats the problem?
> >
> > Amandeep
> >
> >
> >
> > Amandeep Khurana
> > Computer Science Graduate Student
> > University of California, Santa Cruz
> >
>
