Hi Mahadev,
I'm not sure the workaround can solve the problem, because it appears
that a subdirectory is created under that directory with the name of the
hodring host. So if the next allocation made by a different user chooses
the same host, the permission problem might show up again. Unless
obviously each time the user who allocated resources deletes that
directory during deallocation, which would be a solution for this problem.
I filed a bug: HADOOP-2899
Cheers,
Luca
Mahadev Konar wrote:
Hi Luca,
This seems like a bug. The JobTracker process tries to create this
directory if this does nto exist. And if you have two different users
running hod clusters they will both try to create this directory and
since only one succeeds, with permissions the directory is owned by the
user who created the cluster first.
A work around to this solution is to create
Hdfs:///mapredsystem manually and make it world writable.
Please open a bug regarding this issue.
Regards
Mahadev
-----Original Message-----
From: Luca Telloli [mailto:[EMAIL PROTECTED]
Sent: Tuesday, February 26, 2008 9:06 AM
To: [email protected]
Subject: [HOD] hdfs:///mapredsystem directory
Hello everyone,
I have a problem with directory hdfs:///mapredsystem and I'm not
sure
if it's a bug or my fault.
Not sure if this influences what follows, but I have two users, one is
"hadoop" who has sudo privileges on all the nodes, the other one is
"luca", who has normal privileges.
I see that each submitted job creates a hdfs:///mapredsystem
directory,
created by (I guess) the hodring process. Problem is that it's not
cleaned up at the end of the process; for instance a use case would
be:
- user hadoop allocates a cluster, the ringmaster is svr3, so a
/mapredsystem/svr3 directory is created
- user hadoop deallocates the cluster, but that directory is not
cleaned
up
- user luca allocates a cluster, and the first node chosen as
ringmaster
is svr3, so hodring tries to write hdfs:///mapredsystem but it fails
- allocation succeeds, but there's no hodring running; looking at
0-jobtracker/logdir/hadoop.log under the temporary directory I can
read:
2008-02-26 17:28:42,567 WARN org.apache.hadoop.mapred.JobTracker:
Error
starting tracker: org.apache.hadoop.ipc.RemoteException:
org.apache.hadoop.fs.permission.AccessControlException: Permission
denied: user=luca, access=WRITE,
inode="mapredsystem":hadoop:supergroup:rwxr-xr-x
Am I doing anything wrong?
Cheers,
Luca
