ps -ef | grep hadoop shows that it is indeed "hadoopmachine" that is running hadoop.
I su'ed into the user hadoopmachine (which is also the standard user I login with in debian), and I can access the hdfs that way as well. The free space should also not be a problem: hadoopmachine@debian:~$ df -h Filesystem Size Used Avail Use% Mounted on /dev/sda1 7.6G 4.0G 3.2G 56% / tmpfs 1.8G 0 1.8G 0% /lib/init/rw udev 1.8G 172K 1.8G 1% /dev tmpfs 1.8G 0 1.8G 0% /dev/shm I don't know if it is relevant, but it is on a virtual machine. Regards, Bas On Tue, Apr 3, 2012 at 8:17 PM, Harsh J <ha...@cloudera.com> wrote: > The permissions look alright if TT too is run by 'hadoopmachine'. Can > you also check if you have adequate space free, reported by df -h > /home/hadoopmachine? > > On Tue, Apr 3, 2012 at 10:28 PM, Bas Hickendorff > <hickendorff...@gmail.com> wrote: >> Thanks for your help! >> However, as far as I can see, the user has those rights. >> >> I have in mapred-ste.xml : >> >> <property> >> <name>mapred.local.dir</name> >> <value>/home/hadoopmachine/hadoop_data/mapred</value> >> <final>true</final> >> </property> >> >> >> and the directories look like this: >> >> hadoopmachine@debian:~$ cd /home/hadoopmachine/hadoop_data/mapred >> hadoopmachine@debian:~/hadoop_data/mapred$ ls -lah >> total 24K >> drwxr-xr-x 6 hadoopmachine hadoopmachine 4.0K Apr 3 12:11 . >> drwxr-xr-x 6 hadoopmachine hadoopmachine 4.0K Apr 3 08:26 .. >> drwxr-xr-x 2 hadoopmachine hadoopmachine 4.0K Apr 3 12:10 taskTracker >> drwxr-xr-x 2 hadoopmachine hadoopmachine 4.0K Apr 3 12:10 tt_log_tmp >> drwx------ 2 hadoopmachine hadoopmachine 4.0K Apr 3 12:10 ttprivate >> drwxr-xr-x 2 hadoopmachine hadoopmachine 4.0K Apr 3 08:28 userlogs >> >> hadoopmachine@debian:~/hadoop_data/mapred$ cd .. >> hadoopmachine@debian:~/hadoop_data$ ls -lah >> total 24K >> drwxr-xr-x 6 hadoopmachine hadoopmachine 4.0K Apr 3 08:26 . >> drwxr-xr-x 31 hadoopmachine hadoopmachine 4.0K Apr 3 12:08 .. >> drwxr-xr-x 6 hadoopmachine hadoopmachine 4.0K Apr 3 12:10 data >> drwxr-xr-x 6 hadoopmachine hadoopmachine 4.0K Apr 3 12:11 mapred >> drwxr-xr-x 5 hadoopmachine hadoopmachine 4.0K Apr 3 12:09 name >> drwxr-xr-x 4 hadoopmachine hadoopmachine 4.0K Apr 3 10:11 tmp >> >> >> As far as I can see (but my linux permissions knowledge might be >> failing) the user "hadoopmachine" has rights on these folders. I >> confirmed that that user is indeed the user that runs the TaskTracker. >> >> Are there any other things I could check? >> >> >> Regards, >> >> Bas >> >> On Tue, Apr 3, 2012 at 6:12 PM, Harsh J <ha...@cloudera.com> wrote: >>> Some of your TaskTrackers' mapred.local.dirs do not have proper r/w >>> permissions set on them. Make sure they are owned by the user that >>> runs the TT service and have read/write permission at least for that >>> user. >>> >>> On Tue, Apr 3, 2012 at 6:58 PM, Bas Hickendorff >>> <hickendorff...@gmail.com> wrote: >>>> Hello all, >>>> >>>> My map-reduce operation on Hadoop (running on Debian) is correctly >>>> starting and finding the input file. However, just after starting the >>>> map reduce, Hadoop tells me that it cannot find a file. Unfortunately, >>>> it does not state what file it cannot find, or where it is looking. >>>> Does someone now about what file error is? See below for the complete >>>> error. >>>> >>>> Since the java error is in the chmod() function (judging from the >>>> stack in the output), I assume it is a problem with the rights, but >>>> how do I know what rights to change if it gives me no path? >>>> >>>> Thanks in advance, >>>> >>>> Bas >>>> >>>> >>>> >>>> >>>> The output of the job: >>>> >>>> >>>> hadoopmachine@debian:~$ ./hadoop-1.0.1/bin/hadoop jar >>>> hadooptest/main.jar nl.mydomain.hadoop.debian.test.Main >>>> /user/hadoopmachine/input /user/hadoopmachine/output >>>> Warning: $HADOOP_HOME is deprecated. >>>> >>>> 12/04/03 08:05:08 WARN mapred.JobClient: Use GenericOptionsParser for >>>> parsing the arguments. Applications should implement Tool for the >>>> same. >>>> ****hdfs://localhost:9000/user/hadoopmachine/input >>>> 12/04/03 08:05:08 INFO input.FileInputFormat: Total input paths to process >>>> : 1 >>>> 12/04/03 08:05:08 INFO mapred.JobClient: Running job: job_201204030722_0004 >>>> 12/04/03 08:05:09 INFO mapred.JobClient: map 0% reduce 0% >>>> 12/04/03 08:05:13 INFO mapred.JobClient: Task Id : >>>> attempt_201204030722_0004_m_000002_0, Status : FAILED >>>> Error initializing attempt_201204030722_0004_m_000002_0: >>>> ENOENT: No such file or directory >>>> at org.apache.hadoop.io.nativeio.NativeIO.chmod(Native Method) >>>> at >>>> org.apache.hadoop.fs.FileUtil.execSetPermission(FileUtil.java:692) >>>> at org.apache.hadoop.fs.FileUtil.setPermission(FileUtil.java:647) >>>> at >>>> org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:509) >>>> at >>>> org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:344) >>>> at >>>> org.apache.hadoop.mapred.JobLocalizer.initializeJobLogDir(JobLocalizer.java:239) >>>> at >>>> org.apache.hadoop.mapred.DefaultTaskController.initializeJob(DefaultTaskController.java:196) >>>> at org.apache.hadoop.mapred.TaskTracker$4.run(TaskTracker.java:1226) >>>> at java.security.AccessController.doPrivileged(Native Method) >>>> at javax.security.auth.Subject.doAs(Subject.java:416) >>>> at >>>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1093) >>>> at >>>> org.apache.hadoop.mapred.TaskTracker.initializeJob(TaskTracker.java:1201) >>>> at >>>> org.apache.hadoop.mapred.TaskTracker.localizeJob(TaskTracker.java:1116) >>>> at org.apache.hadoop.mapred.TaskTracker$5.run(TaskTracker.java:2404) >>>> at java.lang.Thread.run(Thread.java:636) >>>> >>>> 12/04/03 08:05:13 WARN mapred.JobClient: Error reading task >>>> outputhttp://localhost:50060/tasklog?plaintext=true&attemptid=attempt_201204030722_0004_m_000002_0&filter=stdout >>>> 12/04/03 08:05:13 WARN mapred.JobClient: Error reading task >>>> outputhttp://localhost:50060/tasklog?plaintext=true&attemptid=attempt_201204030722_0004_m_000002_0&filter=stderr >>> >>> >>> >>> -- >>> Harsh J > > > > -- > Harsh J
