[ https://issues.apache.org/jira/browse/MAPREDUCE-7077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16434391#comment-16434391 ]
Suma Shivaprasad commented on MAPREDUCE-7077: --------------------------------------------- This is occuring due to a race condition caused due to maps and reducers writing the jobTokenPassword to the same application directory instead of the tasks's working directory, which was changed in MAPREDUCE-7036 > Pipe mapreduce job fails with Permission denied for jobTokenPassword > -------------------------------------------------------------------- > > Key: MAPREDUCE-7077 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-7077 > Project: Hadoop Map/Reduce > Issue Type: Bug > Reporter: Yesha Vora > Priority: Major > > Steps: > Launch wordcount example with pipe > {code} > /usr/hdp/current/hadoop-client/bin/hadoop pipes > "-Dhadoop.pipes.java.recordreader=true" > "-Dhadoop.pipes.java.recordwriter=true" -input pipeInput -output pipeOutput > -program bin/wordcount{code} > The application fails with below stacktrace > {code:title=AM} > attempt_1517534613368_0041_r_000000_2 is : 0.0 > 2018-02-02 02:40:51,071 ERROR [IPC Server handler 16 on 43391] > org.apache.hadoop.mapred.TaskAttemptListenerImpl: Task: > attempt_1517534613368_0041_r_000000_2 - exited : > java.io.FileNotFoundException: > /grid/0/hadoop/yarn/local/usercache/hrt_qa/appcache/application_1517534613368_0041/jobTokenPassword > (Permission denied) > at java.io.FileOutputStream.open0(Native Method) > at java.io.FileOutputStream.open(FileOutputStream.java:270) > at java.io.FileOutputStream.<init>(FileOutputStream.java:213) > at > org.apache.hadoop.fs.RawLocalFileSystem$LocalFSFileOutputStream.<init>(RawLocalFileSystem.java:236) > at > org.apache.hadoop.fs.RawLocalFileSystem$LocalFSFileOutputStream.<init>(RawLocalFileSystem.java:219) > at > org.apache.hadoop.fs.RawLocalFileSystem.createOutputStreamWithMode(RawLocalFileSystem.java:318) > at > org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:307) > at > org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:338) > at > org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSOutputSummer.<init>(ChecksumFileSystem.java:401) > at > org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:464) > at > org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:443) > at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:1169) > at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:1149) > at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:1038) > at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:1026) > at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:703) > at > org.apache.hadoop.mapred.pipes.Application.writePasswordToLocalFile(Application.java:173) > at org.apache.hadoop.mapred.pipes.Application.<init>(Application.java:109) > at > org.apache.hadoop.mapred.pipes.PipesReducer.startApplication(PipesReducer.java:87) > at org.apache.hadoop.mapred.pipes.PipesReducer.reduce(PipesReducer.java:65) > at org.apache.hadoop.mapred.pipes.PipesReducer.reduce(PipesReducer.java:38) > at org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:445) > at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:393) > at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:174) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1965) > at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:168) > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org