[ https://issues.apache.org/jira/browse/MAPREDUCE-6527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Haibo Chen reassigned MAPREDUCE-6527: ------------------------------------- Assignee: Haibo Chen > Data race on field org.apache.hadoop.mapred.JobConf.credentials > --------------------------------------------------------------- > > Key: MAPREDUCE-6527 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6527 > Project: Hadoop Map/Reduce > Issue Type: Bug > Affects Versions: 2.7.1 > Reporter: Ali Kheradmand > Assignee: Haibo Chen > > I am running the test suite against a dynamic race detector called > RV-Predict. Here is a race report that I got: > {noformat} > Data race on field org.apache.hadoop.mapred.JobConf.credentials: {{{ > Concurrent read in thread T327 (locks held: {}) > ----> at org.apache.hadoop.mapred.JobConf.getCredentials(JobConf.java:505) > at > org.apache.hadoop.mapreduce.task.JobContextImpl.<init>(JobContextImpl.java:70) > at > org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:524) > T327 is created by T22 > at > org.apache.hadoop.mapred.LocalJobRunner$Job.<init>(LocalJobRunner.java:218) > Concurrent write in thread T22 (locks held: {Monitor@496c673a, > Monitor@496319b0}) > ----> at org.apache.hadoop.mapred.JobConf.setCredentials(JobConf.java:510) > at > org.apache.hadoop.mapred.LocalJobRunner.submitJob(LocalJobRunner.java:787) > at > org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:241) > at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1290) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1669) > at org.apache.hadoop.mapreduce.Job.submit(Job.java:1287) > at > org.apache.hadoop.mapreduce.lib.jobcontrol.ControlledJob.submit(ControlledJob.java:335) > locked Monitor@496319b0 at > org.apache.hadoop.mapreduce.lib.jobcontrol.ControlledJob.submit(ControlledJob.java:n/a) > > at > org.apache.hadoop.mapreduce.lib.jobcontrol.JobControl.run(JobControl.java:245) > locked Monitor@496c673a at > org.apache.hadoop.mapreduce.lib.jobcontrol.JobControl.run(JobControl.java:229) > > T22 is created by T1 > at > org.apache.hadoop.mapred.jobcontrol.TestJobControl.doJobControlTest(TestJobControl.java:111) > }}} > {noformat} > In the source code of org.apache.hadoop.mapreduce.JobStatus.submitJob > function, we have the following lines: > {code} > Job job = new Job(JobID.downgrade(jobid), jobSubmitDir); > job.job.setCredentials(credentials); > {code} > It looks a bit suspicious: Job extends thread and at the end of its > constructor it starts a new thread which creates a new instance of > JobContextImpl which reads credentials. However, the first thread > concurrently sets credentials after a creating the Job instance. -- This message was sent by Atlassian JIRA (v6.3.4#6332)