I¹m pretty sure there is no guarantee of isolation on a shared .m2/repository directory for multiple concurrent Maven processes. I¹ve had a theory for a while that one build running ³mvm install² can overwrite the snapshot artifact that was just installed by another concurrent build. This can create bizarre problems, for example if a patch introduces a new class in hadoop-common and then references that class from hadoop-hdfs.
I expect using completely separate work directories for .m2/repository, the patch directory, and the Jenkins workspace could resolve this. The typical cost for this kind of change is increased disk consumption and increased build time, since Maven would need to download dependencies fresh every time. Chris Nauroth Hortonworks http://hortonworks.com/ On 2/12/15, 2:00 PM, "Colin P. McCabe" <cmcc...@apache.org> wrote: >We could potentially use different .m2 directories for each executor. >I think this has been brought up in the past as well. > >I'm not sure how maven handles concurrent access to the .m2 >directory... if it's not using flock or fnctl then it's not really >safe. This might explain some of our missing class error issues. > >Colin > >On Tue, Feb 10, 2015 at 2:13 AM, Steve Loughran <ste...@hortonworks.com> >wrote: >> Mvn is a dark mystery to us all. I wouldn't trust it not pick up things >>from other builds if they ended up published to ~/.m2/repository during >>the process >> >> >> >> On 9 February 2015 at 19:29:06, Colin P. McCabe >>(cmcc...@apache.org<mailto:cmcc...@apache.org>) wrote: >> >> I'm sorry, I don't have any insight into this. With regard to >> HADOOP-11084, I thought that $BUILD_URL would be unique for each >> concurrent build, which would prevent build artifacts from getting >> mixed up between jobs. Based on the value of PATCHPROCESS that Kihwal >> posted, perhaps this is not the case? Perhaps someone can explain how >> this is supposed to work (I am a Jenkins newbie). >> >> regards, >> Colin >> >> On Thu, Feb 5, 2015 at 10:42 AM, Yongjun Zhang <yzh...@cloudera.com> >>wrote: >>> Thanks Kihwal for bringing this up. >>> >>> Seems related to: >>> >>> https://issues.apache.org/jira/browse/HADOOP-11084 >>> >>> Hi Andrew/Arpit/Colin/Steve, you guys worked on this jira before, any >>> insight about the issue Kihwal described? >>> >>> Thanks. >>> >>> --Yongjun >>> >>> >>> On Thu, Feb 5, 2015 at 9:49 AM, Kihwal Lee >>><kih...@yahoo-inc.com.invalid> >>> wrote: >>> >>>> I am sure many of us have seen strange jenkins behavior out of the >>>> precommit builds. >>>> >>>> - build artifacts missing >>>> - serving build artifact belonging to another build. This also causes >>>> wrong precommit results to be posted on the bug. >>>> - etc. >>>> >>>> The latest one I saw is disappearance of the unit test stdout/stderr >>>>file >>>> during a build. After a successful run of unit tests, the file >>>>vanished, so >>>> the script could not cat it. It looked like another build process had >>>> deleted it, while this build was in progress. >>>> >>>> It might have something to do with the fact that the patch-dir is set >>>>like >>>> following: >>>> >>>> >>>>PATCHPROCESS=/home/jenkins/jenkins-slave/workspace/PreCommit-HDFS-Build >>>>/../patchprocessI >>>> don't have access to the jenkins build configs or the build machines, >>>>so I >>>> can't debug it further, but I think we need to take care of it sooner >>>>than >>>> later. Can any one help? >>>> >>>> Kihwal >>>>
