Hi Srikrishna the reason to this issue is you had uploaded assembly jar to HDFS twice.
paste your command could be better diagnosis 田毅 =========================================== 橘云平台产品线 大数据产品部 亚信联创科技（中国）有限公司 手机：13910177261 电话：010－82166322 传真：010－82166617 Q Q：20057509 MSN：yi.t...@hotmail.com 地址：北京市海淀区东北旺西路10号院东区 亚信联创大厦 =========================================== 在 2014年7月9日，上午3:03，Srikrishna S <srikrishna...@gmail.com> 写道： > Hi All, > > > I tried the make distribution script and it worked well. I was able to > compile the spark binary on our CDH5 cluster. Once I compiled Spark, I > copied over the binaries in the dist folder to all the other machines > in the cluster. > > However, I run into an issue while submit a job in yarn-client mode. I > get an error message that says the following > Resource > file:/opt/spark/spark-1.0.0-bin-hadoop2/lib/spark-assembly-1.1.0-SNAPSHOT-hadoop2.3.0.jar" > changed on src filesystem (expected 1404845211000, was 1404845404000) > > My end goal is to submit a job (that uses MLLib) in our Yarn cluster. > > Any thoughts anyone? > > Regards, > Krishna > > > > On Tue, Jul 8, 2014 at 9:49 AM, Sandy Ryza <sandy.r...@cloudera.com> wrote: >> >> Hi Srikrishna, >> >> The binaries are built with something like >> mvn package -Pyarn -Dhadoop.version=2.3.0-cdh5.0.1 >> -Dyarn.version=2.3.0-cdh5.0.1 >> >> -Sandy >> >> >> On Tue, Jul 8, 2014 at 3:14 AM, 田毅 <tia...@asiainfo.com> wrote: >>> >>> try this command: >>> >>> make-distribution.sh --hadoop 2.3.0-cdh5.0.0 --with-yarn --with-hive >>> >>> >>> >>> >>> 田毅 >>> =========================================== >>> 橘云平台产品线 >>> 大数据产品部 >>> 亚信联创科技（中国）有限公司 >>> 手机：13910177261 >>> 电话：010－82166322 >>> 传真：010－82166617 >>> Q Q：20057509 >>> MSN：yi.t...@hotmail.com >>> 地址：北京市海淀区东北旺西路10号院东区 亚信联创大厦 >>> >>> >>> =========================================== >>> >>> 在 2014年7月8日，上午11:53，Krishna Sankar <ksanka...@gmail.com> 写道： >>> >>> Couldn't find any reference of CDH in pom.xml - profiles or the >>> hadoop.version.Am also wondering how the cdh compatible artifact was >>> compiled. >>> Cheers >>> <k/> >>> >>> >>> On Mon, Jul 7, 2014 at 8:07 PM, Srikrishna S <srikrishna...@gmail.com> >>> wrote: >>>> >>>> Hi All, >>>> >>>> Does anyone know what the command line arguments to mvn are to generate >>>> the pre-built binary for spark on Hadoop 2-CHD5. >>>> >>>> I would like to pull in a recent bug fix in spark-master and rebuild the >>>> binaries in the exact same way that was used for that provided on the >>>> website. >>>> >>>> I have tried the following: >>>> >>>> mvn install -Pyarn -Dhadoop.version=2.3.0-cdh5.0.1 >>>> >>>> And it doesn't quite work. >>>> >>>> Any thoughts anyone? >>>> >>> >>> >> >
