Thanks Chris, Yan, Let me try that.
-----Original Message----- From: Chris Riccomini [mailto:[email protected]] Sent: Tuesday, March 11, 2014 10:22 AM To: [email protected] Subject: Re: Failed to package using mvn Hey Yan, Awesome!The location where you can add your .md is here: docs/learn/tutorials/0.7.0/ Here's a link to the code tree: https://git-wip-us.apache.org/repos/asf?p=incubator-samza.git;a=tree;f=docs /learn/tutorials/0.7.0;h=ef117f4066f14a00f50f0f6fca17903130448312;hb=HEAD You can get the code here: git clone http://git-wip-us.apache.org/repos/asf/incubator-samza.git Once you write the .md, just throw it up on a JIRA, and one of us can merge it in. Re: hdfs-site.xml, ah ha, that's what I figured. This is good to know. So you just copy your hdfs-site.xml from your NodeManager's conf directory into your local hdfs-site.xml. Cheers, Chris On 3/11/14 10:16 AM, "Yan Fang" <[email protected]> wrote: >Hi Chris, > >Sure. I just do not know how/where to contribute this page...*_* > >Oh, I mean the same this as you mentioned in the *Cluster >Installation*thread: > >*"2. Get a copy of one of your NM's yarn-site.xml and put it somewhere >on* > >*your desktop (I usually use ~/.yarn/conf/yarn-site.xml). Note that >there'sa "conf" directory there. This is mandatory."* > >So I just copy the hdfs-site.xml to ~/.yarn/conf/hdfs-site.xml. Thank you. > >Cheers, > >Fang, Yan >[email protected] >+1 (206) 849-4108 > > >On Tue, Mar 11, 2014 at 10:10 AM, Chris Riccomini ><[email protected]>wrote: > >> Hey Yan, >> >> Would you be up for contributing a tutorial page that describes this? >>This >> is really useful information. Our docs are just simple .md files in >>the main code base. >> >> Regarding step (3), is the hdfs-site.xml put into the conf folder for >>the NM boxes, or on the client side (where run-job.sh is run)? >> >> Cheers, >> Chris >> >> On 3/11/14 10:07 AM, "Yan Fang" <[email protected]> wrote: >> >> >Hi Sonali, >> > >> >The way I make Samza run with HDFS is following: >> > >> >1. include hdfs jar in Samza jar tar.gz. >> >2. you may also want to make sure the hadoop-common.jar has the same >> >version as your hdfs jar. Otherwise, you may have configuration >> >error popping out. >> >3. then put hdfs-site.xml to conf folder, the same folder as the >> >yarn-site.xml 4. all other steps are not changed. >> > >> >Hope this will help. Thank you. >> > >> >Cheers, >> > >> >Fang, Yan >> >[email protected] >> >+1 (206) 849-4108 >> > >> > >> >On Tue, Mar 11, 2014 at 9:25 AM, Chris Riccomini >> ><[email protected]>wrote: >> > >> >> Hey Sonali, >> >> >> >> I believe that you need to make sure that the HDFS jar is in your >> >>.tar.gz file, as you've said. >> >> >> >> If that doesn't work, you might need to define this setting in >> >> core-site.xml on the machine you're running run-job.sh on: >> >> >> >> <property> >> >> <name>fs.hdfs.impl</name> >> >> <value>org.apache.hadoop.hdfs.DistributedFileSystem</value> >> >> <description>The FileSystem for hdfs: uris.</description> >> >> </property> >> >> >> >> >> >> You might also need to configure your NodeManagers to have the >> >> HDFS >>file >> >> system impl as well. >> >> >> >> I've never run Samza with HDFS, so I'm guessing here. Perhaps >> >>someone else on the list has been successful with this? >> >> >> >> Cheers, >> >> Chris >> >> >> >> On 3/10/14 3:59 PM, "[email protected]" >> >> <[email protected]> wrote: >> >> >> >> >Hello, >> >> > >> >> >I fixed this by starting from scratch with gradlew. But now when >> >> >I >>run >> >>my >> >> >job it throws this error: >> >> >Exception in thread "main" java.io.IOException: No FileSystem for >> >>scheme: >> >> >hdfs >> >> > at >> >> >> >>>>>org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java: >>>>>242 >>>>>1) >> >> > at >> >> >>>org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2428 >>>) >> >> > at >> >>org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:88) >> >> > at >> >> >>>org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:246 >>>7) >> >> > at >> >>org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2449) >> >> > at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:367) >> >> > at org.apache.hadoop.fs.Path.getFileSystem(Path.java:287) >> >> > at >> >> >> >>>>>org.apache.samza.job.yarn.ClientHelper.submitApplication(ClientHelper. >>>>>sc >> >>>al >> >> >a:111) >> >> > at >>org.apache.samza.job.yarn.YarnJob.submit(YarnJob.scala:55) >> >> > at >>org.apache.samza.job.yarn.YarnJob.submit(YarnJob.scala:48) >> >> > at org.apache.samza.job.JobRunner.run(JobRunner.scala:100) >> >> > at org.apache.samza.job.JobRunner$.main(JobRunner.scala:75) >> >> > at org.apache.samza.job.JobRunner.main(JobRunner.scala) >> >> > >> >> >I looked at the samza job tar.gz and it doesn't have a >> >> >Hadoop-hdfs >>jar. >> >> >Is that why I get this error? >> >> > >> >> >Thanks, >> >> >Sonali >> >> > >> >> >From: Parthasarathy, Sonali >> >> >Sent: Monday, March 10, 2014 11:25 AM >> >> >To: [email protected] >> >> >Subject: Failed to package using mvn >> >> > >> >> >Hi, >> >> > >> >> >When I tried to do a mvn clean package of my hello-samza project, >> >> >I >>get >> >> >the following error. Has anyone seen this before? >> >> > >> >> >[ERROR] Failed to execute goal on project samza-wikipedia: Could >> >> >not resolve dependencies for project samza:samza-wikipedia:jar:0.7.0: >>Could >> >> >not find artifact org.apache.samza:samza-kv_2.10:jar:0.7.0 in >> >> >apache-releases >>(https://repository.apache.org/content/groups/public) >> >>-> >> >> >[Help 1] >> >> >[ERROR] >> >> >[ERROR] To see the full stack trace of the errors, re-run Maven >> >> >with >> >>the >> >> >-e switch. >> >> >[ERROR] Re-run Maven using the -X switch to enable full debug >>logging. >> >> >[ERROR] >> >> >[ERROR] For more information about the errors and possible >>solutions, >> >> >please read the following articles: >> >> >[ERROR] [Help 1] >> >> > >> >> >> >> >> >>http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionE >>xce >> >>p >> >> >tion >> >> >[ERROR] >> >> >[ERROR] After correcting the problems, you can resume the build >> >> >with >> >>the >> >> >command >> >> >[ERROR] mvn <goals> -rf :samza-wikipedia >> >> > >> >> >Thanks, >> >> >Sonali >> >> > >> >> >Sonali Parthasarathy >> >> >R&D Developer, Data Insights >> >> >Accenture Technology Labs >> >> >703-341-7432 >> >> > >> >> > >> >> >________________________________ >> >> > >> >> >This message is for the designated recipient only and may contain >> >> >privileged, proprietary, or otherwise confidential information. >> >> >If >>you >> >> >have received it in error, please notify the sender immediately >> >> >and delete the original. Any other use of the e-mail by you is >>prohibited. >> >> >Where allowed by local law, electronic communications with >> >> >Accenture >> >>and >> >> >its affiliates, including e-mail and instant messaging (including >> >> >content), may be scanned by our systems for the purposes of >>information >> >> >security and assessment of internal compliance with Accenture >>policy. >> >> >> >>>>>___________________________________________________________________ >>>>>___ >>>>>__ >> >>>__ >> >> >____________ >> >> > >> >> >www.accenture.com >> >> >> >> >> >> ________________________________ This message is for the designated recipient only and may contain privileged, proprietary, or otherwise confidential information. If you have received it in error, please notify the sender immediately and delete the original. Any other use of the e-mail by you is prohibited. Where allowed by local law, electronic communications with Accenture and its affiliates, including e-mail and instant messaging (including content), may be scanned by our systems for the purposes of information security and assessment of internal compliance with Accenture policy. ______________________________________________________________________________________ www.accenture.com
