I tried pushing the tar file to HDFS but I got an error from hadoop saying that it couldn’t find core-site.xml file.
Have you seen this before? Thanks, On 10 Aug 2014, at 00:20, Telles Nobrega <[email protected]> wrote: > Hi Chris, now I have the tar file in my RM machine, and the yarn path points > to it. I changed the core-site.xml to use HttpFileSystem instead of HDFS now > it is failing with > > Application application_1407640485281_0001 failed 2 times due to AM Container > for appattempt_1407640485281_0001_000002 exited with exitCode:-1000 due to: > java.lang.ClassNotFoundException: Class > org.apache.samza.util.hadoop.HttpFileSystem not found > > I think I can solve this just installing scala files from the samza tutorial, > can you confirm that? > > On 09 Aug 2014, at 08:34, Telles Nobrega <[email protected]> wrote: > >> Hi Chris, >> >> I think the problem is that I forgot to update the yarn.job.package. >> I will try again to see if it works now. >> >> I have one more question, how can I stop (command line) the jobs running in >> my topology, for the experiment that I will run, I need to run the same job >> in 4 minutes intervals. So I need to kill it, clean the kafka topics and >> rerun. >> >> Thanks in advance. >> >> On 08 Aug 2014, at 12:41, Chris Riccomini <[email protected]> >> wrote: >> >>> Hey Telles, >>> >>>>> Do I need to have the job folder on each machine in my cluster? >>> >>> No, you should not need to do this. There are two ways to deploy your >>> tarball to the YARN grid. One is to put it in HDFS, and the other is to >>> put it on an HTTP server. The link to running a Samza job in a multi-node >>> YARN cluster describes how to do both (either HTTP server or HDFS). >>> >>> In both cases, once the tarball is put in on the HTTP/HDFS server(s), you >>> must update yarn.package.path to point to it. From there, the YARN NM >>> should download it for you automatically when you start your job. >>> >>> * Can you send along a paste of your job config? >>> >>> Cheers, >>> Chris >>> >>> On 8/8/14 8:04 AM, "Claudio Martins" <[email protected]> wrote: >>> >>>> Hi Telles, it looks to me that you forgot to update the >>>> "yarn.package.path" >>>> attribute in your config file for the task. >>>> >>>> - Claudio Martins >>>> Head of Engineering >>>> MobileAware USA Inc. / www.mobileaware.com >>>> office: +1 617 986 5060 / mobile: +1 617 480 5288 >>>> linkedin: www.linkedin.com/in/martinsclaudio >>>> >>>> >>>> On Fri, Aug 8, 2014 at 10:55 AM, Telles Nobrega <[email protected]> >>>> wrote: >>>> >>>>> Hi, >>>>> >>>>> this is my first time trying to run a job on a multinode environment. I >>>>> have the cluster set up, I can see in the GUI that all nodes are >>>>> working. >>>>> Do I need to have the job folder on each machine in my cluster? >>>>> - The first time I tried running with the job on the namenode machine >>>>> and >>>>> it failed saying: >>>>> >>>>> Application application_1407509228798_0001 failed 2 times due to AM >>>>> Container for appattempt_1407509228798_0001_000002 exited with exitCode: >>>>> -1000 due to: File >>>>> >>>>> >>>>> file:/home/ubuntu/alarm-samza/samza-job-package/target/samza-job-package- >>>>> 0.7.0-dist.tar.gz >>>>> does not exist >>>>> >>>>> So I copied the folder to each machine in my cluster and got this error: >>>>> >>>>> Application application_1407509228798_0002 failed 2 times due to AM >>>>> Container for appattempt_1407509228798_0002_000002 exited with exitCode: >>>>> -1000 due to: Resource >>>>> >>>>> >>>>> file:/home/ubuntu/alarm-samza/samza-job-package/target/samza-job-package- >>>>> 0.7.0-dist.tar.gz >>>>> changed on src filesystem (expected 1407509168000, was 1407509434000 >>>>> >>>>> What am I missing? >>>>> >>>>> p.s.: I followed this >>>>> <https://github.com/yahoo/samoa/wiki/Executing-SAMOA-with-Apache-Samza> >>>>> tutorial >>>>> and this >>>>> < >>>>> >>>>> http://samza.incubator.apache.org/learn/tutorials/0.7.0/run-in-multi-node >>>>> -yarn.html >>>>>> >>>>> to >>>>> set up the cluster. >>>>> >>>>> Help is much appreciated. >>>>> >>>>> Thanks in advance. >>>>> >>>>> -- >>>>> ------------------------------------------ >>>>> Telles Mota Vidal Nobrega >>>>> M.sc. Candidate at UFCG >>>>> B.sc. in Computer Science at UFCG >>>>> Software Engineer at OpenStack Project - HP/LSD-UFCG >>>>> >>> >> >
