Hey Milinda, Could you start by rebasing to the latest 0.9.0 branch? There have been some changes there to run multiple jobs. For example, jobs are now started as part of the test_* methods, rather than in deployment.py:
https://git-wip-us.apache.org/repos/asf?p=samza.git;a=blob_plain;f=samza-test/src/main/python/deployment.py;hb=HEAD This was done because the YARN NM can only run so many jobs at once, and the remaining jobs just sit in the ACCEPTED state forever. With the new approach, you run job1, wait for it to finish, then run job2, etc. When I run your branch, I see that the second negate-numbers job is getting wedged in the ACCEPTED state. This is because YARN isn't letting it run since another job is running already. If you rebase, you should be able to run two jobs sequentially. Note that this requires that jobs shut themselves down when they're "done". See SAMZA-544, or the negate-number example (on master) on how this is done. I can also rebase the samza-sql Apache branch as well. Cheers, Chris On Tue, Feb 24, 2015 at 9:08 AM, Milinda Pathirage <mpath...@umail.iu.edu> wrote: > Hi Chris, > > You can find my changes at > https://github.com/milinda/samza/tree/samza-integration-tests > > Thanks > Milinda > > On Tue, Feb 24, 2015 at 12:04 PM, Milinda Pathirage <mpath...@umail.iu.edu > > > wrote: > > > Hi Chris, > > > > Will push it to a branch in my Samza fork and send you the link. > > > > Thanks > > Milinda > > > > On Tue, Feb 24, 2015 at 12:02 PM, Chris Riccomini <criccom...@apache.org > > > > wrote: > > > >> Hey Milinda, > >> > >> Could you post your code somewhere? I have two jobs running in > >> smoke_tests.py on SAMZA-544. Maybe that helps? > >> > >> Cheers, > >> Chris > >> > >> On Tue, Feb 24, 2015 at 8:58 AM, Milinda Pathirage < > mpath...@umail.iu.edu > >> > > >> wrote: > >> > >> > Hi Devs, > >> > > >> > I tried to add a new integration test to samza-test and found out that > >> the > >> > second test always fail if that test depends on a different job. I had > >> > two *samza_job_deployer.start > >> > *calls for 'negate_number' job and the test job I implemented. But my > >> test > >> > always failed (but negate_number test worked without any issues) when > I > >> had > >> > both those jobs in deployment.py. But when I remove the > 'negate_number' > >> job > >> > from deployment script, my test passed without any issues. > >> > > >> > Do any of you have experienced this situation before? > >> > > >> > Thanks > >> > Milinda > >> > > >> > -- > >> > Milinda Pathirage > >> > > >> > PhD Student | Research Assistant > >> > School of Informatics and Computing | Data to Insight Center > >> > Indiana University > >> > > >> > twitter: milindalakmal > >> > skype: milinda.pathirage > >> > blog: http://milinda.pathirage.org > >> > > >> > > > > > > > > -- > > Milinda Pathirage > > > > PhD Student | Research Assistant > > School of Informatics and Computing | Data to Insight Center > > Indiana University > > > > twitter: milindalakmal > > skype: milinda.pathirage > > blog: http://milinda.pathirage.org > > > > > > -- > Milinda Pathirage > > PhD Student | Research Assistant > School of Informatics and Computing | Data to Insight Center > Indiana University > > twitter: milindalakmal > skype: milinda.pathirage > blog: http://milinda.pathirage.org >
