Hi Chris, Thanks for the info. I'll rebase to the latest 0.9.0 branch. I think its better if we can rebase the samza-sql branch. I'm trying to get simple query where query working end-to-end with avro messages.
Thanks Milinda On Tue, Feb 24, 2015 at 12:35 PM, Chris Riccomini <criccom...@apache.org> wrote: > Hey Milinda, > > Could you start by rebasing to the latest 0.9.0 branch? There have been > some changes there to run multiple jobs. For example, jobs are now started > as part of the test_* methods, rather than in deployment.py: > > > > https://git-wip-us.apache.org/repos/asf?p=samza.git;a=blob_plain;f=samza-test/src/main/python/deployment.py;hb=HEAD > > This was done because the YARN NM can only run so many jobs at once, and > the remaining jobs just sit in the ACCEPTED state forever. With the new > approach, you run job1, wait for it to finish, then run job2, etc. > > When I run your branch, I see that the second negate-numbers job is getting > wedged in the ACCEPTED state. This is because YARN isn't letting it run > since another job is running already. If you rebase, you should be able to > run two jobs sequentially. Note that this requires that jobs shut > themselves down when they're "done". See SAMZA-544, or the negate-number > example (on master) on how this is done. > > I can also rebase the samza-sql Apache branch as well. > > Cheers, > Chris > > On Tue, Feb 24, 2015 at 9:08 AM, Milinda Pathirage <mpath...@umail.iu.edu> > wrote: > > > Hi Chris, > > > > You can find my changes at > > https://github.com/milinda/samza/tree/samza-integration-tests > > > > Thanks > > Milinda > > > > On Tue, Feb 24, 2015 at 12:04 PM, Milinda Pathirage < > mpath...@umail.iu.edu > > > > > wrote: > > > > > Hi Chris, > > > > > > Will push it to a branch in my Samza fork and send you the link. > > > > > > Thanks > > > Milinda > > > > > > On Tue, Feb 24, 2015 at 12:02 PM, Chris Riccomini < > criccom...@apache.org > > > > > > wrote: > > > > > >> Hey Milinda, > > >> > > >> Could you post your code somewhere? I have two jobs running in > > >> smoke_tests.py on SAMZA-544. Maybe that helps? > > >> > > >> Cheers, > > >> Chris > > >> > > >> On Tue, Feb 24, 2015 at 8:58 AM, Milinda Pathirage < > > mpath...@umail.iu.edu > > >> > > > >> wrote: > > >> > > >> > Hi Devs, > > >> > > > >> > I tried to add a new integration test to samza-test and found out > that > > >> the > > >> > second test always fail if that test depends on a different job. I > had > > >> > two *samza_job_deployer.start > > >> > *calls for 'negate_number' job and the test job I implemented. But > my > > >> test > > >> > always failed (but negate_number test worked without any issues) > when > > I > > >> had > > >> > both those jobs in deployment.py. But when I remove the > > 'negate_number' > > >> job > > >> > from deployment script, my test passed without any issues. > > >> > > > >> > Do any of you have experienced this situation before? > > >> > > > >> > Thanks > > >> > Milinda > > >> > > > >> > -- > > >> > Milinda Pathirage > > >> > > > >> > PhD Student | Research Assistant > > >> > School of Informatics and Computing | Data to Insight Center > > >> > Indiana University > > >> > > > >> > twitter: milindalakmal > > >> > skype: milinda.pathirage > > >> > blog: http://milinda.pathirage.org > > >> > > > >> > > > > > > > > > > > > -- > > > Milinda Pathirage > > > > > > PhD Student | Research Assistant > > > School of Informatics and Computing | Data to Insight Center > > > Indiana University > > > > > > twitter: milindalakmal > > > skype: milinda.pathirage > > > blog: http://milinda.pathirage.org > > > > > > > > > > > -- > > Milinda Pathirage > > > > PhD Student | Research Assistant > > School of Informatics and Computing | Data to Insight Center > > Indiana University > > > > twitter: milindalakmal > > skype: milinda.pathirage > > blog: http://milinda.pathirage.org > > > -- Milinda Pathirage PhD Student | Research Assistant School of Informatics and Computing | Data to Insight Center Indiana University twitter: milindalakmal skype: milinda.pathirage blog: http://milinda.pathirage.org
