Hey Milinda, I rebased earlier this morning, and pushed samza-sql on Apache. If you rebase your fork off of this, you should be seeing the current state of the integration tests.
Cheers, Chris On Tue, Feb 24, 2015 at 12:09 PM, Milinda Pathirage <mpath...@umail.iu.edu> wrote: > Looks like rebasing is already done. I didn't see the commit messages. > > Milinda > > On Tue, Feb 24, 2015 at 3:08 PM, Milinda Pathirage <mpath...@umail.iu.edu> > wrote: > > > Hi Chris, > > > > Thanks for the info. I'll rebase to the latest 0.9.0 branch. I think its > > better if we can rebase the samza-sql branch. I'm trying to get simple > > query where query working end-to-end with avro messages. > > > > Thanks > > Milinda > > > > On Tue, Feb 24, 2015 at 12:35 PM, Chris Riccomini <criccom...@apache.org > > > > wrote: > > > >> Hey Milinda, > >> > >> Could you start by rebasing to the latest 0.9.0 branch? There have been > >> some changes there to run multiple jobs. For example, jobs are now > started > >> as part of the test_* methods, rather than in deployment.py: > >> > >> > >> > >> > https://git-wip-us.apache.org/repos/asf?p=samza.git;a=blob_plain;f=samza-test/src/main/python/deployment.py;hb=HEAD > >> > >> This was done because the YARN NM can only run so many jobs at once, and > >> the remaining jobs just sit in the ACCEPTED state forever. With the new > >> approach, you run job1, wait for it to finish, then run job2, etc. > >> > >> When I run your branch, I see that the second negate-numbers job is > >> getting > >> wedged in the ACCEPTED state. This is because YARN isn't letting it run > >> since another job is running already. If you rebase, you should be able > to > >> run two jobs sequentially. Note that this requires that jobs shut > >> themselves down when they're "done". See SAMZA-544, or the negate-number > >> example (on master) on how this is done. > >> > >> I can also rebase the samza-sql Apache branch as well. > >> > >> Cheers, > >> Chris > >> > >> On Tue, Feb 24, 2015 at 9:08 AM, Milinda Pathirage < > mpath...@umail.iu.edu > >> > > >> wrote: > >> > >> > Hi Chris, > >> > > >> > You can find my changes at > >> > https://github.com/milinda/samza/tree/samza-integration-tests > >> > > >> > Thanks > >> > Milinda > >> > > >> > On Tue, Feb 24, 2015 at 12:04 PM, Milinda Pathirage < > >> mpath...@umail.iu.edu > >> > > > >> > wrote: > >> > > >> > > Hi Chris, > >> > > > >> > > Will push it to a branch in my Samza fork and send you the link. > >> > > > >> > > Thanks > >> > > Milinda > >> > > > >> > > On Tue, Feb 24, 2015 at 12:02 PM, Chris Riccomini < > >> criccom...@apache.org > >> > > > >> > > wrote: > >> > > > >> > >> Hey Milinda, > >> > >> > >> > >> Could you post your code somewhere? I have two jobs running in > >> > >> smoke_tests.py on SAMZA-544. Maybe that helps? > >> > >> > >> > >> Cheers, > >> > >> Chris > >> > >> > >> > >> On Tue, Feb 24, 2015 at 8:58 AM, Milinda Pathirage < > >> > mpath...@umail.iu.edu > >> > >> > > >> > >> wrote: > >> > >> > >> > >> > Hi Devs, > >> > >> > > >> > >> > I tried to add a new integration test to samza-test and found out > >> that > >> > >> the > >> > >> > second test always fail if that test depends on a different job. > I > >> had > >> > >> > two *samza_job_deployer.start > >> > >> > *calls for 'negate_number' job and the test job I implemented. > But > >> my > >> > >> test > >> > >> > always failed (but negate_number test worked without any issues) > >> when > >> > I > >> > >> had > >> > >> > both those jobs in deployment.py. But when I remove the > >> > 'negate_number' > >> > >> job > >> > >> > from deployment script, my test passed without any issues. > >> > >> > > >> > >> > Do any of you have experienced this situation before? > >> > >> > > >> > >> > Thanks > >> > >> > Milinda > >> > >> > > >> > >> > -- > >> > >> > Milinda Pathirage > >> > >> > > >> > >> > PhD Student | Research Assistant > >> > >> > School of Informatics and Computing | Data to Insight Center > >> > >> > Indiana University > >> > >> > > >> > >> > twitter: milindalakmal > >> > >> > skype: milinda.pathirage > >> > >> > blog: http://milinda.pathirage.org > >> > >> > > >> > >> > >> > > > >> > > > >> > > > >> > > -- > >> > > Milinda Pathirage > >> > > > >> > > PhD Student | Research Assistant > >> > > School of Informatics and Computing | Data to Insight Center > >> > > Indiana University > >> > > > >> > > twitter: milindalakmal > >> > > skype: milinda.pathirage > >> > > blog: http://milinda.pathirage.org > >> > > > >> > > >> > > >> > > >> > -- > >> > Milinda Pathirage > >> > > >> > PhD Student | Research Assistant > >> > School of Informatics and Computing | Data to Insight Center > >> > Indiana University > >> > > >> > twitter: milindalakmal > >> > skype: milinda.pathirage > >> > blog: http://milinda.pathirage.org > >> > > >> > > > > > > > > -- > > Milinda Pathirage > > > > PhD Student | Research Assistant > > School of Informatics and Computing | Data to Insight Center > > Indiana University > > > > twitter: milindalakmal > > skype: milinda.pathirage > > blog: http://milinda.pathirage.org > > > > > > -- > Milinda Pathirage > > PhD Student | Research Assistant > School of Informatics and Computing | Data to Insight Center > Indiana University > > twitter: milindalakmal > skype: milinda.pathirage > blog: http://milinda.pathirage.org >
