Hi Trevor you mean adding an abstract layer for basic function like run job, cancel job etc, the we implement these by specific job engine like oozie, Ooyala job server etc, right ?
so I will do two things as below: (1) working on the scheduler edp jobs by oozie coordination so make sure we can running scheduler edp jobs (currently working on ,will submit patch later) (2) abstract the basic function as the abstract layer, and rewrite the specific job engine like oozie and ooyala job server. ( after doing first step, will talk it with you) so do you think that I have caught you meanings? or missed something? 2015-04-22 21:59 GMT+08:00 Trevor McKay <tmc...@redhat.com>: > On Wed, 2015-04-22 at 12:36 +0000, Chen, Ken wrote: > > o more complex workflows (job dependencies, DAGs, etc. Do we rely on > Oozie, or something else? > > >> Huichun is now figuring this. I am not whether you guys already > have some detail ideas about this? If needed we can contribute some effort. > If no details are ready, we can help draw a draft version first. > > I just made a note on the pad > > https://etherpad.openstack.org/p/sahara-liberty-proposed-sessions > > Maybe the right approach here is to develop a mapping notation that can > be expressed as a JSON object (like the proposed job interface mapping). > > If we can develop an abstract way to describe relationships between > jobs, then the individual EDP engines can implement it. For the Oozie > EDP engine, maybe it uses Oozie features in workflows. For Spark, or > Storm, maybe it uses some existing opensource coordinator or one is > written. > > The key idea would be to make job coordination part of the EDP engine, > with a well defined set of objects to describe the relationships. > > What do you think? Just a rough idea. Maybe there is a better way. > > > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >
__________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev