Hi all.

We got several feedbacks about Sahara EDP's future from some China customers.
Here are some ideas we would like to share with you and need your input if we 
can implement them in Sahara(Liberty).

1. Add a schedule feature to run the jobs on time:
This request comes from the customer, they usually run the job in a specific 
time every day. So it should be great if there is a scheduler to help arrange 
the regular job to run.

2. A more complex workflow design in Sahara EDP:
Current EDP only provide one job that is running on one cluster.
But in a real case, it should be more complex, they usually use multiple jobs 
to calculate the data and may use several different type clusters to process it.
For example: Raw Data -> Job A(Cluster A) -> Job B(Cluster B) -> Job C(Cluster 
A) -> Result
Actually in my opinion, this kind of job could be easy to implement by using 
Oozie as a workflow engine. But for current EDP, it doesn't implement this kind 
of complex case.
Another concern is about Spark, for Spark it cannot use Oozie to do this. So we 
need to create an abstract layer to help to implement this kind of scenarios.

However, any suggestion is welcome.
Thanks.

__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Reply via email to