> -----Original Message----- > From: Serge A. Salamanka [mailto:salsa-...@tut.by] > Sent: Wednesday, March 06, 2013 2:16 AM > To: cloudstack-dev@incubator.apache.org > Subject: Re: Building on Cloudstack > > On 06.03.2013 08:50, Seif Eddine Jemli wrote: > > You asked me about the scheduler i would like to implement. > > > > In fact, i have a school project: i have to divide matrix calculus on > > several VM's, the VM's have to calculate then return the results to > > the scheduler for aggregation. > > The scheduler have then to send the aggregated results to the user who > > typed the matrices. > > > > so the scheduler: > > * > > * > > *-have to contact cloudstack and ask for X number of VM's* *-then the > > vm's get work assigned to them crunch the work and return the results > > to the scheduler for aggregation* > > * > > * > > * > > * > > what do you think of this project? > > > > Is it feasible? > > > > Thanks in advance for your help. > > > > The VM's by themselves are just simple OS boxes. To run some tasks on > them you will need a job execution environment. There are plenty of > solutions that could be used for that purpose. > I'm personally looking at using IPython [1] and UNICORE [2] for accessing the > computing resources provided in cloud. There is also an idea to use > QosCosGrid [3] software stack but all this requires experiments with setups. > > We need some way to provision required resources in cloud with specially > prepared VMs. This could be achieved through CloudStack API or with a help > of BOSH [4].
Interesting topic. Yesterday, I read an article about google's borg and twitter's mesos: http://www.wired.com/wiredenterprise/2013/03/google-borg-twitter-mesos/ At the end of the article, it says: "Yes, there are other ways of efficiently spreading workloads across a cluster of servers. You could use virtualization, where you run virtual servers atop your physical machines and then load them with whatever software you like. But with Borg and Mesos, you don't have to worry about juggling all those virtual machines." Are you trying to do the same thing like Borg and Mesos does, but using CloudStack + Virtual Machine? What CloudStack can provide is to allocate resources(virtual machines), e.g. I want 2GHz cpu, 2G memory, then you can call cloudstack api to create a VM with the same service offering. After VM is booted up, need a way to distribute workloads(e.g. Hadoop/Spark jobs) into VM. Seems, it's possible to port Mesos's hadoop code (https://svn.apache.org/repos/asf/incubator/mesos/trunk/hadoop/mesos/) to use CloudStack. > > These are just my thoughts on the subject. > > See how Amazon provides HPC resources [5, 6] > > #Serge > > [1] http://ipython.org/ipython-doc/rel-0.13.1/parallel/parallel_intro.html > [2] http://www.unicore.eu/ > [3] https://wiki.egi.eu/wiki/QosCosGrid_Platform > [4] https://github.com/cloudfoundry/bosh > [5] http://aws.amazon.com/hpc-applications/ > [6] http://www.youtube.com/embed/YfCgK1bmCjw
