Re: Building on Cloudstack
On 06.03.2013 15:15, Sebastien Goasguen wrote: If you don't need to multiplex regular hpc jobs and VMs then things gets easier. Just build a CloudStack cloud, and setup your HPC cluster via templates. One template for a head node and one template for the compute nodes. Put MPI or whatever libraries you need and your are up. I don't need to multiplex HPC jobs and VMs. This is exactly the setup I want to make. This way we can provide users the ability to create their own HPC clusters with their own templates for worker nodes as configurable environment. CloudStack won't help with the actual parallelizing, that's not what it's made for. CloudStack can help to create highly configurable virtual infrastructure for HPC jobs that can be used by organizations that care about strong privacy of the submitted information and by those individuals that want root access to their HPC nodes. If you wanted an optimized VM instance, for true HPC, you would need to set it up on hypervisors with PCI passthrough, some type of low latency network like infiniband. In addition you could have GPU instances (like on Amazon), if you had GPUs on your hardware. Infiniband and GPU nodes are available. The task is to make them available through CloudStack. I would appreciate more information on configuring optimized VMs. #Serge
Re: Building on Cloudstack
On 06.03.2013 02:00, Alex Huang wrote: Serge, In CloudStack, you can control where VM's are deployed. If you don't want to do any programming, the easiest is to do host-tags and storage-tags to control where vms run and their storage deployed. However, that gives you very static information. If you're fine with writing code, CloudStack offers three different types of plugins to fine tune the placement. You would write one of these plugins, change the componentcontext.xml to use your implementation and you can control the placement of the VMs. - DeploymentPlanner: Called by CloudStack to find placement for VMs. - HostAllocator: Called by DeploymentPlanner to find a hypervisor host for VMs. - StoragePoolAllocator: Called by DeploymentPlanner to find storage for volumes. Generally, we advise that DeploymentPlanner should be used for heuristics and the allocators are used for actual limitations of the hypervisor or storage. For example, in your case, you may want to write a service of your own to keep track of actual usage of cpu and peak times and then use that information to feed it back to CloudStack via a SuperComputingDeploymentPlanner. It's a little hard to be clear without the specifics. Let me know if you need more information. --Alex Alex, I cannot provide you specific information at the moment. What you have just explained is probably good for one idea I have. The idea is to add some functionality to the UNICORE Rich Client [1] for creating HPC infrastructure in the Cloud which can be used for job submition aftewards. I will certainly come back to this if there is time. For now I want to learn the process of creation and customization of system VMs to provide some specific services to user created VMs in Cloud. #Serge [1] http://www.unicore.eu/unicore/architecture/client-layer.php#anchor_urc
Re: Building on Cloudstack
On Mar 12, 2013, at 7:41 AM, Serge A. Salamanka salsa-...@tut.by wrote: On 06.03.2013 15:15, Sebastien Goasguen wrote: If you don't need to multiplex regular hpc jobs and VMs then things gets easier. Just build a CloudStack cloud, and setup your HPC cluster via templates. One template for a head node and one template for the compute nodes. Put MPI or whatever libraries you need and your are up. I don't need to multiplex HPC jobs and VMs. This is exactly the setup I want to make. This way we can provide users the ability to create their own HPC clusters with their own templates for worker nodes as configurable environment. CloudStack won't help with the actual parallelizing, that's not what it's made for. CloudStack can help to create highly configurable virtual infrastructure for HPC jobs that can be used by organizations that care about strong privacy of the submitted information and by those individuals that want root access to their HPC nodes. If you wanted an optimized VM instance, for true HPC, you would need to set it up on hypervisors with PCI passthrough, some type of low latency network like infiniband. In addition you could have GPU instances (like on Amazon), if you had GPUs on your hardware. Infiniband and GPU nodes are available. The task is to make them available through CloudStack. I would appreciate more information on configuring optimized VMs. The way I would do it is to first work without CloudStack, focusing on the hypervisor layer. You need to be able to use infiniband properly through a VM, I believe there are special drivers, but could be that virtio is now enough to get the low latency. Once you know how to do that, you can create a service offering in cloudstack. For GPUs, I still believe you need to setup a pass through (quick search): http://superuser.com/questions/427918/virtual-machine-with-dedicated-gpu-is-it-possible-private-gpu-cloud and that you will have to dedicate GPUs to a single user. With bare metal functionality in CS 4.1, you might want to look into provisioning a bare metal cluster from the nodes that have GPUs. #Serge
Re: Building on Cloudstack
On Mar 6, 2013, at 12:50 AM, Seif Eddine Jemli seifeddineje...@gmail.com wrote: You asked me about the scheduler i would like to implement. In fact, i have a school project: i have to divide matrix calculus on several VM's, the VM's have to calculate then return the results to the scheduler for aggregation. The scheduler have then to send the aggregated results to the user who typed the matrices. so the scheduler: * * *-have to contact cloudstack and ask for X number of VM's* *-then the vm's get work assigned to them crunch the work and return the results to the scheduler for aggregation* * * * * what do you think of this project? Is it feasible? Seif, yes this is definitely feasible but the parallelizing of tasks is not provided by CloudStack. What you should do: 1-Deploy CloudStack so you have a cloud 2-Prepare a template that has the batch scheduler that you want and the parallel libraries needed (e.g MPI) 3-Launch several VMs via CloudStack using the prepared template. 4-Access the head node of your virtual cluster (that you just started) and submit your parallel job Again CloudStack won't help with the actual parallelizing, that's not what it's made for. If you wanted an optimized VM instance, for true HPC, you would need to set it up on hypervisors with PCI passthrough, some type of low latency network like infiniband. In addition you could have GPU instances (like on Amazon), if you had GPUs on your hardware. Hope that helps, -sebastien Thanks in advance for your help.
Re: Building on Cloudstack
On Mar 5, 2013, at 4:31 PM, Serge A. Salamanka salsa-...@tut.by wrote: Hi, Alex I've joined the list couple days ago. Interested in this same feature. At the supercomputing center where I work we have lots of computing power that could be used as a virtual supercomputer through virtualization with CloudStack. I'm just thinking how it can be realized with the help of CloudStack. I guess Seif is doing something similar? I found an article [1] that explains pretty much all that I wish to do with CloudStack for that purpose. Regards, Serge [1] http://sebgoa.blogspot.com/2012/11/i-was-asked-other-day-what-was.html Hi Serge, I wrote that blog. Yes it can be done, I am not aware of a large scale deployment for HPC purposes yet, however I don't foresee any huge obstacles. The biggest issue is to make CloudStack play nicely with your current infrastructure, especially if you want to keep traditional jobs running at the same time than having VMs running on the hosts. Im my previous job what we had done, is write a simple batch job that would get nodes from the scheduler, then once we would get the nodes we would run the cloud agent to basically build a cloud-infrastructure on top of the existing supercomputer. The cloud would just appear has a regular HPC job and be destroyed once the wall time expires. The issue tends to be with the networking, you may want to setup a specific VLAN for you transient cloud. If you don't need to multiplex regular hpc jobs and VMs then things gets easier. Just build a CloudStack cloud, and setup your HPC cluster via templates. One template for a head node and one template for the compute nodes. Put MPI or whatever libraries you need and your are up. -sebastien On 06.03.2013 00:20, Alex Huang wrote: Seif, What is the function of your scheduler? Can you go into it a little more? Thanks. --Alex -Original Message- From: Seif Eddine Jemli [mailto:seifeddineje...@gmail.com] Sent: Tuesday, March 5, 2013 12:36 PM To: cloudstack-dev@incubator.apache.org Subject: Building on Cloudstack Hi, I am working on a project using Cloudstack. I'll need to implement a scheduler to control the virtual machines generated by Cloudstack. (in fact, the project is about parallel calculus on different virtual machines). So, it seems that i have to use the Cloudstack API. i wanted to ask for some tips for my project. any volunteers? :) thanks in advance.
Re: Building on Cloudstack
On Wed, Mar 6, 2013 at 1:17 PM, Edison Su edison...@citrix.com wrote: -Original Message- From: Serge A. Salamanka [mailto:salsa-...@tut.by] Sent: Wednesday, March 06, 2013 2:16 AM To: cloudstack-dev@incubator.apache.org Subject: Re: Building on Cloudstack On 06.03.2013 08:50, Seif Eddine Jemli wrote: You asked me about the scheduler i would like to implement. In fact, i have a school project: i have to divide matrix calculus on several VM's, the VM's have to calculate then return the results to the scheduler for aggregation. The scheduler have then to send the aggregated results to the user who typed the matrices. so the scheduler: * * *-have to contact cloudstack and ask for X number of VM's* *-then the vm's get work assigned to them crunch the work and return the results to the scheduler for aggregation* * * * * what do you think of this project? Is it feasible? Thanks in advance for your help. The VM's by themselves are just simple OS boxes. To run some tasks on them you will need a job execution environment. There are plenty of solutions that could be used for that purpose. I'm personally looking at using IPython [1] and UNICORE [2] for accessing the computing resources provided in cloud. There is also an idea to use QosCosGrid [3] software stack but all this requires experiments with setups. We need some way to provision required resources in cloud with specially prepared VMs. This could be achieved through CloudStack API or with a help of BOSH [4]. Interesting topic. Yesterday, I read an article about google's borg and twitter's mesos: http://www.wired.com/wiredenterprise/2013/03/google-borg-twitter-mesos/ At the end of the article, it says: Yes, there are other ways of efficiently spreading workloads across a cluster of servers. You could use virtualization, where you run virtual servers atop your physical machines and then load them with whatever software you like. But with Borg and Mesos, you don't have to worry about juggling all those virtual machines. Are you trying to do the same thing like Borg and Mesos does, but using CloudStack + Virtual Machine? What CloudStack can provide is to allocate resources(virtual machines), e.g. I want 2GHz cpu, 2G memory, then you can call cloudstack api to create a VM with the same service offering. After VM is booted up, need a way to distribute workloads(e.g. Hadoop/Spark jobs) into VM. Seems, it's possible to port Mesos's hadoop code (https://svn.apache.org/repos/asf/incubator/mesos/trunk/hadoop/mesos/) to use CloudStack. Mesos is now part of the incubator: http://incubator.apache.org/mesos/ --David
Re: Building on Cloudstack
that's exactly what i am trying to do, distribute calculus on the VM's created by cloudstack. what about HADOOP??how should i use it?
Re: Building on Cloudstack
What language are you using? I just learned how to use the CloudStack API and have working examples in Java. On Tue, Mar 5, 2013 at 1:36 PM, Seif Eddine Jemli seifeddineje...@gmail.com wrote: Hi, I am working on a project using Cloudstack. I'll need to implement a scheduler to control the virtual machines generated by Cloudstack. (in fact, the project is about parallel calculus on different virtual machines). So, it seems that i have to use the Cloudstack API. i wanted to ask for some tips for my project. any volunteers? :) thanks in advance. -- *Mike Tutkowski* *Senior CloudStack Developer, SolidFire Inc.* e: mike.tutkow...@solidfire.com o: 303.746.7302 Advancing the way the world uses the cloudhttp://solidfire.com/solution/overview/?video=play *™*
Re: Building on Cloudstack
Would this example code help? I think if I send it out as an attachment to the e-mail list that it will be blocked, though. Perhaps I should send it directly to you? On Tue, Mar 5, 2013 at 1:40 PM, Seif Eddine Jemli seifeddineje...@gmail.com wrote: i prefer java -- *Mike Tutkowski* *Senior CloudStack Developer, SolidFire Inc.* e: mike.tutkow...@solidfire.com o: 303.746.7302 Advancing the way the world uses the cloudhttp://solidfire.com/solution/overview/?video=play *™*
Re: Building on Cloudstack
send it directly to me if you don't mind. thanks, i already sent you a direct email
Re: Building on Cloudstack
I know we prefer that conversations be held on the e-mail list (as opposed to direct communications), so here's a copy/paste of what I sent over directly (I sent it directly because I included an attachment and I think the e-mail list woud have blocked it): Hi! I've attached that Java code. A lot of it is currently commented out as I was commenting some of it in and out and executing parts of it to test different APIs. You will notice a little of it is related to the XenServer API. Feel free to ignore that. :) You will also notice a little of it is related to the SolidFire API. Feel free to ignore that, too. :) I work at a company in Boulder, CO called SolidFire (http://solidfire.com). We develop software that ties all-SSD storage nodes together into a clustered and fault-tolerant SAN. I'm working on CloudStack integrating SolidFire features into CloudStack and also helping out on CloudStack where I can. As an FYI, there is also a CloudStack CLI called CloudMonkey that might be worth checking out in your situation: https://cwiki.apache.org/confluence/display/CLOUDSTACK/CloudStack+cloudmonkey+CLI I haven't used it before, but it looks pretty useful. Here is a link to the API Developer's Guide: http://incubator.apache.org/cloudstack/docs/en-US/Apache_CloudStack/4.0.1-incubating/html/API_Developers_Guide/index.html That explains the weird logic I have to create a signature for the API call. Feel free to send me questions directly, but I believe the CloudStack community prefers that we e-mail over their e-mail list so that others can chime in and possibly learn from the experience. By the way, the name CreateVolume.java is legacy. Originally I created this class to create a SolidFire volume only. It has expanded as I've looked into the CloudStack and XenServer APIs. Talk to you later! On Tue, Mar 5, 2013 at 1:42 PM, Seif Eddine Jemli seifeddineje...@gmail.com wrote: Hi, i am working on a project, trying to establish distributed calculus using the virtual machines. What did you use the cloudstack API for? thanks :) On Tue, Mar 5, 2013 at 1:43 PM, Seif Eddine Jemli seifeddineje...@gmail.com wrote: send it directly to me if you don't mind. thanks, i already sent you a direct email -- *Mike Tutkowski* *Senior CloudStack Developer, SolidFire Inc.* e: mike.tutkow...@solidfire.com o: 303.746.7302 Advancing the way the world uses the cloudhttp://solidfire.com/solution/overview/?video=play *™*
Re: Building on Cloudstack
I forgot to mention that I was - at one time - passing in parameters to that app. You may see that some of those parameters are still used while others are hard coded. On Tue, Mar 5, 2013 at 1:58 PM, Mike Tutkowski mike.tutkow...@solidfire.com wrote: I know we prefer that conversations be held on the e-mail list (as opposed to direct communications), so here's a copy/paste of what I sent over directly (I sent it directly because I included an attachment and I think the e-mail list woud have blocked it): Hi! I've attached that Java code. A lot of it is currently commented out as I was commenting some of it in and out and executing parts of it to test different APIs. You will notice a little of it is related to the XenServer API. Feel free to ignore that. :) You will also notice a little of it is related to the SolidFire API. Feel free to ignore that, too. :) I work at a company in Boulder, CO called SolidFire (http://solidfire.com). We develop software that ties all-SSD storage nodes together into a clustered and fault-tolerant SAN. I'm working on CloudStack integrating SolidFire features into CloudStack and also helping out on CloudStack where I can. As an FYI, there is also a CloudStack CLI called CloudMonkey that might be worth checking out in your situation: https://cwiki.apache.org/confluence/display/CLOUDSTACK/CloudStack+cloudmonkey+CLI I haven't used it before, but it looks pretty useful. Here is a link to the API Developer's Guide: http://incubator.apache.org/cloudstack/docs/en-US/Apache_CloudStack/4.0.1-incubating/html/API_Developers_Guide/index.html That explains the weird logic I have to create a signature for the API call. Feel free to send me questions directly, but I believe the CloudStack community prefers that we e-mail over their e-mail list so that others can chime in and possibly learn from the experience. By the way, the name CreateVolume.java is legacy. Originally I created this class to create a SolidFire volume only. It has expanded as I've looked into the CloudStack and XenServer APIs. Talk to you later! On Tue, Mar 5, 2013 at 1:42 PM, Seif Eddine Jemli seifeddineje...@gmail.com wrote: Hi, i am working on a project, trying to establish distributed calculus using the virtual machines. What did you use the cloudstack API for? thanks :) On Tue, Mar 5, 2013 at 1:43 PM, Seif Eddine Jemli seifeddineje...@gmail.com wrote: send it directly to me if you don't mind. thanks, i already sent you a direct email -- *Mike Tutkowski* *Senior CloudStack Developer, SolidFire Inc.* e: mike.tutkow...@solidfire.com o: 303.746.7302 Advancing the way the world uses the cloudhttp://solidfire.com/solution/overview/?video=play *™* -- *Mike Tutkowski* *Senior CloudStack Developer, SolidFire Inc.* e: mike.tutkow...@solidfire.com o: 303.746.7302 Advancing the way the world uses the cloudhttp://solidfire.com/solution/overview/?video=play *™*
Re: Building on Cloudstack
This is probably the most relevant method for your purposes: private static String executeCloudStackCommand(final String strCsIpAddress, final String strCsPort, final String strCommand, final String strCsApiKey, final String strCsSecretKey) throws Exception On Tue, Mar 5, 2013 at 2:06 PM, Mike Tutkowski mike.tutkow...@solidfire.com wrote: I forgot to mention that I was - at one time - passing in parameters to that app. You may see that some of those parameters are still used while others are hard coded. On Tue, Mar 5, 2013 at 1:58 PM, Mike Tutkowski mike.tutkow...@solidfire.com wrote: I know we prefer that conversations be held on the e-mail list (as opposed to direct communications), so here's a copy/paste of what I sent over directly (I sent it directly because I included an attachment and I think the e-mail list woud have blocked it): Hi! I've attached that Java code. A lot of it is currently commented out as I was commenting some of it in and out and executing parts of it to test different APIs. You will notice a little of it is related to the XenServer API. Feel free to ignore that. :) You will also notice a little of it is related to the SolidFire API. Feel free to ignore that, too. :) I work at a company in Boulder, CO called SolidFire (http://solidfire.com). We develop software that ties all-SSD storage nodes together into a clustered and fault-tolerant SAN. I'm working on CloudStack integrating SolidFire features into CloudStack and also helping out on CloudStack where I can. As an FYI, there is also a CloudStack CLI called CloudMonkey that might be worth checking out in your situation: https://cwiki.apache.org/confluence/display/CLOUDSTACK/CloudStack+cloudmonkey+CLI I haven't used it before, but it looks pretty useful. Here is a link to the API Developer's Guide: http://incubator.apache.org/cloudstack/docs/en-US/Apache_CloudStack/4.0.1-incubating/html/API_Developers_Guide/index.html That explains the weird logic I have to create a signature for the API call. Feel free to send me questions directly, but I believe the CloudStack community prefers that we e-mail over their e-mail list so that others can chime in and possibly learn from the experience. By the way, the name CreateVolume.java is legacy. Originally I created this class to create a SolidFire volume only. It has expanded as I've looked into the CloudStack and XenServer APIs. Talk to you later! On Tue, Mar 5, 2013 at 1:42 PM, Seif Eddine Jemli seifeddineje...@gmail.com wrote: Hi, i am working on a project, trying to establish distributed calculus using the virtual machines. What did you use the cloudstack API for? thanks :) On Tue, Mar 5, 2013 at 1:43 PM, Seif Eddine Jemli seifeddineje...@gmail.com wrote: send it directly to me if you don't mind. thanks, i already sent you a direct email -- *Mike Tutkowski* *Senior CloudStack Developer, SolidFire Inc.* e: mike.tutkow...@solidfire.com o: 303.746.7302 Advancing the way the world uses the cloudhttp://solidfire.com/solution/overview/?video=play *™* -- *Mike Tutkowski* *Senior CloudStack Developer, SolidFire Inc.* e: mike.tutkow...@solidfire.com o: 303.746.7302 Advancing the way the world uses the cloudhttp://solidfire.com/solution/overview/?video=play *™* -- *Mike Tutkowski* *Senior CloudStack Developer, SolidFire Inc.* e: mike.tutkow...@solidfire.com o: 303.746.7302 Advancing the way the world uses the cloudhttp://solidfire.com/solution/overview/?video=play *™*
RE: Building on Cloudstack
Seif, What is the function of your scheduler? Can you go into it a little more? Thanks. --Alex -Original Message- From: Seif Eddine Jemli [mailto:seifeddineje...@gmail.com] Sent: Tuesday, March 5, 2013 12:36 PM To: cloudstack-dev@incubator.apache.org Subject: Building on Cloudstack Hi, I am working on a project using Cloudstack. I'll need to implement a scheduler to control the virtual machines generated by Cloudstack. (in fact, the project is about parallel calculus on different virtual machines). So, it seems that i have to use the Cloudstack API. i wanted to ask for some tips for my project. any volunteers? :) thanks in advance.
Re: Building on Cloudstack
Hi, Alex I've joined the list couple days ago. Interested in this same feature. At the supercomputing center where I work we have lots of computing power that could be used as a virtual supercomputer through virtualization with CloudStack. I'm just thinking how it can be realized with the help of CloudStack. I guess Seif is doing something similar? I found an article [1] that explains pretty much all that I wish to do with CloudStack for that purpose. Regards, Serge [1] http://sebgoa.blogspot.com/2012/11/i-was-asked-other-day-what-was.html On 06.03.2013 00:20, Alex Huang wrote: Seif, What is the function of your scheduler? Can you go into it a little more? Thanks. --Alex -Original Message- From: Seif Eddine Jemli [mailto:seifeddineje...@gmail.com] Sent: Tuesday, March 5, 2013 12:36 PM To: cloudstack-dev@incubator.apache.org Subject: Building on Cloudstack Hi, I am working on a project using Cloudstack. I'll need to implement a scheduler to control the virtual machines generated by Cloudstack. (in fact, the project is about parallel calculus on different virtual machines). So, it seems that i have to use the Cloudstack API. i wanted to ask for some tips for my project. any volunteers? :) thanks in advance.
RE: Building on Cloudstack
Serge, In CloudStack, you can control where VM's are deployed. If you don't want to do any programming, the easiest is to do host-tags and storage-tags to control where vms run and their storage deployed. However, that gives you very static information. If you're fine with writing code, CloudStack offers three different types of plugins to fine tune the placement. You would write one of these plugins, change the componentcontext.xml to use your implementation and you can control the placement of the VMs. - DeploymentPlanner: Called by CloudStack to find placement for VMs. - HostAllocator: Called by DeploymentPlanner to find a hypervisor host for VMs. - StoragePoolAllocator: Called by DeploymentPlanner to find storage for volumes. Generally, we advise that DeploymentPlanner should be used for heuristics and the allocators are used for actual limitations of the hypervisor or storage. For example, in your case, you may want to write a service of your own to keep track of actual usage of cpu and peak times and then use that information to feed it back to CloudStack via a SuperComputingDeploymentPlanner. It's a little hard to be clear without the specifics. Let me know if you need more information. --Alex -Original Message- From: Serge A. Salamanka [mailto:salsa-...@tut.by] Sent: Tuesday, March 5, 2013 1:32 PM To: cloudstack-dev@incubator.apache.org Subject: Re: Building on Cloudstack Hi, Alex I've joined the list couple days ago. Interested in this same feature. At the supercomputing center where I work we have lots of computing power that could be used as a virtual supercomputer through virtualization with CloudStack. I'm just thinking how it can be realized with the help of CloudStack. I guess Seif is doing something similar? I found an article [1] that explains pretty much all that I wish to do with CloudStack for that purpose. Regards, Serge [1] http://sebgoa.blogspot.com/2012/11/i-was-asked-other-day-what- was.html On 06.03.2013 00:20, Alex Huang wrote: Seif, What is the function of your scheduler? Can you go into it a little more? Thanks. --Alex -Original Message- From: Seif Eddine Jemli [mailto:seifeddineje...@gmail.com] Sent: Tuesday, March 5, 2013 12:36 PM To: cloudstack-dev@incubator.apache.org Subject: Building on Cloudstack Hi, I am working on a project using Cloudstack. I'll need to implement a scheduler to control the virtual machines generated by Cloudstack. (in fact, the project is about parallel calculus on different virtual machines). So, it seems that i have to use the Cloudstack API. i wanted to ask for some tips for my project. any volunteers? :) thanks in advance.
Re: Building on Cloudstack
You asked me about the scheduler i would like to implement. In fact, i have a school project: i have to divide matrix calculus on several VM's, the VM's have to calculate then return the results to the scheduler for aggregation. The scheduler have then to send the aggregated results to the user who typed the matrices. so the scheduler: * * *-have to contact cloudstack and ask for X number of VM's* *-then the vm's get work assigned to them crunch the work and return the results to the scheduler for aggregation* * * * * what do you think of this project? Is it feasible? Thanks in advance for your help.
