Re: [openstack-dev] [Nova] Meetup Summary
Agreed. I'm just thinking on the opportunity of providing a REST API on top of the scheduler RPC API with a 1:1 matching, so that the Gantt project would step up by itself. I don't think it's a hard stuff, provided I already did that stuff for Climate (providing Pecan/WSME API). What do you think about it ? Even if it's not top priority, that's a quickwin. Well, Im not sure about quickwin, though J I think that we should focus on the main objective of having a self-contained Gantt working with Nova first. Some of the interaction issues still worry me, especially the host_state host_update queries. These issues will have impact on the Gantt API (at least for Nova to use), so Im not sure the current RPC API will hold up either. But I will not discourage any personal effort J De : Sylvain Bauza [mailto:sylvain.ba...@gmail.com] Envoyé : mardi 18 février 2014 22:41 À : OpenStack Development Mailing List (not for usage questions) Objet : Re: [openstack-dev] [Nova] Meetup Summary Hi Don, 2014-02-18 21:28 GMT+01:00 Dugger, Donald D donald.d.dug...@intel.com: Sylvain- As you can tell from the meeting today the scheduler sub-group is really not the gantt group meeting, I try to make sure that messages for things like the agenda and what not include both `gantt and `scheduler in the subject so its clear were talking about the same thing. That's the main reason why I was unable to attend the previous scheduler meetings... Now that I attended this today meeting, that's quite clear to me. I apologize for this misunderstanding, but as I can't dedicate all my time on Gantt/Nova, I have to make sure the time I'm taking on it can be worth it. Now that we agreed on a plan for next steps, I think it's important to put the infos on Gantt blueprints, even if most of the changes are related to Nova. The current etherpad is huge, and frightening people who would want to contribute IMHO. Note that our ultimate goal is to create a scheduler that is usable by other projects, not just nova, but that is a second task. The first task is to create a separate scheduler that will be usable by nova at a minimum. (World domination will follow later J Agreed. I'm just thinking on the opportunity of providing a REST API on top of the scheduler RPC API with a 1:1 matching, so that the Gantt project would step up by itself. I don't think it's a hard stuff, provided I already did that stuff for Climate (providing Pecan/WSME API). What do you think about it ? Even if it's not top priority, that's a quickwin. -Sylvain -- Don Dugger Censeo Toto nos in Kansa esse decisse. - D. Gale Ph: 303/443-3786 tel:303%2F443-3786 From: Sylvain Bauza [mailto:sylvain.ba...@gmail.com] Sent: Monday, February 17, 2014 4:26 PM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [Nova] Meetup Summary Hi Russell and Don, 2014-02-17 23:41 GMT+01:00 Russell Bryant rbry...@redhat.com: Greetings, 2) Gantt - We discussed the progress of the Gantt effort. After discussing the problems encountered so far and the other scheduler work going on, the consensus was that we really need to focus on decoupling the scheduler from the rest of Nova while it's still in the Nova tree. Don was still interested in working on the existing gantt tree to learn what he can about the coupling of the scheduler to the rest of Nova. Nobody had a problem with that, but it doesn't sound like we'll be ready to regenerate the gantt tree to be the real gantt tree soon. We probably need another cycle of development before it will be ready. As a follow-up to this, I wonder if we should rename the current gantt repository from openstack/gantt to stackforge/gantt to avoid any possible confusion. We should make it clear that we don't expect the current repo to be used yet. There is currently no precise meeting timeslot for Gantt but the one with Nova scheduler subteam. Would it be possible to have a status on the current path for Gantt so that people interested in joining the effort would be able to get in ? There is currently a discussion on how Gantt and Nova should interact, in particular regarding HostState and how Nova Computes could update their status so as Gantt would be able to filter on them. There are also other discussions about testing, API, etc. so I'm just wondering how to help and where. On a side note, if Gantt is becoming a Stackforge project planning to have Nova scheduling first, could we also assume that we could also implement this service for being used by other projects (such as Climate) in parallel with Nova ? The current utilization-aware-scheduling blueprint is nearly done so that it can be used for other queries than just Nova scheduling, but unfortunately as the scheduler is still part of Nova and without a REST API, it can't be leveraged on third-party projects. Thanks, -Sylvain [1] : https://blueprints.launchpad.net/nova/+spec
Re: [openstack-dev] [Nova] Meetup Summary
Hi Toan-Tran, 2014-02-19 9:40 GMT+01:00 Khanh-Toan Tran khanh-toan.t...@cloudwatt.com: Agreed. I'm just thinking on the opportunity of providing a REST API on top of the scheduler RPC API with a 1:1 matching, so that the Gantt project would step up by itself. I don't think it's a hard stuff, provided I already did that stuff for Climate (providing Pecan/WSME API). What do you think about it ? Even if it's not top priority, that's a quickwin. Well, I'm not sure about quickwin, though J I think that we should focus on the main objective of having a self-contained Gantt working with Nova first. Some of the interaction issues still worry me, especially the host_state host_update queries. These issues will have impact on the Gantt API (at least for Nova to use), so I'm not sure the current RPC API will hold up either. But I will not discourage any personal effort J Well, about the 2 things : first, the REST API would just be a mapping 1:1 of the scheduler RPC API, ie select_destinations(), run_instance() and prep_resize(). The arguments passed to the object would just be serialized in a POST query with JSON/XML data and deserialized/passed to the RPC API. Think it as a REST wrapper, no hard stuff. About the host_state that's something that has been discussed yesterday, Gantt first has to provide a python lib for the calls to update_from_compute_node, but that could also be managed thanks to a CLI python binding later on. -Sylvain ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Meetup Summary
On 2/17/2014 4:41 PM, Russell Bryant wrote: Greetings, Last week we had an in-person Nova meetup. Bluehost was a wonderful and generous host. Many thanks to them. :-) Here's some observations and a summary of some of the things that we discussed: 1) Mark McClain (Neutron PTL) and and Mark Washenberger (Glance PTL) both attended. Having some cross-project discussion time was *incredibly* useful, so I'm thankful they attended. This makes me very optimistic about our plans to have a cross-project day at the Atlanta design summit. We need to try to get as many opportunities as possible for this sort of collaboration. 2) Gantt - We discussed the progress of the Gantt effort. After discussing the problems encountered so far and the other scheduler work going on, the consensus was that we really need to focus on decoupling the scheduler from the rest of Nova while it's still in the Nova tree. Don was still interested in working on the existing gantt tree to learn what he can about the coupling of the scheduler to the rest of Nova. Nobody had a problem with that, but it doesn't sound like we'll be ready to regenerate the gantt tree to be the real gantt tree soon. We probably need another cycle of development before it will be ready. As a follow-up to this, I wonder if we should rename the current gantt repository from openstack/gantt to stackforge/gantt to avoid any possible confusion. We should make it clear that we don't expect the current repo to be used yet. 3) v3 API - We discussed the current status of this effort, including the tasks API, and all other v3 work. There are some notes here: https://etherpad.openstack.org/p/NovaV3APIDoneCriteria I actually think we need to talk about this some more before we mark v3 as stable. I'll get notes together and start another thread soon. 4) We talked about Nova's integration with Neutron and made some good progress. We came up with a blueprint (ideally for Icehouse) to improve Nova-Neutron interaction. There are two cases we need to improve that have been particularly painful. The first is the network info cache. Neutron can issue an API callback to Nova to let us know that we need to refresh the cache. The second is knowing that VIF setup is complete. Right now we have cases where we issue a request to Neutron and it is processed asynchronously. We have no way to know when it has finished. For example, we really need to know that VIF plumbing is set up before we boot an instance and it tries its DHCP request. We can do this with nova-network, but with Neutron it's just a giant race. I'm actually surprised we've made it this long without fixing this. One or both of these issues (thinking VIF readiness) is also causing a gate failure in master and stable/havana: https://bugs.launchpad.net/nova/+bug/1210483 I'd like to propose skipping that test if Tempest is configured with Neutron until we get the bug fixed/blueprint resolved. By the way, can I get a link to the blueprint to reference in the bug (or vice-versa)? 5) Driver CI - We talked about the ongoing effort to set up CI for all of the compute drivers. The discussion was mostly a status review. At this point, the Xenserver and Docker drivers are both at risk of being removed from Nova for the Icehouse release if CI is not up and running in time. 6) Upgrades - we discussed the state of upgrading Nova. It was mostly a review of the excellent progress being made this cycle. Dan Smith has been doing a lot of work to get us closer to where we can upgrade the control services at once with downtime, but roll through upgrading the computes later after service is back up. Joe Gordon has been working on automating the testing of this to make sure we don't break it, so that should be running soon. Lastly, everyone in attendance seemed to really enjoy it, and the overwhelming vote in the room was for doing the same thing again during the Juno cycle. Dates and location TBD. +1 Thanks, -- Thanks, Matt Riedemann ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Meetup Summary
On 02/18/2014 12:36 PM, Matt Riedemann wrote: 4) We talked about Nova's integration with Neutron and made some good progress. We came up with a blueprint (ideally for Icehouse) to improve Nova-Neutron interaction. There are two cases we need to improve that have been particularly painful. The first is the network info cache. Neutron can issue an API callback to Nova to let us know that we need to refresh the cache. The second is knowing that VIF setup is complete. Right now we have cases where we issue a request to Neutron and it is processed asynchronously. We have no way to know when it has finished. For example, we really need to know that VIF plumbing is set up before we boot an instance and it tries its DHCP request. We can do this with nova-network, but with Neutron it's just a giant race. I'm actually surprised we've made it this long without fixing this. One or both of these issues (thinking VIF readiness) is also causing a gate failure in master and stable/havana: https://bugs.launchpad.net/nova/+bug/1210483 I'd like to propose skipping that test if Tempest is configured with Neutron until we get the bug fixed/blueprint resolved. By the way, can I get a link to the blueprint to reference in the bug (or vice-versa)? I haven't seen a blueprint for this yet. Mark, is that something you were planning on driving? -- Russell Bryant ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Meetup Summary
Sylvain- As you can tell from the meeting today the scheduler sub-group is really not the gantt group meeting, I try to make sure that messages for things like the agenda and what not include both `gantt' and `scheduler' in the subject so it's clear we're talking about the same thing. Note that our ultimate goal is to create a scheduler that is usable by other projects, not just nova, but that is a second task. The first task is to create a separate scheduler that will be usable by nova at a minimum. (World domination will follow later :) -- Don Dugger Censeo Toto nos in Kansa esse decisse. - D. Gale Ph: 303/443-3786 From: Sylvain Bauza [mailto:sylvain.ba...@gmail.com] Sent: Monday, February 17, 2014 4:26 PM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [Nova] Meetup Summary Hi Russell and Don, 2014-02-17 23:41 GMT+01:00 Russell Bryant rbry...@redhat.commailto:rbry...@redhat.com: Greetings, 2) Gantt - We discussed the progress of the Gantt effort. After discussing the problems encountered so far and the other scheduler work going on, the consensus was that we really need to focus on decoupling the scheduler from the rest of Nova while it's still in the Nova tree. Don was still interested in working on the existing gantt tree to learn what he can about the coupling of the scheduler to the rest of Nova. Nobody had a problem with that, but it doesn't sound like we'll be ready to regenerate the gantt tree to be the real gantt tree soon. We probably need another cycle of development before it will be ready. As a follow-up to this, I wonder if we should rename the current gantt repository from openstack/gantt to stackforge/gantt to avoid any possible confusion. We should make it clear that we don't expect the current repo to be used yet. There is currently no precise meeting timeslot for Gantt but the one with Nova scheduler subteam. Would it be possible to have a status on the current path for Gantt so that people interested in joining the effort would be able to get in ? There is currently a discussion on how Gantt and Nova should interact, in particular regarding HostState and how Nova Computes could update their status so as Gantt would be able to filter on them. There are also other discussions about testing, API, etc. so I'm just wondering how to help and where. On a side note, if Gantt is becoming a Stackforge project planning to have Nova scheduling first, could we also assume that we could also implement this service for being used by other projects (such as Climate) in parallel with Nova ? The current utilization-aware-scheduling blueprint is nearly done so that it can be used for other queries than just Nova scheduling, but unfortunately as the scheduler is still part of Nova and without a REST API, it can't be leveraged on third-party projects. Thanks, -Sylvain [1] : https://blueprints.launchpad.net/nova/+spec/utilization-aware-scheduling ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Meetup Summary
On 2/18/2014 1:12 PM, Russell Bryant wrote: On 02/18/2014 12:36 PM, Matt Riedemann wrote: 4) We talked about Nova's integration with Neutron and made some good progress. We came up with a blueprint (ideally for Icehouse) to improve Nova-Neutron interaction. There are two cases we need to improve that have been particularly painful. The first is the network info cache. Neutron can issue an API callback to Nova to let us know that we need to refresh the cache. The second is knowing that VIF setup is complete. Right now we have cases where we issue a request to Neutron and it is processed asynchronously. We have no way to know when it has finished. For example, we really need to know that VIF plumbing is set up before we boot an instance and it tries its DHCP request. We can do this with nova-network, but with Neutron it's just a giant race. I'm actually surprised we've made it this long without fixing this. One or both of these issues (thinking VIF readiness) is also causing a gate failure in master and stable/havana: https://bugs.launchpad.net/nova/+bug/1210483 I'd like to propose skipping that test if Tempest is configured with Neutron until we get the bug fixed/blueprint resolved. By the way, can I get a link to the blueprint to reference in the bug (or vice-versa)? I haven't seen a blueprint for this yet. Mark, is that something you were planning on driving? Here we go: https://blueprints.launchpad.net/nova/+spec/check-neutron-port-status There are two patches up for it now from Aaron, still needs (exception) approval for Icehouse. -- Thanks, Matt Riedemann ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Meetup Summary
From: Russell Bryant [rbry...@redhat.com] Sent: 17 February 2014 22:41 To: OpenStack Development Mailing List Subject: [openstack-dev] [Nova] Meetup Summary 5) Driver CI - We talked about the ongoing effort to set up CI for all of the compute drivers. The discussion was mostly a status review. At this point, the Xenserver and Docker drivers are both at risk of being removed from Nova for the Icehouse release if CI is not up and running in time. Just a quick update on this - the Citrix CI is up and running and commenting on jobs that pass full tempest using XenServer 6.2 with the XenAPI Driver. We are not currently commenting on jobs that fail as I think there are some false negatives we need to iron out - or convince ourselves they are the same as the bugs that are hitting the official gate - over the next few days. The CI is currently working through a backlog of jobs, but it's running tests against nova, devstack and tempest. Bob ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Meetup Summary
Hi Don, 2014-02-18 21:28 GMT+01:00 Dugger, Donald D donald.d.dug...@intel.com: Sylvain- As you can tell from the meeting today the scheduler sub-group is really not the gantt group meeting, I try to make sure that messages for things like the agenda and what not include both `gantt' and `scheduler' in the subject so it's clear we're talking about the same thing. That's the main reason why I was unable to attend the previous scheduler meetings... Now that I attended this today meeting, that's quite clear to me. I apologize for this misunderstanding, but as I can't dedicate all my time on Gantt/Nova, I have to make sure the time I'm taking on it can be worth it. Now that we agreed on a plan for next steps, I think it's important to put the infos on Gantt blueprints, even if most of the changes are related to Nova. The current etherpad is huge, and frightening people who would want to contribute IMHO. Note that our ultimate goal is to create a scheduler that is usable by other projects, not just nova, but that is a second task. The first task is to create a separate scheduler that will be usable by nova at a minimum. (World domination will follow later J Agreed. I'm just thinking on the opportunity of providing a REST API on top of the scheduler RPC API with a 1:1 matching, so that the Gantt project would step up by itself. I don't think it's a hard stuff, provided I already did that stuff for Climate (providing Pecan/WSME API). What do you think about it ? Even if it's not top priority, that's a quickwin. -Sylvain -- Don Dugger Censeo Toto nos in Kansa esse decisse. - D. Gale Ph: 303/443-3786 *From:* Sylvain Bauza [mailto:sylvain.ba...@gmail.com] *Sent:* Monday, February 17, 2014 4:26 PM *To:* OpenStack Development Mailing List (not for usage questions) *Subject:* Re: [openstack-dev] [Nova] Meetup Summary Hi Russell and Don, 2014-02-17 23:41 GMT+01:00 Russell Bryant rbry...@redhat.com: Greetings, 2) Gantt - We discussed the progress of the Gantt effort. After discussing the problems encountered so far and the other scheduler work going on, the consensus was that we really need to focus on decoupling the scheduler from the rest of Nova while it's still in the Nova tree. Don was still interested in working on the existing gantt tree to learn what he can about the coupling of the scheduler to the rest of Nova. Nobody had a problem with that, but it doesn't sound like we'll be ready to regenerate the gantt tree to be the real gantt tree soon. We probably need another cycle of development before it will be ready. As a follow-up to this, I wonder if we should rename the current gantt repository from openstack/gantt to stackforge/gantt to avoid any possible confusion. We should make it clear that we don't expect the current repo to be used yet. There is currently no precise meeting timeslot for Gantt but the one with Nova scheduler subteam. Would it be possible to have a status on the current path for Gantt so that people interested in joining the effort would be able to get in ? There is currently a discussion on how Gantt and Nova should interact, in particular regarding HostState and how Nova Computes could update their status so as Gantt would be able to filter on them. There are also other discussions about testing, API, etc. so I'm just wondering how to help and where. On a side note, if Gantt is becoming a Stackforge project planning to have Nova scheduling first, could we also assume that we could also implement this service for being used by other projects (such as Climate) in parallel with Nova ? The current utilization-aware-scheduling blueprint is nearly done so that it can be used for other queries than just Nova scheduling, but unfortunately as the scheduler is still part of Nova and without a REST API, it can't be leveraged on third-party projects. Thanks, -Sylvain [1] : https://blueprints.launchpad.net/nova/+spec/utilization-aware-scheduling ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
