Re: [openstack-dev] We need a new version of hacking for Icehouse, or provide compatibility with oslo.sphinx in oslosphinx

[Sergey Lukjanov]

I've created change request to bump min hacking version to 0.8.1 -
https://review.openstack.org/82339

On Sat, Mar 22, 2014 at 10:52 AM, Thomas Goirand  wrote:
> On 03/22/2014 12:58 AM, Joe Gordon wrote:
>>
>> So it sounds like we need:
>>
>> * Hacking 0.8.1 to fix the oslo.sphinx  oslosphinx issue for Icehouse.
>> Since we cap hacking versions at 0.9 [1] this will  get used in icehouse.
>> * Hacking 0.9 to release all the new hacking goodness. This will be
>> targeted for use in Juno.
>
> I agree with this plan.
>
> Thomas
>
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



-- 
Sincerely yours,
Sergey Lukjanov
Sahara Technical Lead
(OpenStack Data Processing)
Mirantis Inc.

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [NOVA][VMWare][live-migration] VCDriver live migration problem

[laserjetyang]

I think we might need to discuss the VMware driver to refactor progress in
IRC meeting as well, and what is our overall plan. We have been keeping
seeing the vmware code broken.


On Sun, Mar 23, 2014 at 7:49 AM, Jay Lau  wrote:

> Thanks Shawn, what you proposed is exactly I want ;-) Cool!
>
> We can discuss more during the IRC meeting.
>
> Thanks!
>
>
> 2014-03-22 20:22 GMT+08:00 Jay Lau :
>
> Thanks Shawn, I have updated the title with VMWare.
>>
>> Yes, I know that live migration "works". But the problem is when a
>> cluster admin want to live migrate a VM instance, s/he will not know the
>> target host where to migrate, as s/he cannot get target host from nova
>> compute because currently VCDriver can only report cluster or resource pool
>> as hypervisor host but not ESX server.
>>
>> IMHO, the VCDriver should support live migration between cluster,
>> resource pool and ESX host, so we may need do at least the following
>> enhancements:
>> 1) Enable live migration with even one nova compute. My current thinking
>> is that enhance target host as host:node when live migrate a VM instance
>> and the live migration task need
>> 2) Enable VCDriver report all ESX servers.
>>
>> We can discuss more during next week's IRC meeting.
>>
>> Thanks!
>>
>>
>> 2014-03-22 17:13 GMT+08:00 Shawn Hartsock :
>>
>> Hi Jay. We usually use [vmware] to tag discussion of VMware things. I
>>> almost didn't see this message.
>>>
>>> In short, there is a plan and we're currently blocked because we have
>>> to address several other pressing issues in the driver before we can
>>> address this one. Part of this is due to the fact that we can't press
>>> harder on blueprints or changes to the VCDriver right now.
>>>
>>> I actually reported this bug and we've discussed this at
>>> https://wiki.openstack.org/wiki/Meetings/VMwareAPI the basic problem
>>> is that live-migration actually "works" but you can't presently
>>> formulate a command that activates the feature from the CLI under some
>>> configurations. That's because of the introduction of clusters in the
>>> VCDriver in Havana.
>>>
>>> To fix this, we have to come up with a way to target a host inside the
>>> cluster (as I pointed out in the bug) or we have to have some way for
>>> a live migration to occur between clusters and a way to validate that
>>> this can happen first.
>>>
>>> As for the priority of this bug, it's been set to Medium which puts it
>>> well behind many of the Critical or High tasks on our radar. As for
>>> fixing the bug, no new outward behaviors or API are going to be
>>> introduced and this was working at one point and now it's stopped. To
>>> call this a new feature seems a bit strange.
>>>
>>> So, moving forward... perhaps we need to re-evaluate the priority
>>> order on some of these things. I tabled Juno planning during the last
>>> VMwareAPI subteam meeting but I plan on starting the discussion next
>>> week. We have a priority order for blueprints that we set as a team
>>> and these are publicly recorded in our meeting logs and on the wiki.
>>> I'll try to do better advertising these things. You are of course
>>> invited... and yeah... if you're interested in what we're fixing next
>>> in the VCDriver that next IRC meeting is where we'll start the
>>> discussion.
>>>
>>> On Sat, Mar 22, 2014 at 1:18 AM, Jay Lau  wrote:
>>> > Hi,
>>> >
>>> > Currently we cannot do live migration with VCDriver in nova, live
>>> migration
>>> > is really an important feature, so any plan to fix this?
>>> >
>>> > I noticed that there is already bug tracing this but seems no progress
>>> since
>>> > last year's November: https://bugs.launchpad.net/nova/+bug/1192192
>>> >
>>> > Here just bring this problem up to see if there are any plan to fix
>>> this.
>>> > After some investigation, I think that this might deserve to be a
>>> blueprint
>>> > but not a bug.
>>> >
>>> > We may need to resolve issues for the following cases:
>>> > 1) How to live migration with only one nova compute? (one nova compute
>>> can
>>> > manage multiple clusters and there can be multi hosts in one cluster)
>>> > 2) Support live migration between clusters
>>> > 3) Support live migration between resource pools
>>> > 4) Support live migration between hosts
>>> > 5) Support live migration between cluster and host
>>> > 6) Support live migration between cluster and resource pool
>>> > 7) Support live migration between resource pool and host
>>> > 8) Might be more cases.
>>> >
>>> > Please show your comments if any and correct me if anything is not
>>> correct.
>>> >
>>> > --
>>> > Thanks,
>>> >
>>> > Jay
>>> >
>>> > ___
>>> > OpenStack-dev mailing list
>>> > OpenStack-dev@lists.openstack.org
>>> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>> >
>>>
>>>
>>>
>>> --
>>> # Shawn.Hartsock - twitter: @hartsock - plus.google.com/+ShawnHartsock
>>>
>>> ___
>>> OpenStack-dev mailing list
>>> 

Re: [openstack-dev] [NOVA][VMWare][live-migration] VCDriver live migration problem

[Jay Lau]

Thanks Shawn, what you proposed is exactly I want ;-) Cool!

We can discuss more during the IRC meeting.

Thanks!


2014-03-22 20:22 GMT+08:00 Jay Lau :

> Thanks Shawn, I have updated the title with VMWare.
>
> Yes, I know that live migration "works". But the problem is when a cluster
> admin want to live migrate a VM instance, s/he will not know the target
> host where to migrate, as s/he cannot get target host from nova compute
> because currently VCDriver can only report cluster or resource pool as
> hypervisor host but not ESX server.
>
> IMHO, the VCDriver should support live migration between cluster, resource
> pool and ESX host, so we may need do at least the following enhancements:
> 1) Enable live migration with even one nova compute. My current thinking
> is that enhance target host as host:node when live migrate a VM instance
> and the live migration task need
> 2) Enable VCDriver report all ESX servers.
>
> We can discuss more during next week's IRC meeting.
>
> Thanks!
>
>
> 2014-03-22 17:13 GMT+08:00 Shawn Hartsock :
>
> Hi Jay. We usually use [vmware] to tag discussion of VMware things. I
>> almost didn't see this message.
>>
>> In short, there is a plan and we're currently blocked because we have
>> to address several other pressing issues in the driver before we can
>> address this one. Part of this is due to the fact that we can't press
>> harder on blueprints or changes to the VCDriver right now.
>>
>> I actually reported this bug and we've discussed this at
>> https://wiki.openstack.org/wiki/Meetings/VMwareAPI the basic problem
>> is that live-migration actually "works" but you can't presently
>> formulate a command that activates the feature from the CLI under some
>> configurations. That's because of the introduction of clusters in the
>> VCDriver in Havana.
>>
>> To fix this, we have to come up with a way to target a host inside the
>> cluster (as I pointed out in the bug) or we have to have some way for
>> a live migration to occur between clusters and a way to validate that
>> this can happen first.
>>
>> As for the priority of this bug, it's been set to Medium which puts it
>> well behind many of the Critical or High tasks on our radar. As for
>> fixing the bug, no new outward behaviors or API are going to be
>> introduced and this was working at one point and now it's stopped. To
>> call this a new feature seems a bit strange.
>>
>> So, moving forward... perhaps we need to re-evaluate the priority
>> order on some of these things. I tabled Juno planning during the last
>> VMwareAPI subteam meeting but I plan on starting the discussion next
>> week. We have a priority order for blueprints that we set as a team
>> and these are publicly recorded in our meeting logs and on the wiki.
>> I'll try to do better advertising these things. You are of course
>> invited... and yeah... if you're interested in what we're fixing next
>> in the VCDriver that next IRC meeting is where we'll start the
>> discussion.
>>
>> On Sat, Mar 22, 2014 at 1:18 AM, Jay Lau  wrote:
>> > Hi,
>> >
>> > Currently we cannot do live migration with VCDriver in nova, live
>> migration
>> > is really an important feature, so any plan to fix this?
>> >
>> > I noticed that there is already bug tracing this but seems no progress
>> since
>> > last year's November: https://bugs.launchpad.net/nova/+bug/1192192
>> >
>> > Here just bring this problem up to see if there are any plan to fix
>> this.
>> > After some investigation, I think that this might deserve to be a
>> blueprint
>> > but not a bug.
>> >
>> > We may need to resolve issues for the following cases:
>> > 1) How to live migration with only one nova compute? (one nova compute
>> can
>> > manage multiple clusters and there can be multi hosts in one cluster)
>> > 2) Support live migration between clusters
>> > 3) Support live migration between resource pools
>> > 4) Support live migration between hosts
>> > 5) Support live migration between cluster and host
>> > 6) Support live migration between cluster and resource pool
>> > 7) Support live migration between resource pool and host
>> > 8) Might be more cases.
>> >
>> > Please show your comments if any and correct me if anything is not
>> correct.
>> >
>> > --
>> > Thanks,
>> >
>> > Jay
>> >
>> > ___
>> > OpenStack-dev mailing list
>> > OpenStack-dev@lists.openstack.org
>> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> >
>>
>>
>>
>> --
>> # Shawn.Hartsock - twitter: @hartsock - plus.google.com/+ShawnHartsock
>>
>> ___
>> OpenStack-dev mailing list
>> OpenStack-dev@lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
>
>
> --
> Thanks,
>
> Jay
>



-- 
Thanks,

Jay
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] An analysis of code review in Nova

[Shawn Hartsock]

On Sat, Mar 22, 2014 at 11:55 AM, Matt Riedemann
 wrote:
>
>
> On 3/22/2014 5:19 AM, Shawn Hartsock wrote:
>>
>> On Fri, Mar 14, 2014 at 6:58 PM, Dan Smith  wrote:
>>>
>>> Review latency will be directly affected by how good the refactoring
>>> changes are staged. If they are small, on-topic and easy to validate,
>>> they will go quickly. They should be linearized unless there are some
>>> places where multiple sequences of changes make sense (i.e. refactoring
>>> a single file that results in no changes required to others).
>>
>>
>>
>> I'm going to bring this to the next
>> https://wiki.openstack.org/wiki/Meetings/VMwareAPI we can start
>> working on how we'll set the order for this kind of work. Currently we
>> have a whole blueprint for refactoring a single method. That seems
>> silly. I'll want to come up with a plan around how to restructure the
>> driver so we can avoid some of the messes we've seen in the past.
>
>
> I think the point of starting with refactoring the nested method mess in the
> spawn method was it (1) seemed relatively trivial (think fast review
> turnaround) and (2) should be a good "bang for the buck" kind of change, as
> a lot of the original complaint was related to how hard it is to verify
> changes in the giant spawn method are tested - which you also point out
> below.
>
>
>>
>> I want to avoid one big refactor effort that drags on, but I also want
>> to address bigger problems we have inside the driver. For example,
>
>
> I also want to avoid a big refactor effort dragging on, and I like the
> thinking on design changes, but are they doing going to be happening at the
> same time?  Or is the complete re-design going to supersede the refactoring?
> My only concern is biting off more than can be chewed in juno-1.
>
> Plus there is the refactor to use oslo.vmware, where does that fit into
> this?
>
>
>> vm_util.py seems to have become burdened with work that it shouldn't
>> have. It also performs a great number of unnecessary round trips using
>> a vm_ref to pull individual vm details over one at a time. Introducing
>> a VirtualMachine object that held all these references would simplify
>> some operations (I'm not the first person to suggest this and it
>> wasn't novel to me either when it was presented.)
>>
>> It would seem Juno-1 would be the time to make these changes and we
>> need to serialize this work to keep reviewers from losing their
>> marbles trying to track it all. I would like to work out a plan for
>> this in conjunction with interested core-reviewers who would be
>> willing to more or less sponsor this work. Because of the problems
>> Matt points out, I don't want to tackle this in a haphazard or
>> piece-meal way since it could completely disrupt any new blueprint
>> work people may have targeted for Juno.
>
>
> Yeah, definitely need a plan here.  I'd like to see things prioritized based
> on what can be fixed in a relatively isolated way which gives a good return
> on coding/reviewing investment, e.g. pulling those nested methods out of
> spawn so they can be unit tested individually with mock rather than a large,
> seemingly rigid and scaffolded test framework.
>

Just because you said something about it :-)

I wrote this up off-the-cuff:
https://blueprints.launchpad.net/nova/+spec/vmware-vm-ref-refactor

This is about the size of each of these refactors that I'm thinking
of. I don't want to get much bigger than this and I don't want to
force a round-trip up to oslo.vmware and back to Nova for this size of
change. I don't really think we have to table the inclusion of
oslo.vmware integration in Juno in order to do this kind of work.
However, just as we pointed out at the top of the thread, there's a
danger that this kind of work can stall new feature adds so the timing
and inclusion of it is sensitive.

>
>>
>> Having said that, on this driver, new blueprints in the last several
>> cycles have introduced serious feature regressions. Several minor bug
>> fixes have altered and introduced key architectural components that
>> have broken multiple critical features. In my professional opinion
>> this has a root cause based on the drivers tightly coupled and
>> non-cohesive internal design.
>>
>> The driver design is tightly coupled in that a change in one area
>> forces multiple updates and changes *throughout* the rest of the
>> driver. This is true in testing as well. The testing design often
>> requires you to trace the entire codebase if you add a single optional
>> parameter to a single method. This does not have to be true.
>
>
> Yup, this is my major complaint and ties into what I'm saying above, I find
> it really difficult to determine most of the time where a change is tested.
> Because of the nature of the driver code and my lack of actually writing
> features in it, as a reviewer I don't know if a change in X is going to
> break Y, so I rely on solid test coverage and the testing needs to be more
> natural to follow than it currently is.
>
>
>>
>> Th

Re: [openstack-dev] OpenStack/GSoC 2014 - Reviewing Applications

[Davanum Srinivas]

Sriram,

We'll have to check with a student if there are able to see it during
this phase of the GSoC where mentors are evaluating the proposals.
There is some information/guidance here:
http://www.google-melange.com/document/show/gsoc_program/google/gsoc2011/userguide#depth_appreview
http://en.flossmanuals.net/melange/student-application-period/

-- dims

On Sat, Mar 22, 2014 at 6:34 PM, Sriram Subramanian
 wrote:
> Davanum, are the scores private or is there a way to make them private?
>
>
> On Sat, Mar 22, 2014 at 12:07 PM, Davanum Srinivas 
> wrote:
>>
>> Dear Students,
>>
>> We received the following proposals:
>>
>> AMES-Cloud (Nagashree Bhat)
>> A pre-caching system for OpenStack (Anastasis Andronidis)
>> Proposal for Implementing an application-level FWaaS driver
>> ((Zorp) Dániel Csubák)
>> Cassandra and Redis storage backend for Openstack marconi (Jeremy
>> Henriques)
>> Openstack-OSLO :Add a New Backend to Oslo.Cache (sai krishna)
>> Implement a Fuzz testing framework that can be run on Tempest
>> (Manishanker Talusani)
>> How to detect network anomalies from telemetry data within Openstack
>> (mst89)
>> Implement a re-usable shared library for VMware(oslo.vmware) (Masaru
>> Nomura)
>> Cross-services Scheduler project (Artem Shepelev)
>> OpenStack/Marconi: Py3k support (Nataliia)
>> Improve benchmarking context mechanism in Rally (Kumar Rishabh)
>> Develop a benchmarking suite and new storage backends to OpenStack
>> Marconi (Prashanth Raghu)
>> Adding Redis as a Storage Backend to OpenStack Marconi (Chenchong Qin)
>> Auto Benchmarking System for OpenStack (RobberPhex)
>> Developing Benchmarks for Virtual Machines of OpenStack with Rally
>> (Tzanetos Balitsaris)
>> Add a new storage backend to the OpenStack Message Queuing Service
>> (Victoria Martínez de la Cruz)
>>
>> Dear Mentors
>> (ddutta/flwang/julim/hughsaunders/greghaynes/annegentle/sriramhere/arnaudleg/coroner/boris_42/blint/ybudupi/cppcabrera),
>>
>> Please log into the Google GSoC web-site and review **all** proposals
>> within a week (our own deadline - say 29th).
>>
>> 1) Please click "Wish to mentor" on the left hand side, if you are
>> willing to mentor this project.
>>Yes, we should have more than one mentor. This is a way to mention
>> that you are willing to mentor,
>>actual assignment will happen later i believe. So please switch
>> this on for all projects you are
>>interested in.
>>
>> 2) If you have a question about a proposal and need a response from
>> the candidate, please leave
>>a comment for them. If you want to ask a observation/question to
>> other mentors or me, leave a
>>comment marked "Private"
>>
>> 3) To assign a score. Please click on the number of start in "My
>> score:" at the bottom
>> 5 = amazing applicant, could be a module maintainer on completing
>> the program
>> 4 = strong applicant, will do a good job
>> 3 = good applicant, but is somewhat inexperienced
>> 2 = is unlikely to do a good job
>> 1 = not a good candidate
>>
>> Please feel free to leave detailed notes on candidates you know well
>> for other mentors (Please mark comments as "Private")
>>
>> If you are not able to help with this exercise, please let me know ASAP.
>>
>> If anyone else would like to step up and be a Mentor for any of these
>> projects/students, please register in the Google GSoC site and let me
>> know your "username".
>>
>> Thanks,
>> dims
>>
>>
>>
>> --
>> Davanum Srinivas :: http://davanum.wordpress.com
>>
>> ___
>> OpenStack-dev mailing list
>> OpenStack-dev@lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
>
> --
> Thanks,
> -Sriram
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



-- 
Davanum Srinivas :: http://davanum.wordpress.com

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] OpenStack/GSoC 2014 - Reviewing Applications

[Sriram Subramanian]

Davanum, are the scores private or is there a way to make them private?


On Sat, Mar 22, 2014 at 12:07 PM, Davanum Srinivas wrote:

> Dear Students,
>
> We received the following proposals:
>
> AMES-Cloud (Nagashree Bhat)
> A pre-caching system for OpenStack (Anastasis Andronidis)
> Proposal for Implementing an application-level FWaaS driver
> ((Zorp) Dániel Csubák)
> Cassandra and Redis storage backend for Openstack marconi (Jeremy
> Henriques)
> Openstack-OSLO :Add a New Backend to Oslo.Cache (sai krishna)
> Implement a Fuzz testing framework that can be run on Tempest
> (Manishanker Talusani)
> How to detect network anomalies from telemetry data within Openstack
> (mst89)
> Implement a re-usable shared library for VMware(oslo.vmware) (Masaru
> Nomura)
> Cross-services Scheduler project (Artem Shepelev)
> OpenStack/Marconi: Py3k support (Nataliia)
> Improve benchmarking context mechanism in Rally (Kumar Rishabh)
> Develop a benchmarking suite and new storage backends to OpenStack
> Marconi (Prashanth Raghu)
> Adding Redis as a Storage Backend to OpenStack Marconi (Chenchong Qin)
> Auto Benchmarking System for OpenStack (RobberPhex)
> Developing Benchmarks for Virtual Machines of OpenStack with Rally
> (Tzanetos Balitsaris)
> Add a new storage backend to the OpenStack Message Queuing Service
> (Victoria Martínez de la Cruz)
>
> Dear Mentors
> (ddutta/flwang/julim/hughsaunders/greghaynes/annegentle/sriramhere/arnaudleg/coroner/boris_42/blint/ybudupi/cppcabrera),
>
> Please log into the Google GSoC web-site and review **all** proposals
> within a week (our own deadline - say 29th).
>
> 1) Please click "Wish to mentor" on the left hand side, if you are
> willing to mentor this project.
>Yes, we should have more than one mentor. This is a way to mention
> that you are willing to mentor,
>actual assignment will happen later i believe. So please switch
> this on for all projects you are
>interested in.
>
> 2) If you have a question about a proposal and need a response from
> the candidate, please leave
>a comment for them. If you want to ask a observation/question to
> other mentors or me, leave a
>comment marked "Private"
>
> 3) To assign a score. Please click on the number of start in "My
> score:" at the bottom
> 5 = amazing applicant, could be a module maintainer on completing
> the program
> 4 = strong applicant, will do a good job
> 3 = good applicant, but is somewhat inexperienced
> 2 = is unlikely to do a good job
> 1 = not a good candidate
>
> Please feel free to leave detailed notes on candidates you know well
> for other mentors (Please mark comments as "Private")
>
> If you are not able to help with this exercise, please let me know ASAP.
>
> If anyone else would like to step up and be a Mentor for any of these
> projects/students, please register in the Google GSoC site and let me
> know your "username".
>
> Thanks,
> dims
>
>
>
> --
> Davanum Srinivas :: http://davanum.wordpress.com
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



-- 
Thanks,
-Sriram
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] An analysis of code review in Nova

[Shawn Hartsock]

On Sat, Mar 22, 2014 at 2:03 PM, Chris Behrens  wrote:
> I'd like to get spawn broken up sooner rather than later, personally. It has 
> additional benefits of being able to do better orchestration of builds from 
> conductor, etc.
>

I plan on delivery of that occurring as soon as or within a week after
Juno opens for commits. I understand we should see this in a week.
I've been holding on the refactor to avoid splitting reviewer
attention.

-- 
# Shawn.Hartsock - twitter: @hartsock - plus.google.com/+ShawnHartsock

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [NOVA][VMWare][live-migration] VCDriver live migration problem

[Shawn Hartsock]

Jay,

On point number 2 I proposed
https://blueprints.launchpad.net/nova/+spec/vmware-auto-inventory soon
after we merged the Cluster patch because of just the issues you are
highlighting. I may have named it badly and it's purpose maybe
outdated now... so I should definitely work on clarifying things like
that.

My original plan for what I called "auto-inventory" was to enable the
VCDriver to pull out individual ESX hosts and present them as nodes to
Nova.  So I mention this just to say, many of us have the same ideas
but we've not acted on them yet. I think the time will be right to
start working on some of these issues in Juno.

Talk to you Wednesday. We won't finish the discussion then, but we can
at least start having them.

On Sat, Mar 22, 2014 at 8:22 AM, Jay Lau  wrote:
> Thanks Shawn, I have updated the title with VMWare.
>
> Yes, I know that live migration "works". But the problem is when a cluster
> admin want to live migrate a VM instance, s/he will not know the target host
> where to migrate, as s/he cannot get target host from nova compute because
> currently VCDriver can only report cluster or resource pool as hypervisor
> host but not ESX server.
>
> IMHO, the VCDriver should support live migration between cluster, resource
> pool and ESX host, so we may need do at least the following enhancements:
> 1) Enable live migration with even one nova compute. My current thinking is
> that enhance target host as host:node when live migrate a VM instance and
> the live migration task need
> 2) Enable VCDriver report all ESX servers.
>
> We can discuss more during next week's IRC meeting.
>
> Thanks!
>
>
> 2014-03-22 17:13 GMT+08:00 Shawn Hartsock :
>
>> Hi Jay. We usually use [vmware] to tag discussion of VMware things. I
>> almost didn't see this message.
>>
>> In short, there is a plan and we're currently blocked because we have
>> to address several other pressing issues in the driver before we can
>> address this one. Part of this is due to the fact that we can't press
>> harder on blueprints or changes to the VCDriver right now.
>>
>> I actually reported this bug and we've discussed this at
>> https://wiki.openstack.org/wiki/Meetings/VMwareAPI the basic problem
>> is that live-migration actually "works" but you can't presently
>> formulate a command that activates the feature from the CLI under some
>> configurations. That's because of the introduction of clusters in the
>> VCDriver in Havana.
>>
>> To fix this, we have to come up with a way to target a host inside the
>> cluster (as I pointed out in the bug) or we have to have some way for
>> a live migration to occur between clusters and a way to validate that
>> this can happen first.
>>
>> As for the priority of this bug, it's been set to Medium which puts it
>> well behind many of the Critical or High tasks on our radar. As for
>> fixing the bug, no new outward behaviors or API are going to be
>> introduced and this was working at one point and now it's stopped. To
>> call this a new feature seems a bit strange.
>>
>> So, moving forward... perhaps we need to re-evaluate the priority
>> order on some of these things. I tabled Juno planning during the last
>> VMwareAPI subteam meeting but I plan on starting the discussion next
>> week. We have a priority order for blueprints that we set as a team
>> and these are publicly recorded in our meeting logs and on the wiki.
>> I'll try to do better advertising these things. You are of course
>> invited... and yeah... if you're interested in what we're fixing next
>> in the VCDriver that next IRC meeting is where we'll start the
>> discussion.
>>
>> On Sat, Mar 22, 2014 at 1:18 AM, Jay Lau  wrote:
>> > Hi,
>> >
>> > Currently we cannot do live migration with VCDriver in nova, live
>> > migration
>> > is really an important feature, so any plan to fix this?
>> >
>> > I noticed that there is already bug tracing this but seems no progress
>> > since
>> > last year's November: https://bugs.launchpad.net/nova/+bug/1192192
>> >
>> > Here just bring this problem up to see if there are any plan to fix
>> > this.
>> > After some investigation, I think that this might deserve to be a
>> > blueprint
>> > but not a bug.
>> >
>> > We may need to resolve issues for the following cases:
>> > 1) How to live migration with only one nova compute? (one nova compute
>> > can
>> > manage multiple clusters and there can be multi hosts in one cluster)
>> > 2) Support live migration between clusters
>> > 3) Support live migration between resource pools
>> > 4) Support live migration between hosts
>> > 5) Support live migration between cluster and host
>> > 6) Support live migration between cluster and resource pool
>> > 7) Support live migration between resource pool and host
>> > 8) Might be more cases.
>> >
>> > Please show your comments if any and correct me if anything is not
>> > correct.
>> >
>> > --
>> > Thanks,
>> >
>> > Jay
>> >
>> > ___
>> > OpenStack-dev mailing li

[openstack-dev] OpenStack/GSoC 2014 - Reviewing Applications

[Davanum Srinivas]

Dear Students,

We received the following proposals:

AMES-Cloud (Nagashree Bhat)
A pre-caching system for OpenStack (Anastasis Andronidis)
Proposal for Implementing an application-level FWaaS driver
((Zorp) Dániel Csubák)
Cassandra and Redis storage backend for Openstack marconi (Jeremy Henriques)
Openstack-OSLO :Add a New Backend to Oslo.Cache (sai krishna)
Implement a Fuzz testing framework that can be run on Tempest
(Manishanker Talusani)
How to detect network anomalies from telemetry data within Openstack (mst89)
Implement a re-usable shared library for VMware(oslo.vmware) (Masaru Nomura)
Cross-services Scheduler project (Artem Shepelev)
OpenStack/Marconi: Py3k support (Nataliia)
Improve benchmarking context mechanism in Rally (Kumar Rishabh)
Develop a benchmarking suite and new storage backends to OpenStack
Marconi (Prashanth Raghu)
Adding Redis as a Storage Backend to OpenStack Marconi (Chenchong Qin)
Auto Benchmarking System for OpenStack (RobberPhex)
Developing Benchmarks for Virtual Machines of OpenStack with Rally
(Tzanetos Balitsaris)
Add a new storage backend to the OpenStack Message Queuing Service
(Victoria Martínez de la Cruz)

Dear Mentors 
(ddutta/flwang/julim/hughsaunders/greghaynes/annegentle/sriramhere/arnaudleg/coroner/boris_42/blint/ybudupi/cppcabrera),

Please log into the Google GSoC web-site and review **all** proposals
within a week (our own deadline - say 29th).

1) Please click "Wish to mentor" on the left hand side, if you are
willing to mentor this project.
   Yes, we should have more than one mentor. This is a way to mention
that you are willing to mentor,
   actual assignment will happen later i believe. So please switch
this on for all projects you are
   interested in.

2) If you have a question about a proposal and need a response from
the candidate, please leave
   a comment for them. If you want to ask a observation/question to
other mentors or me, leave a
   comment marked "Private"

3) To assign a score. Please click on the number of start in "My
score:" at the bottom
5 = amazing applicant, could be a module maintainer on completing
the program
4 = strong applicant, will do a good job
3 = good applicant, but is somewhat inexperienced
2 = is unlikely to do a good job
1 = not a good candidate

Please feel free to leave detailed notes on candidates you know well
for other mentors (Please mark comments as "Private")

If you are not able to help with this exercise, please let me know ASAP.

If anyone else would like to step up and be a Mentor for any of these
projects/students, please register in the Google GSoC site and let me
know your "username".

Thanks,
dims



-- 
Davanum Srinivas :: http://davanum.wordpress.com

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] An analysis of code review in Nova

[Chris Behrens]

I'd like to get spawn broken up sooner rather than later, personally. It has 
additional benefits of being able to do better orchestration of builds from 
conductor, etc.

On Mar 14, 2014, at 3:58 PM, Dan Smith  wrote:

>> Just to answer this point, despite the review latency, please don't be
>> tempted to think one big change will get in quicker than a series of
>> little, easy to review, changes. All changes are not equal. A large
>> change often scares me away to easier to review patches.
>> 
>> Seems like, for Juno-1, it would be worth cancelling all non-urgent
>> bug fixes, and doing the refactoring we need.
>> 
>> I think the aim here should be better (and easier to understand) unit
>> test coverage. Thats a great way to drive good code structure.
> 
> Review latency will be directly affected by how good the refactoring
> changes are staged. If they are small, on-topic and easy to validate,
> they will go quickly. They should be linearized unless there are some
> places where multiple sequences of changes make sense (i.e. refactoring
> a single file that results in no changes required to others).
> 
> As John says, if it's just a big "change everything" patch, or a ton of
> smaller ones that don't fit a plan or process, then it will be slow and
> painful (for everyone).
> 
>> +1 sounds like a good first step is to move to oslo.vmware
> 
> I'm not sure whether I think that refactoring spawn would be better done
> first or second. My gut tells me that doing spawn first would mean that
> we could more easily validate the oslo refactors because (a) spawn is
> impossible to follow right now and (b) refactoring it to smaller methods
> should be fairly easy. The tests for spawn are equally hard to follow
> and refactoring it first would yield a bunch of more unit-y tests that
> would help us follow the oslo refactoring.
> 
> However, it sounds like the osloificastion has maybe already started and
> that refactoring spawn will have to take a backseat to that.
> 
> --Dan
> 
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] An analysis of code review in Nova

[Matt Riedemann]

On 3/22/2014 5:19 AM, Shawn Hartsock wrote:

On Fri, Mar 14, 2014 at 6:58 PM, Dan Smith  wrote:

Review latency will be directly affected by how good the refactoring
changes are staged. If they are small, on-topic and easy to validate,
they will go quickly. They should be linearized unless there are some
places where multiple sequences of changes make sense (i.e. refactoring
a single file that results in no changes required to others).



I'm going to bring this to the next
https://wiki.openstack.org/wiki/Meetings/VMwareAPI we can start
working on how we'll set the order for this kind of work. Currently we
have a whole blueprint for refactoring a single method. That seems
silly. I'll want to come up with a plan around how to restructure the
driver so we can avoid some of the messes we've seen in the past.


I think the point of starting with refactoring the nested method mess in 
the spawn method was it (1) seemed relatively trivial (think fast review 
turnaround) and (2) should be a good "bang for the buck" kind of change, 
as a lot of the original complaint was related to how hard it is to 
verify changes in the giant spawn method are tested - which you also 
point out below.




I want to avoid one big refactor effort that drags on, but I also want
to address bigger problems we have inside the driver. For example,


I also want to avoid a big refactor effort dragging on, and I like the 
thinking on design changes, but are they doing going to be happening at 
the same time?  Or is the complete re-design going to supersede the 
refactoring?  My only concern is biting off more than can be chewed in 
juno-1.


Plus there is the refactor to use oslo.vmware, where does that fit into 
this?



vm_util.py seems to have become burdened with work that it shouldn't
have. It also performs a great number of unnecessary round trips using
a vm_ref to pull individual vm details over one at a time. Introducing
a VirtualMachine object that held all these references would simplify
some operations (I'm not the first person to suggest this and it
wasn't novel to me either when it was presented.)

It would seem Juno-1 would be the time to make these changes and we
need to serialize this work to keep reviewers from losing their
marbles trying to track it all. I would like to work out a plan for
this in conjunction with interested core-reviewers who would be
willing to more or less sponsor this work. Because of the problems
Matt points out, I don't want to tackle this in a haphazard or
piece-meal way since it could completely disrupt any new blueprint
work people may have targeted for Juno.


Yeah, definitely need a plan here.  I'd like to see things prioritized 
based on what can be fixed in a relatively isolated way which gives a 
good return on coding/reviewing investment, e.g. pulling those nested 
methods out of spawn so they can be unit tested individually with mock 
rather than a large, seemingly rigid and scaffolded test framework.




Having said that, on this driver, new blueprints in the last several
cycles have introduced serious feature regressions. Several minor bug
fixes have altered and introduced key architectural components that
have broken multiple critical features. In my professional opinion
this has a root cause based on the drivers tightly coupled and
non-cohesive internal design.

The driver design is tightly coupled in that a change in one area
forces multiple updates and changes *throughout* the rest of the
driver. This is true in testing as well. The testing design often
requires you to trace the entire codebase if you add a single optional
parameter to a single method. This does not have to be true.


Yup, this is my major complaint and ties into what I'm saying above, I 
find it really difficult to determine most of the time where a change is 
tested.  Because of the nature of the driver code and my lack of 
actually writing features in it, as a reviewer I don't know if a change 
in X is going to break Y, so I rely on solid test coverage and the 
testing needs to be more natural to follow than it currently is.




The driver design is non-cohesive in that important details and
related information is spread throughout the driver. You must be aware
at all times (for example) whether or not your current operation
requires you to check if your vm_ref is outdated (we just worked on
several last minute critical bugs for RC1 where myself and others
pulled all nighters to fix the issue in a bad case of Heroic
Programming).

I would like to stop the http://c2.com/cgi/wiki?CodeVomit please. May we?



I know this isn't going to be easy so I'm really glad you're planning on 
tackling it in Juno.  I'll tentatively sign up to help sponsor this but 
I'm not going to be able to commit all of my review bandwidth to a ton 
of changes for refactor and re-design.  Hopefully targets will become 
more clear as the team gets the plans in place.


--

Thanks,

Matt Riedemann


_

Re: [openstack-dev] Preparing for 2013.2.3 -- branches freeze March 27th

[Thomas Goirand]

On 03/22/2014 03:32 AM, Adam Gandelman wrote:
> Hi All-
> 
> We'll be freezing the stable/havana branches for integrated projects this
> Thursday March 27th in preparation for the 2013.2.3 stable release on
> Thursday April 3rd.  You can view the current queue of proposed patches
> on gerrit [1].  I'd like to request all interested parties review current
> bugs affecting Havana and help ensure any relevant fixes be proposed
> soon and merged by Thursday, or notify the stable-maint team of
> anything critical that may land late.
> 
> Thanks,
> Adam
> 
> [1] https://review.openstack.org/#/q/status:open+branch:stable/havana,n,z

Hi,

There's indeed 2 bugs which I would very much like to be fixed.

The first one would be the backport of oauth2 to oauthlib in Keystone,
which is IMO critical for security reasons:
https://review.openstack.org/#/c/70750/

It is my view that core reviewers should have paid more attention to
this backport, and that it shouldn't have taken that long... :/

Note that I currently embed a Debian specific patch in the package, but
I really would love to get rid of it, and use something that has gone
through the review process.

The 2nd one would be the removal of minified Javascript in Horizon:
https://review.openstack.org/#/c/67096/

I'm not sure that 2nd one has been backported yet, though it is
important that it lands into the stable release, because otherwise that
makes Horizon non-free. Therefore, I just created a patch for the
stable/havana branch over here:

https://review.openstack.org/82298

I'd really like to have it included in the next stable release of
Horizon. I don't think this one is hard to deal with, and I hope it will
go through even with the lack of time.

Cheers,

Thomas Goirand (zigo)

P.S: I'm not registered to the openstack-stable-maint list.


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [NOVA][VMWare][live-migration] VCDriver live migration problem

[Jay Lau]

Thanks Shawn, I have updated the title with VMWare.

Yes, I know that live migration "works". But the problem is when a cluster
admin want to live migrate a VM instance, s/he will not know the target
host where to migrate, as s/he cannot get target host from nova compute
because currently VCDriver can only report cluster or resource pool as
hypervisor host but not ESX server.

IMHO, the VCDriver should support live migration between cluster, resource
pool and ESX host, so we may need do at least the following enhancements:
1) Enable live migration with even one nova compute. My current thinking is
that enhance target host as host:node when live migrate a VM instance and
the live migration task need
2) Enable VCDriver report all ESX servers.

We can discuss more during next week's IRC meeting.

Thanks!


2014-03-22 17:13 GMT+08:00 Shawn Hartsock :

> Hi Jay. We usually use [vmware] to tag discussion of VMware things. I
> almost didn't see this message.
>
> In short, there is a plan and we're currently blocked because we have
> to address several other pressing issues in the driver before we can
> address this one. Part of this is due to the fact that we can't press
> harder on blueprints or changes to the VCDriver right now.
>
> I actually reported this bug and we've discussed this at
> https://wiki.openstack.org/wiki/Meetings/VMwareAPI the basic problem
> is that live-migration actually "works" but you can't presently
> formulate a command that activates the feature from the CLI under some
> configurations. That's because of the introduction of clusters in the
> VCDriver in Havana.
>
> To fix this, we have to come up with a way to target a host inside the
> cluster (as I pointed out in the bug) or we have to have some way for
> a live migration to occur between clusters and a way to validate that
> this can happen first.
>
> As for the priority of this bug, it's been set to Medium which puts it
> well behind many of the Critical or High tasks on our radar. As for
> fixing the bug, no new outward behaviors or API are going to be
> introduced and this was working at one point and now it's stopped. To
> call this a new feature seems a bit strange.
>
> So, moving forward... perhaps we need to re-evaluate the priority
> order on some of these things. I tabled Juno planning during the last
> VMwareAPI subteam meeting but I plan on starting the discussion next
> week. We have a priority order for blueprints that we set as a team
> and these are publicly recorded in our meeting logs and on the wiki.
> I'll try to do better advertising these things. You are of course
> invited... and yeah... if you're interested in what we're fixing next
> in the VCDriver that next IRC meeting is where we'll start the
> discussion.
>
> On Sat, Mar 22, 2014 at 1:18 AM, Jay Lau  wrote:
> > Hi,
> >
> > Currently we cannot do live migration with VCDriver in nova, live
> migration
> > is really an important feature, so any plan to fix this?
> >
> > I noticed that there is already bug tracing this but seems no progress
> since
> > last year's November: https://bugs.launchpad.net/nova/+bug/1192192
> >
> > Here just bring this problem up to see if there are any plan to fix this.
> > After some investigation, I think that this might deserve to be a
> blueprint
> > but not a bug.
> >
> > We may need to resolve issues for the following cases:
> > 1) How to live migration with only one nova compute? (one nova compute
> can
> > manage multiple clusters and there can be multi hosts in one cluster)
> > 2) Support live migration between clusters
> > 3) Support live migration between resource pools
> > 4) Support live migration between hosts
> > 5) Support live migration between cluster and host
> > 6) Support live migration between cluster and resource pool
> > 7) Support live migration between resource pool and host
> > 8) Might be more cases.
> >
> > Please show your comments if any and correct me if anything is not
> correct.
> >
> > --
> > Thanks,
> >
> > Jay
> >
> > ___
> > OpenStack-dev mailing list
> > OpenStack-dev@lists.openstack.org
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >
>
>
>
> --
> # Shawn.Hartsock - twitter: @hartsock - plus.google.com/+ShawnHartsock
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



-- 
Thanks,

Jay
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Ceilometer][QA][Tempest][Infra] Ceilometer tempest testing in gate

[Sean Dague]

On 03/21/2014 05:11 PM, Joe Gordon wrote:
> 
> 
> 
> On Fri, Mar 21, 2014 at 4:04 AM, Sean Dague  > wrote:
> 
> On 03/20/2014 06:18 PM, Joe Gordon wrote:
> >
> >
> >
> > On Thu, Mar 20, 2014 at 3:03 PM, Alexei Kornienko
> > mailto:alexei.kornie...@gmail.com>
>  >> wrote:
> >
> > Hello,
> >
> > We've done some profiling and results are quite interesting:
> > during 1,5 hour ceilometer inserted 59755 events (59755 calls to
> > record_metering_data)
> > this calls resulted in total 2591573 SQL queries.
> >
> > And the most interesting part is that 291569 queries were ROLLBACK
> > queries.
> > We do around 5 rollbacks to record a single event!
> >
> > I guess it means that MySQL backend is currently totally
> unusable in
> > production environment.
> >
> >
> > It should be noticed that SQLAlchemy is horrible for performance, in
> > nova we usually see sqlalchemy overheads of well over 10x (time
> > nova.db.api call vs the time MySQL measures when slow log is recording
> > everything).
> 
> That's not really a fair assessment. Python object inflation takes time.
> I do get that there is SQLA overhead here, but even if you trimmed it
> out you would not get the the mysql query time.
> 
> 
> To give an example from nova:
> 
> doing a nova list with no servers:
> 
> stack@devstack:~/devstack$ nova --timing list 
> 
> | GET
> http://10.0.0.16:8774/v2/a82ededa9a934b93a7184d06f302d745/servers/detail
> | 0.0817470550537 |
> 
> So nova command takes 0.0817470550537 seconds.
> 
> Inside the nova logs (when putting a timer around all nova.db.api calls
> [1] ), nova.db.api.instance_get_all_by_filters takes 0.06 seconds:
> 
> 2014-03-21 20:58:46.760 DEBUG nova.db.api
> [req-91879f86-7665-4943-8953-41c92c42c030 demo demo]
> 'instance_get_all_by_filters' 0.06 seconds timed
> /mnt/stack/nova/nova/db/api.py:1940
> 
> But the sql slow long reports the same query takes only 0.001006 seconds
> with a lock_time of 0.000269 for a total of  0.00127 seconds.
> 
> # Query_time: 0.001006  Lock_time: 0.000269 Rows_sent: 0
>  Rows_examined: 0
> 
> 
> So in this case only 2% of the time
> that  nova.db.api.instance_get_all_by_filters takes is spent inside of
> mysql. Or to put it differently  nova.db.api.instance_get_all_by_filters
> is 47 times slower then the raw DB call underneath.
> 
> Yes I agree that that turning raw sql data into python objects should
> take time, but I just don't think it should take 98% of the time.
> 
> [1] 
> https://github.com/jogo/nova/commit/7743ee366bbf8746f1c0f634f29ebf73bff16ea1
> 
> That being said, having Ceilometer's write path be highly tuned and not
> use SQLA (and written for every back end natively) is probably
> appropriate.
> 
> 
> While I like this idea, they loose free postgresql support by dropping
> SQLA. But that is a solvable problem.

Joe, you're just trolling now, right? :)

I mean you picked the most pathological case possible. An empty table
with no data ever returned. So no actual work was done anywhere, and
this is just measure side effects which in no way are commensurate with
actual read / write profiles of a real system.

I 100% agree that SQLA provides overhead. However, removing SQLA is the
last in a series of optimizations that you do on a system. Because
taking it out doesn't solve having bad data usage (getting more data
than you need), bad schema, or bad queries. I would expect substantial
gains could be made tackling those first.

If after that, fast path drivers sounded like a good idea, go for it.

But realize that a fast path driver is more work to write and maintain.
And has the energy hasn't gone into optimizing things yet, I think a
proposal to put even more work on the team to write a new set of harder
to maintain drivers, is just a non starter.

All I'm asking is that we need profiling. Ceilometer is suppose to be
high performance / low overhead metrics collection. We have some
indication that it's not meeting that desire based on our gate runs.
Which means we can reproduce it. Which is great, because reproducing
means things are fixable, and we can easily know if we did fix it.

Optimizing is hard, but I think it's the right time to do it. Not just
with elasticity, but with old fashion analysis.

-Sean

-- 
Sean Dague
Samsung Research America
s...@dague.net / sean.da...@samsung.com
http://dague.net



signature.asc
Description: OpenPGP digital signature
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] An analysis of code review in Nova

[Shawn Hartsock]

On Fri, Mar 14, 2014 at 6:58 PM, Dan Smith  wrote:
> Review latency will be directly affected by how good the refactoring
> changes are staged. If they are small, on-topic and easy to validate,
> they will go quickly. They should be linearized unless there are some
> places where multiple sequences of changes make sense (i.e. refactoring
> a single file that results in no changes required to others).


I'm going to bring this to the next
https://wiki.openstack.org/wiki/Meetings/VMwareAPI we can start
working on how we'll set the order for this kind of work. Currently we
have a whole blueprint for refactoring a single method. That seems
silly. I'll want to come up with a plan around how to restructure the
driver so we can avoid some of the messes we've seen in the past.

I want to avoid one big refactor effort that drags on, but I also want
to address bigger problems we have inside the driver. For example,
vm_util.py seems to have become burdened with work that it shouldn't
have. It also performs a great number of unnecessary round trips using
a vm_ref to pull individual vm details over one at a time. Introducing
a VirtualMachine object that held all these references would simplify
some operations (I'm not the first person to suggest this and it
wasn't novel to me either when it was presented.)

It would seem Juno-1 would be the time to make these changes and we
need to serialize this work to keep reviewers from losing their
marbles trying to track it all. I would like to work out a plan for
this in conjunction with interested core-reviewers who would be
willing to more or less sponsor this work. Because of the problems
Matt points out, I don't want to tackle this in a haphazard or
piece-meal way since it could completely disrupt any new blueprint
work people may have targeted for Juno.

Having said that, on this driver, new blueprints in the last several
cycles have introduced serious feature regressions. Several minor bug
fixes have altered and introduced key architectural components that
have broken multiple critical features. In my professional opinion
this has a root cause based on the drivers tightly coupled and
non-cohesive internal design.

The driver design is tightly coupled in that a change in one area
forces multiple updates and changes *throughout* the rest of the
driver. This is true in testing as well. The testing design often
requires you to trace the entire codebase if you add a single optional
parameter to a single method. This does not have to be true.

The driver design is non-cohesive in that important details and
related information is spread throughout the driver. You must be aware
at all times (for example) whether or not your current operation
requires you to check if your vm_ref is outdated (we just worked on
several last minute critical bugs for RC1 where myself and others
pulled all nighters to fix the issue in a bad case of Heroic
Programming).

I would like to stop the http://c2.com/cgi/wiki?CodeVomit please. May we?

-- 
# Shawn.Hartsock - twitter: @hartsock - plus.google.com/+ShawnHartsock

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova] Updates to Juno blueprint review process

[Sean Dague]

On 03/21/2014 02:55 PM, Stefano Maffulli wrote:
> On 03/20/2014 03:50 PM, Jay Lau wrote:
>> It is better that we can have some diagram workflow just like
>> Gerrit_Workflow  to
>> show the new process.
> 
> Indeed, I think it would help.
> 
> While I'm here, and for the records, I think that creating a new
> workflow 'temporarily' only until we have Storyboard usable, is a *huge*
> mistake. It seems to me that you're ignoring or at least underestimating
> the amount of *people* that will need to be retrained, the amount of
> documentation that need to be fixed/adjusted. And the confusion that
> this will create on the 'long tail' developers.
> 
> A change like this, done with a couple of announcements on a mailing
> list and a few mentions on IRC is not enough to steer the ~400
> developers who may be affected by this change. And then we'll have to
> manage the change again when we switch to Storyboard. If I were you, I'd
> focus on getting storyboard ready to use asap, instead.
> 
> There, I said it, and I'm now going back to my cave.
> 
> .stef

Honestly, I largely disagree. This is applying some process where there
clearly wasn't any before. For people that aren't creating new
blueprints, nothing changes, except the blueprints they see in launchpad
are 100x more useful to understand what features the community is
interested in.

For people submitting blueprints, there is now actual guidance, instead
of randomly throw things into launchpad and not realize why your patch
is -2ed at milestone 3.

Storyboard remains vaporware. I will be enormously happy when it is not.
But you can't have people sit around waiting on vaporware, ever. Because
software that doesn't exist and people don't have to use always
magically seems to solve problems better than software that does exist.

I don't think anything about this process is incompatible with
Storyboard. The workflow will be very similar given any tracker. And I
expect the workflow might evolve over time if Storyboard can help more.
I honestly think spec review was completely outside of it's use case. So
I look forward to evolving the process with Storyboard once it's
actually a tool we use and not a Unicorn we talk about.

-Sean

-- 
Sean Dague
Samsung Research America
s...@dague.net / sean.da...@samsung.com
http://dague.net



signature.asc
Description: OpenPGP digital signature
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [OpenStack-dev][NOVA][VCDriver][live-migration] VCDriver live migration problem

[Shawn Hartsock]

Hi Jay. We usually use [vmware] to tag discussion of VMware things. I
almost didn't see this message.

In short, there is a plan and we're currently blocked because we have
to address several other pressing issues in the driver before we can
address this one. Part of this is due to the fact that we can't press
harder on blueprints or changes to the VCDriver right now.

I actually reported this bug and we've discussed this at
https://wiki.openstack.org/wiki/Meetings/VMwareAPI the basic problem
is that live-migration actually "works" but you can't presently
formulate a command that activates the feature from the CLI under some
configurations. That's because of the introduction of clusters in the
VCDriver in Havana.

To fix this, we have to come up with a way to target a host inside the
cluster (as I pointed out in the bug) or we have to have some way for
a live migration to occur between clusters and a way to validate that
this can happen first.

As for the priority of this bug, it's been set to Medium which puts it
well behind many of the Critical or High tasks on our radar. As for
fixing the bug, no new outward behaviors or API are going to be
introduced and this was working at one point and now it's stopped. To
call this a new feature seems a bit strange.

So, moving forward... perhaps we need to re-evaluate the priority
order on some of these things. I tabled Juno planning during the last
VMwareAPI subteam meeting but I plan on starting the discussion next
week. We have a priority order for blueprints that we set as a team
and these are publicly recorded in our meeting logs and on the wiki.
I'll try to do better advertising these things. You are of course
invited... and yeah... if you're interested in what we're fixing next
in the VCDriver that next IRC meeting is where we'll start the
discussion.

On Sat, Mar 22, 2014 at 1:18 AM, Jay Lau  wrote:
> Hi,
>
> Currently we cannot do live migration with VCDriver in nova, live migration
> is really an important feature, so any plan to fix this?
>
> I noticed that there is already bug tracing this but seems no progress since
> last year's November: https://bugs.launchpad.net/nova/+bug/1192192
>
> Here just bring this problem up to see if there are any plan to fix this.
> After some investigation, I think that this might deserve to be a blueprint
> but not a bug.
>
> We may need to resolve issues for the following cases:
> 1) How to live migration with only one nova compute? (one nova compute can
> manage multiple clusters and there can be multi hosts in one cluster)
> 2) Support live migration between clusters
> 3) Support live migration between resource pools
> 4) Support live migration between hosts
> 5) Support live migration between cluster and host
> 6) Support live migration between cluster and resource pool
> 7) Support live migration between resource pool and host
> 8) Might be more cases.
>
> Please show your comments if any and correct me if anything is not correct.
>
> --
> Thanks,
>
> Jay
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



-- 
# Shawn.Hartsock - twitter: @hartsock - plus.google.com/+ShawnHartsock

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev