Re: [openstack-dev] [nova] Pulling nova/virt/hardware.py into nova/objects/
I spent a little time trying to work out a good way to include this kind of data in the ComputeNode object. You will have seen that I added the supported_instances reported to the RT by the virt drivers as a list of HVSpec – where HVSpec is a new nova object I created for the purpose. The rationale behind two parallel data model hiercharies is that the format the virt drivers report data in, is not likely to be exactly the same as the format that the resoure tracker / scheduler wishes to use in the database. Yeah, and in cases where we know where that line is, it makes sense to use the lighter-weight modeling for sure. Something that happens in some cases in RT is the data reported by the virt driver is modified by the RT – this is certainly the case for stats (which should probably not have been used by the virt driver – ironic in this case). In other cases memory, cpu, disk etc are split out into other fields at the moment (which is where Jay Pipes is working on a model for resources in general). Where the data in an object field is not simple (e.g. lists of something) it is not really easy to work with in an object. You can’t just add to a list that is already stored in an object field, you need to make sure the object knows it has been updated. So the easiest way to work is to add entire fields to the object and not change them in situ. So that seems to me like dealing with non-nova object data types makes sense as something separate to, say, the ComputeNode object. An alternative would be to work on the way nova object handle nested data structures (i.e. recognizing updates in nested structures, api allowing for list manipulation etc.) It depends whether you think the objects are just doing versioning to support upgrade/mixed versions or are a general purpose object model. Note that this happens with NUMA data structures and PCI as well at the moment. FWIW, my patch series is logically split up into two parts. THe first 10 or so patches are just thought of as general cleanup and useful to Nova regardless of what we decide todo. The second 10 or so patches are where the objects start appearing getting used the controversial bits needing mor detailed discussion. Right, so after some discussion I think we should go ahead and merge the bottom of this set (all of them are now acked I think) and continue the discussion on the top half where the modeling is introduced. Paul ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Pulling nova/virt/hardware.py into nova/objects/
On 10/21/2014 05:44 AM, Nikola Đipanov wrote: On 10/20/2014 07:38 PM, Jay Pipes wrote: Hi Dan, Dan, Nikola, all Nova devs, OK, so in reviewing Dan B's patch series that refactors the virt driver's get_available_resource() method [1], I am stuck between two concerns. I like (love even) much of the refactoring work involved in Dan's patches. They replace a whole bunch of our nested dicts that are used in the resource tracker with real objects -- and this is something I've been harping on for months that really hinders developer's understanding of Nova's internals. However, all of the object classes that Dan B has introduced have been unversioned objects -- i.e. they have not derived from nova.objects.base.NovaObject. This means that these objects cannot be sent over the wire via an RPC API call. In practical terms, this issue has not yet reared its head, because the resource tracker still sends a dictified JSON representation of the object's fields directly over the wire, in the same format as Icehouse, therefore there have been no breakages in RPC API compatibility. The problems with having all these objects not modelled by deriving from nova.objects.base.NovaObject are two-fold: * The object's fields/schema cannot be changed -- or rather, cannot be changed without introducing upgrade problems. * The objects introduce a different way of serializing the object contents than is used in nova/objects -- it's not that much different, but it's different, and only has not caused a problem because the serialization routines are not yet being used to transfer data over the wire So, what to do? Clearly, I think the nova/virt/hardware.py objects are badly needed. However, one of (the top?) priorities of the Nova project is upgradeability, and by not deriving from nova.objects.base.NovaObject, these nova.virt.hardware objects are putting that mission in jeopardy, IMO. My proposal is that before we go and approve any BPs or patches that add to nova/virt/hardware.py, we first put together a patch series that moves the object models in nova/virt/hardware.py to being full-fledged objects in nova/objects/* I think that we should have both in some cases, and although it makes sense to have them only as objects in some cases - having them as separate classes for some and not others may be confusing. So when does it make sense to have them as separate classes? Well basically whenever there is a need for driver-agnostic logic that will be used outside of the driver (scheduler/claims/API/). Can this stuff go in objects? Technically yes, but objects are really not a good place for such logic as they may already be trying to solve too much (data versioning and downgrading when there is a multi version cloud running, database access for compute, and there are at least 2 more features considered to be part of objects - cells integration and schema data migrations). Take CPU pinning as an example [1] - none of that logic would benefit from living in the NovaObject child class itself, and will make it quite bloated. Having it in the separate module objects can call into is definitely beneficial, while we definitely should stay with objects for versioning/backporting support. So I say in a number of cases we need both. Both is exactly what I did for NUMA, with the exception of the compute node side (we are hopping to start the json blob cleanup in K so I did not concern myself with it for the sake of getting things done, but we will need it). This is what I am doing now with CPU pinning. The question I did not touch upon is what kind of interface does that leave poor Nova developers with. Having everything as objects would allow us to write things like (in the CPU pinning case): instance.cpu_pinning = compute.cpu_pinning.get_pinning_for_instance( instance) Pretty slick, no? While keeping it completely separate would make us do things like cpu_pinning = compute.cpu_pinning.topology_from_obj() if cpu_pinning: instance_pinning = cpu_pinning.get_pinning_for_instance( instance.cpu_pinning.topology_from_obj()) instance.cpu_pinning = objects.InstanceCPUPinning.obj_from_topology( instance_pinning) Way less slick, but can be easily fixed with a level of indirection. Note that the above holds only when we are objectified everywhere - until then - we pretty much *have* to have both. So to sum up - what I think we should do is: 1) Don't bloat the object code with low level stuff By low-level stuff, if you mean methods that *do* something with the data in an object, I agree. However, the nova.objects framework should be used to represent the *data fields* of any object model that can potentially be transferred over the RPC API wires or stored in backend storage (DB or otherwise). The reason is that the data fields of these objects will all surely need to undergo some changes -- field renames/adds/deletes/re-types, etc -- and that is where the nova.objects framework
Re: [openstack-dev] [nova] Pulling nova/virt/hardware.py into nova/objects/
On 10/21/2014 04:51 PM, Dan Smith wrote: The rationale behind two parallel data model hiercharies is that the format the virt drivers report data in, is not likely to be exactly the same as the format that the resoure tracker / scheduler wishes to use in the database. Yeah, and in cases where we know where that line is, it makes sense to use the lighter-weight modeling for sure. FWIW, my patch series is logically split up into two parts. THe first 10 or so patches are just thought of as general cleanup and useful to Nova regardless of what we decide todo. The second 10 or so patches are where the objects start appearing getting used the controversial bits needing mor detailed discussion. Right, so after some discussion I think we should go ahead and merge the bottom of this set (all of them are now acked I think) and continue the discussion on the top half where the modeling is introduced. Agreed. -jay ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Pulling nova/virt/hardware.py into nova/objects/
Le 20 oct. 2014 20:13, Dan Smith d...@danplanet.com a écrit : OK, so in reviewing Dan B's patch series that refactors the virt driver's get_available_resource() method [1], I am stuck between two concerns. I like (love even) much of the refactoring work involved in Dan's patches. They replace a whole bunch of our nested dicts that are used in the resource tracker with real objects -- and this is something I've been harping on for months that really hinders developer's understanding of Nova's internals. dict['line1'] = 'Agreed, this is extremely important stuff.' dict['line2'] = 'The current dict mess that we have there is ' dict['line3'] = 'really obscure and confusing.' reply = jsonutils.dumps(dict) However, all of the object classes that Dan B has introduced have been unversioned objects -- i.e. they have not derived from nova.objects.base.NovaObject. This means that these objects cannot be sent over the wire via an RPC API call. In practical terms, this issue has not yet reared its head, because the resource tracker still sends a dictified JSON representation of the object's fields directly over the wire, in the same format as Icehouse, therefore there have been no breakages in RPC API compatibility. Right, so the blueprint for this work states that it's not to be sent over the RPC wire or stored in the database. However, it already is in some cases (at least the ComputeNode object has the unversioned JSONified version of some of these hardware models in it). If the modeling is purely for internal-to-compute-node purposes, then it's all good. However, it surely seems like with the pending scheduler isolation work, we're in a spot where we are building two parallel model hierarchies, and I'm not really sure why. As there are multiple interfaces using non versioned dicts and as we are looking at reducing technical debt by Kilo, there are different blueprints which can be worked in parallel. Here, the virt-to-RT interface has to be objectified, hence Dan's work. On the other end of the RT, the RT-to-scheduler interface has to be objectified, hence Jay and mine's work. I hope we will provide a clear big picture and a roadmap for the Summit so we could give you more insights. My proposal is that before we go and approve any BPs or patches that add to nova/virt/hardware.py, we first put together a patch series that moves the object models in nova/virt/hardware.py to being full-fledged objects in nova/objects/* I'm not sure that just converting them all to NovaObjects is really necessary here. If it's all stuff that is going to go over the wire eventually as part of the resource tracker's expansion, then probably so. If there are bits of the model that only serve to let the resource tracker do its calculations, then perhaps it doesn't make sense to require those be NovaObjects. Totally agreed. Here there is no need to version the interface as the virt/Rt interface is not RPC based and purely internal to nova-compute. We just need to objectify the interface to explicitetly provide what kind of resources are sent but that's it. Regardless, it sounds like we need some discussion on how best to proceed here. Since it's entirely wrapped up in the scheduler work, we should definitely try to make sure that what we're doing here fits with those plans. Last I heard, we weren't sure where we were going to draw the line between nova bits and scheduler bits, so erring on the side of more versioned interfaces seems safest to me. Again, we hope to give you all a better understanding at the Summit. I can't develop further as I'm in vacation until next Wed so I totally assume my last paragraph as an horrible teaser unless someone else from the gang adds more details. -Sylvain --Dan ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Pulling nova/virt/hardware.py into nova/objects/
On 10/20/2014 07:38 PM, Jay Pipes wrote: Hi Dan, Dan, Nikola, all Nova devs, OK, so in reviewing Dan B's patch series that refactors the virt driver's get_available_resource() method [1], I am stuck between two concerns. I like (love even) much of the refactoring work involved in Dan's patches. They replace a whole bunch of our nested dicts that are used in the resource tracker with real objects -- and this is something I've been harping on for months that really hinders developer's understanding of Nova's internals. However, all of the object classes that Dan B has introduced have been unversioned objects -- i.e. they have not derived from nova.objects.base.NovaObject. This means that these objects cannot be sent over the wire via an RPC API call. In practical terms, this issue has not yet reared its head, because the resource tracker still sends a dictified JSON representation of the object's fields directly over the wire, in the same format as Icehouse, therefore there have been no breakages in RPC API compatibility. The problems with having all these objects not modelled by deriving from nova.objects.base.NovaObject are two-fold: * The object's fields/schema cannot be changed -- or rather, cannot be changed without introducing upgrade problems. * The objects introduce a different way of serializing the object contents than is used in nova/objects -- it's not that much different, but it's different, and only has not caused a problem because the serialization routines are not yet being used to transfer data over the wire So, what to do? Clearly, I think the nova/virt/hardware.py objects are badly needed. However, one of (the top?) priorities of the Nova project is upgradeability, and by not deriving from nova.objects.base.NovaObject, these nova.virt.hardware objects are putting that mission in jeopardy, IMO. My proposal is that before we go and approve any BPs or patches that add to nova/virt/hardware.py, we first put together a patch series that moves the object models in nova/virt/hardware.py to being full-fledged objects in nova/objects/* I think that we should have both in some cases, and although it makes sense to have them only as objects in some cases - having them as separate classes for some and not others may be confusing. So when does it make sense to have them as separate classes? Well basically whenever there is a need for driver-agnostic logic that will be used outside of the driver (scheduler/claims/API/). Can this stuff go in objects? Technically yes, but objects are really not a good place for such logic as they may already be trying to solve too much (data versioning and downgrading when there is a multi version cloud running, database access for compute, and there are at least 2 more features considered to be part of objects - cells integration and schema data migrations). Take CPU pinning as an example [1] - none of that logic would benefit from living in the NovaObject child class itself, and will make it quite bloated. Having it in the separate module objects can call into is definitely beneficial, while we definitely should stay with objects for versioning/backporting support. So I say in a number of cases we need both. Both is exactly what I did for NUMA, with the exception of the compute node side (we are hopping to start the json blob cleanup in K so I did not concern myself with it for the sake of getting things done, but we will need it). This is what I am doing now with CPU pinning. The question I did not touch upon is what kind of interface does that leave poor Nova developers with. Having everything as objects would allow us to write things like (in the CPU pinning case): instance.cpu_pinning = compute.cpu_pinning.get_pinning_for_instance( instance) Pretty slick, no? While keeping it completely separate would make us do things like cpu_pinning = compute.cpu_pinning.topology_from_obj() if cpu_pinning: instance_pinning = cpu_pinning.get_pinning_for_instance( instance.cpu_pinning.topology_from_obj()) instance.cpu_pinning = objects.InstanceCPUPinning.obj_from_topology( instance_pinning) Way less slick, but can be easily fixed with a level of indirection. Note that the above holds only when we are objectified everywhere - until then - we pretty much *have* to have both. So to sum up - what I think we should do is: 1) Don't bloat the object code with low level stuff 2) Do have objects for versioning everything 3) Make nice APIs that developers can enjoy (after we've converted all the code to use objects). N. [1] https://review.openstack.org/#/c/128738/4/nova/virt/hardware.py Thoughts? -jay [1] https://review.openstack.org/#/q/status:open+project:openstack/nova+branch:master+topic:bp/virt-driver-get-available-resources-object,n,z ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org
Re: [openstack-dev] [nova] Pulling nova/virt/hardware.py into nova/objects/
As there are multiple interfaces using non versioned dicts and as we are looking at reducing technical debt by Kilo, there are different blueprints which can be worked in parallel. I don't think I disagree with anything above, but I'm not sure what you're getting at. I think the parallelism we should avoid is building models that mirror things we need to send to a remote service and just require conversion. If we're modeling things between the virt driver and the RT (or interface to the RT) that get aggregated or transformed in some way before they leave the service, then that's fine. Here, the virt-to-RT interface has to be objectified, hence Dan's work. On the other end of the RT, the RT-to-scheduler interface has to be objectified, hence Jay and mine's work. Right, but if the RT moves out of the compute service, then we'll need to be using versioned models, regardless of if it's RPC or REST or whatever. So *if* that's going to happen, building an unversioned object hierarchy that then necessarily has to be converted to another form might be work we don't need to do. I hope we will provide a clear big picture and a roadmap for the Summit so we could give you more insights. Right, since that part is still not fully defined, it's hard to know the best course of action going forward. I'd hate to delay any of this work until after summit, but if you think that would be the most efficient, I guess it's only a couple weeks away. Totally agreed. Here there is no need to version the interface as the virt/Rt interface is not RPC based and purely internal to nova-compute. Well, unless the RT is moved outside the compute node, which is (I think) what is being proposed, no? --Dan signature.asc Description: OpenPGP digital signature ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Pulling nova/virt/hardware.py into nova/objects/
On Mon, Oct 20, 2014 at 01:38:46PM -0400, Jay Pipes wrote: Hi Dan, Dan, Nikola, all Nova devs, OK, so in reviewing Dan B's patch series that refactors the virt driver's get_available_resource() method [1], I am stuck between two concerns. I like (love even) much of the refactoring work involved in Dan's patches. They replace a whole bunch of our nested dicts that are used in the resource tracker with real objects -- and this is something I've been harping on for months that really hinders developer's understanding of Nova's internals. Yep, as you say one of the problems with understanding WTF.com is going on in the code is that the interface between resource_tracker.py and the virt/driver.py was a completely undocumented dict. Some of the data in the dict got directly copied into the database in whatever format the virt driver sent it in. Other data fields in the dict got over-written by the resource tracker. Other fields got converted into a slightly different format, with extra info added to them. However, all of the object classes that Dan B has introduced have been unversioned objects -- i.e. they have not derived from nova.objects.base.NovaObject. This means that these objects cannot be sent over the wire via an RPC API call. In practical terms, this issue has not yet reared its head, because the resource tracker still sends a dictified JSON representation of the object's fields directly over the wire, in the same format as Icehouse, therefore there have been no breakages in RPC API compatibility. If all the data from the virt driver was going straight into the database or out over the wire, unchanged, then I'd agree that using the versioned objects would clearly make sense. When I started the cleanup though, I got the impression that most the data from the virt driver got changed/munged in some way before hitting the database or RPC layer. There is also the long standing discussions about the extensible resource tracker, that would represent data in the database in a completely generic abstracted way as a list of key/value pairs. So I was imagining that long term what's put in the database by the resource tracker would be in a completely different structure than the data coming out fo the virt drivers. Based on that understanding I felt it would be better to define a clear set of classes solely for the data that's coming out of the virt driver, and de-couple this from the objects used for storing stuff in the database. Of course I've not attempted to tackle the full problemspace of cleaning up the entire resource tracker codebase. I just focused in the interface to the virt drivers. So as you note, in order to maintain compatibility, I was careful to ensure that the classes I defined were able to serialize into the same JSON format as is currently used in the horrible undocumented dicts. I was not really expecting that the to_dict/from_dict/to_json/from_json methods in the virt/hardware.py classes be something we use long termm though. I was just thinking of them as a temporary stepping stone, and that the rest of the people working on the (extensible) resource tracker would eventually convert the RT code to directly read the attributes in the hardware.py classes and use them to populate whatever data format the RT wants to use long term. In particular what I'd like to see is that the virt driver be decoupled from long term changes in the resource tracker code data formats. eg if someone comes along in the L cycle and decides the resource tracker/scheduler would be much more effective if the data was persisted in a new format X, then we ought to avoid having to changing the virt/driver.py virt/hardware.py APIs/classes. The RT code would just use the existing classes and convert into whatever fancy new format is better. The problems with having all these objects not modelled by deriving from nova.objects.base.NovaObject are two-fold: * The object's fields/schema cannot be changed -- or rather, cannot be changed without introducing upgrade problems. * The objects introduce a different way of serializing the object contents than is used in nova/objects -- it's not that much different, but it's different, and only has not caused a problem because the serialization routines are not yet being used to transfer data over the wire So, what to do? Clearly, I think the nova/virt/hardware.py objects are badly needed. However, one of (the top?) priorities of the Nova project is upgradeability, and by not deriving from nova.objects.base.NovaObject, these nova.virt.hardware objects are putting that mission in jeopardy, IMO. My proposal is that before we go and approve any BPs or patches that add to nova/virt/hardware.py, we first put together a patch series that moves the object models in nova/virt/hardware.py to being full-fledged objects in nova/objects/* I really it really depends on how we see the resource tracker data model evolving over the next few
Re: [openstack-dev] [nova] Pulling nova/virt/hardware.py into nova/objects/
On Mon, Oct 20, 2014 at 11:12:57AM -0700, Dan Smith wrote: OK, so in reviewing Dan B's patch series that refactors the virt driver's get_available_resource() method [1], I am stuck between two concerns. I like (love even) much of the refactoring work involved in Dan's patches. They replace a whole bunch of our nested dicts that are used in the resource tracker with real objects -- and this is something I've been harping on for months that really hinders developer's understanding of Nova's internals. dict['line1'] = 'Agreed, this is extremely important stuff.' dict['line2'] = 'The current dict mess that we have there is ' dict['line3'] = 'really obscure and confusing.' reply = jsonutils.dumps(dict) However, all of the object classes that Dan B has introduced have been unversioned objects -- i.e. they have not derived from nova.objects.base.NovaObject. This means that these objects cannot be sent over the wire via an RPC API call. In practical terms, this issue has not yet reared its head, because the resource tracker still sends a dictified JSON representation of the object's fields directly over the wire, in the same format as Icehouse, therefore there have been no breakages in RPC API compatibility. Right, so the blueprint for this work states that it's not to be sent over the RPC wire or stored in the database. However, it already is in some cases (at least the ComputeNode object has the unversioned JSONified version of some of these hardware models in it). If the modeling is purely for internal-to-compute-node purposes, then it's all good. However, it surely seems like with the pending scheduler isolation work, we're in a spot where we are building two parallel model hierarchies, and I'm not really sure why. The rationale behind two parallel data model hiercharies is that the format the virt drivers report data in, is not likely to be exactly the same as the format that the resoure tracker / scheduler wishes to use in the database. If we have a single hierarchy, then whenever we need to update the data format for the schedular to improve its performance or flexibility then we have a ripple effect where we'd have to update all the virt driver implementations too. Based on what I've seen about the extensible resource tracker it seems like over time there was going to be greater divergance between what the virt drivers support and how the ERT wants to persist it after transforming it into a easier to deal with structure. My proposal is that before we go and approve any BPs or patches that add to nova/virt/hardware.py, we first put together a patch series that moves the object models in nova/virt/hardware.py to being full-fledged objects in nova/objects/* I'm not sure that just converting them all to NovaObjects is really necessary here. If it's all stuff that is going to go over the wire eventually as part of the resource tracker's expansion, then probably so. If there are bits of the model that only serve to let the resource tracker do its calculations, then perhaps it doesn't make sense to require those be NovaObjects. Yep, pretty much agree with that position. What's difficult is that we're in a bit of a transition stage where stuff that's reported now does get stuffed straight into the database, but in the future might well undergo translation/calculation prior to being stuff in the database. If we convert everything into NovaObjects based on what is directly stored in the DB today, long term this might have proved to be uneccessary. So I took the pragmatic decision that I'd define some plain Python classes, rather than NovaObject clases on the basis that it was still an improvement over an undocumented dict. And as when the scheduler/resource tracker refactor settled down we could either stick with the plain python classes, or convert them into full Nova objects as applicable. Regardless, it sounds like we need some discussion on how best to proceed here. Since it's entirely wrapped up in the scheduler work, we should definitely try to make sure that what we're doing here fits with those plans. Last I heard, we weren't sure where we were going to draw the line between nova bits and scheduler bits, so erring on the side of more versioned interfaces seems safest to me. FWIW, my patch series is logically split up into two parts. THe first 10 or so patches are just thought of as general cleanup and useful to Nova regardless of what we decide todo. The second 10 or so patches are where the objects start appearing getting used the controversial bits needing mor detailed discussion. Regards, Daniel -- |: http://berrange.com -o-http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
Re: [openstack-dev] [nova] Pulling nova/virt/hardware.py into nova/objects/
On Tue, Oct 21, 2014 at 08:46:10AM -0700, Dan Smith wrote: As there are multiple interfaces using non versioned dicts and as we are looking at reducing technical debt by Kilo, there are different blueprints which can be worked in parallel. I don't think I disagree with anything above, but I'm not sure what you're getting at. I think the parallelism we should avoid is building models that mirror things we need to send to a remote service and just require conversion. If we're modeling things between the virt driver and the RT (or interface to the RT) that get aggregated or transformed in some way before they leave the service, then that's fine. Here, the virt-to-RT interface has to be objectified, hence Dan's work. On the other end of the RT, the RT-to-scheduler interface has to be objectified, hence Jay and mine's work. Right, but if the RT moves out of the compute service, then we'll need to be using versioned models, regardless of if it's RPC or REST or whatever. So *if* that's going to happen, building an unversioned object hierarchy that then necessarily has to be converted to another form might be work we don't need to do. Yep, that's a bit I'm still pretty fuzzy on myself. I have been imaginging that while the scheduler would split out of nova, the resource_tracker.py code would still remain part of Nova because it is the bit that interfaces with the nova virt driver API. We have historically always considered the Nova virt driver API to be an internal only thing that we retain the right to change at will so anything talking to it would be in-tree. If the resource_tracker.py were to move out of Nova, then this now implies that the get_available_resources() method in virt/driver.py is now defacto stable API between Nova the schedulr project, that we have to preserve compatibility for. If that is ineed the case then clearly using versioned NovaObjects for that method's return value is going to be required. IMHO though this is a not a desirable split of responsibility. Personally I was always expecting that resource_tracker.py would stay part of Nova. It would use the internal get_available_resources() API to talk to the virt driver, and transform the data it got back into whatever format the external scheduler project wants to consume via some formal API and these formats would likely be completely separate. In that world view, there is no need for the get_available_resources() method to use NovaObject. I hope we will provide a clear big picture and a roadmap for the Summit so we could give you more insights. Right, since that part is still not fully defined, it's hard to know the best course of action going forward. I'd hate to delay any of this work until after summit, but if you think that would be the most efficient, I guess it's only a couple weeks away. As mentioned in my other mail, my 20 patch series is split into two real pieces. The first 10 or so patches are just generall cleanup / prep work that is hopefully fairly uncontroversial to consider merging. If the second 10 or so patches have to wait until after the summit, so be it, that's not a big problem. Getting the code structure right is more important long term than a fast merge. Totally agreed. Here there is no need to version the interface as the virt/Rt interface is not RPC based and purely internal to nova-compute. Well, unless the RT is moved outside the compute node, which is (I think) what is being proposed, no? Regards, Daniel -- |: http://berrange.com -o-http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :| ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Pulling nova/virt/hardware.py into nova/objects/
The rationale behind two parallel data model hiercharies is that the format the virt drivers report data in, is not likely to be exactly the same as the format that the resoure tracker / scheduler wishes to use in the database. Yeah, and in cases where we know where that line is, it makes sense to use the lighter-weight modeling for sure. FWIW, my patch series is logically split up into two parts. THe first 10 or so patches are just thought of as general cleanup and useful to Nova regardless of what we decide todo. The second 10 or so patches are where the objects start appearing getting used the controversial bits needing mor detailed discussion. Right, so after some discussion I think we should go ahead and merge the bottom of this set (all of them are now acked I think) and continue the discussion on the top half where the modeling is introduced. Thanks! --Dan signature.asc Description: OpenPGP digital signature ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [nova] Pulling nova/virt/hardware.py into nova/objects/
Hi Dan, Dan, Nikola, all Nova devs, OK, so in reviewing Dan B's patch series that refactors the virt driver's get_available_resource() method [1], I am stuck between two concerns. I like (love even) much of the refactoring work involved in Dan's patches. They replace a whole bunch of our nested dicts that are used in the resource tracker with real objects -- and this is something I've been harping on for months that really hinders developer's understanding of Nova's internals. However, all of the object classes that Dan B has introduced have been unversioned objects -- i.e. they have not derived from nova.objects.base.NovaObject. This means that these objects cannot be sent over the wire via an RPC API call. In practical terms, this issue has not yet reared its head, because the resource tracker still sends a dictified JSON representation of the object's fields directly over the wire, in the same format as Icehouse, therefore there have been no breakages in RPC API compatibility. The problems with having all these objects not modelled by deriving from nova.objects.base.NovaObject are two-fold: * The object's fields/schema cannot be changed -- or rather, cannot be changed without introducing upgrade problems. * The objects introduce a different way of serializing the object contents than is used in nova/objects -- it's not that much different, but it's different, and only has not caused a problem because the serialization routines are not yet being used to transfer data over the wire So, what to do? Clearly, I think the nova/virt/hardware.py objects are badly needed. However, one of (the top?) priorities of the Nova project is upgradeability, and by not deriving from nova.objects.base.NovaObject, these nova.virt.hardware objects are putting that mission in jeopardy, IMO. My proposal is that before we go and approve any BPs or patches that add to nova/virt/hardware.py, we first put together a patch series that moves the object models in nova/virt/hardware.py to being full-fledged objects in nova/objects/* Thoughts? -jay [1] https://review.openstack.org/#/q/status:open+project:openstack/nova+branch:master+topic:bp/virt-driver-get-available-resources-object,n,z ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Pulling nova/virt/hardware.py into nova/objects/
OK, so in reviewing Dan B's patch series that refactors the virt driver's get_available_resource() method [1], I am stuck between two concerns. I like (love even) much of the refactoring work involved in Dan's patches. They replace a whole bunch of our nested dicts that are used in the resource tracker with real objects -- and this is something I've been harping on for months that really hinders developer's understanding of Nova's internals. dict['line1'] = 'Agreed, this is extremely important stuff.' dict['line2'] = 'The current dict mess that we have there is ' dict['line3'] = 'really obscure and confusing.' reply = jsonutils.dumps(dict) However, all of the object classes that Dan B has introduced have been unversioned objects -- i.e. they have not derived from nova.objects.base.NovaObject. This means that these objects cannot be sent over the wire via an RPC API call. In practical terms, this issue has not yet reared its head, because the resource tracker still sends a dictified JSON representation of the object's fields directly over the wire, in the same format as Icehouse, therefore there have been no breakages in RPC API compatibility. Right, so the blueprint for this work states that it's not to be sent over the RPC wire or stored in the database. However, it already is in some cases (at least the ComputeNode object has the unversioned JSONified version of some of these hardware models in it). If the modeling is purely for internal-to-compute-node purposes, then it's all good. However, it surely seems like with the pending scheduler isolation work, we're in a spot where we are building two parallel model hierarchies, and I'm not really sure why. My proposal is that before we go and approve any BPs or patches that add to nova/virt/hardware.py, we first put together a patch series that moves the object models in nova/virt/hardware.py to being full-fledged objects in nova/objects/* I'm not sure that just converting them all to NovaObjects is really necessary here. If it's all stuff that is going to go over the wire eventually as part of the resource tracker's expansion, then probably so. If there are bits of the model that only serve to let the resource tracker do its calculations, then perhaps it doesn't make sense to require those be NovaObjects. Regardless, it sounds like we need some discussion on how best to proceed here. Since it's entirely wrapped up in the scheduler work, we should definitely try to make sure that what we're doing here fits with those plans. Last I heard, we weren't sure where we were going to draw the line between nova bits and scheduler bits, so erring on the side of more versioned interfaces seems safest to me. --Dan signature.asc Description: OpenPGP digital signature ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
