Re: [openstack-dev] [Nova] Need help with a gnarly Object Version issue
Hi Dan, > > > On a compute manager that is still running the old version of the code > > (i.e using the previous object version), if a method that hasn't yet > > been converted to objects gets a dict created from the new version of > > the object (e.g. rescue, get_console_output), then object_compat() > > decorator will call the _/from_db/_object() method in > > objects.Instance. Because this is the old version of the object > > code, it expects user_data to be a field in dict, and throws a key error. > > Yeah, so the versioning rules are that for a minor version, you can only add > things to the object, not remove them. Ah - Ok. That probably explains why it doesn't work then ;-( > > > 1) Rather than removing the user_data field from the object just > > set it to a null value if its not requested. > > Objects have a notion of "unset" which is what you'd want here. Fields that > are not set can be lazy-loaded when touched, which might be a reasonable > way out of the box here if user_data is really only used in one place. It > would > mean that older clients would lazy-load it when needed, and going forward > we'd be specific about asking for it when we want. > > However, the problem is that instance defines the fields it's willing to lazy- > load, and user_data isn't one of them. That'd mean that we need to backport > a change to allow it to be lazy-loaded, which means we should probably just > backport the thing that requests user_data when needed instead. > Not quite sure I follow. The list of "can be lazy loaded" fields is defined by INSTANCE_OPTIONAL_ATTRS right ? I moved user_data into that set of fields as part of my patch, but the problem I have is with mix of objects and non objects, such as the sequence where: Client:Gets an Object (of new version) RPCAPI: Converts Object to a Dict (because the specific RPC method hasn't been converted to take an Object yet) Manager: Converts dict to an Object (of the old version) via the @object_compat decorator The last step fails because "_from_db_object()" runs just in the not-yet-updated manager, and hence hits a key error. I don't think lazy loading helps here, because the code that fails is trying to create the object form a dict, not trying to access into an Object - or am I missing something ? > > 2) Add object versioning in the client side of the RPC layer for > > those methods that don't take objects. > > I'm not sure what you mean here. > In terms of the above scenario I was thinking that the RPCAPI layer could make sure the object was the right version before it converts it to a dict. > > I'm open to other ideas, and general guidance around how deletion of > > fields from Objects is meant to be handled ? > > It's meant to be handled by rev-ing the major version, since removing > something isn't a compatible operation. > > Note that *conductor* has knowledge of the client-side version of an object > on which the remotable_classmethod is being called, but that is not exposed > to the actual object implementation in any way. We could, perhaps, figure > out a sneaky way to expose that, which would let you honor the old behavior > if we know the object is old, or the new behavior otherwise. > I think the problem is that I don't have an object at the point where I get the failure, I have a dict that is trying to be mapped into an object, so it doesn't call back into conductor. I'm thinking now that as the problem is the size and/or data in user_data - and that is only in a very few specific places I could just set the user_data contest in the instance Object to None or "X" if its not requested when the object is created. (Setting it to "X" would probably make it easier to debug if something that does what it gets missed) Phil ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Need help with a gnarly Object Version issue
> On a compute manager that is still running the old version of the code > (i.e using the previous object version), if a method that hasn’t yet > been converted to objects gets a dict created from the new version of > the object (e.g. rescue, get_console_output), then object_compat() > decorator will call the _/from_db/_object() method in > objects.Instance. Because this is the old version of the object > code, it expects user_data to be a field in dict, and throws a key error. Yeah, so the versioning rules are that for a minor version, you can only add things to the object, not remove them. > 1) Rather than removing the user_data field from the object just > set it to a null value if its not requested. Objects have a notion of "unset" which is what you'd want here. Fields that are not set can be lazy-loaded when touched, which might be a reasonable way out of the box here if user_data is really only used in one place. It would mean that older clients would lazy-load it when needed, and going forward we'd be specific about asking for it when we want. However, the problem is that instance defines the fields it's willing to lazy-load, and user_data isn't one of them. That'd mean that we need to backport a change to allow it to be lazy-loaded, which means we should probably just backport the thing that requests user_data when needed instead. > 2) Add object versioning in the client side of the RPC layer for > those methods that don’t take objects. I'm not sure what you mean here. > I’m open to other ideas, and general guidance around how deletion of > fields from Objects is meant to be handled ? It's meant to be handled by rev-ing the major version, since removing something isn't a compatible operation. Note that *conductor* has knowledge of the client-side version of an object on which the remotable_classmethod is being called, but that is not exposed to the actual object implementation in any way. We could, perhaps, figure out a sneaky way to expose that, which would let you honor the old behavior if we know the object is old, or the new behavior otherwise. --Dan ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [Nova] Need help with a gnarly Object Version issue
Hi Folks, I've been working on a change to make the user_data field an optional part of the Instance object since passing it around everywhere seems a bad idea since: - It can be huge - It's only used when getting metadata - It can contain user sensitive data - https://review.openstack.org/#/c/92623/9 I've included the object version changes, and that all works fine - but I'm left with one issue that I'm not sure how to proceed with: On a compute manager that is still running the old version of the code (i.e using the previous object version), if a method that hasn't yet been converted to objects gets a dict created from the new version of the object (e.g. rescue, get_console_output), then object_compat() decorator will call the _from_db_object() method in objects.Instance. Because this is the old version of the object code, it expects user_data to be a field in dict, and throws a key error. I can think of a number of possible fixes - but I'm not sure any of them are very elegant (and of course they have to fix the problem before the data is sent to the compute manager): 1) Rather than removing the user_data field from the object just set it to a null value if its not requested. 2) Add object versioning in the client side of the RPC layer for those methods that don't take objects. I'm open to other ideas, and general guidance around how deletion of fields from Objects is meant to be handled ? Phil ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
