Re: [vdsm] [virt-node] RFC: API Supportability
On Thu, Jun 21, 2012 at 01:20:40PM +0300, Dan Kenigsberg wrote: On Wed, Jun 20, 2012 at 10:42:16AM -0500, Adam Litke wrote: On Tue, Jun 19, 2012 at 10:17:28AM -0400, Saggi Mizrahi wrote: I've opened a wiki page [1] for the stable API and extracted some of the TODO points so we don't forget. Everyone can feel free to add more stuff. [1] http://ovirt.org/wiki/VDSM_Stable_API_Plan Rest of the comments inline - Original Message - From: Adam Litke a...@us.ibm.com To: Saggi Mizrahi smizr...@redhat.com Cc: VDSM Project Development vdsm-devel@lists.fedorahosted.org, Barak Azulay bazu...@redhat.com, Itamar Heim ih...@redhat.com, Ayal Baron aba...@redhat.com, Anthony Liguori aligu...@redhat.com Sent: Monday, June 18, 2012 12:23:10 PM Subject: Re: [virt-node] RFC: API Supportability On Mon, Jun 18, 2012 at 11:02:25AM -0400, Saggi Mizrahi wrote: The first thing we need to decide is API supportabiliy. I'll list the questions that need to be answered. The decision made here will have great effect on transport selection (espscially API change process and versioning) so try and think about this without going to specfic technicalities (eg. X can't be done on REST). Thanks for sending this out. I will take a crack at these questions... I would like to pose an additional question to be answered: - Should API parameter and return value constraints be formally defined? If so, how? Think of this as defining an API schema. For example: When creating a VM, which parameters are required/optional? What are the valid formats for specifying a VM disk? What are all of the possible task states? Has to be part of response to the call that retrieves the state. This will allow us to change the states in a BC manner. I am not sure I agree. I think it should be a part of the schema but not transmitted along with each API response involving a task. This would increase traffic and make responses unnecessarily verbose. Is there a maximum length for the storage domain description? I totally agree, how depends on the transport of choice but in any case I think the definition should be done in a declarative manner (XML\JSON) using concrete types (important for binding with C\Java) and have some *code to enforce* that the input is correct. This will prevent clients from not adhering to the schema exploiting python's relative lax approach to types. We already had issues with the engine wrongly sending numbers as strings and having this break internally because of some change in the python code made it not handle the conversion very well. Our schema should fully define a set of simple types and complex types. Each defined simple type will have an internal validation function to verify conformity of a given input. Complex types consist of nested lists and dicts of simple types. They are validated first by validating members as simple types and then checking for missing and/or extra data. When designing a dependable API, we should not desert our agility. ovirt-Engine has enjoyed the possibility of saying hey, we want another field reported in getVdsStats and presto, here it was. Complex types should be easily extendible (with a proper update of the API minor version, or a capabilities set). +1 -- Adam Litke a...@us.ibm.com IBM Linux Technology Center ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] [virt-node] RFC: API Supportability
I tired to sum everything in the wiki page [1] Please review the page and see if there is something I missed or that you don't agree with. - Original Message - From: Adam Litke a...@us.ibm.com To: Dan Kenigsberg dan...@redhat.com Cc: Saggi Mizrahi smizr...@redhat.com, VDSM Project Development vdsm-devel@lists.fedorahosted.org, Daniel Veillard veill...@redhat.com, Anthony Liguori aligu...@redhat.com Sent: Thursday, June 21, 2012 10:41:36 AM Subject: Re: [vdsm] [virt-node] RFC: API Supportability On Thu, Jun 21, 2012 at 01:20:40PM +0300, Dan Kenigsberg wrote: On Wed, Jun 20, 2012 at 10:42:16AM -0500, Adam Litke wrote: On Tue, Jun 19, 2012 at 10:17:28AM -0400, Saggi Mizrahi wrote: I've opened a wiki page [1] for the stable API and extracted some of the TODO points so we don't forget. Everyone can feel free to add more stuff. [1] http://ovirt.org/wiki/VDSM_Stable_API_Plan Rest of the comments inline - Original Message - From: Adam Litke a...@us.ibm.com To: Saggi Mizrahi smizr...@redhat.com Cc: VDSM Project Development vdsm-devel@lists.fedorahosted.org, Barak Azulay bazu...@redhat.com, Itamar Heim ih...@redhat.com, Ayal Baron aba...@redhat.com, Anthony Liguori aligu...@redhat.com Sent: Monday, June 18, 2012 12:23:10 PM Subject: Re: [virt-node] RFC: API Supportability On Mon, Jun 18, 2012 at 11:02:25AM -0400, Saggi Mizrahi wrote: The first thing we need to decide is API supportabiliy. I'll list the questions that need to be answered. The decision made here will have great effect on transport selection (espscially API change process and versioning) so try and think about this without going to specfic technicalities (eg. X can't be done on REST). Thanks for sending this out. I will take a crack at these questions... I would like to pose an additional question to be answered: - Should API parameter and return value constraints be formally defined? If so, how? Think of this as defining an API schema. For example: When creating a VM, which parameters are required/optional? What are the valid formats for specifying a VM disk? What are all of the possible task states? Has to be part of response to the call that retrieves the state. This will allow us to change the states in a BC manner. I am not sure I agree. I think it should be a part of the schema but not transmitted along with each API response involving a task. This would increase traffic and make responses unnecessarily verbose. Is there a maximum length for the storage domain description? I totally agree, how depends on the transport of choice but in any case I think the definition should be done in a declarative manner (XML\JSON) using concrete types (important for binding with C\Java) and have some *code to enforce* that the input is correct. This will prevent clients from not adhering to the schema exploiting python's relative lax approach to types. We already had issues with the engine wrongly sending numbers as strings and having this break internally because of some change in the python code made it not handle the conversion very well. Our schema should fully define a set of simple types and complex types. Each defined simple type will have an internal validation function to verify conformity of a given input. Complex types consist of nested lists and dicts of simple types. They are validated first by validating members as simple types and then checking for missing and/or extra data. When designing a dependable API, we should not desert our agility. ovirt-Engine has enjoyed the possibility of saying hey, we want another field reported in getVdsStats and presto, here it was. Complex types should be easily extendible (with a proper update of the API minor version, or a capabilities set). +1 -- Adam Litke a...@us.ibm.com IBM Linux Technology Center ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] [virt-node] RFC: API Supportability
I've opened a wiki page [1] for the stable API and extracted some of the TODO points so we don't forget. Everyone can feel free to add more stuff. [1] http://ovirt.org/wiki/VDSM_Stable_API_Plan Rest of the comments inline - Original Message - From: Adam Litke a...@us.ibm.com To: Saggi Mizrahi smizr...@redhat.com Cc: VDSM Project Development vdsm-devel@lists.fedorahosted.org, Barak Azulay bazu...@redhat.com, Itamar Heim ih...@redhat.com, Ayal Baron aba...@redhat.com, Anthony Liguori aligu...@redhat.com Sent: Monday, June 18, 2012 12:23:10 PM Subject: Re: [virt-node] RFC: API Supportability On Mon, Jun 18, 2012 at 11:02:25AM -0400, Saggi Mizrahi wrote: The first thing we need to decide is API supportabiliy. I'll list the questions that need to be answered. The decision made here will have great effect on transport selection (espscially API change process and versioning) so try and think about this without going to specfic technicalities (eg. X can't be done on REST). Thanks for sending this out. I will take a crack at these questions... I would like to pose an additional question to be answered: - Should API parameter and return value constraints be formally defined? If so, how? Think of this as defining an API schema. For example: When creating a VM, which parameters are required/optional? What are the valid formats for specifying a VM disk? What are all of the possible task states? Has to be part of response to the call that retrieves the state. This will allow us to change the states in a BC manner. Is there a maximum length for the storage domain description? I totally agree, how depends on the transport of choice but in any case I think the definition should be done in a declarative manner (XML\JSON) using concrete types (important for binding with C\Java) and have some *code to enforce* that the input is correct. This will prevent clients from not adhering to the schema exploiting python's relative lax approach to types. We already had issues with the engine wrongly sending numbers as strings and having this break internally because of some change in the python code made it not handle the conversion very well. New API acceptance process == - What is the process to suggest new API calls? New APIs should be proposed on a mailing list. This allows everyone to participate in the conversation and preserves the comments. If the API is simple, a patch the provides a concrete example of implementation is recommended. Once the API design is agreed upon, patches can be submitted via the standard method (gerrit) to implement a new experimental API based on the design. The submitter of the patches should reply to the design discussion thread to notify participants of the available code. +1 - Who can ack such a change? API changes should be subject to wider approval than a simple change to an internal component. I believe that the +1,-1 system works well here and we should seek approvals from all participants in the design discussion if possible. I will add that you need a +1 from at least 2 maintainers for an API change. Also someone has to test that the change did not break old clients. - Does someone have veto rights? Anyone can NACK an API design. Same rules as for normal code. +1 - Are there experimental APIs? Yes! Dave Allan has mentioned that from his experience in libvirt, it would be very nice to have experimental APIs that can be improved before being baked into the supportable API. I definitely agree. In fact, all new APIs should go through a period of being experimental. Experimental API functions should begin with '_'. Once deemed stable, the '_' can be removed. I don't like this specific mangling scheme but I do agree that we need experimental calls. I also think that you need a mechanism to turn them on similar to `import __future__` in python so that you make sure API user knows it's experimental. API deprecation process === - Do we allow deprecation? I would like to allow deprecation because it grants us an avenue to clean up the API from time to time. That being said, I am not aware of a clean way to do this without breaking old clients too badly. At a minimum, an API would need to be deprecated for at least 2 years before it can be removed. How will this decision influence the initial API design? Are there features that we can build into an API that can ease the burden of deprecation on API consumers? Deprecation is tricky. We also need a mechanism for a client to know that his version of the API no longer exists so it can check for that at host connection and fail if the client is to old. To do that we could either have API group versions and expose which versions are supported in full. We could also take the opengl route of querying for call. But this might be to
Re: [vdsm] [virt-node] RFC: API Supportability
On Mon, Jun 18, 2012 at 11:02:25AM -0400, Saggi Mizrahi wrote: The first thing we need to decide is API supportabiliy. I'll list the questions that need to be answered. The decision made here will have great effect on transport selection (espscially API change process and versioning) so try and think about this without going to specfic technicalities (eg. X can't be done on REST). Thanks for sending this out. I will take a crack at these questions... I would like to pose an additional question to be answered: - Should API parameter and return value constraints be formally defined? If so, how? Think of this as defining an API schema. For example: When creating a VM, which parameters are required/optional? What are the valid formats for specifying a VM disk? What are all of the possible task states? Is there a maximum length for the storage domain description? New API acceptance process == - What is the process to suggest new API calls? New APIs should be proposed on a mailing list. This allows everyone to participate in the conversation and preserves the comments. If the API is simple, a patch the provides a concrete example of implementation is recommended. Once the API design is agreed upon, patches can be submitted via the standard method (gerrit) to implement a new experimental API based on the design. The submitter of the patches should reply to the design discussion thread to notify participants of the available code. - Who can ack such a change? API changes should be subject to wider approval than a simple change to an internal component. I believe that the +1,-1 system works well here and we should seek approvals from all participants in the design discussion if possible. - Does someone have veto rights? Anyone can NACK an API design. Same rules as for normal code. - Are there experimental APIs? Yes! Dave Allan has mentioned that from his experience in libvirt, it would be very nice to have experimental APIs that can be improved before being baked into the supportable API. I definitely agree. In fact, all new APIs should go through a period of being experimental. Experimental API functions should begin with '_'. Once deemed stable, the '_' can be removed. API deprecation process === - Do we allow deprecation? I would like to allow deprecation because it grants us an avenue to clean up the API from time to time. That being said, I am not aware of a clean way to do this without breaking old clients too badly. At a minimum, an API would need to be deprecated for at least 2 years before it can be removed. How will this decision influence the initial API design? Are there features that we can build into an API that can ease the burden of deprecation on API consumers? - When can an API call be deprecated? As a matter of last resort. Deprecation must last for an agreed-upon minumum time period (for supportability) before an API may be removed. - Who can ack such a change? - Does someone have veto rights? Same as above. API change process == - Can calls be modified or no symbol can ever repeat in a different form I would suggest libvirt's model here. APIs can be extended by passing flags to enable new behavior. New parameters can be added to an API as long as they are optional. - When can an API call be deprecated? See above. - Who can ack such a change? - Does someone have veto rights? Same as above. API versioning == - Is the API versioned as a whole, is it per subsystem (storage, networking, etc..) or is each call versioned by itself. The API should be versioned as a whole. This version should be the same as the software release version. Additionally, the API should support a list of capabilities that can be used to express features that can not be expressed by the presense of a function signature. For example, the supported storage domain types. The ovirt-engine rest API has a decent capabilities API. - What happens when old clients connects to a newer server. The API is append-only (with two exceptions) so there should never be a problem with this. Exceptions: - Experimental functions may be missing - Deprecated functions may have been removed - What happens when a new client connects to an older sever. This is a bit harder. We need to handle the following cases: 1) Server wants to use an API call that may not exist on an old version. An attempt to call a non-existing function should result in a standardized error that the client can handle appropriately. In such a case, a client that wants to support older servers would need to handle the error and perform the needed operation using an alternate method. To ease this process for clients, our API should have a registry of allowed methods. In REST this would be RSDL. It could also be exported as a component of the capabilities API. 2) Server wants to supply
