Re: [vdsm] [virt-node] VDSM as a general purpose virt host manager
On 06/26/2012 11:26 AM, Adam Litke wrote: On Tue, Jun 26, 2012 at 11:11:51PM +0800, Shu Ming wrote: On 2012-6-26 20:45, Adam Litke wrote: On Tue, Jun 26, 2012 at 09:53:10AM +0800, Xu He Jie wrote: On 06/26/2012 05:19 AM, Adam Litke wrote: On Mon, Jun 25, 2012 at 05:53:31PM +0300, Dan Kenigsberg wrote: On Mon, Jun 25, 2012 at 08:28:29AM -0500, Adam Litke wrote: On Fri, Jun 22, 2012 at 06:45:43PM -0400, Andrew Cathrow wrote: - Original Message - From: "Ryan Harper" To: "Adam Litke" Cc: "Anthony Liguori", "VDSM Project Development" Sent: Friday, June 22, 2012 12:45:42 PM Subject: Re: [vdsm] [virt-node] VDSM as a general purpose virt host manager * Adam Litke [2012-06-22 11:35]: On Thu, Jun 21, 2012 at 12:17:19PM +0300, Dor Laor wrote: On 06/19/2012 08:12 PM, Saggi Mizrahi wrote: - Original Message - From: "Deepak C Shetty" To: "Ryan Harper" Cc: "Saggi Mizrahi", "Anthony Liguori" , "VDSM Project Development" Sent: Tuesday, June 19, 2012 10:58:47 AM Subject: Re: [vdsm] [virt-node] VDSM as a general purpose virt host manager On 06/19/2012 01:13 AM, Ryan Harper wrote: * Saggi Mizrahi [2012-06-18 10:05]: I would like to put on to the table for descussion the growing need for a way to more easily reuse of the functionality of VDSM in order to service projects other than Ovirt-Engine. Originally VDSM was created as a proprietary agent for the sole purpose of serving the then proprietary version of what is known as ovirt-engine. Red Hat, after acquiring the technology, pressed on with it's commitment to open source ideals and released the code. But just releasing code into the wild doesn't build a community or makes a project successful. Further more when building open source software you should aspire to build reusable components instead of monolithic stacks. Saggi, Thanks for sending this out. I've been trying to pull together some thoughts on what else is needed for vdsm as a community. I know that for some time downstream has been the driving force for all of the work and now with a community there are challenges in finding our own way. While we certainly don't want to make downstream efforts harder, I think we need to develop and support our own vision for what vdsm can be come, some what independent of downstream and other exploiters. Revisiting the API is definitely a much needed endeavor and I think adding some use-cases or sample applications would be useful in demonstrating whether or not we're evolving the API into something easier to use for applications beyond engine. We would like to expose a stable, documented, well supported API. This gives us a chance to rethink the VDSM API from the ground up. There is already work in progress of making the internal logic of VDSM separate enough >from the API layer so we could continue feature development and bug fixing while designing the API of the future. In order to achieve this though we need to do several things: 1. Declare API supportability guidelines 2. Decide on an API transport (e.g. REST, ZMQ, AMQP) 3. Make the API easily consumable (e.g. proper docs, example code, extending the API, etc) 4. Implement the API itself In the earlier we'd discussed working to have similarities in the modeling between the oVirt API and VDSM but that seems to have dropped off the radar. Yes, the current REST API has attempted to be compatible with the current ovirt-engine API. Going forward, I am not sure how easy this will be to maintain given than engine is built on Java and vdsm is built on Python. Could you elaborate why the language difference is an issue? Isn't this what APIs are supposed to solve? The main language issue is that ovirt-engine has built their API using a set of Java-specific frameworks (JAXB and its dependents). It's true, if you google for 'python jaxb' you will find some sourceforge projects that attempt to bring the jaxb interface to python but I don't think that's the right approach. If you're writing a java project, do things the java way. If you're writing a python project, do them the python way. Right now I am focused on defining the current API (API.py/xmlrpc) mechanically (creating a schema and API documentation). XSD is not the correct language for that task (which is why I forsee a divergence at least at first). I want to take a stab at defining the API in a beneficial, long-term manner. Adam, Can you explain why you think XSD is not the correct language? Is it because of the lacking of full python language code generator? Is it possible to modify the existing code generator to address that issue? What is the benefit to introduce a new schema/code generator to oVirt instead of changing the existing code generator? XSD is designed to specify a schema for XML documents. The
Re: [vdsm] [virt-node] VDSM as a general purpose virt host manager
On 2012-6-27 0:18, Adam Litke wrote: On Tue, Jun 26, 2012 at 04:47:35PM +0100, Daniel P. Berrange wrote: On Tue, Jun 26, 2012 at 05:37:26PM +0300, Dan Kenigsberg wrote: On Mon, Jun 25, 2012 at 04:19:28PM -0500, Adam Litke wrote: 1) Completely define the current XMLRPC API including all functions, parameters, and return values. Complex data structures can be broken down into their basic types. These are: int, str, bool, list, dict, typed-dict, enum. I have already started this process and am using Qemu's QAPI schema language. You can see that here [1]. For an example of what that looks like describing the vdsm API see this snippet [2]. 2) Import the parser/generator code from qemu for the above schema. Vdsm will require a few extensions such as typed-dictionaries, tuples, and type aliases. Adapt the generator so that it can produce a libvdsm which provides API language bindings for python, c, and java. 3) Implement a vdsm shell in terms of libvdsm. In fact, this can be largely auto-generated from the schema and accompanying documentation. This can serve to model how new transports can be written. For example, an AMQP implementation can be implemented entirely outside of the vdsm project if we wished. It only needs to talk to vdsm via libvdsm. Easy as 1,2,3 :) [1] http://git.qemu.org/?p=qemu.git;a=blob;f=qapi-schema.json;h=3b6e3468b440b4b681f321c9525a3d83bea2137a;hb=HEAD [2] http://fpaste.org/rt96/ Probably more than you bargained for when asking for more info... :) Indeed! I am still at a loss why the languages take such a prominent place in your choice for an API. A good API is easily consumable by any language. I think you are both right here. A good API is easily consumed from any language, but this doesn't mean there is zero cost to starting to consume it from a client. You either way to be able to auto-generate code for the client side APIs in all your languages of choice, or even better, you want the client side APIs to be just do runtime dynamic dispatch based on published schema. Thanks for commenting! On one hand, dynamic dispatch seems attractive but I think dramatically increases complexity on both the client and server sides. Does anyone know of a prominent open source project that has been successful with dynamic dispatch? I am inclined to go with the C library approach because it is tried and tested and it fits the model of other virtualization libraries that I am familiar with. If you go down the route of writing a C based libvdsm for VDSM, then my recommendation would be to use the GObject APIs. You can then take full advantage of the GObject Introspection capabilities to have full dynamic dispatch in languages like python, perl, javascript, or full auto-generation of code in Vala, C#, etc Ahh, thanks for reminding me of this. GObject definitely seems like the way to go. I assume there are no real implications for the schema definition and that the heavy-lifting for GObject support would be limited to the C code generator. Time to take a closer look at the GObject stuff. Even if we can generate the various API bindings from the schema, we still need to change the transportation layers to connect the new APIs from the client to the VDSM services, whether or not the layer is XMLRPC, REST or any other way. I really hate to change the transportation layer from time to time to know new APIs. It is better to let the transportation layer to figure out the new API automatically and keep zero change with new APIs. I certainly wouldn't waste time writing your own code-generator for all the various languages, since that's just reinventing the wheel that GObject Introspection already provides for the most part. Agreed. I would love to avoid this! -- Shu Ming IBM China Systems and Technology Laboratory ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] [virt-node] VDSM as a general purpose virt host manager
On Tue, Jun 26, 2012 at 04:47:35PM +0100, Daniel P. Berrange wrote: > On Tue, Jun 26, 2012 at 05:37:26PM +0300, Dan Kenigsberg wrote: > > On Mon, Jun 25, 2012 at 04:19:28PM -0500, Adam Litke wrote: > > > 1) Completely define the current XMLRPC API including all functions, > > > parameters, > > > and return values. Complex data structures can be broken down into their > > > basic > > > types. These are: int, str, bool, list, dict, typed-dict, enum. I have > > > already > > > started this process and am using Qemu's QAPI schema language. You can > > > see that > > > here [1]. For an example of what that looks like describing the vdsm API > > > see > > > this snippet [2]. > > > > > > 2) Import the parser/generator code from qemu for the above schema. Vdsm > > > will > > > require a few extensions such as typed-dictionaries, tuples, and type > > > aliases. > > > Adapt the generator so that it can produce a libvdsm which provides API > > > language > > > bindings for python, c, and java. > > > > > > 3) Implement a vdsm shell in terms of libvdsm. In fact, this can be > > > largely > > > auto-generated from the schema and accompanying documentation. This can > > > serve > > > to model how new transports can be written. For example, an AMQP > > > implementation > > > can be implemented entirely outside of the vdsm project if we wished. It > > > only > > > needs to talk to vdsm via libvdsm. > > > > > > Easy as 1,2,3 :) > > > > > > [1] > > > http://git.qemu.org/?p=qemu.git;a=blob;f=qapi-schema.json;h=3b6e3468b440b4b681f321c9525a3d83bea2137a;hb=HEAD > > > [2] http://fpaste.org/rt96/ > > > > > > Probably more than you bargained for when asking for more info... :) > > > > Indeed! > > > > I am still at a loss why the languages take such a prominent place in > > your choice for an API. A good API is easily consumable by any language. > > I think you are both right here. A good API is easily consumed from any > language, but this doesn't mean there is zero cost to starting to consume > it from a client. You either way to be able to auto-generate code for the > client side APIs in all your languages of choice, or even better, you want > the client side APIs to be just do runtime dynamic dispatch based on > published schema. Thanks for commenting! On one hand, dynamic dispatch seems attractive but I think dramatically increases complexity on both the client and server sides. Does anyone know of a prominent open source project that has been successful with dynamic dispatch? I am inclined to go with the C library approach because it is tried and tested and it fits the model of other virtualization libraries that I am familiar with. > If you go down the route of writing a C based libvdsm for VDSM, then my > recommendation would be to use the GObject APIs. You can then take full > advantage of the GObject Introspection capabilities to have full dynamic > dispatch in languages like python, perl, javascript, or full auto-generation > of code in Vala, C#, etc Ahh, thanks for reminding me of this. GObject definitely seems like the way to go. I assume there are no real implications for the schema definition and that the heavy-lifting for GObject support would be limited to the C code generator. Time to take a closer look at the GObject stuff. > I certainly wouldn't waste time writing your own code-generator for all > the various languages, since that's just reinventing the wheel that > GObject Introspection already provides for the most part. Agreed. I would love to avoid this! -- Adam Litke IBM Linux Technology Center ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] [virt-node] VDSM as a general purpose virt host manager
On Tue, Jun 26, 2012 at 05:37:26PM +0300, Dan Kenigsberg wrote: > On Mon, Jun 25, 2012 at 04:19:28PM -0500, Adam Litke wrote: > > 1) Completely define the current XMLRPC API including all functions, > > parameters, > > and return values. Complex data structures can be broken down into their > > basic > > types. These are: int, str, bool, list, dict, typed-dict, enum. I have > > already > > started this process and am using Qemu's QAPI schema language. You can see > > that > > here [1]. For an example of what that looks like describing the vdsm API > > see > > this snippet [2]. > > > > 2) Import the parser/generator code from qemu for the above schema. Vdsm > > will > > require a few extensions such as typed-dictionaries, tuples, and type > > aliases. > > Adapt the generator so that it can produce a libvdsm which provides API > > language > > bindings for python, c, and java. > > > > 3) Implement a vdsm shell in terms of libvdsm. In fact, this can be largely > > auto-generated from the schema and accompanying documentation. This can > > serve > > to model how new transports can be written. For example, an AMQP > > implementation > > can be implemented entirely outside of the vdsm project if we wished. It > > only > > needs to talk to vdsm via libvdsm. > > > > Easy as 1,2,3 :) > > > > [1] > > http://git.qemu.org/?p=qemu.git;a=blob;f=qapi-schema.json;h=3b6e3468b440b4b681f321c9525a3d83bea2137a;hb=HEAD > > [2] http://fpaste.org/rt96/ > > > > Probably more than you bargained for when asking for more info... :) > > Indeed! > > I am still at a loss why the languages take such a prominent place in > your choice for an API. A good API is easily consumable by any language. I think you are both right here. A good API is easily consumed from any language, but this doesn't mean there is zero cost to starting to consume it from a client. You either way to be able to auto-generate code for the client side APIs in all your languages of choice, or even better, you want the client side APIs to be just do runtime dynamic dispatch based on published schema. If you go down the route of writing a C based libvdsm for VDSM, then my recommendation would be to use the GObject APIs. You can then take full advantage of the GObject Introspection capabilities to have full dynamic dispatch in languages like python, perl, javascript, or full auto-generation of code in Vala, C#, etc I certainly wouldn't waste time writing your own code-generator for all the various languages, since that's just reinventing the wheel that GObject Introspection already provides for the most part. Daniel -- |: http://berrange.com -o-http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :| ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] [virt-node] VDSM as a general purpose virt host manager
On Tue, Jun 26, 2012 at 11:11:51PM +0800, Shu Ming wrote: > On 2012-6-26 20:45, Adam Litke wrote: > >On Tue, Jun 26, 2012 at 09:53:10AM +0800, Xu He Jie wrote: > >>On 06/26/2012 05:19 AM, Adam Litke wrote: > >>>On Mon, Jun 25, 2012 at 05:53:31PM +0300, Dan Kenigsberg wrote: > >>>>On Mon, Jun 25, 2012 at 08:28:29AM -0500, Adam Litke wrote: > >>>>>On Fri, Jun 22, 2012 at 06:45:43PM -0400, Andrew Cathrow wrote: > >>>>>>- Original Message - > >>>>>>>From: "Ryan Harper" > >>>>>>>To: "Adam Litke" > >>>>>>>Cc: "Anthony Liguori" , "VDSM Project > >>>>>>>Development" > >>>>>>>Sent: Friday, June 22, 2012 12:45:42 PM > >>>>>>>Subject: Re: [vdsm] [virt-node] VDSM as a general purpose virt host > >>>>>>>manager > >>>>>>> > >>>>>>>* Adam Litke [2012-06-22 11:35]: > >>>>>>>>On Thu, Jun 21, 2012 at 12:17:19PM +0300, Dor Laor wrote: > >>>>>>>>>On 06/19/2012 08:12 PM, Saggi Mizrahi wrote: > >>>>>>>>>>- Original Message - > >>>>>>>>>>>From: "Deepak C Shetty" > >>>>>>>>>>>To: "Ryan Harper" > >>>>>>>>>>>Cc: "Saggi Mizrahi" , "Anthony Liguori" > >>>>>>>>>>>, "VDSM Project Development" > >>>>>>>>>>> > >>>>>>>>>>>Sent: Tuesday, June 19, 2012 10:58:47 AM > >>>>>>>>>>>Subject: Re: [vdsm] [virt-node] VDSM as a general purpose virt > >>>>>>>>>>>host manager > >>>>>>>>>>> > >>>>>>>>>>>On 06/19/2012 01:13 AM, Ryan Harper wrote: > >>>>>>>>>>>>* Saggi Mizrahi [2012-06-18 10:05]: > >>>>>>>>>>>>>I would like to put on to the table for descussion the > >>>>>>>>>>>>>growing > >>>>>>>>>>>>>need for a way > >>>>>>>>>>>>>to more easily reuse of the functionality of VDSM in order to > >>>>>>>>>>>>>service projects > >>>>>>>>>>>>>other than Ovirt-Engine. > >>>>>>>>>>>>> > >>>>>>>>>>>>>Originally VDSM was created as a proprietary agent for the > >>>>>>>>>>>>>sole > >>>>>>>>>>>>>purpose of > >>>>>>>>>>>>>serving the then proprietary version of what is known as > >>>>>>>>>>>>>ovirt-engine. Red Hat, > >>>>>>>>>>>>>after acquiring the technology, pressed on with it's > >>>>>>>>>>>>>commitment to > >>>>>>>>>>>>>open source > >>>>>>>>>>>>>ideals and released the code. But just releasing code into > >>>>>>>>>>>>>the > >>>>>>>>>>>>>wild doesn't > >>>>>>>>>>>>>build a community or makes a project successful. Further more > >>>>>>>>>>>>>when > >>>>>>>>>>>>>building > >>>>>>>>>>>>>open source software you should aspire to build reusable > >>>>>>>>>>>>>components instead of > >>>>>>>>>>>>>monolithic stacks. > >>>>>>>>>>>>> > >>>>>>>>>>>>Saggi, > >>>>>>>>>>>> > >>>>>>>>>>>>Thanks for sending this out. I've been trying to pull > >>>>>>>>>>>>together > >>>>>>>>>>>>some > >>>>>>>>>>>>thoughts on what else is needed for vdsm as a community. I > >>>>>>>>>>>>know > >>>>>>>>>>>>that > >>>>>>>>>
Re: [vdsm] [virt-node] VDSM as a general purpose virt host manager
On Tue, Jun 26, 2012 at 05:37:26PM +0300, Dan Kenigsberg wrote: > On Mon, Jun 25, 2012 at 04:19:28PM -0500, Adam Litke wrote: > > On Mon, Jun 25, 2012 at 05:53:31PM +0300, Dan Kenigsberg wrote: > > > On Mon, Jun 25, 2012 at 08:28:29AM -0500, Adam Litke wrote: > > > > On Fri, Jun 22, 2012 at 06:45:43PM -0400, Andrew Cathrow wrote: > > > > > > > > > > > > > > > - Original Message - > > > > > > From: "Ryan Harper" > > > > > > To: "Adam Litke" > > > > > > Cc: "Anthony Liguori" , "VDSM Project > > > > > > Development" > > > > > > Sent: Friday, June 22, 2012 12:45:42 PM > > > > > > Subject: Re: [vdsm] [virt-node] VDSM as a general purpose virt host > > > > > > manager > > > > > > > > > > > > * Adam Litke [2012-06-22 11:35]: > > > > > > > On Thu, Jun 21, 2012 at 12:17:19PM +0300, Dor Laor wrote: > > > > > > > > On 06/19/2012 08:12 PM, Saggi Mizrahi wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > >- Original Message - > > > > > > > > >>From: "Deepak C Shetty" > > > > > > > > >>To: "Ryan Harper" > > > > > > > > >>Cc: "Saggi Mizrahi" , "Anthony Liguori" > > > > > > > > >>, "VDSM Project Development" > > > > > > > > >> > > > > > > > > >>Sent: Tuesday, June 19, 2012 10:58:47 AM > > > > > > > > >>Subject: Re: [vdsm] [virt-node] VDSM as a general purpose virt > > > > > > > > >>host manager > > > > > > > > >> > > > > > > > > >>On 06/19/2012 01:13 AM, Ryan Harper wrote: > > > > > > > > >>>* Saggi Mizrahi [2012-06-18 10:05]: > > > > > > > > >>>>I would like to put on to the table for descussion the > > > > > > > > >>>>growing > > > > > > > > >>>>need for a way > > > > > > > > >>>>to more easily reuse of the functionality of VDSM in order > > > > > > > > >>>>to > > > > > > > > >>>>service projects > > > > > > > > >>>>other than Ovirt-Engine. > > > > > > > > >>>> > > > > > > > > >>>>Originally VDSM was created as a proprietary agent for the > > > > > > > > >>>>sole > > > > > > > > >>>>purpose of > > > > > > > > >>>>serving the then proprietary version of what is known as > > > > > > > > >>>>ovirt-engine. Red Hat, > > > > > > > > >>>>after acquiring the technology, pressed on with it's > > > > > > > > >>>>commitment to > > > > > > > > >>>>open source > > > > > > > > >>>>ideals and released the code. But just releasing code into > > > > > > > > >>>>the > > > > > > > > >>>>wild doesn't > > > > > > > > >>>>build a community or makes a project successful. Further > > > > > > > > >>>>more > > > > > > > > >>>>when > > > > > > > > >>>>building > > > > > > > > >>>>open source software you should aspire to build reusable > > > > > > > > >>>>components instead of > > > > > > > > >>>>monolithic stacks. > > > > > > > > >>>> > > > > > > > > >>>Saggi, > > > > > > > > >>> > > > > > > > > >>>Thanks for sending this out. I've been trying to pull > > > > > > > > >>>together > > > > > > > > >>>some > > > > > > > > >>>thoughts on what else is needed for vdsm as a community. I > > > > > > > > >>>know > > > > > > > &g
Re: [vdsm] [virt-node] VDSM as a general purpose virt host manager
On 2012-6-26 20:45, Adam Litke wrote: On Tue, Jun 26, 2012 at 09:53:10AM +0800, Xu He Jie wrote: On 06/26/2012 05:19 AM, Adam Litke wrote: On Mon, Jun 25, 2012 at 05:53:31PM +0300, Dan Kenigsberg wrote: On Mon, Jun 25, 2012 at 08:28:29AM -0500, Adam Litke wrote: On Fri, Jun 22, 2012 at 06:45:43PM -0400, Andrew Cathrow wrote: - Original Message - From: "Ryan Harper" To: "Adam Litke" Cc: "Anthony Liguori" , "VDSM Project Development" Sent: Friday, June 22, 2012 12:45:42 PM Subject: Re: [vdsm] [virt-node] VDSM as a general purpose virt host manager * Adam Litke [2012-06-22 11:35]: On Thu, Jun 21, 2012 at 12:17:19PM +0300, Dor Laor wrote: On 06/19/2012 08:12 PM, Saggi Mizrahi wrote: - Original Message - From: "Deepak C Shetty" To: "Ryan Harper" Cc: "Saggi Mizrahi" , "Anthony Liguori" , "VDSM Project Development" Sent: Tuesday, June 19, 2012 10:58:47 AM Subject: Re: [vdsm] [virt-node] VDSM as a general purpose virt host manager On 06/19/2012 01:13 AM, Ryan Harper wrote: * Saggi Mizrahi [2012-06-18 10:05]: I would like to put on to the table for descussion the growing need for a way to more easily reuse of the functionality of VDSM in order to service projects other than Ovirt-Engine. Originally VDSM was created as a proprietary agent for the sole purpose of serving the then proprietary version of what is known as ovirt-engine. Red Hat, after acquiring the technology, pressed on with it's commitment to open source ideals and released the code. But just releasing code into the wild doesn't build a community or makes a project successful. Further more when building open source software you should aspire to build reusable components instead of monolithic stacks. Saggi, Thanks for sending this out. I've been trying to pull together some thoughts on what else is needed for vdsm as a community. I know that for some time downstream has been the driving force for all of the work and now with a community there are challenges in finding our own way. While we certainly don't want to make downstream efforts harder, I think we need to develop and support our own vision for what vdsm can be come, some what independent of downstream and other exploiters. Revisiting the API is definitely a much needed endeavor and I think adding some use-cases or sample applications would be useful in demonstrating whether or not we're evolving the API into something easier to use for applications beyond engine. We would like to expose a stable, documented, well supported API. This gives us a chance to rethink the VDSM API from the ground up. There is already work in progress of making the internal logic of VDSM separate enough >from the API layer so we could continue feature development and bug fixing while designing the API of the future. In order to achieve this though we need to do several things: 1. Declare API supportability guidelines 2. Decide on an API transport (e.g. REST, ZMQ, AMQP) 3. Make the API easily consumable (e.g. proper docs, example code, extending the API, etc) 4. Implement the API itself In the earlier we'd discussed working to have similarities in the modeling between the oVirt API and VDSM but that seems to have dropped off the radar. Yes, the current REST API has attempted to be compatible with the current ovirt-engine API. Going forward, I am not sure how easy this will be to maintain given than engine is built on Java and vdsm is built on Python. Could you elaborate why the language difference is an issue? Isn't this what APIs are supposed to solve? The main language issue is that ovirt-engine has built their API using a set of Java-specific frameworks (JAXB and its dependents). It's true, if you google for 'python jaxb' you will find some sourceforge projects that attempt to bring the jaxb interface to python but I don't think that's the right approach. If you're writing a java project, do things the java way. If you're writing a python project, do them the python way. Right now I am focused on defining the current API (API.py/xmlrpc) mechanically (creating a schema and API documentation). XSD is not the correct language for that task (which is why I forsee a divergence at least at first). I want to take a stab at defining the API in a beneficial, long-term manner. Adam, Can you explain why you think XSD is not the correct language? Is it because of the lacking of full python language code generator? Is it possible to modify the existing code generator to address that issue? What is the benefit to introduce a new schema/code generator to oVirt instead of changing the existing code generator? 1) Completely define the current XMLRPC API including all functions, parameters, and return values. Complex data structures can be broken down into t
Re: [vdsm] [virt-node] VDSM as a general purpose virt host manager
On Mon, Jun 25, 2012 at 04:19:28PM -0500, Adam Litke wrote: > On Mon, Jun 25, 2012 at 05:53:31PM +0300, Dan Kenigsberg wrote: > > On Mon, Jun 25, 2012 at 08:28:29AM -0500, Adam Litke wrote: > > > On Fri, Jun 22, 2012 at 06:45:43PM -0400, Andrew Cathrow wrote: > > > > > > > > > > > > - Original Message - > > > > > From: "Ryan Harper" > > > > > To: "Adam Litke" > > > > > Cc: "Anthony Liguori" , "VDSM Project > > > > > Development" > > > > > Sent: Friday, June 22, 2012 12:45:42 PM > > > > > Subject: Re: [vdsm] [virt-node] VDSM as a general purpose virt host > > > > > manager > > > > > > > > > > * Adam Litke [2012-06-22 11:35]: > > > > > > On Thu, Jun 21, 2012 at 12:17:19PM +0300, Dor Laor wrote: > > > > > > > On 06/19/2012 08:12 PM, Saggi Mizrahi wrote: > > > > > > > > > > > > > > > > > > > > > > > >- Original Message - > > > > > > > >>From: "Deepak C Shetty" > > > > > > > >>To: "Ryan Harper" > > > > > > > >>Cc: "Saggi Mizrahi" , "Anthony Liguori" > > > > > > > >>, "VDSM Project Development" > > > > > > > >> > > > > > > > >>Sent: Tuesday, June 19, 2012 10:58:47 AM > > > > > > > >>Subject: Re: [vdsm] [virt-node] VDSM as a general purpose virt > > > > > > > >>host manager > > > > > > > >> > > > > > > > >>On 06/19/2012 01:13 AM, Ryan Harper wrote: > > > > > > > >>>* Saggi Mizrahi [2012-06-18 10:05]: > > > > > > > >>>>I would like to put on to the table for descussion the > > > > > > > >>>>growing > > > > > > > >>>>need for a way > > > > > > > >>>>to more easily reuse of the functionality of VDSM in order to > > > > > > > >>>>service projects > > > > > > > >>>>other than Ovirt-Engine. > > > > > > > >>>> > > > > > > > >>>>Originally VDSM was created as a proprietary agent for the > > > > > > > >>>>sole > > > > > > > >>>>purpose of > > > > > > > >>>>serving the then proprietary version of what is known as > > > > > > > >>>>ovirt-engine. Red Hat, > > > > > > > >>>>after acquiring the technology, pressed on with it's > > > > > > > >>>>commitment to > > > > > > > >>>>open source > > > > > > > >>>>ideals and released the code. But just releasing code into > > > > > > > >>>>the > > > > > > > >>>>wild doesn't > > > > > > > >>>>build a community or makes a project successful. Further more > > > > > > > >>>>when > > > > > > > >>>>building > > > > > > > >>>>open source software you should aspire to build reusable > > > > > > > >>>>components instead of > > > > > > > >>>>monolithic stacks. > > > > > > > >>>> > > > > > > > >>>Saggi, > > > > > > > >>> > > > > > > > >>>Thanks for sending this out. I've been trying to pull > > > > > > > >>>together > > > > > > > >>>some > > > > > > > >>>thoughts on what else is needed for vdsm as a community. I > > > > > > > >>>know > > > > > > > >>>that > > > > > > > >>>for some time downstream has been the driving force for all of > > > > > > > >>>the > > > > > > > >>>work > > > > > > > >>>and now with a community there are challenges in finding our > > > > > > > >>>own > > > > > > > >>>way. > > > > > > > >>> > &g
Re: [vdsm] [virt-node] VDSM as a general purpose virt host manager
On Tue, Jun 26, 2012 at 09:53:10AM +0800, Xu He Jie wrote: > On 06/26/2012 05:19 AM, Adam Litke wrote: > >On Mon, Jun 25, 2012 at 05:53:31PM +0300, Dan Kenigsberg wrote: > >>On Mon, Jun 25, 2012 at 08:28:29AM -0500, Adam Litke wrote: > >>>On Fri, Jun 22, 2012 at 06:45:43PM -0400, Andrew Cathrow wrote: > >>>> > >>>>- Original Message - > >>>>>From: "Ryan Harper" > >>>>>To: "Adam Litke" > >>>>>Cc: "Anthony Liguori" , "VDSM Project Development" > >>>>> > >>>>>Sent: Friday, June 22, 2012 12:45:42 PM > >>>>>Subject: Re: [vdsm] [virt-node] VDSM as a general purpose virt host > >>>>>manager > >>>>> > >>>>>* Adam Litke [2012-06-22 11:35]: > >>>>>>On Thu, Jun 21, 2012 at 12:17:19PM +0300, Dor Laor wrote: > >>>>>>>On 06/19/2012 08:12 PM, Saggi Mizrahi wrote: > >>>>>>>> > >>>>>>>>- Original Message - > >>>>>>>>>From: "Deepak C Shetty" > >>>>>>>>>To: "Ryan Harper" > >>>>>>>>>Cc: "Saggi Mizrahi" , "Anthony Liguori" > >>>>>>>>>, "VDSM Project Development" > >>>>>>>>> > >>>>>>>>>Sent: Tuesday, June 19, 2012 10:58:47 AM > >>>>>>>>>Subject: Re: [vdsm] [virt-node] VDSM as a general purpose virt > >>>>>>>>>host manager > >>>>>>>>> > >>>>>>>>>On 06/19/2012 01:13 AM, Ryan Harper wrote: > >>>>>>>>>>* Saggi Mizrahi [2012-06-18 10:05]: > >>>>>>>>>>>I would like to put on to the table for descussion the > >>>>>>>>>>>growing > >>>>>>>>>>>need for a way > >>>>>>>>>>>to more easily reuse of the functionality of VDSM in order to > >>>>>>>>>>>service projects > >>>>>>>>>>>other than Ovirt-Engine. > >>>>>>>>>>> > >>>>>>>>>>>Originally VDSM was created as a proprietary agent for the > >>>>>>>>>>>sole > >>>>>>>>>>>purpose of > >>>>>>>>>>>serving the then proprietary version of what is known as > >>>>>>>>>>>ovirt-engine. Red Hat, > >>>>>>>>>>>after acquiring the technology, pressed on with it's > >>>>>>>>>>>commitment to > >>>>>>>>>>>open source > >>>>>>>>>>>ideals and released the code. But just releasing code into > >>>>>>>>>>>the > >>>>>>>>>>>wild doesn't > >>>>>>>>>>>build a community or makes a project successful. Further more > >>>>>>>>>>>when > >>>>>>>>>>>building > >>>>>>>>>>>open source software you should aspire to build reusable > >>>>>>>>>>>components instead of > >>>>>>>>>>>monolithic stacks. > >>>>>>>>>>> > >>>>>>>>>>Saggi, > >>>>>>>>>> > >>>>>>>>>>Thanks for sending this out. I've been trying to pull > >>>>>>>>>>together > >>>>>>>>>>some > >>>>>>>>>>thoughts on what else is needed for vdsm as a community. I > >>>>>>>>>>know > >>>>>>>>>>that > >>>>>>>>>>for some time downstream has been the driving force for all of > >>>>>>>>>>the > >>>>>>>>>>work > >>>>>>>>>>and now with a community there are challenges in finding our > >>>>>>>>>>own > >>>>>>>>>>way. > >>>>>>>>>> > >>>>>>>>>>While we certainly don't want to make downstream efforts > >>&g
Re: [vdsm] [virt-node] VDSM as a general purpose virt host manager
On 06/26/2012 05:19 AM, Adam Litke wrote: On Mon, Jun 25, 2012 at 05:53:31PM +0300, Dan Kenigsberg wrote: On Mon, Jun 25, 2012 at 08:28:29AM -0500, Adam Litke wrote: On Fri, Jun 22, 2012 at 06:45:43PM -0400, Andrew Cathrow wrote: - Original Message - From: "Ryan Harper" To: "Adam Litke" Cc: "Anthony Liguori" , "VDSM Project Development" Sent: Friday, June 22, 2012 12:45:42 PM Subject: Re: [vdsm] [virt-node] VDSM as a general purpose virt host manager * Adam Litke [2012-06-22 11:35]: On Thu, Jun 21, 2012 at 12:17:19PM +0300, Dor Laor wrote: On 06/19/2012 08:12 PM, Saggi Mizrahi wrote: - Original Message - From: "Deepak C Shetty" To: "Ryan Harper" Cc: "Saggi Mizrahi" , "Anthony Liguori" , "VDSM Project Development" Sent: Tuesday, June 19, 2012 10:58:47 AM Subject: Re: [vdsm] [virt-node] VDSM as a general purpose virt host manager On 06/19/2012 01:13 AM, Ryan Harper wrote: * Saggi Mizrahi [2012-06-18 10:05]: I would like to put on to the table for descussion the growing need for a way to more easily reuse of the functionality of VDSM in order to service projects other than Ovirt-Engine. Originally VDSM was created as a proprietary agent for the sole purpose of serving the then proprietary version of what is known as ovirt-engine. Red Hat, after acquiring the technology, pressed on with it's commitment to open source ideals and released the code. But just releasing code into the wild doesn't build a community or makes a project successful. Further more when building open source software you should aspire to build reusable components instead of monolithic stacks. Saggi, Thanks for sending this out. I've been trying to pull together some thoughts on what else is needed for vdsm as a community. I know that for some time downstream has been the driving force for all of the work and now with a community there are challenges in finding our own way. While we certainly don't want to make downstream efforts harder, I think we need to develop and support our own vision for what vdsm can be come, some what independent of downstream and other exploiters. Revisiting the API is definitely a much needed endeavor and I think adding some use-cases or sample applications would be useful in demonstrating whether or not we're evolving the API into something easier to use for applications beyond engine. We would like to expose a stable, documented, well supported API. This gives us a chance to rethink the VDSM API from the ground up. There is already work in progress of making the internal logic of VDSM separate enough >from the API layer so we could continue feature development and bug fixing while designing the API of the future. In order to achieve this though we need to do several things: 1. Declare API supportability guidelines 2. Decide on an API transport (e.g. REST, ZMQ, AMQP) 3. Make the API easily consumable (e.g. proper docs, example code, extending the API, etc) 4. Implement the API itself In the earlier we'd discussed working to have similarities in the modeling between the oVirt API and VDSM but that seems to have dropped off the radar. Yes, the current REST API has attempted to be compatible with the current ovirt-engine API. Going forward, I am not sure how easy this will be to maintain given than engine is built on Java and vdsm is built on Python. Could you elaborate why the language difference is an issue? Isn't this what APIs are supposed to solve? The main language issue is that ovirt-engine has built their API using a set of Java-specific frameworks (JAXB and its dependents). It's true, if you google for 'python jaxb' you will find some sourceforge projects that attempt to bring the jaxb interface to python but I don't think that's the right approach. If you're writing a java project, do things the java way. If you're writing a python project, do them the python way. Right now I am focused on defining the current API (API.py/xmlrpc) mechanically (creating a schema and API documentation). XSD is not the correct language for that task (which is why I forsee a divergence at least at first). I want to take a stab at defining the API in a beneficial, long-term manner. 1) Completely define the current XMLRPC API including all functions, parameters, and return values. Complex data structures can be broken down into their basic types. These are: int, str, bool, list, dict, typed-dict, enum. I have already started this process and am using Qemu's QAPI schema language. You can see that here [1]. For an example of what that looks like describing the vdsm API see this snippet [2]. 2) Import the parser/generator code from qemu for the above schema. Vdsm will require a few extensions such as typed-dictionaries, tuples, and type aliases. Adapt the
Re: [vdsm] [virt-node] VDSM as a general purpose virt host manager
On Mon, Jun 25, 2012 at 05:53:31PM +0300, Dan Kenigsberg wrote: > On Mon, Jun 25, 2012 at 08:28:29AM -0500, Adam Litke wrote: > > On Fri, Jun 22, 2012 at 06:45:43PM -0400, Andrew Cathrow wrote: > > > > > > > > > - Original Message - > > > > From: "Ryan Harper" > > > > To: "Adam Litke" > > > > Cc: "Anthony Liguori" , "VDSM Project Development" > > > > > > > > Sent: Friday, June 22, 2012 12:45:42 PM > > > > Subject: Re: [vdsm] [virt-node] VDSM as a general purpose virt host > > > > manager > > > > > > > > * Adam Litke [2012-06-22 11:35]: > > > > > On Thu, Jun 21, 2012 at 12:17:19PM +0300, Dor Laor wrote: > > > > > > On 06/19/2012 08:12 PM, Saggi Mizrahi wrote: > > > > > > > > > > > > > > > > > > > > >- Original Message - > > > > > > >>From: "Deepak C Shetty" > > > > > > >>To: "Ryan Harper" > > > > > > >>Cc: "Saggi Mizrahi" , "Anthony Liguori" > > > > > > >>, "VDSM Project Development" > > > > > > >> > > > > > > >>Sent: Tuesday, June 19, 2012 10:58:47 AM > > > > > > >>Subject: Re: [vdsm] [virt-node] VDSM as a general purpose virt > > > > > > >>host manager > > > > > > >> > > > > > > >>On 06/19/2012 01:13 AM, Ryan Harper wrote: > > > > > > >>>* Saggi Mizrahi [2012-06-18 10:05]: > > > > > > >>>>I would like to put on to the table for descussion the > > > > > > >>>>growing > > > > > > >>>>need for a way > > > > > > >>>>to more easily reuse of the functionality of VDSM in order to > > > > > > >>>>service projects > > > > > > >>>>other than Ovirt-Engine. > > > > > > >>>> > > > > > > >>>>Originally VDSM was created as a proprietary agent for the > > > > > > >>>>sole > > > > > > >>>>purpose of > > > > > > >>>>serving the then proprietary version of what is known as > > > > > > >>>>ovirt-engine. Red Hat, > > > > > > >>>>after acquiring the technology, pressed on with it's > > > > > > >>>>commitment to > > > > > > >>>>open source > > > > > > >>>>ideals and released the code. But just releasing code into > > > > > > >>>>the > > > > > > >>>>wild doesn't > > > > > > >>>>build a community or makes a project successful. Further more > > > > > > >>>>when > > > > > > >>>>building > > > > > > >>>>open source software you should aspire to build reusable > > > > > > >>>>components instead of > > > > > > >>>>monolithic stacks. > > > > > > >>>> > > > > > > >>>Saggi, > > > > > > >>> > > > > > > >>>Thanks for sending this out. I've been trying to pull > > > > > > >>>together > > > > > > >>>some > > > > > > >>>thoughts on what else is needed for vdsm as a community. I > > > > > > >>>know > > > > > > >>>that > > > > > > >>>for some time downstream has been the driving force for all of > > > > > > >>>the > > > > > > >>>work > > > > > > >>>and now with a community there are challenges in finding our > > > > > > >>>own > > > > > > >>>way. > > > > > > >>> > > > > > > >>>While we certainly don't want to make downstream efforts > > > > > > >>>harder, I > > > > > > >>>think > > > > > > >>>we need to develop and support our own vision for what vdsm > > > > > > >>>can be > > > > > > >>>come, > > > >
Re: [vdsm] [virt-node] VDSM as a general purpose virt host manager
On Mon, Jun 25, 2012 at 08:28:29AM -0500, Adam Litke wrote: > On Fri, Jun 22, 2012 at 06:45:43PM -0400, Andrew Cathrow wrote: > > > > > > - Original Message - > > > From: "Ryan Harper" > > > To: "Adam Litke" > > > Cc: "Anthony Liguori" , "VDSM Project Development" > > > > > > Sent: Friday, June 22, 2012 12:45:42 PM > > > Subject: Re: [vdsm] [virt-node] VDSM as a general purpose virt host > > > manager > > > > > > * Adam Litke [2012-06-22 11:35]: > > > > On Thu, Jun 21, 2012 at 12:17:19PM +0300, Dor Laor wrote: > > > > > On 06/19/2012 08:12 PM, Saggi Mizrahi wrote: > > > > > > > > > > > > > > > > > >- Original Message - > > > > > >>From: "Deepak C Shetty" > > > > > >>To: "Ryan Harper" > > > > > >>Cc: "Saggi Mizrahi" , "Anthony Liguori" > > > > > >>, "VDSM Project Development" > > > > > >> > > > > > >>Sent: Tuesday, June 19, 2012 10:58:47 AM > > > > > >>Subject: Re: [vdsm] [virt-node] VDSM as a general purpose virt > > > > > >>host manager > > > > > >> > > > > > >>On 06/19/2012 01:13 AM, Ryan Harper wrote: > > > > > >>>* Saggi Mizrahi [2012-06-18 10:05]: > > > > > >>>>I would like to put on to the table for descussion the > > > > > >>>>growing > > > > > >>>>need for a way > > > > > >>>>to more easily reuse of the functionality of VDSM in order to > > > > > >>>>service projects > > > > > >>>>other than Ovirt-Engine. > > > > > >>>> > > > > > >>>>Originally VDSM was created as a proprietary agent for the > > > > > >>>>sole > > > > > >>>>purpose of > > > > > >>>>serving the then proprietary version of what is known as > > > > > >>>>ovirt-engine. Red Hat, > > > > > >>>>after acquiring the technology, pressed on with it's > > > > > >>>>commitment to > > > > > >>>>open source > > > > > >>>>ideals and released the code. But just releasing code into > > > > > >>>>the > > > > > >>>>wild doesn't > > > > > >>>>build a community or makes a project successful. Further more > > > > > >>>>when > > > > > >>>>building > > > > > >>>>open source software you should aspire to build reusable > > > > > >>>>components instead of > > > > > >>>>monolithic stacks. > > > > > >>>> > > > > > >>>Saggi, > > > > > >>> > > > > > >>>Thanks for sending this out. I've been trying to pull > > > > > >>>together > > > > > >>>some > > > > > >>>thoughts on what else is needed for vdsm as a community. I > > > > > >>>know > > > > > >>>that > > > > > >>>for some time downstream has been the driving force for all of > > > > > >>>the > > > > > >>>work > > > > > >>>and now with a community there are challenges in finding our > > > > > >>>own > > > > > >>>way. > > > > > >>> > > > > > >>>While we certainly don't want to make downstream efforts > > > > > >>>harder, I > > > > > >>>think > > > > > >>>we need to develop and support our own vision for what vdsm > > > > > >>>can be > > > > > >>>come, > > > > > >>>some what independent of downstream and other exploiters. > > > > > >>> > > > > > >>>Revisiting the API is definitely a much needed endeavor and I > > > > > >>>think > > > > > >>>adding some use-cases or sample applications would be useful > > > > > >>>in > > > > > >>>demonstrating whether or not we're evolving the API into > > > > > >>>something > > > > > >>>easier to use for applications beyond engine. > > > > > >>> > > > > > >>>>We would like to expose a stable, documented, well supported > > > > > >>>>API. > > > > > >>>>This gives > > > > > >>>>us a chance to rethink the VDSM API from the ground up. There > > > > > >>>>is > > > > > >>>>already work > > > > > > > >>>>in progress of making the internal logic of VDSM separate > > > > > >>>>enough > > > > > >>>>from the API > > > > > >>>>layer so we could continue feature development and bug fixing > > > > > >>>>while designing > > > > > >>>>the API of the future. > > > > > >>>> > > > > > >>>>In order to achieve this though we need to do several things: > > > > > >>>> 1. Declare API supportability guidelines > > > > > >>>> 2. Decide on an API transport (e.g. REST, ZMQ, AMQP) > > > > > >>>> 3. Make the API easily consumable (e.g. proper docs, > > > > > >>>> example > > > > > >>>> code, extending > > > > > >>>>the API, etc) > > > > > >>>> 4. Implement the API itself > > > > In the earlier we'd discussed working to have similarities in the modeling > > between the oVirt API and VDSM but that seems to have dropped off the radar. > > Yes, the current REST API has attempted to be compatible with the current > ovirt-engine API. Going forward, I am not sure how easy this will be to > maintain given than engine is built on Java and vdsm is built on Python. Could you elaborate why the language difference is an issue? Isn't this what APIs are supposed to solve? ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] [virt-node] VDSM as a general purpose virt host manager
On Fri, Jun 22, 2012 at 06:45:43PM -0400, Andrew Cathrow wrote: > > > - Original Message - > > From: "Ryan Harper" > > To: "Adam Litke" > > Cc: "Anthony Liguori" , "VDSM Project Development" > > > > Sent: Friday, June 22, 2012 12:45:42 PM > > Subject: Re: [vdsm] [virt-node] VDSM as a general purpose virt host manager > > > > * Adam Litke [2012-06-22 11:35]: > > > On Thu, Jun 21, 2012 at 12:17:19PM +0300, Dor Laor wrote: > > > > On 06/19/2012 08:12 PM, Saggi Mizrahi wrote: > > > > > > > > > > > > > > >- Original Message - > > > > >>From: "Deepak C Shetty" > > > > >>To: "Ryan Harper" > > > > >>Cc: "Saggi Mizrahi" , "Anthony Liguori" > > > > >>, "VDSM Project Development" > > > > >> > > > > >>Sent: Tuesday, June 19, 2012 10:58:47 AM > > > > >>Subject: Re: [vdsm] [virt-node] VDSM as a general purpose virt > > > > >>host manager > > > > >> > > > > >>On 06/19/2012 01:13 AM, Ryan Harper wrote: > > > > >>>* Saggi Mizrahi [2012-06-18 10:05]: > > > > >>>>I would like to put on to the table for descussion the > > > > >>>>growing > > > > >>>>need for a way > > > > >>>>to more easily reuse of the functionality of VDSM in order to > > > > >>>>service projects > > > > >>>>other than Ovirt-Engine. > > > > >>>> > > > > >>>>Originally VDSM was created as a proprietary agent for the > > > > >>>>sole > > > > >>>>purpose of > > > > >>>>serving the then proprietary version of what is known as > > > > >>>>ovirt-engine. Red Hat, > > > > >>>>after acquiring the technology, pressed on with it's > > > > >>>>commitment to > > > > >>>>open source > > > > >>>>ideals and released the code. But just releasing code into > > > > >>>>the > > > > >>>>wild doesn't > > > > >>>>build a community or makes a project successful. Further more > > > > >>>>when > > > > >>>>building > > > > >>>>open source software you should aspire to build reusable > > > > >>>>components instead of > > > > >>>>monolithic stacks. > > > > >>>> > > > > >>>Saggi, > > > > >>> > > > > >>>Thanks for sending this out. I've been trying to pull > > > > >>>together > > > > >>>some > > > > >>>thoughts on what else is needed for vdsm as a community. I > > > > >>>know > > > > >>>that > > > > >>>for some time downstream has been the driving force for all of > > > > >>>the > > > > >>>work > > > > >>>and now with a community there are challenges in finding our > > > > >>>own > > > > >>>way. > > > > >>> > > > > >>>While we certainly don't want to make downstream efforts > > > > >>>harder, I > > > > >>>think > > > > >>>we need to develop and support our own vision for what vdsm > > > > >>>can be > > > > >>>come, > > > > >>>some what independent of downstream and other exploiters. > > > > >>> > > > > >>>Revisiting the API is definitely a much needed endeavor and I > > > > >>>think > > > > >>>adding some use-cases or sample applications would be useful > > > > >>>in > > > > >>>demonstrating whether or not we're evolving the API into > > > > >>>something > > > > >>>easier to use for applications beyond engine. > > > > >>> > > > > >>>>We would like to expose a stable, documented, well supported > > > > >>>>API. > > > > >>>>This gives > > > > >>>>us a chance to rethink the VDSM API from the ground up. There > > > > >>>>is > > > > >>>>already work > > > > > >>>>in progress of making the internal logic of VDSM separate > > > > >>>>enough > > > > >>>>from the API > > > > >>>>layer so we could continue feature development and bug fixing > > > > >>>>while designing > > > > >>>>the API of the future. > > > > >>>> > > > > >>>>In order to achieve this though we need to do several things: > > > > >>>> 1. Declare API supportability guidelines > > > > >>>> 2. Decide on an API transport (e.g. REST, ZMQ, AMQP) > > > > >>>> 3. Make the API easily consumable (e.g. proper docs, > > > > >>>> example > > > > >>>> code, extending > > > > >>>>the API, etc) > > > > >>>> 4. Implement the API itself > > In the earlier we'd discussed working to have similarities in the modeling > between the oVirt API and VDSM but that seems to have dropped off the radar. Yes, the current REST API has attempted to be compatible with the current ovirt-engine API. Going forward, I am not sure how easy this will be to maintain given than engine is built on Java and vdsm is built on Python. -- Adam Litke IBM Linux Technology Center ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] [virt-node] VDSM as a general purpose virt host manager
- Original Message - > From: "Ryan Harper" > To: "Adam Litke" > Cc: "Anthony Liguori" , "VDSM Project Development" > > Sent: Friday, June 22, 2012 12:45:42 PM > Subject: Re: [vdsm] [virt-node] VDSM as a general purpose virt host manager > > * Adam Litke [2012-06-22 11:35]: > > On Thu, Jun 21, 2012 at 12:17:19PM +0300, Dor Laor wrote: > > > On 06/19/2012 08:12 PM, Saggi Mizrahi wrote: > > > > > > > > > > > >- Original Message - > > > >>From: "Deepak C Shetty" > > > >>To: "Ryan Harper" > > > >>Cc: "Saggi Mizrahi" , "Anthony Liguori" > > > >>, "VDSM Project Development" > > > >> > > > >>Sent: Tuesday, June 19, 2012 10:58:47 AM > > > >>Subject: Re: [vdsm] [virt-node] VDSM as a general purpose virt > > > >>host manager > > > >> > > > >>On 06/19/2012 01:13 AM, Ryan Harper wrote: > > > >>>* Saggi Mizrahi [2012-06-18 10:05]: > > > >>>>I would like to put on to the table for descussion the > > > >>>>growing > > > >>>>need for a way > > > >>>>to more easily reuse of the functionality of VDSM in order to > > > >>>>service projects > > > >>>>other than Ovirt-Engine. > > > >>>> > > > >>>>Originally VDSM was created as a proprietary agent for the > > > >>>>sole > > > >>>>purpose of > > > >>>>serving the then proprietary version of what is known as > > > >>>>ovirt-engine. Red Hat, > > > >>>>after acquiring the technology, pressed on with it's > > > >>>>commitment to > > > >>>>open source > > > >>>>ideals and released the code. But just releasing code into > > > >>>>the > > > >>>>wild doesn't > > > >>>>build a community or makes a project successful. Further more > > > >>>>when > > > >>>>building > > > >>>>open source software you should aspire to build reusable > > > >>>>components instead of > > > >>>>monolithic stacks. > > > >>>> > > > >>>Saggi, > > > >>> > > > >>>Thanks for sending this out. I've been trying to pull > > > >>>together > > > >>>some > > > >>>thoughts on what else is needed for vdsm as a community. I > > > >>>know > > > >>>that > > > >>>for some time downstream has been the driving force for all of > > > >>>the > > > >>>work > > > >>>and now with a community there are challenges in finding our > > > >>>own > > > >>>way. > > > >>> > > > >>>While we certainly don't want to make downstream efforts > > > >>>harder, I > > > >>>think > > > >>>we need to develop and support our own vision for what vdsm > > > >>>can be > > > >>>come, > > > >>>some what independent of downstream and other exploiters. > > > >>> > > > >>>Revisiting the API is definitely a much needed endeavor and I > > > >>>think > > > >>>adding some use-cases or sample applications would be useful > > > >>>in > > > >>>demonstrating whether or not we're evolving the API into > > > >>>something > > > >>>easier to use for applications beyond engine. > > > >>> > > > >>>>We would like to expose a stable, documented, well supported > > > >>>>API. > > > >>>>This gives > > > >>>>us a chance to rethink the VDSM API from the ground up. There > > > >>>>is > > > >>>>already work > > > >>>>in progress of making the internal logic of VDSM separate > > > >>>>enough > > > >>>>from the API > > > >>>>layer so we could continue feature development and bug fixing > > > >>>>while designing > > > >>>>the API of the future. > > > >>>> > > >
Re: [vdsm] [virt-node] VDSM as a general purpose virt host manager
* Adam Litke [2012-06-22 11:35]: > On Thu, Jun 21, 2012 at 12:17:19PM +0300, Dor Laor wrote: > > On 06/19/2012 08:12 PM, Saggi Mizrahi wrote: > > > > > > > > >- Original Message - > > >>From: "Deepak C Shetty" > > >>To: "Ryan Harper" > > >>Cc: "Saggi Mizrahi" , "Anthony Liguori" > > >>, "VDSM Project Development" > > >> > > >>Sent: Tuesday, June 19, 2012 10:58:47 AM > > >>Subject: Re: [vdsm] [virt-node] VDSM as a general purpose virt host > > >>manager > > >> > > >>On 06/19/2012 01:13 AM, Ryan Harper wrote: > > >>>* Saggi Mizrahi [2012-06-18 10:05]: > > >>>>I would like to put on to the table for descussion the growing > > >>>>need for a way > > >>>>to more easily reuse of the functionality of VDSM in order to > > >>>>service projects > > >>>>other than Ovirt-Engine. > > >>>> > > >>>>Originally VDSM was created as a proprietary agent for the sole > > >>>>purpose of > > >>>>serving the then proprietary version of what is known as > > >>>>ovirt-engine. Red Hat, > > >>>>after acquiring the technology, pressed on with it's commitment to > > >>>>open source > > >>>>ideals and released the code. But just releasing code into the > > >>>>wild doesn't > > >>>>build a community or makes a project successful. Further more when > > >>>>building > > >>>>open source software you should aspire to build reusable > > >>>>components instead of > > >>>>monolithic stacks. > > >>>> > > >>>Saggi, > > >>> > > >>>Thanks for sending this out. I've been trying to pull together > > >>>some > > >>>thoughts on what else is needed for vdsm as a community. I know > > >>>that > > >>>for some time downstream has been the driving force for all of the > > >>>work > > >>>and now with a community there are challenges in finding our own > > >>>way. > > >>> > > >>>While we certainly don't want to make downstream efforts harder, I > > >>>think > > >>>we need to develop and support our own vision for what vdsm can be > > >>>come, > > >>>some what independent of downstream and other exploiters. > > >>> > > >>>Revisiting the API is definitely a much needed endeavor and I think > > >>>adding some use-cases or sample applications would be useful in > > >>>demonstrating whether or not we're evolving the API into something > > >>>easier to use for applications beyond engine. > > >>> > > >>>>We would like to expose a stable, documented, well supported API. > > >>>>This gives > > >>>>us a chance to rethink the VDSM API from the ground up. There is > > >>>>already work > > >>>>in progress of making the internal logic of VDSM separate enough > > >>>>from the API > > >>>>layer so we could continue feature development and bug fixing > > >>>>while designing > > >>>>the API of the future. > > >>>> > > >>>>In order to achieve this though we need to do several things: > > >>>> 1. Declare API supportability guidelines > > >>>> 2. Decide on an API transport (e.g. REST, ZMQ, AMQP) > > >>>> 3. Make the API easily consumable (e.g. proper docs, example > > >>>> code, extending > > >>>>the API, etc) > > >>>> 4. Implement the API itself > > >>>I agree with the list, but I'd like to work on the redesign > > >>>discussion so > > >>>that we're not doing all of 1-4 around the existing API that's > > >>>engine-focused. > > >>> > > >>>I'm over due for posting a feature page on vdsm standalone mode, > > >>>and I > > >>>have some other thoughts on various uses. > > >>> > > >>>Some other paths of thought for use-cases I've been mulling over: > > >>> > > >>> - Simplifying using QEMU/KVM > > >>>
Re: [vdsm] [virt-node] VDSM as a general purpose virt host manager
On Thu, Jun 21, 2012 at 12:17:19PM +0300, Dor Laor wrote: > On 06/19/2012 08:12 PM, Saggi Mizrahi wrote: > > > > > >- Original Message - > >>From: "Deepak C Shetty" > >>To: "Ryan Harper" > >>Cc: "Saggi Mizrahi" , "Anthony Liguori" > >>, "VDSM Project Development" > >> > >>Sent: Tuesday, June 19, 2012 10:58:47 AM > >>Subject: Re: [vdsm] [virt-node] VDSM as a general purpose virt host manager > >> > >>On 06/19/2012 01:13 AM, Ryan Harper wrote: > >>>* Saggi Mizrahi [2012-06-18 10:05]: > >>>>I would like to put on to the table for descussion the growing > >>>>need for a way > >>>>to more easily reuse of the functionality of VDSM in order to > >>>>service projects > >>>>other than Ovirt-Engine. > >>>> > >>>>Originally VDSM was created as a proprietary agent for the sole > >>>>purpose of > >>>>serving the then proprietary version of what is known as > >>>>ovirt-engine. Red Hat, > >>>>after acquiring the technology, pressed on with it's commitment to > >>>>open source > >>>>ideals and released the code. But just releasing code into the > >>>>wild doesn't > >>>>build a community or makes a project successful. Further more when > >>>>building > >>>>open source software you should aspire to build reusable > >>>>components instead of > >>>>monolithic stacks. > >>>> > >>>Saggi, > >>> > >>>Thanks for sending this out. I've been trying to pull together > >>>some > >>>thoughts on what else is needed for vdsm as a community. I know > >>>that > >>>for some time downstream has been the driving force for all of the > >>>work > >>>and now with a community there are challenges in finding our own > >>>way. > >>> > >>>While we certainly don't want to make downstream efforts harder, I > >>>think > >>>we need to develop and support our own vision for what vdsm can be > >>>come, > >>>some what independent of downstream and other exploiters. > >>> > >>>Revisiting the API is definitely a much needed endeavor and I think > >>>adding some use-cases or sample applications would be useful in > >>>demonstrating whether or not we're evolving the API into something > >>>easier to use for applications beyond engine. > >>> > >>>>We would like to expose a stable, documented, well supported API. > >>>>This gives > >>>>us a chance to rethink the VDSM API from the ground up. There is > >>>>already work > >>>>in progress of making the internal logic of VDSM separate enough > >>>>from the API > >>>>layer so we could continue feature development and bug fixing > >>>>while designing > >>>>the API of the future. > >>>> > >>>>In order to achieve this though we need to do several things: > >>>> 1. Declare API supportability guidelines > >>>> 2. Decide on an API transport (e.g. REST, ZMQ, AMQP) > >>>> 3. Make the API easily consumable (e.g. proper docs, example > >>>> code, extending > >>>>the API, etc) > >>>> 4. Implement the API itself > >>>I agree with the list, but I'd like to work on the redesign > >>>discussion so > >>>that we're not doing all of 1-4 around the existing API that's > >>>engine-focused. > >>> > >>>I'm over due for posting a feature page on vdsm standalone mode, > >>>and I > >>>have some other thoughts on various uses. > >>> > >>>Some other paths of thought for use-cases I've been mulling over: > >>> > >>> - Simplifying using QEMU/KVM > >>> - consuming qemu via command line > >>> - can we manage/support developers launching qemu > >>> directly > >>> - consuming qemu via libvirt > >>> - can we integrate with systems that are already using > >>> libvirt > >>> > >>> - Addressing issues with libvirt > >>> - are there kvm specific features we c
Re: [vdsm] [virt-node] VDSM as a general purpose virt host manager
On 06/19/2012 08:12 PM, Saggi Mizrahi wrote: - Original Message - From: "Deepak C Shetty" To: "Ryan Harper" Cc: "Saggi Mizrahi" , "Anthony Liguori" , "VDSM Project Development" Sent: Tuesday, June 19, 2012 10:58:47 AM Subject: Re: [vdsm] [virt-node] VDSM as a general purpose virt host manager On 06/19/2012 01:13 AM, Ryan Harper wrote: * Saggi Mizrahi [2012-06-18 10:05]: I would like to put on to the table for descussion the growing need for a way to more easily reuse of the functionality of VDSM in order to service projects other than Ovirt-Engine. Originally VDSM was created as a proprietary agent for the sole purpose of serving the then proprietary version of what is known as ovirt-engine. Red Hat, after acquiring the technology, pressed on with it's commitment to open source ideals and released the code. But just releasing code into the wild doesn't build a community or makes a project successful. Further more when building open source software you should aspire to build reusable components instead of monolithic stacks. Saggi, Thanks for sending this out. I've been trying to pull together some thoughts on what else is needed for vdsm as a community. I know that for some time downstream has been the driving force for all of the work and now with a community there are challenges in finding our own way. While we certainly don't want to make downstream efforts harder, I think we need to develop and support our own vision for what vdsm can be come, some what independent of downstream and other exploiters. Revisiting the API is definitely a much needed endeavor and I think adding some use-cases or sample applications would be useful in demonstrating whether or not we're evolving the API into something easier to use for applications beyond engine. We would like to expose a stable, documented, well supported API. This gives us a chance to rethink the VDSM API from the ground up. There is already work in progress of making the internal logic of VDSM separate enough from the API layer so we could continue feature development and bug fixing while designing the API of the future. In order to achieve this though we need to do several things: 1. Declare API supportability guidelines 2. Decide on an API transport (e.g. REST, ZMQ, AMQP) 3. Make the API easily consumable (e.g. proper docs, example code, extending the API, etc) 4. Implement the API itself I agree with the list, but I'd like to work on the redesign discussion so that we're not doing all of 1-4 around the existing API that's engine-focused. I'm over due for posting a feature page on vdsm standalone mode, and I have some other thoughts on various uses. Some other paths of thought for use-cases I've been mulling over: - Simplifying using QEMU/KVM - consuming qemu via command line - can we manage/support developers launching qemu directly - consuming qemu via libvirt - can we integrate with systems that are already using libvirt - Addressing issues with libvirt - are there kvm specific features we can exploit that libvirt doesn't? - Scale-up/fail-over - can we support a single vdsm node, but allow for building up clusters/groups without bringing in something like ovirt-engine - can we look at decentralized fail-over for reliability without a central mgmt server? - pluggability - can we support an API that allows for third-party plugins to support new features or changes in implementation? Pluggability feature would be nice. Even nicer would be the ability to introspect and figure whats supported by VDSM. For eg: It would be That's a good idea. I don't see folks editing the wiki page [1] w/ their comments. This wiki page doesn't contain all the verb for the api too. It would be pity to lose data suggested on the mailing list, please add it all to the wiki. IIRC there was a length suggestion in the past for REST api, can we start from it or at least the requirement/verbs it offers? Regards, Dor [1] http://ovirt.org/wiki/VDSM_Stable_API_Plan nice to query what plugins/capabilities are supported and accordingly the client can take a decision and/or call the appropriate APIs w/o worrying about ENOTSUPP kind of error. It does becomes blur when we talk about Repository Engines... that was also targetted to provide pluggaibility in managing Images.. how will that co-exist with API level pluggability ? IIUC, StorageProvisioning (via libstoragemgmt) can be one such optional support that can fit as a plug-in nicely, right ? You will have have an introspective verb to get supported storage engines. Without the engine the hosts will n
Re: [vdsm] [virt-node] VDSM as a general purpose virt host manager
On 2012-6-19 22:19, Saggi Mizrahi wrote: - Original Message - From: "Ryan Harper" To: "Saggi Mizrahi" Cc: "Ryan Harper" , "VDSM Project Development" , "Anthony Liguori" Sent: Tuesday, June 19, 2012 9:30:08 AM Subject: Re: [vdsm] [virt-node] VDSM as a general purpose virt host manager * Saggi Mizrahi [2012-06-18 16:09]: Ryan, thanks for commenting. Sadly I feel that your points, though important, are a bit of a digression from the main discussion. Internal architectural changes to VDSM are out of the scope as this should be done on a very tight schedual. I don't think I was suggesting internal architectural changes. I may not yet be familiar enough with to code to understand that modifying the exist API will result in architectural changes. I do worry about what we expect to accomplish here if we have a tight schedule and also include the idea of "general purpose virt host manager". Maybe your opening was too wide for the specific purpose you were intending (your numbered list). If you're strictly focused on something around Fedora18 timeline wise, I would agree that there isn't much runway to make big changes. With that in mind, I'd say we need to add a topic to your list: 5. API versioning and deprecation This is part of the supportability discussion. Please join in if you have something to add. The supportability email was sent to the list as well. Now, VDSM version was highly bound with oVirt Engine version. In order to make oVirt to work, both VDSM and ovirt engine should be synced with the latest binary, no back compatibility yet. If we want to break this binding out, we should classify the level of the VDSM APIs like these: 1) public stable 2) public evolving 3) undocumented volatile And the we should make sure public stable interface should be supported in a very long time as possible as we can. public evolving interfaces should keep the compatibility in the same major release(ie, 4.x.x). undocumented volatile is not recommended to the application and it is the responsibility of the application to take the risk. I believe you've got a number of questions in this space on your other thread so I'll move over there. This is going to be a critical dicussion on how we move forward. Seeing as this is a pretty good list of things that need to be done\discussed in VDSM anyway. I took the liberty of putting them in a wiki page [1] so we don't forget and others can add\comment on the ideas. Thanks. In any case you can feel free to raise those issues on the list separately. Specifically, 3rd party plugins might be very topical with the undergoing gluster integration work. [1] http://www.ovirt.org/wiki/VDSM_Potential_Features - Original Message - From: "Ryan Harper" To: "Saggi Mizrahi" Cc: "VDSM Project Development" , "Anthony Liguori" Sent: Monday, June 18, 2012 3:43:42 PM Subject: Re: [vdsm] [virt-node] VDSM as a general purpose virt host manager * Saggi Mizrahi [2012-06-18 10:05]: I would like to put on to the table for descussion the growing need for a way to more easily reuse of the functionality of VDSM in order to service projects other than Ovirt-Engine. Originally VDSM was created as a proprietary agent for the sole purpose of serving the then proprietary version of what is known as ovirt-engine. Red Hat, after acquiring the technology, pressed on with it's commitment to open source ideals and released the code. But just releasing code into the wild doesn't build a community or makes a project successful. Further more when building open source software you should aspire to build reusable components instead of monolithic stacks. Saggi, Thanks for sending this out. I've been trying to pull together some thoughts on what else is needed for vdsm as a community. I know that for some time downstream has been the driving force for all of the work and now with a community there are challenges in finding our own way. While we certainly don't want to make downstream efforts harder, I think we need to develop and support our own vision for what vdsm can be come, some what independent of downstream and other exploiters. Revisiting the API is definitely a much needed endeavor and I think adding some use-cases or sample applications would be useful in demonstrating whether or not we're evolving the API into something easier to use for applications beyond engine. We would like to expose a stable, documented, well supported API. This gives us a chance to rethink the VDSM API from the ground up. There is already work in progress of making the internal logic of VDSM separate enough from the API layer so we could continue feature development and bug fixing while designing the API of the future. In order to achieve this though we need to do several things: 1. Declare
Re: [vdsm] [virt-node] VDSM as a general purpose virt host manager
- Original Message - > From: "Deepak C Shetty" > To: "Ryan Harper" > Cc: "Saggi Mizrahi" , "Anthony Liguori" > , "VDSM Project Development" > > Sent: Tuesday, June 19, 2012 10:58:47 AM > Subject: Re: [vdsm] [virt-node] VDSM as a general purpose virt host manager > > On 06/19/2012 01:13 AM, Ryan Harper wrote: > > * Saggi Mizrahi [2012-06-18 10:05]: > >> I would like to put on to the table for descussion the growing > >> need for a way > >> to more easily reuse of the functionality of VDSM in order to > >> service projects > >> other than Ovirt-Engine. > >> > >> Originally VDSM was created as a proprietary agent for the sole > >> purpose of > >> serving the then proprietary version of what is known as > >> ovirt-engine. Red Hat, > >> after acquiring the technology, pressed on with it's commitment to > >> open source > >> ideals and released the code. But just releasing code into the > >> wild doesn't > >> build a community or makes a project successful. Further more when > >> building > >> open source software you should aspire to build reusable > >> components instead of > >> monolithic stacks. > >> > > Saggi, > > > > Thanks for sending this out. I've been trying to pull together > > some > > thoughts on what else is needed for vdsm as a community. I know > > that > > for some time downstream has been the driving force for all of the > > work > > and now with a community there are challenges in finding our own > > way. > > > > While we certainly don't want to make downstream efforts harder, I > > think > > we need to develop and support our own vision for what vdsm can be > > come, > > some what independent of downstream and other exploiters. > > > > Revisiting the API is definitely a much needed endeavor and I think > > adding some use-cases or sample applications would be useful in > > demonstrating whether or not we're evolving the API into something > > easier to use for applications beyond engine. > > > >> We would like to expose a stable, documented, well supported API. > >> This gives > >> us a chance to rethink the VDSM API from the ground up. There is > >> already work > >> in progress of making the internal logic of VDSM separate enough > >> from the API > >> layer so we could continue feature development and bug fixing > >> while designing > >> the API of the future. > >> > >> In order to achieve this though we need to do several things: > >> 1. Declare API supportability guidelines > >> 2. Decide on an API transport (e.g. REST, ZMQ, AMQP) > >> 3. Make the API easily consumable (e.g. proper docs, example > >> code, extending > >>the API, etc) > >> 4. Implement the API itself > > I agree with the list, but I'd like to work on the redesign > > discussion so > > that we're not doing all of 1-4 around the existing API that's > > engine-focused. > > > > I'm over due for posting a feature page on vdsm standalone mode, > > and I > > have some other thoughts on various uses. > > > > Some other paths of thought for use-cases I've been mulling over: > > > > - Simplifying using QEMU/KVM > > - consuming qemu via command line > > - can we manage/support developers launching qemu > > directly > > - consuming qemu via libvirt > > - can we integrate with systems that are already using > > libvirt > > > > - Addressing issues with libvirt > > - are there kvm specific features we can exploit that > > libvirt > > doesn't? > > > > - Scale-up/fail-over > > - can we support a single vdsm node, but allow for > > building up > > clusters/groups without bringing in something like > > ovirt-engine > > - can we look at decentralized fail-over for reliability > > without > > a central mgmt server? > > > > - pluggability > > - can we support an API that allows for third-party > > plugins to > > support new features or changes in implementation? > > Pluggability feature would be nice. Even nicer would be the ability > to > introspect and figure whats sup
Re: [vdsm] [virt-node] VDSM as a general purpose virt host manager
On 06/19/2012 01:13 AM, Ryan Harper wrote: * Saggi Mizrahi [2012-06-18 10:05]: I would like to put on to the table for descussion the growing need for a way to more easily reuse of the functionality of VDSM in order to service projects other than Ovirt-Engine. Originally VDSM was created as a proprietary agent for the sole purpose of serving the then proprietary version of what is known as ovirt-engine. Red Hat, after acquiring the technology, pressed on with it's commitment to open source ideals and released the code. But just releasing code into the wild doesn't build a community or makes a project successful. Further more when building open source software you should aspire to build reusable components instead of monolithic stacks. Saggi, Thanks for sending this out. I've been trying to pull together some thoughts on what else is needed for vdsm as a community. I know that for some time downstream has been the driving force for all of the work and now with a community there are challenges in finding our own way. While we certainly don't want to make downstream efforts harder, I think we need to develop and support our own vision for what vdsm can be come, some what independent of downstream and other exploiters. Revisiting the API is definitely a much needed endeavor and I think adding some use-cases or sample applications would be useful in demonstrating whether or not we're evolving the API into something easier to use for applications beyond engine. We would like to expose a stable, documented, well supported API. This gives us a chance to rethink the VDSM API from the ground up. There is already work in progress of making the internal logic of VDSM separate enough from the API layer so we could continue feature development and bug fixing while designing the API of the future. In order to achieve this though we need to do several things: 1. Declare API supportability guidelines 2. Decide on an API transport (e.g. REST, ZMQ, AMQP) 3. Make the API easily consumable (e.g. proper docs, example code, extending the API, etc) 4. Implement the API itself I agree with the list, but I'd like to work on the redesign discussion so that we're not doing all of 1-4 around the existing API that's engine-focused. I'm over due for posting a feature page on vdsm standalone mode, and I have some other thoughts on various uses. Some other paths of thought for use-cases I've been mulling over: - Simplifying using QEMU/KVM - consuming qemu via command line - can we manage/support developers launching qemu directly - consuming qemu via libvirt - can we integrate with systems that are already using libvirt - Addressing issues with libvirt - are there kvm specific features we can exploit that libvirt doesn't? - Scale-up/fail-over - can we support a single vdsm node, but allow for building up clusters/groups without bringing in something like ovirt-engine - can we look at decentralized fail-over for reliability without a central mgmt server? - pluggability - can we support an API that allows for third-party plugins to support new features or changes in implementation? Pluggability feature would be nice. Even nicer would be the ability to introspect and figure whats supported by VDSM. For eg: It would be nice to query what plugins/capabilities are supported and accordingly the client can take a decision and/or call the appropriate APIs w/o worrying about ENOTSUPP kind of error. It does becomes blur when we talk about Repository Engines... that was also targetted to provide pluggaibility in managing Images.. how will that co-exist with API level pluggability ? IIUC, StorageProvisioning (via libstoragemgmt) can be one such optional support that can fit as a plug-in nicely, right ? - kvm tool integration into the API - there are lots of different kvm virt tools for various tasks and they are all stand-alone tools. Can we integrate their use into the node level API. Think libguestfs, virt-install, p2v/v2v tooling. All of these are available, but there isn't an easy way to use this tools through an API. - host management operations - vdsm already does some host level configuration (see networking e.g.) it would be good to think about extending the API to cover other areas of configuration and updates - hardware enumeration - driver level information - storage configuration (we've got a bit of a discussion going around libstoragemgmt here) - performance monitoring/debugging - is the host collecting enough information to do debug/perf analysis - can we support specific configurations of a host that optimize for specific worklo
Re: [vdsm] [virt-node] VDSM as a general purpose virt host manager
- Original Message - > From: "Ryan Harper" > To: "Saggi Mizrahi" > Cc: "Ryan Harper" , "VDSM Project Development" > , "Anthony > Liguori" > Sent: Tuesday, June 19, 2012 9:30:08 AM > Subject: Re: [vdsm] [virt-node] VDSM as a general purpose virt host manager > > * Saggi Mizrahi [2012-06-18 16:09]: > > Ryan, thanks for commenting. > > > > Sadly I feel that your points, though important, are a bit of a > > digression from the main discussion. Internal architectural > > changes > > to VDSM are out of the scope as this should be done on a very tight > > schedual. > > I don't think I was suggesting internal architectural changes. I may > not yet be familiar enough with to code to understand that modifying > the > exist API will result in architectural changes. > > I do worry about what we expect to accomplish here if we have a tight > schedule and also include the idea of "general purpose virt host > manager". Maybe your opening was too wide for the specific purpose > you > were intending (your numbered list). > > If you're strictly focused on something around Fedora18 timeline > wise, I > would agree that there isn't much runway to make big changes. > > With that in mind, I'd say we need to add a topic to your list: > > 5. API versioning and deprecation This is part of the supportability discussion. Please join in if you have something to add. The supportability email was sent to the list as well. > > I believe you've got a number of questions in this space on your > other > thread so I'll move over there. This is going to be a critical > dicussion on how we move forward. > > > > > Seeing as this is a pretty good list of things that need to be > > done\discussed in VDSM anyway. I took the liberty of putting them > > in a > > wiki page [1] so we don't forget and others can add\comment on the > > ideas. > > Thanks. > > > > > In any case you can feel free to raise those issues on the list > > separately. Specifically, 3rd party plugins might be very topical > > with the undergoing gluster integration work. > > > > [1] http://www.ovirt.org/wiki/VDSM_Potential_Features > > > > - Original Message - > > > From: "Ryan Harper" > > > To: "Saggi Mizrahi" > > > Cc: "VDSM Project Development" > > > , "Anthony Liguori" > > > > > > Sent: Monday, June 18, 2012 3:43:42 PM > > > Subject: Re: [vdsm] [virt-node] VDSM as a general purpose virt > > > host manager > > > > > > * Saggi Mizrahi [2012-06-18 10:05]: > > > > I would like to put on to the table for descussion the growing > > > > need > > > > for a way > > > > to more easily reuse of the functionality of VDSM in order to > > > > service projects > > > > other than Ovirt-Engine. > > > > > > > > Originally VDSM was created as a proprietary agent for the sole > > > > purpose of > > > > serving the then proprietary version of what is known as > > > > ovirt-engine. Red Hat, > > > > after acquiring the technology, pressed on with it's commitment > > > > to > > > > open source > > > > ideals and released the code. But just releasing code into the > > > > wild > > > > doesn't > > > > build a community or makes a project successful. Further more > > > > when > > > > building > > > > open source software you should aspire to build reusable > > > > components > > > > instead of > > > > monolithic stacks. > > > > > > > > > > Saggi, > > > > > > Thanks for sending this out. I've been trying to pull together > > > some > > > thoughts on what else is needed for vdsm as a community. I know > > > that > > > for some time downstream has been the driving force for all of > > > the > > > work > > > and now with a community there are challenges in finding our own > > > way. > > > > > > While we certainly don't want to make downstream efforts harder, > > > I > > > think > > > we need to develop and support our own vision for what vdsm can > > > be > > > come, > > > some what independent of downstream and other exploiters. > > > > > > Revisiting the API is definitely a
Re: [vdsm] [virt-node] VDSM as a general purpose virt host manager
* Saggi Mizrahi [2012-06-18 16:09]: > Ryan, thanks for commenting. > > Sadly I feel that your points, though important, are a bit of a > digression from the main discussion. Internal architectural changes > to VDSM are out of the scope as this should be done on a very tight > schedual. I don't think I was suggesting internal architectural changes. I may not yet be familiar enough with to code to understand that modifying the exist API will result in architectural changes. I do worry about what we expect to accomplish here if we have a tight schedule and also include the idea of "general purpose virt host manager". Maybe your opening was too wide for the specific purpose you were intending (your numbered list). If you're strictly focused on something around Fedora18 timeline wise, I would agree that there isn't much runway to make big changes. With that in mind, I'd say we need to add a topic to your list: 5. API versioning and deprication I believe you've got a number of questions in this space on your other thread so I'll move over there. This is going to be a critical dicussion on how we move forward. > > Seeing as this is a pretty good list of things that need to be > done\discussed in VDSM anyway. I took the liberty of putting them in a > wiki page [1] so we don't forget and others can add\comment on the > ideas. Thanks. > > In any case you can feel free to raise those issues on the list separately. > Specifically, 3rd party plugins might be very topical with the undergoing > gluster integration work. > > [1] http://www.ovirt.org/wiki/VDSM_Potential_Features > > - Original Message - > > From: "Ryan Harper" > > To: "Saggi Mizrahi" > > Cc: "VDSM Project Development" , > > "Anthony Liguori" > > Sent: Monday, June 18, 2012 3:43:42 PM > > Subject: Re: [vdsm] [virt-node] VDSM as a general purpose virt host manager > > > > * Saggi Mizrahi [2012-06-18 10:05]: > > > I would like to put on to the table for descussion the growing need > > > for a way > > > to more easily reuse of the functionality of VDSM in order to > > > service projects > > > other than Ovirt-Engine. > > > > > > Originally VDSM was created as a proprietary agent for the sole > > > purpose of > > > serving the then proprietary version of what is known as > > > ovirt-engine. Red Hat, > > > after acquiring the technology, pressed on with it's commitment to > > > open source > > > ideals and released the code. But just releasing code into the wild > > > doesn't > > > build a community or makes a project successful. Further more when > > > building > > > open source software you should aspire to build reusable components > > > instead of > > > monolithic stacks. > > > > > > > Saggi, > > > > Thanks for sending this out. I've been trying to pull together some > > thoughts on what else is needed for vdsm as a community. I know that > > for some time downstream has been the driving force for all of the > > work > > and now with a community there are challenges in finding our own way. > > > > While we certainly don't want to make downstream efforts harder, I > > think > > we need to develop and support our own vision for what vdsm can be > > come, > > some what independent of downstream and other exploiters. > > > > Revisiting the API is definitely a much needed endeavor and I think > > adding some use-cases or sample applications would be useful in > > demonstrating whether or not we're evolving the API into something > > easier to use for applications beyond engine. > > > > > We would like to expose a stable, documented, well supported API. > > > This gives > > > us a chance to rethink the VDSM API from the ground up. There is > > > already work > > > in progress of making the internal logic of VDSM separate enough > > > from the API > > > layer so we could continue feature development and bug fixing while > > > designing > > > the API of the future. > > > > > > In order to achieve this though we need to do several things: > > >1. Declare API supportability guidelines > > >2. Decide on an API transport (e.g. REST, ZMQ, AMQP) > > >3. Make the API easily consumable (e.g. proper docs, example > > >code, extending > > > the API, etc) > > >4. Implement the API itself > > > > I agree with the list, but I'd like t
Re: [vdsm] [virt-node] VDSM as a general purpose virt host manager
On Mon, Jun 18, 2012 at 03:14:15PM -0500, Anthony Liguori wrote: > On 06/18/2012 10:02 AM, Saggi Mizrahi wrote: > >I would like to put on to the table for descussion the growing need for a way > >to more easily reuse of the functionality of VDSM in order to service > >projects > >other than Ovirt-Engine. > > > >Originally VDSM was created as a proprietary agent for the sole purpose of > >serving the then proprietary version of what is known as ovirt-engine. Red > >Hat, > >after acquiring the technology, pressed on with it's commitment to open > >source > >ideals and released the code. But just releasing code into the wild doesn't > >build a community or makes a project successful. Further more when building > >open source software you should aspire to build reusable components instead > >of > >monolithic stacks. > > > >We would like to expose a stable, documented, well supported API. This gives > >us a chance to rethink the VDSM API from the ground up. There is already work > >in progress of making the internal logic of VDSM separate enough from the API > >layer so we could continue feature development and bug fixing while designing > >the API of the future. > > > >In order to achieve this though we need to do several things: > >1. Declare API supportability guidelines > > Adding danpb and DV as I think they can provide good advice here. > > Practically speaking, I think the most important thing to do is > clearly declare what's supported and not supported in more detail > than you probably want to. Realistically, you have to just support > whatever you have. I don't know that designing a "supportable" > interface can be really successful unless you start with that > tomorrow. > > So basically, unless you plan on removing the XML-RPC interface in > the next release, you should plan on supporting it forever... Agreed, whether you like it or not, whatever you ship becomes the supportable interface. If there are warts you don't like then at least document your deprecation procedure. If deprecating something you /must/ provide apps an alternative viable API for the same functionality, or you'll find they just continue using the deprecated stuff & you'll never be able to get rid of it. Also when deprecating, you need to leave a couple of releases where both old & new APIs exist, to give apps time to adjust. > >2. Decide on an API transport (e.g. REST, ZMQ, AMQP) > > We spent so much time trying to find the best transport in QEMU with > the resulting being something I'm ultimately unhappy with. > > The best decision we've made recently on this front is to move to a > schema-based RPC mechanism where the transport code is all > autogenerated. Python has an advantage in that it supports > introspection although a disadvantage in that it's easy to end up > with an ad-hoc interface by relying on passing around dictionaries. My advice here would be to pick a primary/native transport that - Easily consumed by the widest possible set of platforms - Has minimal (near zero) infrastructure requirements - Is easily bindable/translatable to other API transports Once the primary transport is decided, declare that any other transports are to be added as external agents which bridge from one model to the other. Don't try to implement multiple primary transports internally, or you'll fall into the trap where each transports has different subsets of your functionality exposed. If you do pick a message bus based transport (ZMQ/AMQP) as the primary one, then you definitely want a REST or XML-RPC bridge to that to make it more easily consumable > > >3. Make the API easily consumable (e.g. proper docs, example code, > > extending > > the API, etc) > > Documentation is by far the most important thing IMHO. I actually > think that simply taking the existing XML-RPC interface and adding > documentation ought to be the first step even.. Yep, good docs are important. libvirt kinda sucks in this regard, (but somehow we've suceeded despite this flaw). > >4. Implement the API itself > > I think the biggest risk in an effort like this is letting perfect > become the enemy of good. If the goal is to open VDSM up to other > applications, you can start today but just documenting what you have > with plans to deprecate and improve later. Agreed, from my experiance both with libvirt & QMP I'd say don't worry too much about getting APIs wrong. Expect that you'll make mistakes and you'll need to add new APIs later to correct these. I don't mean accept any old API that is proposed - certainly do critical analysis of all new APIs, but don't hold up APIs indefinitely because they are not quite perfect. This really hurt us with QMP for a period of time. > Honestly, worrying about XML-RPC vs. REST vs. AMQP is likely going > to result in a lot of bike shedding and grand plans. Pick REST ;-P Daniel -- |: http://berrange.com -o-http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.o
Re: [vdsm] [virt-node] VDSM as a general purpose virt host manager
Saggi, Thanks for writing these down in wiki pages. http://www.ovirt.org/wiki/VDSM_Potential_Features Another possible feature would be make each node in the clusters to be able to maintain the storage domain master data in parallel without a bottle neck in one SPM node. By this way, the storage domain maintain workload will be dispersed into different nodes. On 2012-6-19 5:09, Saggi Mizrahi wrote: Ryan, thanks for commenting. Sadly I feel that your points, though important, are a bit of a digression from the main discussion. Internal architectural changes to VDSM are out of the scope as this should be done on a very tight schedual. Seeing as this is a pretty good list of things that need to be done\discussed in VDSM anyway. I took the liberty of putting them in a wiki page [1] so we don't forget and others can add\comment on the ideas. In any case you can feel free to raise those issues on the list separately. Specifically, 3rd party plugins might be very topical with the undergoing gluster integration work. [1] http://www.ovirt.org/wiki/VDSM_Potential_Features - Original Message - From: "Ryan Harper" To: "Saggi Mizrahi" Cc: "VDSM Project Development", "Anthony Liguori" Sent: Monday, June 18, 2012 3:43:42 PM Subject: Re: [vdsm] [virt-node] VDSM as a general purpose virt host manager * Saggi Mizrahi [2012-06-18 10:05]: I would like to put on to the table for descussion the growing need for a way to more easily reuse of the functionality of VDSM in order to service projects other than Ovirt-Engine. Originally VDSM was created as a proprietary agent for the sole purpose of serving the then proprietary version of what is known as ovirt-engine. Red Hat, after acquiring the technology, pressed on with it's commitment to open source ideals and released the code. But just releasing code into the wild doesn't build a community or makes a project successful. Further more when building open source software you should aspire to build reusable components instead of monolithic stacks. Saggi, Thanks for sending this out. I've been trying to pull together some thoughts on what else is needed for vdsm as a community. I know that for some time downstream has been the driving force for all of the work and now with a community there are challenges in finding our own way. While we certainly don't want to make downstream efforts harder, I think we need to develop and support our own vision for what vdsm can be come, some what independent of downstream and other exploiters. Revisiting the API is definitely a much needed endeavor and I think adding some use-cases or sample applications would be useful in demonstrating whether or not we're evolving the API into something easier to use for applications beyond engine. We would like to expose a stable, documented, well supported API. This gives us a chance to rethink the VDSM API from the ground up. There is already work in progress of making the internal logic of VDSM separate enough from the API layer so we could continue feature development and bug fixing while designing the API of the future. In order to achieve this though we need to do several things: 1. Declare API supportability guidelines 2. Decide on an API transport (e.g. REST, ZMQ, AMQP) 3. Make the API easily consumable (e.g. proper docs, example code, extending the API, etc) 4. Implement the API itself I agree with the list, but I'd like to work on the redesign discussion so that we're not doing all of 1-4 around the existing API that's engine-focused. I'm over due for posting a feature page on vdsm standalone mode, and I have some other thoughts on various uses. Some other paths of thought for use-cases I've been mulling over: - Simplifying using QEMU/KVM - consuming qemu via command line - can we manage/support developers launching qemu directly - consuming qemu via libvirt - can we integrate with systems that are already using libvirt - Addressing issues with libvirt - are there kvm specific features we can exploit that libvirt doesn't? - Scale-up/fail-over - can we support a single vdsm node, but allow for building up clusters/groups without bringing in something like ovirt-engine - can we look at decentralized fail-over for reliability without a central mgmt server? - pluggability - can we support an API that allows for third-party plugins to support new features or changes in implementation? - kvm tool integration into the API - there are lots of different kvm virt tools for various tasks and they are all stand-alone tools. Can we integrate their use into the node level API. Think libguest
Re: [vdsm] [virt-node] VDSM as a general purpose virt host manager
The decision to declare the current API as supported or not, or opening ourselves to more then one API transport is directly related to how we decide to handle deprecation (if any), API versioning and forward\backward compatibility. If we discover we clear path to evolve the API or support multiple transports and adhere to the (soon to be) agreed upon supportability guidelines we might choose the easy way of supporting the current API. This is why deciding how we are going to support things is the first step in the process. As a side note, having the XML-RPC operational for a version or two until the engine starts to use the new API is a non issue IMHO. - Original Message - > From: "Anthony Liguori" > To: "Saggi Mizrahi" > Cc: "VDSM Project Development" , "Daniel > P. Berrange" , > "Daniel Veillard" > Sent: Monday, June 18, 2012 4:14:15 PM > Subject: Re: [vdsm] [virt-node] VDSM as a general purpose virt host manager > > On 06/18/2012 10:02 AM, Saggi Mizrahi wrote: > > I would like to put on to the table for descussion the growing need > > for a way > > to more easily reuse of the functionality of VDSM in order to > > service projects > > other than Ovirt-Engine. > > > > Originally VDSM was created as a proprietary agent for the sole > > purpose of > > serving the then proprietary version of what is known as > > ovirt-engine. Red Hat, > > after acquiring the technology, pressed on with it's commitment to > > open source > > ideals and released the code. But just releasing code into the wild > > doesn't > > build a community or makes a project successful. Further more when > > building > > open source software you should aspire to build reusable components > > instead of > > monolithic stacks. > > > > We would like to expose a stable, documented, well supported API. > > This gives > > us a chance to rethink the VDSM API from the ground up. There is > > already work > > in progress of making the internal logic of VDSM separate enough > > from the API > > layer so we could continue feature development and bug fixing while > > designing > > the API of the future. > > > > In order to achieve this though we need to do several things: > > 1. Declare API supportability guidelines > > Adding danpb and DV as I think they can provide good advice here. > > Practically speaking, I think the most important thing to do is > clearly declare > what's supported and not supported in more detail than you probably > want to. > Realistically, you have to just support whatever you have. I don't > know that > designing a "supportable" interface can be really successful unless > you start > with that tomorrow. > > So basically, unless you plan on removing the XML-RPC interface in > the next > release, you should plan on supporting it forever... > > > 2. Decide on an API transport (e.g. REST, ZMQ, AMQP) > > We spent so much time trying to find the best transport in QEMU with > the > resulting being something I'm ultimately unhappy with. > > The best decision we've made recently on this front is to move to a > schema-based > RPC mechanism where the transport code is all autogenerated. Python > has an > advantage in that it supports introspection although a disadvantage > in that it's > easy to end up with an ad-hoc interface by relying on passing around > dictionaries. > > > 3. Make the API easily consumable (e.g. proper docs, example > > code, extending > >the API, etc) > > Documentation is by far the most important thing IMHO. I actually > think that > simply taking the existing XML-RPC interface and adding documentation > ought to > be the first step even.. > > > 4. Implement the API itself > > I think the biggest risk in an effort like this is letting perfect > become the > enemy of good. If the goal is to open VDSM up to other applications, > you can > start today but just documenting what you have with plans to > deprecate and > improve later. > > Honestly, worrying about XML-RPC vs. REST vs. AMQP is likely going to > result in > a lot of bike shedding and grand plans. > > Regards, > > Anthony Liguori > > > All of these are dependent on one another and the permutations are > > endless. > > This is why I think we should try and work on each one separately. > > All > > discussions will be done openly on the mailing list and until the > > final version > > comes out nothing is set in stone. > > > >
Re: [vdsm] [virt-node] VDSM as a general purpose virt host manager
Ryan, thanks for commenting. Sadly I feel that your points, though important, are a bit of a digression from the main discussion. Internal architectural changes to VDSM are out of the scope as this should be done on a very tight schedual. Seeing as this is a pretty good list of things that need to be done\discussed in VDSM anyway. I took the liberty of putting them in a wiki page [1] so we don't forget and others can add\comment on the ideas. In any case you can feel free to raise those issues on the list separately. Specifically, 3rd party plugins might be very topical with the undergoing gluster integration work. [1] http://www.ovirt.org/wiki/VDSM_Potential_Features - Original Message - > From: "Ryan Harper" > To: "Saggi Mizrahi" > Cc: "VDSM Project Development" , "Anthony > Liguori" > Sent: Monday, June 18, 2012 3:43:42 PM > Subject: Re: [vdsm] [virt-node] VDSM as a general purpose virt host manager > > * Saggi Mizrahi [2012-06-18 10:05]: > > I would like to put on to the table for descussion the growing need > > for a way > > to more easily reuse of the functionality of VDSM in order to > > service projects > > other than Ovirt-Engine. > > > > Originally VDSM was created as a proprietary agent for the sole > > purpose of > > serving the then proprietary version of what is known as > > ovirt-engine. Red Hat, > > after acquiring the technology, pressed on with it's commitment to > > open source > > ideals and released the code. But just releasing code into the wild > > doesn't > > build a community or makes a project successful. Further more when > > building > > open source software you should aspire to build reusable components > > instead of > > monolithic stacks. > > > > Saggi, > > Thanks for sending this out. I've been trying to pull together some > thoughts on what else is needed for vdsm as a community. I know that > for some time downstream has been the driving force for all of the > work > and now with a community there are challenges in finding our own way. > > While we certainly don't want to make downstream efforts harder, I > think > we need to develop and support our own vision for what vdsm can be > come, > some what independent of downstream and other exploiters. > > Revisiting the API is definitely a much needed endeavor and I think > adding some use-cases or sample applications would be useful in > demonstrating whether or not we're evolving the API into something > easier to use for applications beyond engine. > > > We would like to expose a stable, documented, well supported API. > > This gives > > us a chance to rethink the VDSM API from the ground up. There is > > already work > > in progress of making the internal logic of VDSM separate enough > > from the API > > layer so we could continue feature development and bug fixing while > > designing > > the API of the future. > > > > In order to achieve this though we need to do several things: > >1. Declare API supportability guidelines > >2. Decide on an API transport (e.g. REST, ZMQ, AMQP) > >3. Make the API easily consumable (e.g. proper docs, example > >code, extending > > the API, etc) > >4. Implement the API itself > > I agree with the list, but I'd like to work on the redesign > discussion so > that we're not doing all of 1-4 around the existing API that's > engine-focused. > > I'm over due for posting a feature page on vdsm standalone mode, and > I > have some other thoughts on various uses. > > Some other paths of thought for use-cases I've been mulling over: > > - Simplifying using QEMU/KVM > - consuming qemu via command line > - can we manage/support developers launching qemu > directly > - consuming qemu via libvirt > - can we integrate with systems that are already using > libvirt > > - Addressing issues with libvirt > - are there kvm specific features we can exploit that libvirt > doesn't? > > - Scale-up/fail-over > - can we support a single vdsm node, but allow for building > up > clusters/groups without bringing in something like > ovirt-engine > - can we look at decentralized fail-over for reliability > without > a central mgmt server? > > - pluggability > - can we support an API that allows for third-party plugins > to > support new features or changes in impl
Re: [vdsm] [virt-node] VDSM as a general purpose virt host manager
On 06/18/2012 10:02 AM, Saggi Mizrahi wrote: I would like to put on to the table for descussion the growing need for a way to more easily reuse of the functionality of VDSM in order to service projects other than Ovirt-Engine. Originally VDSM was created as a proprietary agent for the sole purpose of serving the then proprietary version of what is known as ovirt-engine. Red Hat, after acquiring the technology, pressed on with it's commitment to open source ideals and released the code. But just releasing code into the wild doesn't build a community or makes a project successful. Further more when building open source software you should aspire to build reusable components instead of monolithic stacks. We would like to expose a stable, documented, well supported API. This gives us a chance to rethink the VDSM API from the ground up. There is already work in progress of making the internal logic of VDSM separate enough from the API layer so we could continue feature development and bug fixing while designing the API of the future. In order to achieve this though we need to do several things: 1. Declare API supportability guidelines Adding danpb and DV as I think they can provide good advice here. Practically speaking, I think the most important thing to do is clearly declare what's supported and not supported in more detail than you probably want to. Realistically, you have to just support whatever you have. I don't know that designing a "supportable" interface can be really successful unless you start with that tomorrow. So basically, unless you plan on removing the XML-RPC interface in the next release, you should plan on supporting it forever... 2. Decide on an API transport (e.g. REST, ZMQ, AMQP) We spent so much time trying to find the best transport in QEMU with the resulting being something I'm ultimately unhappy with. The best decision we've made recently on this front is to move to a schema-based RPC mechanism where the transport code is all autogenerated. Python has an advantage in that it supports introspection although a disadvantage in that it's easy to end up with an ad-hoc interface by relying on passing around dictionaries. 3. Make the API easily consumable (e.g. proper docs, example code, extending the API, etc) Documentation is by far the most important thing IMHO. I actually think that simply taking the existing XML-RPC interface and adding documentation ought to be the first step even.. 4. Implement the API itself I think the biggest risk in an effort like this is letting perfect become the enemy of good. If the goal is to open VDSM up to other applications, you can start today but just documenting what you have with plans to deprecate and improve later. Honestly, worrying about XML-RPC vs. REST vs. AMQP is likely going to result in a lot of bike shedding and grand plans. Regards, Anthony Liguori All of these are dependent on one another and the permutations are endless. This is why I think we should try and work on each one separately. All discussions will be done openly on the mailing list and until the final version comes out nothing is set in stone. If you think you have anything to contribute to this process, please do so either by commenting on the discussions or by sending code/docs/whatever patches. Once the API solidifies it will be quite difficult to change fundamental things, so speak now or forever hold your peace. Note that this is just an introductory email. There will be a quick follow up email to kick start the discussions. ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] [virt-node] VDSM as a general purpose virt host manager
* Saggi Mizrahi [2012-06-18 10:05]: > I would like to put on to the table for descussion the growing need for a way > to more easily reuse of the functionality of VDSM in order to service projects > other than Ovirt-Engine. > > Originally VDSM was created as a proprietary agent for the sole purpose of > serving the then proprietary version of what is known as ovirt-engine. Red > Hat, > after acquiring the technology, pressed on with it's commitment to open source > ideals and released the code. But just releasing code into the wild doesn't > build a community or makes a project successful. Further more when building > open source software you should aspire to build reusable components instead of > monolithic stacks. > Saggi, Thanks for sending this out. I've been trying to pull together some thoughts on what else is needed for vdsm as a community. I know that for some time downstream has been the driving force for all of the work and now with a community there are challenges in finding our own way. While we certainly don't want to make downstream efforts harder, I think we need to develop and support our own vision for what vdsm can be come, some what independent of downstream and other exploiters. Revisiting the API is definitely a much needed endeavor and I think adding some use-cases or sample applications would be useful in demonstrating whether or not we're evolving the API into something easier to use for applications beyond engine. > We would like to expose a stable, documented, well supported API. This gives > us a chance to rethink the VDSM API from the ground up. There is already work > in progress of making the internal logic of VDSM separate enough from the API > layer so we could continue feature development and bug fixing while designing > the API of the future. > > In order to achieve this though we need to do several things: >1. Declare API supportability guidelines >2. Decide on an API transport (e.g. REST, ZMQ, AMQP) >3. Make the API easily consumable (e.g. proper docs, example code, > extending > the API, etc) >4. Implement the API itself I agree with the list, but I'd like to work on the redesign discussion so that we're not doing all of 1-4 around the existing API that's engine-focused. I'm over due for posting a feature page on vdsm standalone mode, and I have some other thoughts on various uses. Some other paths of thought for use-cases I've been mulling over: - Simplifying using QEMU/KVM - consuming qemu via command line - can we manage/support developers launching qemu directly - consuming qemu via libvirt - can we integrate with systems that are already using libvirt - Addressing issues with libvirt - are there kvm specific features we can exploit that libvirt doesn't? - Scale-up/fail-over - can we support a single vdsm node, but allow for building up clusters/groups without bringing in something like ovirt-engine - can we look at decentralized fail-over for reliability without a central mgmt server? - pluggability - can we support an API that allows for third-party plugins to support new features or changes in implementation? - kvm tool integration into the API - there are lots of different kvm virt tools for various tasks and they are all stand-alone tools. Can we integrate their use into the node level API. Think libguestfs, virt-install, p2v/v2v tooling. All of these are available, but there isn't an easy way to use this tools through an API. - host management operations - vdsm already does some host level configuration (see networking e.g.) it would be good to think about extending the API to cover other areas of configuration and updates - hardware enumeration - driver level information - storage configuration (we've got a bit of a discussion going around libstoragemgmt here) - performance monitoring/debugging - is the host collecting enough information to do debug/perf analysis - can we support specific configurations of a host that optimize for specific workloads - and can we do this in the API such that third-parties can supply and maintain specific workload configurations > > All of these are dependent on one another and the permutations are endless. > This is why I think we should try and work on each one separately. All > discussions will be done openly on the mailing list and until the final > version > comes out nothing is set in stone. > > If you think you have anything to contribute to this process, please do so > either by commenting on the discussions or by sending code/docs/whatever > patches. Once the API solidifies it will be quite difficult to change > fundamental
Re: [vdsm] [virt-node] VDSM as a general purpose virt host manager
To the question of "What blocks us using the current VDSM API?". The main issue is supportability, This is also why it's the first point of discussion. The current API has no supportability guidelines and there is no way we could support it for the long run. Further more the current API, apart from being outdated is highly engine-specific. A lot of the decisions are related to VDSM having to slow down development to accommodate the slower pace of movement of the giant the is the ovirt-engine. This means, for example having confusing verbs and argument names (eg. destroy, and iscsi portals). Having redundant steps in the setup of things (eg. storage domain creation). Arbitrary limitations (eg. storage pools, iso\export domains), etc. In order to give a well supported API, we need to think about what we expose and how we expose it. Every verb should be thoroughly examined. This was not the case when the original API was created because, as I already noted, it was built to a case where the supported API is at the Engine level and not the VDSM level. This made creating\removing verbs a lot quicker and less "expensive", accepting things we knew were not ideal with the knowledge we can change them the next version. This cannot be the case with a supported public API. - Original Message - > From: "Deepak C Shetty" > To: "Saggi Mizrahi" > Cc: "VDSM Project Development" , "Anthony > Liguori" > Sent: Monday, June 18, 2012 1:35:21 PM > Subject: Re: [vdsm] [virt-node] VDSM as a general purpose virt host manager > > On 06/18/2012 08:32 PM, Saggi Mizrahi wrote: > > I would like to put on to the table for descussion the growing need > > for a way > > to more easily reuse of the functionality of VDSM in order to > > service projects > > other than Ovirt-Engine. > > > > Originally VDSM was created as a proprietary agent for the sole > > purpose of > > serving the then proprietary version of what is known as > > ovirt-engine. Red Hat, > > after acquiring the technology, pressed on with it's commitment to > > open source > > ideals and released the code. But just releasing code into the wild > > doesn't > > build a community or makes a project successful. Further more when > > building > > open source software you should aspire to build reusable components > > instead of > > monolithic stacks. > > > > Can you list issues that block tools (other than ovirt-engine) in > using > VDSM ? > That will help provide more clarity and scope of work described here. > > I understand the lack of REST API, which is where Adam's work comes > in. > With REST API support for vdsm, other tools can integrate upwardly > with > VDSM and exploit it. What else ? How does the current API layer > design/implementation inhibit tools other than ovirt-engine to use > VDSM ? > > > We would like to expose a stable, documented, well supported API. > > This gives > > us a chance to rethink the VDSM API from the ground up. There is > > already work > > in progress of making the internal logic of VDSM separate enough > > from the API > > layer so we could continue feature development and bug fixing while > > designing > > the API of the future. > > > > In order to achieve this though we need to do several things: > > 1. Declare API supportability guidelines > > 2. Decide on an API transport (e.g. REST, ZMQ, AMQP) > > 3. Make the API easily consumable (e.g. proper docs, example > > code, extending > >the API, etc) > > 4. Implement the API itself > > > > All of these are dependent on one another and the permutations are > > endless. > > This is why I think we should try and work on each one separately. > > All > > discussions will be done openly on the mailing list and until the > > final version > > comes out nothing is set in stone. > > > > If you think you have anything to contribute to this process, > > please do so > > either by commenting on the discussions or by sending > > code/docs/whatever > > patches. Once the API solidifies it will be quite difficult to > > change > > fundamental things, so speak now or forever hold your peace. Note > > that this is > > just an introductory email. There will be a quick follow up email > > to kick start > > the discussions. > > ___ > > vdsm-devel mailing list > > vdsm-devel@lists.fedorahosted.org > > https://fedorahosted.org/mailman/listinfo/vdsm-devel > > ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel
Re: [vdsm] [virt-node] VDSM as a general purpose virt host manager
On 06/18/2012 08:32 PM, Saggi Mizrahi wrote: I would like to put on to the table for descussion the growing need for a way to more easily reuse of the functionality of VDSM in order to service projects other than Ovirt-Engine. Originally VDSM was created as a proprietary agent for the sole purpose of serving the then proprietary version of what is known as ovirt-engine. Red Hat, after acquiring the technology, pressed on with it's commitment to open source ideals and released the code. But just releasing code into the wild doesn't build a community or makes a project successful. Further more when building open source software you should aspire to build reusable components instead of monolithic stacks. Can you list issues that block tools (other than ovirt-engine) in using VDSM ? That will help provide more clarity and scope of work described here. I understand the lack of REST API, which is where Adam's work comes in. With REST API support for vdsm, other tools can integrate upwardly with VDSM and exploit it. What else ? How does the current API layer design/implementation inhibit tools other than ovirt-engine to use VDSM ? We would like to expose a stable, documented, well supported API. This gives us a chance to rethink the VDSM API from the ground up. There is already work in progress of making the internal logic of VDSM separate enough from the API layer so we could continue feature development and bug fixing while designing the API of the future. In order to achieve this though we need to do several things: 1. Declare API supportability guidelines 2. Decide on an API transport (e.g. REST, ZMQ, AMQP) 3. Make the API easily consumable (e.g. proper docs, example code, extending the API, etc) 4. Implement the API itself All of these are dependent on one another and the permutations are endless. This is why I think we should try and work on each one separately. All discussions will be done openly on the mailing list and until the final version comes out nothing is set in stone. If you think you have anything to contribute to this process, please do so either by commenting on the discussions or by sending code/docs/whatever patches. Once the API solidifies it will be quite difficult to change fundamental things, so speak now or forever hold your peace. Note that this is just an introductory email. There will be a quick follow up email to kick start the discussions. ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel
[vdsm] [virt-node] VDSM as a general purpose virt host manager
I would like to put on to the table for descussion the growing need for a way to more easily reuse of the functionality of VDSM in order to service projects other than Ovirt-Engine. Originally VDSM was created as a proprietary agent for the sole purpose of serving the then proprietary version of what is known as ovirt-engine. Red Hat, after acquiring the technology, pressed on with it's commitment to open source ideals and released the code. But just releasing code into the wild doesn't build a community or makes a project successful. Further more when building open source software you should aspire to build reusable components instead of monolithic stacks. We would like to expose a stable, documented, well supported API. This gives us a chance to rethink the VDSM API from the ground up. There is already work in progress of making the internal logic of VDSM separate enough from the API layer so we could continue feature development and bug fixing while designing the API of the future. In order to achieve this though we need to do several things: 1. Declare API supportability guidelines 2. Decide on an API transport (e.g. REST, ZMQ, AMQP) 3. Make the API easily consumable (e.g. proper docs, example code, extending the API, etc) 4. Implement the API itself All of these are dependent on one another and the permutations are endless. This is why I think we should try and work on each one separately. All discussions will be done openly on the mailing list and until the final version comes out nothing is set in stone. If you think you have anything to contribute to this process, please do so either by commenting on the discussions or by sending code/docs/whatever patches. Once the API solidifies it will be quite difficult to change fundamental things, so speak now or forever hold your peace. Note that this is just an introductory email. There will be a quick follow up email to kick start the discussions. ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel
