Re: Proposed next-generation vdsm API
- Original Message - > On Thu, Dec 01, 2011 at 02:02:27AM -0500, Ayal Baron wrote: > > > > > > - Original Message - > > > > > > > > > > -Original Message- > > > > From: Adam Litke [mailto:a...@us.ibm.com] > > > > Sent: Thursday, December 01, 2011 0:41 AM > > > > To: vdsm-devel@lists.fedorahosted.org; engine-de...@ovirt.org > > > > Cc: Daniel P. Berrange; Chris Wright; Dan Kenigsberg; Itamar > > > > Heim > > > > Subject: Proposed next-generation vdsm API > > > > > > > > Recently we've had some very productive discussions concerning > > > > the > > > > VDSM > > > API. I > > > > want to attempt to refocus the discussion around an emerging > > > > proposal > > > and see if > > > > we can agree on a sensible path forward. > > > > > > > > Based on the discussion, I have identified the following > > > > requirements > > > that > > > > a new API for vdsm should have: > > > > > > > > 1.) Single API that can be consumed by ovirt-engine and ISVs > > > > - We don't want to maintain multiple parallel APIs > > > > - To develop a vendor ecosystem, we must have a robust > > > > external > > > > API to > > > >vdsm > > > > > > > > 2.) Full vdsm capabilities are exposed without requiring > > > > ovirt-engine > > > > - ovirt components should be modular and independently useful > > > > - Some deployments might want to manage nodes without > > > > ovirt-engine > > > > > > > > 3.) Standardized protocol with low overhead > > > > - Required for widespread adoption > > > > > > > > 4.) Support for asynchronous tasks and events > > > > - Needed by ovirt-engine and other consumers > > > > > > > > Based on these requirements, the following proposal has started > > > > to > > > emerge: > > > > > > > > Create a REST API that will provide all of the functionality > > > > that > > > > is > > > currently > > > > available via the xmlrpc interface (with the goal of > > > > deprecating > > > > xmlrpc > > > once it > > > > becomes mature enough). To support advanced clustering > > > > features > > > > that > > > > ovirt-engine is planning, we'll write an QMF broker that can > > > > proxy > > > > the > > > REST API > > > > onto a message bus. ovirt-engine will interact with vdsm > > > > exclusively > > > over this > > > > bus but the REST API will be the principle API and the entry > > > > point > > > > for > > > ISV apps. > > > > A REST API provides a light-weight and standard way to access > > > > all > > > > of the > > > vdsm > > > > functionality. > > > > > > > > The REST API will handle events by exposing a new 'events' > > > > collection at > > > the api > > > > root. REST users will use some sort of polling to collect > > > > these > > > > events. > > > The > > > > details of this interface are being worked on. Several ways > > > > for > > > minimizing the > > > > impact of polling have been discussed. The QMF broker can > > > > expose a > > > > publish/subscribe model for events as appropriate. > > > > > > > > Is this model an acceptable way to improve the vdsm API? I > > > > would > > > > like > > > to hear > > > > the opinions of ovirt-engine developers, vdsm developers, and > > > > other > > > > stakeholders. Thanks for providing feedback on this proposal! > > > > > > Why things non native to REST and wrap it in QMF, rather than do > > > the > > > reverse? > > > Or just to them in parallel, since it sounds like both are going > > > to > > > be > > > first class citizens? > > > > This was more my understanding from our discussion on IRC > > yesterday. > > I'm afraid I did not follow that discussion to your conclusions... > > > REST API - everything that is relevant for single node management > > QMF - same API as above + multi-node relevant API calls. I don't > > see any > > reason for doing weird things over REST to support the latter. > > A QMF broker runs on the vdsm host and talks to the REST API. It > connects to a > bus and exposes an API to ovirt-engine on this bus using a vdsm-base > schema. > ovirt-engine wants additional clustering functionality. This API > should be > implemented completely between the QMF broker and ovirt-engine using > a separate > vdsm-cluster schema. > > > In fact, I don't even see any real reason for going through the > > REST API when > > using QMF. > > Because we want to avoid the proliferation of APIs. I would prefer a > mostly > vertical chain of API components to a vdsm with several independent > APIs (which > are sure to diverge or be neglected by individual > developers/patches). > > > If you take a look at today's API you will see that there is > > nothing there > > that limits it to XML-RPC and we could easily expose all the calls > > using REST > > or anything else. In python, exposing a new verb in the various > > APIs can be > > automatic so this would require very little maintenance. Any > > multi-node or > > transport specific calls can be decorated as such and would be > > automatically > > ignored/picked up by t
Re: Proposed next-generation vdsm API
On Thu, Dec 01, 2011 at 02:02:27AM -0500, Ayal Baron wrote: > > > - Original Message - > > > > > > > -Original Message- > > > From: Adam Litke [mailto:a...@us.ibm.com] > > > Sent: Thursday, December 01, 2011 0:41 AM > > > To: vdsm-devel@lists.fedorahosted.org; engine-de...@ovirt.org > > > Cc: Daniel P. Berrange; Chris Wright; Dan Kenigsberg; Itamar Heim > > > Subject: Proposed next-generation vdsm API > > > > > > Recently we've had some very productive discussions concerning the > > > VDSM > > API. I > > > want to attempt to refocus the discussion around an emerging > > > proposal > > and see if > > > we can agree on a sensible path forward. > > > > > > Based on the discussion, I have identified the following > > > requirements > > that > > > a new API for vdsm should have: > > > > > > 1.) Single API that can be consumed by ovirt-engine and ISVs > > > - We don't want to maintain multiple parallel APIs > > > - To develop a vendor ecosystem, we must have a robust external > > > API to > > >vdsm > > > > > > 2.) Full vdsm capabilities are exposed without requiring > > > ovirt-engine > > > - ovirt components should be modular and independently useful > > > - Some deployments might want to manage nodes without ovirt-engine > > > > > > 3.) Standardized protocol with low overhead > > > - Required for widespread adoption > > > > > > 4.) Support for asynchronous tasks and events > > > - Needed by ovirt-engine and other consumers > > > > > > Based on these requirements, the following proposal has started to > > emerge: > > > > > > Create a REST API that will provide all of the functionality that > > > is > > currently > > > available via the xmlrpc interface (with the goal of deprecating > > > xmlrpc > > once it > > > becomes mature enough). To support advanced clustering features > > > that > > > ovirt-engine is planning, we'll write an QMF broker that can proxy > > > the > > REST API > > > onto a message bus. ovirt-engine will interact with vdsm > > > exclusively > > over this > > > bus but the REST API will be the principle API and the entry point > > > for > > ISV apps. > > > A REST API provides a light-weight and standard way to access all > > > of the > > vdsm > > > functionality. > > > > > > The REST API will handle events by exposing a new 'events' > > > collection at > > the api > > > root. REST users will use some sort of polling to collect these > > > events. > > The > > > details of this interface are being worked on. Several ways for > > minimizing the > > > impact of polling have been discussed. The QMF broker can expose a > > > publish/subscribe model for events as appropriate. > > > > > > Is this model an acceptable way to improve the vdsm API? I would > > > like > > to hear > > > the opinions of ovirt-engine developers, vdsm developers, and other > > > stakeholders. Thanks for providing feedback on this proposal! > > > > Why things non native to REST and wrap it in QMF, rather than do the > > reverse? > > Or just to them in parallel, since it sounds like both are going to > > be > > first class citizens? > > This was more my understanding from our discussion on IRC yesterday. I'm afraid I did not follow that discussion to your conclusions... > REST API - everything that is relevant for single node management > QMF - same API as above + multi-node relevant API calls. I don't see any > reason for doing weird things over REST to support the latter. A QMF broker runs on the vdsm host and talks to the REST API. It connects to a bus and exposes an API to ovirt-engine on this bus using a vdsm-base schema. ovirt-engine wants additional clustering functionality. This API should be implemented completely between the QMF broker and ovirt-engine using a separate vdsm-cluster schema. > In fact, I don't even see any real reason for going through the REST API when > using QMF. Because we want to avoid the proliferation of APIs. I would prefer a mostly vertical chain of API components to a vdsm with several independent APIs (which are sure to diverge or be neglected by individual developers/patches). > If you take a look at today's API you will see that there is nothing there > that limits it to XML-RPC and we could easily expose all the calls using REST > or anything else. In python, exposing a new verb in the various APIs can be > automatic so this would require very little maintenance. Any multi-node or > transport specific calls can be decorated as such and would be automatically > ignored/picked up by the relevant API layer. This way, we could also easily > enable using different bus protocols assuming a customer already has a > deployment as was suggested yesterday. I don't think this will be as automatic as you suggest in practice. It sounds like it will increase code complexity, obfuscation, and maintenance burden. -- Adam Litke IBM Linux Technology Center ___ vdsm-devel mailing list vdsm-devel@l
Re: Proposed next-generation vdsm API
On 11/30/2011 05:40 PM, Adam Litke wrote: > Recently we've had some very productive discussions concerning the VDSM API. > I > want to attempt to refocus the discussion around an emerging proposal and see > if > we can agree on a sensible path forward. > > Based on the discussion, I have identified the following requirements that > a new API for vdsm should have: > > 1.) Single API that can be consumed by ovirt-engine and ISVs > - We don't want to maintain multiple parallel APIs > - To develop a vendor ecosystem, we must have a robust external API to > vdsm > > 2.) Full vdsm capabilities are exposed without requiring ovirt-engine > - ovirt components should be modular and independently useful > - Some deployments might want to manage nodes without ovirt-engine > > 3.) Standardized protocol with low overhead > - Required for widespread adoption > > 4.) Support for asynchronous tasks and events > - Needed by ovirt-engine and other consumers > > Based on these requirements, the following proposal has started to emerge: > > Create a REST API that will provide all of the functionality that is currently > available via the xmlrpc interface I just want to stress that it WILL deprecate and significantly change some aspects of the API most obvious are the task aspects of the API. Even though the commands might be the same the current async\sync task semantics might change to better accommodate the QMF wrapper and other users. I don't think it's needed to go into details now but I am stressing that there WILL be API calls the will not migrate or significantly change in the REST API. > (with the goal of deprecating xmlrpc once it > becomes mature enough). To support advanced clustering features that > ovirt-engine is planning, we'll write an QMF broker that can proxy the REST > API > onto a message bus. ovirt-engine will interact with vdsm exclusively over > this > bus but the REST API will be the principle API and the entry point for ISV > apps. > A REST API provides a light-weight and standard way to access all of the vdsm > functionality. > > The REST API will handle events by exposing a new 'events' collection at the > api > root. REST users will use some sort of polling to collect these events. The > details of this interface are being worked on. Several ways for minimizing > the > impact of polling have been discussed. The QMF broker can expose a > publish/subscribe model for events as appropriate. > > Is this model an acceptable way to improve the vdsm API? I would like to hear > the opinions of ovirt-engine developers, vdsm developers, and other > stakeholders. Thanks for providing feedback on this proposal! > ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel
Re: Proposed next-generation vdsm API
On Wed, Nov 30, 2011 at 11:32:55PM -0500, Itamar Heim wrote: > > > > -Original Message- > > From: Adam Litke [mailto:a...@us.ibm.com] > > Sent: Thursday, December 01, 2011 0:41 AM > > To: vdsm-devel@lists.fedorahosted.org; engine-de...@ovirt.org > > Cc: Daniel P. Berrange; Chris Wright; Dan Kenigsberg; Itamar Heim > > Subject: Proposed next-generation vdsm API > > > > Recently we've had some very productive discussions concerning the VDSM > API. I > > want to attempt to refocus the discussion around an emerging proposal > and see if > > we can agree on a sensible path forward. > > > > Based on the discussion, I have identified the following requirements > that > > a new API for vdsm should have: > > > > 1.) Single API that can be consumed by ovirt-engine and ISVs > > - We don't want to maintain multiple parallel APIs > > - To develop a vendor ecosystem, we must have a robust external API to > >vdsm > > > > 2.) Full vdsm capabilities are exposed without requiring ovirt-engine > > - ovirt components should be modular and independently useful > > - Some deployments might want to manage nodes without ovirt-engine > > > > 3.) Standardized protocol with low overhead > > - Required for widespread adoption > > > > 4.) Support for asynchronous tasks and events > > - Needed by ovirt-engine and other consumers > > > > Based on these requirements, the following proposal has started to > emerge: > > > > Create a REST API that will provide all of the functionality that is > currently > > available via the xmlrpc interface (with the goal of deprecating xmlrpc > once it > > becomes mature enough). To support advanced clustering features that > > ovirt-engine is planning, we'll write an QMF broker that can proxy the > REST API > > onto a message bus. ovirt-engine will interact with vdsm exclusively > over this > > bus but the REST API will be the principle API and the entry point for > ISV apps. > > A REST API provides a light-weight and standard way to access all of the > vdsm > > functionality. > > > > The REST API will handle events by exposing a new 'events' collection at > the api > > root. REST users will use some sort of polling to collect these events. > The > > details of this interface are being worked on. Several ways for > minimizing the > > impact of polling have been discussed. The QMF broker can expose a > > publish/subscribe model for events as appropriate. > > > > Is this model an acceptable way to improve the vdsm API? I would like > to hear > > the opinions of ovirt-engine developers, vdsm developers, and other > > stakeholders. Thanks for providing feedback on this proposal! > > Why things non native to REST and wrap it in QMF, rather than do the > reverse? > Or just to them in parallel, since it sounds like both are going to be > first class citizens? Parallel APIs mean dual maintenance. There will be inherent incompatibilities as each API would naturally have small differences. The reason for beginning with REST because of its low overhead and simplicity. Users of the REST API would not need to concern themselves with QMF at all but if that extra set of features is desired it can be easily added. -- Adam Litke IBM Linux Technology Center ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel
Re: Proposed next-generation vdsm API
- Original Message - > > > > -Original Message- > > From: Adam Litke [mailto:a...@us.ibm.com] > > Sent: Thursday, December 01, 2011 0:41 AM > > To: vdsm-devel@lists.fedorahosted.org; engine-de...@ovirt.org > > Cc: Daniel P. Berrange; Chris Wright; Dan Kenigsberg; Itamar Heim > > Subject: Proposed next-generation vdsm API > > > > Recently we've had some very productive discussions concerning the > > VDSM > API. I > > want to attempt to refocus the discussion around an emerging > > proposal > and see if > > we can agree on a sensible path forward. > > > > Based on the discussion, I have identified the following > > requirements > that > > a new API for vdsm should have: > > > > 1.) Single API that can be consumed by ovirt-engine and ISVs > > - We don't want to maintain multiple parallel APIs > > - To develop a vendor ecosystem, we must have a robust external > > API to > >vdsm > > > > 2.) Full vdsm capabilities are exposed without requiring > > ovirt-engine > > - ovirt components should be modular and independently useful > > - Some deployments might want to manage nodes without ovirt-engine > > > > 3.) Standardized protocol with low overhead > > - Required for widespread adoption > > > > 4.) Support for asynchronous tasks and events > > - Needed by ovirt-engine and other consumers > > > > Based on these requirements, the following proposal has started to > emerge: > > > > Create a REST API that will provide all of the functionality that > > is > currently > > available via the xmlrpc interface (with the goal of deprecating > > xmlrpc > once it > > becomes mature enough). To support advanced clustering features > > that > > ovirt-engine is planning, we'll write an QMF broker that can proxy > > the > REST API > > onto a message bus. ovirt-engine will interact with vdsm > > exclusively > over this > > bus but the REST API will be the principle API and the entry point > > for > ISV apps. > > A REST API provides a light-weight and standard way to access all > > of the > vdsm > > functionality. > > > > The REST API will handle events by exposing a new 'events' > > collection at > the api > > root. REST users will use some sort of polling to collect these > > events. > The > > details of this interface are being worked on. Several ways for > minimizing the > > impact of polling have been discussed. The QMF broker can expose a > > publish/subscribe model for events as appropriate. > > > > Is this model an acceptable way to improve the vdsm API? I would > > like > to hear > > the opinions of ovirt-engine developers, vdsm developers, and other > > stakeholders. Thanks for providing feedback on this proposal! > > Why things non native to REST and wrap it in QMF, rather than do the > reverse? > Or just to them in parallel, since it sounds like both are going to > be > first class citizens? This was more my understanding from our discussion on IRC yesterday. REST API - everything that is relevant for single node management QMF - same API as above + multi-node relevant API calls. I don't see any reason for doing weird things over REST to support the latter. In fact, I don't even see any real reason for going through the REST API when using QMF. If you take a look at today's API you will see that there is nothing there that limits it to XML-RPC and we could easily expose all the calls using REST or anything else. In python, exposing a new verb in the various APIs can be automatic so this would require very little maintenance. Any multi-node or transport specific calls can be decorated as such and would be automatically ignored/picked up by the relevant API layer. This way, we could also easily enable using different bus protocols assuming a customer already has a deployment as was suggested yesterday. > ___ > vdsm-devel mailing list > vdsm-devel@lists.fedorahosted.org > https://fedorahosted.org/mailman/listinfo/vdsm-devel > ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel
RE: Proposed next-generation vdsm API
> -Original Message- > From: Adam Litke [mailto:a...@us.ibm.com] > Sent: Thursday, December 01, 2011 0:41 AM > To: vdsm-devel@lists.fedorahosted.org; engine-de...@ovirt.org > Cc: Daniel P. Berrange; Chris Wright; Dan Kenigsberg; Itamar Heim > Subject: Proposed next-generation vdsm API > > Recently we've had some very productive discussions concerning the VDSM API. I > want to attempt to refocus the discussion around an emerging proposal and see if > we can agree on a sensible path forward. > > Based on the discussion, I have identified the following requirements that > a new API for vdsm should have: > > 1.) Single API that can be consumed by ovirt-engine and ISVs > - We don't want to maintain multiple parallel APIs > - To develop a vendor ecosystem, we must have a robust external API to >vdsm > > 2.) Full vdsm capabilities are exposed without requiring ovirt-engine > - ovirt components should be modular and independently useful > - Some deployments might want to manage nodes without ovirt-engine > > 3.) Standardized protocol with low overhead > - Required for widespread adoption > > 4.) Support for asynchronous tasks and events > - Needed by ovirt-engine and other consumers > > Based on these requirements, the following proposal has started to emerge: > > Create a REST API that will provide all of the functionality that is currently > available via the xmlrpc interface (with the goal of deprecating xmlrpc once it > becomes mature enough). To support advanced clustering features that > ovirt-engine is planning, we'll write an QMF broker that can proxy the REST API > onto a message bus. ovirt-engine will interact with vdsm exclusively over this > bus but the REST API will be the principle API and the entry point for ISV apps. > A REST API provides a light-weight and standard way to access all of the vdsm > functionality. > > The REST API will handle events by exposing a new 'events' collection at the api > root. REST users will use some sort of polling to collect these events. The > details of this interface are being worked on. Several ways for minimizing the > impact of polling have been discussed. The QMF broker can expose a > publish/subscribe model for events as appropriate. > > Is this model an acceptable way to improve the vdsm API? I would like to hear > the opinions of ovirt-engine developers, vdsm developers, and other > stakeholders. Thanks for providing feedback on this proposal! Why things non native to REST and wrap it in QMF, rather than do the reverse? Or just to them in parallel, since it sounds like both are going to be first class citizens? ___ vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/vdsm-devel
