Re: [Openstack] Openstack Nova/Quantum :; api-paste.ini file
On Thu, 2012-12-06 at 16:11 +0530, Trinath Somanchi wrote: What is the significance of api-paste.ini file in the configuration of nova and quantum and other modules of openstack? How this configuration is parsed and used? by which api of the openstack modules? So, api-paste.ini is parsed by the PasteDeploy package. As a first step to understanding this file, see this section of the PasteDeploy documentation: http://pythonpaste.org/deploy/#config-uris (Note: the file is formatted as a standard INI file, and I believe PasteDeploy uses the standard Python package ConfigParser to read it…) -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Openstack Nova/Quantum :; api-paste.ini file
Hi Kevin- Thanks for the reply.. But, few of my doubts are left ... [1] What is the significance of the api-paste.ini file in the configuration of nova/quantum and other modules of ipenstack? [2] How do the modules use these API configuration options? How they are used different from normal .conf files? kindly help me understand these.. - Trinath On Thu, Dec 6, 2012 at 10:19 PM, Kevin L. Mitchell kevin.mitch...@rackspace.com wrote: On Thu, 2012-12-06 at 16:11 +0530, Trinath Somanchi wrote: What is the significance of api-paste.ini file in the configuration of nova and quantum and other modules of openstack? How this configuration is parsed and used? by which api of the openstack modules? So, api-paste.ini is parsed by the PasteDeploy package. As a first step to understanding this file, see this section of the PasteDeploy documentation: http://pythonpaste.org/deploy/#config-uris (Note: the file is formatted as a standard INI file, and I believe PasteDeploy uses the standard Python package ConfigParser to read it…) -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp -- Regards, -- Trinath Somanchi, +91 9866 235 130 ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Openstack Nova/Quantum :; api-paste.ini file
Honestly, I don't understand your questions; I figured the documentation I pointed you to would answer them, and the fact it doesn't suggests that you're not asking what I thought you were asking. Maybe an approach from the beginning: Nova, Quantum, Glance, Keystone, etc. all have, as components, a REST API. They use the PasteDeploy package to build this API; PasteDeploy provides a means of building a WSGI stack (WSGI is the Python Web Server Gateway Interface, an interface by which HTTP requests can be presented to a Python application; it allows for not only an application, but also a set of middleware, which wraps the application and can provide enhancements). The various configuration files you reference are used by PasteDeploy to construct the WSGI stack for the API; that is, the configuration file tells PasteDeploy that the nova-api service is composed of a specified controller, wrapped by middleware that implements exception trap translation, authentication checks, ratelimit enforcement, etc., all in a specific order. In essence, the configuration file acts sort of like code, rather than configuration; it expresses the structure of the final application. (Although configuration can also be expressed in the file, we're trying to avoid that, so that we don't mix configuration with code.) Does that help you some? On Thu, 2012-12-06 at 22:29 +0530, Trinath Somanchi wrote: [1] What is the significance of the api-paste.ini file in the configuration of nova/quantum and other modules of ipenstack? [2] How do the modules use these API configuration options? How they are used different from normal .conf files? -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Openstack Nova/Quantum :; api-paste.ini file
It's probably best to ask these sorts of questions on the email list, as it gives an opportunity to others to answer them, as well as allowing others who may have similar questions to see the answers in the first place. On Thu, 2012-12-06 at 23:24 +0530, Trinath Somanchi wrote: [1] In nova or quantum api, We can access the .conf params, This way... cfg.Conf.x as per the soutce code... We can get the api-paste-config too.. But i wonder how we can get the paste api confs values too accessible this way Like, admin_user . PasteDeploy passes configuration options as arguments to the constructors/factories for the various applications and middleware. But, as I say, we're trying to avoid relying on this data in nova; the only consumer of it I am aware of is the Keystone auth_token middleware, and it has the capability now of specifying its necessary configuration in the [keystone_authtoken] section of the nova/glance/quantum/cinder configuration files. (I suspect the Keystone team is deprecating the configuration through api-paste.ini.) This should all be documented in the PasteDeploy manual… [2] since nova/quantum run as services, how do webob and wsgi play a role to prepare the request dict? At this point, we leave behind PasteDeploy. To answer your second question first, WSGI is an interface specification; it describes how a web application can be called by the server which receives the HTTP request. You can find out more about WSGI from PEP-333, at: http://www.python.org/dev/peps/pep-0333/ As for webob, that is another package used by nova, etc., which changes the interface we actually implement; that is, a WSGI application is a callable taking a dictionary with the environment and a start_response callback, but webob takes these two arguments and encapsulates them in a Request class, which provides simplified access to the environment data and some utility methods. In essence, webob implements the strange-looking parts of the WSGI interface spec for us, and we can concentrate on getting the job done. [3] When does( at what level )keystone authentication happens for given RESTful request... Keystone authentication happens, for most projects, in two separate pieces of middleware. The first is auth_token, contained in the python-keystoneclient package (it was just moved from the keystone package); this piece of middleware grabs the token out of the incoming request, verifies that it is a valid and unexpired token, then inserts various authentication data needed by the project (user and tenant IDs, for instance). The second piece of authentication is more or less a shim between the Keystone auth_token and the project; it extracts the data that auth_token injected into the request, then builds a project-specific authentication context. This context is how the various projects keep track of what user made the request, and is used in authorization checks (Does this user have permission to take this action on this resource?). -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Openstack Nova/Quantum :; api-paste.ini file
Thanks a lot kevin.. Your explanation has cleared my doubts.. Keeping togethor what i understand... Suppose, we have a resquest to Nova.. The following steps are performed... 1. The request is captured by webob and is authenticated by keystone and is decorated to wsgi app 2. Nova-api maps the url params to extensions 3. Nova-api extensions return the data dict.. Which webob returns as response to the request in json/xml format... 4. Paste-api helps the keystone and other modules for update of the request... This the request is served... In the above steps, i might be misunderstanding the concepts.. Kindly please help me by validating my understanding ... - Trinath On Dec 6, 2012 11:42 PM, Kevin L. Mitchell kevin.mitch...@rackspace.com wrote: It's probably best to ask these sorts of questions on the email list, as it gives an opportunity to others to answer them, as well as allowing others who may have similar questions to see the answers in the first place. On Thu, 2012-12-06 at 23:24 +0530, Trinath Somanchi wrote: [1] In nova or quantum api, We can access the .conf params, This way... cfg.Conf.x as per the soutce code... We can get the api-paste-config too.. But i wonder how we can get the paste api confs values too accessible this way Like, admin_user . PasteDeploy passes configuration options as arguments to the constructors/factories for the various applications and middleware. But, as I say, we're trying to avoid relying on this data in nova; the only consumer of it I am aware of is the Keystone auth_token middleware, and it has the capability now of specifying its necessary configuration in the [keystone_authtoken] section of the nova/glance/quantum/cinder configuration files. (I suspect the Keystone team is deprecating the configuration through api-paste.ini.) This should all be documented in the PasteDeploy manual… [2] since nova/quantum run as services, how do webob and wsgi play a role to prepare the request dict? At this point, we leave behind PasteDeploy. To answer your second question first, WSGI is an interface specification; it describes how a web application can be called by the server which receives the HTTP request. You can find out more about WSGI from PEP-333, at: http://www.python.org/dev/peps/pep-0333/ As for webob, that is another package used by nova, etc., which changes the interface we actually implement; that is, a WSGI application is a callable taking a dictionary with the environment and a start_response callback, but webob takes these two arguments and encapsulates them in a Request class, which provides simplified access to the environment data and some utility methods. In essence, webob implements the strange-looking parts of the WSGI interface spec for us, and we can concentrate on getting the job done. [3] When does( at what level )keystone authentication happens for given RESTful request... Keystone authentication happens, for most projects, in two separate pieces of middleware. The first is auth_token, contained in the python-keystoneclient package (it was just moved from the keystone package); this piece of middleware grabs the token out of the incoming request, verifies that it is a valid and unexpired token, then inserts various authentication data needed by the project (user and tenant IDs, for instance). The second piece of authentication is more or less a shim between the Keystone auth_token and the project; it extracts the data that auth_token injected into the request, then builds a project-specific authentication context. This context is how the various projects keep track of what user made the request, and is used in authorization checks (Does this user have permission to take this action on this resource?). -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Openstack Nova/Quantum :; api-paste.ini file
Seems like a good start to a wiki page to me :) Sent from my digital shackles On Dec 6, 2012, at 11:27 AM, Kevin L. Mitchell kevin.mitch...@rackspace.com wrote: Honestly, I don't understand your questions; I figured the documentation I pointed you to would answer them, and the fact it doesn't suggests that you're not asking what I thought you were asking. Maybe an approach from the beginning: Nova, Quantum, Glance, Keystone, etc. all have, as components, a REST API. They use the PasteDeploy package to build this API; PasteDeploy provides a means of building a WSGI stack (WSGI is the Python Web Server Gateway Interface, an interface by which HTTP requests can be presented to a Python application; it allows for not only an application, but also a set of middleware, which wraps the application and can provide enhancements). The various configuration files you reference are used by PasteDeploy to construct the WSGI stack for the API; that is, the configuration file tells PasteDeploy that the nova-api service is composed of a specified controller, wrapped by middleware that implements exception trap translation, authentication checks, ratelimit enforcement, etc., all in a specific order. In essence, the configuration file acts sort of like code, rather than configuration; it expresses the structure of the final application. (Although configuration can also be expressed in the file, we're trying to avoid that, so that we don't mix configuration with code.) Does that help you some? On Thu, 2012-12-06 at 22:29 +0530, Trinath Somanchi wrote: [1] What is the significance of the api-paste.ini file in the configuration of nova/quantum and other modules of ipenstack? [2] How do the modules use these API configuration options? How they are used different from normal .conf files? -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Openstack Nova/Quantum :; api-paste.ini file
On Thu, 2012-12-06 at 23:58 +0530, Trinath Somanchi wrote: Suppose, we have a resquest to Nova.. The following steps are performed... 1. The request is captured by webob and is authenticated by keystone and is decorated to wsgi app Not quite correct; webob decorates (some of) the functions called, so all functions in the WSGI stack end up having the WSGI calling convention (func(env, start_response)). The bulk of the middleware uses the webob wsgify decorator, but there are some exceptions (auth_token being one of them). Other than that point, this is correct. 2. Nova-api maps the url params to extensions nova-api maps the URIs to controller classes and methods on those classes (it uses the routes package to accomplish this). Some of those classes are extensions, rather than core; some of those interfaces are further extended by the extensions (the extensions infrastructure can accomplish both). IOW, you are essentially correct… 3. Nova-api extensions return the data dict.. Which webob returns as response to the request in json/xml format... Well, it's nova that serializes the data dict to the appropriate format; webob just handles the mechanics of sending the serialized data back, along with appropriate HTTP headers. The serialization framework is a little complicated, so let's omit it for now… 4. Paste-api helps the keystone and other modules for update of the request... PasteDeploy builds the processing pipeline based on the values in api-paste.ini and friends, putting the middleware into the correct order, with the final application at the end of the chain. (Note that middleware is *not* extension, but rather additional processing done on the request as a whole.) Kindly please help me by validating my understanding ... I think you've fairly well understood most of it, aside from some subtleties that I've tried to correct above. -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Openstack Nova/Quantum :; api-paste.ini file
Hi Kevin- Thanks for the reply and making me understand the data flow. I have one more doubt in plate. I see that in api-paste.ini, with respect to the online available documentation https://github.com/mseknibilel/OpenStack-Folsom-Install-guide/blob/master/OpenStack_Folsom_Install_Guide_WebVersion.rst I see that the auth_host, auth_port, admin_tenant_name, admin_user, admin_password etc... are all hard coded. But then, suppose that we have N number of compute nodes.where we have agents running which might need these variables for all the variables, these options must be manually hardcoded. Also we set these variables as environmental variables. How can we refer the Environmental variables for the variables in the configuration files? Can you guide me understand the issue. Thanks in advance - Trinath On Fri, Dec 7, 2012 at 2:09 AM, Kevin L. Mitchell kevin.mitch...@rackspace.com wrote: On Thu, 2012-12-06 at 23:58 +0530, Trinath Somanchi wrote: Suppose, we have a resquest to Nova.. The following steps are performed... 1. The request is captured by webob and is authenticated by keystone and is decorated to wsgi app Not quite correct; webob decorates (some of) the functions called, so all functions in the WSGI stack end up having the WSGI calling convention (func(env, start_response)). The bulk of the middleware uses the webob wsgify decorator, but there are some exceptions (auth_token being one of them). Other than that point, this is correct. 2. Nova-api maps the url params to extensions nova-api maps the URIs to controller classes and methods on those classes (it uses the routes package to accomplish this). Some of those classes are extensions, rather than core; some of those interfaces are further extended by the extensions (the extensions infrastructure can accomplish both). IOW, you are essentially correct… 3. Nova-api extensions return the data dict.. Which webob returns as response to the request in json/xml format... Well, it's nova that serializes the data dict to the appropriate format; webob just handles the mechanics of sending the serialized data back, along with appropriate HTTP headers. The serialization framework is a little complicated, so let's omit it for now… 4. Paste-api helps the keystone and other modules for update of the request... PasteDeploy builds the processing pipeline based on the values in api-paste.ini and friends, putting the middleware into the correct order, with the final application at the end of the chain. (Note that middleware is *not* extension, but rather additional processing done on the request as a whole.) Kindly please help me by validating my understanding ... I think you've fairly well understood most of it, aside from some subtleties that I've tried to correct above. -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp -- Regards, -- Trinath Somanchi, +91 9866 235 130 ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
