Re: [Openstack] IMPORTANT: Openstack List Migration (Please read)
On Wed, 2013-07-24 at 10:19 -0600, Paul Hummer wrote: Here's the timeline: 100UTC Friday - The Launchpad group ~openstack will be put in invite-only, so no new users will be able to sign up. At this point, I'll get a Launchpad Admin to provide all the data from the mailing list, and migrate it to lists.openstack.org 100UTC Saturday - The mailing list migration will be complete, and all users will be migrated over to lists.openstack.org. From then on, openstack@lists.launchpad.net will be a dead list, and openst...@lists.openstack.org will be the actual list. If you continue to send emails to the Launchpad list, this will be you: http://i.imgur.com/MQUmmqo.gif Do you mean 1000 UTC or 0100 UTC? 100UTC could be interpreted either way… -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] security blueprint related to os binaries
On Tue, 2013-05-14 at 18:38 +0300, Vasiliy Khomenko wrote: Attacker can put binary in /usr/local/bin for example. on ubuntu that path located before /usr/bin. If the attacker has write access to /usr/local/bin, it's already game over; I don't see what we can do to nova that can mitigate something that disastrous. -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] Turnstile updates
Greetings. I've been working on some scalability enhancements to Turnstile[1], and I believe it's about time to announce that work here. I'm hoping that people here find it useful, not to mention help with the final debugging :) Turnstile is a distributed rate-limiting middleware, which replaces Nova's built-in RateLimitingMiddleware with a version that can apply rate limiting across multiple nova-api nodes. (Turnstile itself is actually more general, and can be used for rate limiting with any WSGI application.) It uses an external Redis server for storing data about requests. To use Turnstile with Nova requires the nova_limits[2] package (another such package exists for using Turnstile with Keystone; I'm hoping the developer of that package will chime in with the appropriate link, since I've forgotten it…). My recent work has focused on enhancing Turnstile's scalability; in particular, I've been working on sharding the ephemeral request data across multiple Redis servers. To do that, it will be necessary to use a Redis proxy called Nutcracker[3]. Turnstile is not 100% compatible with Nutcracker, but fortunately the incompatible bits can be worked around easily, and so NutJob[4] was created. The final piece of the scalability work I have is Subway[5], which allows the rate limit configuration to be mirrored across multiple Redis servers. (Why not use Redis's master/slave? Well, Subway also forwards the messages that are used to notify Turnstile of when the limits configuration needs to be reloaded.) Here's hoping ya'll find these projects useful! [1] https://github.com/klmitch/turnstile [2] https://github.com/klmitch/nova_limits [3] Also known as twemproxy; https://github.com/twitter/twemproxy [4] Yeah, I know, bad pun; https://github.com/klmitch/nutjob [5] Because it carries rate limit configuration from Turnstile to Turnstile: https://github.com/klmitch/subway -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Absolute limits is quotas?
On Wed, 2013-04-17 at 14:19 +0300, Vasiliy Khomenko wrote: Official documentation says: The name of the absolute limit uniquely identifies the limit within a deployment., but my experiments shows that limits affects only within tenants, as quotas do. absolute limits are just another name for quotas. I'm not certain why the difference in terminology; it's probably a hold-over from nova's precursors. -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Service RBAC policy.json documentation and usability
On Tue, 2013-04-16 at 15:04 -0400, boden wrote: Ideally all of the roles would've been documented in a centralized location to make this experience more user friendly. Maybe a py annotation in the source files which document the roles used by the class and are then consolidated into a centralized document during the doc build or something... I do realize some core projects document (a portion) their roles on the wiki page, but it does not seem to be a consistent process. Yeah, this is one of the problems with the policy.json file; I've proposed a blueprint for addressing this, but haven't had an opportunity to really work on it. If any other developer wants to take up the task, see: https://blueprints.launchpad.net/oslo/+spec/self-documenting-policies -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] git review failure
On Mon, 2013-03-18 at 10:06 -0700, Ronak Shah wrote: debug1: Connecting to review.openstack.org [198.101.231.251] port 29418. debug1: connect to address 198.101.231.251 port 29418: Connection timed out ssh: connect to host review.openstack.org port 29418: Connection timed out This sounds like a firewall issue… -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Grizzly Dashboard Quota Problem...
On Mon, 2013-03-18 at 14:57 -0300, Martinx - ジェームズ wrote: I'm reinstalling everything (Grizzly from PPA) from scratch again, if I hit the BUG one more time, I'll let you guys know. I believe the quota settings error is a bug in nova, rather than in horizon or novaclient. The problem is that, recently, a change went in that causes nova to reject quota update requests that have unrecognized quotas. The problem is that older versions of nova had two quota resources (gigabytes and volumes) that have been removed in Grizzly, because of the nova/cinder split. Thus, all clients that operate against pre-Grizzly nova will fail to work with Grizzly nova…and it is also the case that novaclient and probably horizon have not been updated to remove those two quota resources. The correct fix will probably be to revert the nova merge that causes this HttpBadRequest to be raised, and to subsequently apply the IETF mantra: Be liberal in what you accept and conservative in what you send. -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Swift]A design draft of Storage Quota
On Wed, 2013-02-20 at 18:11 +0800, Alex Yang wrote: Storage Quotas Design This is the design draft of Storage Quota. Implementation of this design is https://github.com/AlexYangYu/StackLab-swift/tree/dev-quota I'll also point out Boson: https://wiki.openstack.org/wiki/Boson and https://github.com/klmitch/boson with some initial work. Unfortunately, I'm not able to work on Boson at the moment due to higher-priority tasks… -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Swift]A design draft of Storage Quota
On Wed, 2013-02-20 at 21:09 +0100, Chmouel Boudjnah wrote: On Wed, Feb 20, 2013 at 5:26 PM, Kevin L. Mitchell kevin.mitch...@rackspace.com wrote: I'll also point out Boson: https://wiki.openstack.org/wiki/Boson and https://github.com/klmitch/boson with some initial work. Unfortunately, I'm not able to work on Boson at the moment due to higher-priority tasks… From a quick look of it why can't we do the same as Boson without synaps[1]+ceilometer+swift_container_update. I don't know very well those but from the look of it you could have synaps generating alerts based on resources collection from ceilometer and set the enforcement using the native service mechanism? I do not understand your question. Quotas have nothing to do with notifications, as far as I understand it; quotas limit the maximum amount of a given resource a given user can have, while ceilometer just notifies other consumers about actions, right? -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Reinstating Trey Morris for Nova Core
On Tue, 2013-01-22 at 23:38 +, Matt Dietz wrote: I think Trey Morris has been doing really well on reviews again, so I'd like to propose him to be reinstated for Nova core. Thoughts? +1. We need more reviewers, IMO; there are constantly 2 pages of pending code reviews, and I've even seen reviews with a +2 get auto-abandoned after 2 weeks because no one else has reviewed them. -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] Bark logging middleware
I have just completed writing a piece of middleware for logging requests in WSGI stacks. I have dubbed this useful piece of code, Bark, and it is available on PyPi. Here are the links: * http://pypi.python.org/pypi/bark * https://github.com/klmitch/bark I've written an extensive README describing what Bark does and how it does it, but here's a quick summary: Bark is a logging middleware. That is, you place it into your WSGI pipeline (typically at the head of the pipeline, rather than close to the application at the tail) and define one or more log streams. Each log stream is configured with an Apache-compatible format string. Log streams can send the formatted log messages to files, syslog, TCP or UDP sockets, even email. Bark is also easily extensible; it is possible to add both new format string conversions and log stream types by simply defining new entry points. Why use Bark? Bark can be used with any WSGI application (not just nova) and can log virtually any information associated with the request, and do it independently of normal application logging. Moreover, since the format strings are Apache-compatible, it should be possible to use any tool designed to analyze Apache logs with Bark-generated log files. Bark also implements proxy validation, to allow the proper originating IP address of a client to be recorded. Caveats: Bark can only log data provided by the underlying WSGI implementation. For instance, the normal WSGI server used by Nova makes the remote IP address available in the REMOTE_ADDR environment variable, but the port number is not made available (Bark expects it to be placed in REMOTE_PORT if available). Also, certain Apache conversions and modifiers don't make sense for Bark (they are ignored for compatibility). For a full write-up, see the README, available at: http://pypi.python.org/pypi/bark -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Openstack Nova/Quantum :; api-paste.ini file
On Thu, 2012-12-06 at 16:11 +0530, Trinath Somanchi wrote: What is the significance of api-paste.ini file in the configuration of nova and quantum and other modules of openstack? How this configuration is parsed and used? by which api of the openstack modules? So, api-paste.ini is parsed by the PasteDeploy package. As a first step to understanding this file, see this section of the PasteDeploy documentation: http://pythonpaste.org/deploy/#config-uris (Note: the file is formatted as a standard INI file, and I believe PasteDeploy uses the standard Python package ConfigParser to read it…) -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Openstack Nova/Quantum :; api-paste.ini file
Honestly, I don't understand your questions; I figured the documentation I pointed you to would answer them, and the fact it doesn't suggests that you're not asking what I thought you were asking. Maybe an approach from the beginning: Nova, Quantum, Glance, Keystone, etc. all have, as components, a REST API. They use the PasteDeploy package to build this API; PasteDeploy provides a means of building a WSGI stack (WSGI is the Python Web Server Gateway Interface, an interface by which HTTP requests can be presented to a Python application; it allows for not only an application, but also a set of middleware, which wraps the application and can provide enhancements). The various configuration files you reference are used by PasteDeploy to construct the WSGI stack for the API; that is, the configuration file tells PasteDeploy that the nova-api service is composed of a specified controller, wrapped by middleware that implements exception trap translation, authentication checks, ratelimit enforcement, etc., all in a specific order. In essence, the configuration file acts sort of like code, rather than configuration; it expresses the structure of the final application. (Although configuration can also be expressed in the file, we're trying to avoid that, so that we don't mix configuration with code.) Does that help you some? On Thu, 2012-12-06 at 22:29 +0530, Trinath Somanchi wrote: [1] What is the significance of the api-paste.ini file in the configuration of nova/quantum and other modules of ipenstack? [2] How do the modules use these API configuration options? How they are used different from normal .conf files? -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Openstack Nova/Quantum :; api-paste.ini file
It's probably best to ask these sorts of questions on the email list, as it gives an opportunity to others to answer them, as well as allowing others who may have similar questions to see the answers in the first place. On Thu, 2012-12-06 at 23:24 +0530, Trinath Somanchi wrote: [1] In nova or quantum api, We can access the .conf params, This way... cfg.Conf.x as per the soutce code... We can get the api-paste-config too.. But i wonder how we can get the paste api confs values too accessible this way Like, admin_user . PasteDeploy passes configuration options as arguments to the constructors/factories for the various applications and middleware. But, as I say, we're trying to avoid relying on this data in nova; the only consumer of it I am aware of is the Keystone auth_token middleware, and it has the capability now of specifying its necessary configuration in the [keystone_authtoken] section of the nova/glance/quantum/cinder configuration files. (I suspect the Keystone team is deprecating the configuration through api-paste.ini.) This should all be documented in the PasteDeploy manual… [2] since nova/quantum run as services, how do webob and wsgi play a role to prepare the request dict? At this point, we leave behind PasteDeploy. To answer your second question first, WSGI is an interface specification; it describes how a web application can be called by the server which receives the HTTP request. You can find out more about WSGI from PEP-333, at: http://www.python.org/dev/peps/pep-0333/ As for webob, that is another package used by nova, etc., which changes the interface we actually implement; that is, a WSGI application is a callable taking a dictionary with the environment and a start_response callback, but webob takes these two arguments and encapsulates them in a Request class, which provides simplified access to the environment data and some utility methods. In essence, webob implements the strange-looking parts of the WSGI interface spec for us, and we can concentrate on getting the job done. [3] When does( at what level )keystone authentication happens for given RESTful request... Keystone authentication happens, for most projects, in two separate pieces of middleware. The first is auth_token, contained in the python-keystoneclient package (it was just moved from the keystone package); this piece of middleware grabs the token out of the incoming request, verifies that it is a valid and unexpired token, then inserts various authentication data needed by the project (user and tenant IDs, for instance). The second piece of authentication is more or less a shim between the Keystone auth_token and the project; it extracts the data that auth_token injected into the request, then builds a project-specific authentication context. This context is how the various projects keep track of what user made the request, and is used in authorization checks (Does this user have permission to take this action on this resource?). -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Openstack Nova/Quantum :; api-paste.ini file
On Thu, 2012-12-06 at 23:58 +0530, Trinath Somanchi wrote: Suppose, we have a resquest to Nova.. The following steps are performed... 1. The request is captured by webob and is authenticated by keystone and is decorated to wsgi app Not quite correct; webob decorates (some of) the functions called, so all functions in the WSGI stack end up having the WSGI calling convention (func(env, start_response)). The bulk of the middleware uses the webob wsgify decorator, but there are some exceptions (auth_token being one of them). Other than that point, this is correct. 2. Nova-api maps the url params to extensions nova-api maps the URIs to controller classes and methods on those classes (it uses the routes package to accomplish this). Some of those classes are extensions, rather than core; some of those interfaces are further extended by the extensions (the extensions infrastructure can accomplish both). IOW, you are essentially correct… 3. Nova-api extensions return the data dict.. Which webob returns as response to the request in json/xml format... Well, it's nova that serializes the data dict to the appropriate format; webob just handles the mechanics of sending the serialized data back, along with appropriate HTTP headers. The serialization framework is a little complicated, so let's omit it for now… 4. Paste-api helps the keystone and other modules for update of the request... PasteDeploy builds the processing pipeline based on the values in api-paste.ini and friends, putting the middleware into the correct order, with the final application at the end of the chain. (Note that middleware is *not* extension, but rather additional processing done on the request as a whole.) Kindly please help me by validating my understanding ... I think you've fairly well understood most of it, aside from some subtleties that I've tried to correct above. -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Distributed rate-limiting
On Wed, 2012-12-05 at 14:12 +, Karajgi, Rohit wrote: My understanding is Turnstile manages the situation where, the in-memory rate limits that are configured on load balanced API servers are imposed properly on the incoming requests, so each API server is correctly updated/synced with the used rate limits. Can you please confirm this understanding? Yes. Turnstile uses Redis to coordinate rate limit configuration and bucket data, in order to provide rate limiting. Also, I don't think this is part of the Openstack trunk code, and if so, is there any reason why it's not part of Nova, as it was meant to be a replacement? I wrote Turnstile to be general; it can be used for Nova, Keystone, or any other system for which rate limiting is desired. (I in fact designed it with a goal of being able to use it for some personal projects which are not OpenStack-related.) This is the primary reason it's not a direct part of any OpenStack repository. That said, it is hosted on github and I welcome pull-requests…and I'm not at all adverse to the suggestion that it become an OpenStack project; I'm just not convinced that that would be generally desired, or that it would be generally beneficial… -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Glance] config_file parameter in glance-registry.conf file
On Wed, 2012-10-31 at 16:46 -0500, Ahmed Al-Mehdi wrote: I am following the steps in Openstack install manual for Ubuntu. Section 6 ( http://docs.openstack.org/trunk/openstack-compute/install/apt/content/configure-glance-files.html ) states to set the config_file parameter in glance-registry.conf file as follows: [snip] However, based on the commented line in the file (in the unmodified file after install), should config_file be set as follows: config_file = /etc/glance/glance-registry-paste.ini You are correct, it should be glance-registry-paste.ini when you're discussing glance-registry.conf. Just verifying if there is a typo in the doc. Looks like it is a typo. Would you mind logging a doc bug on that? -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Quotas in folsom
On Tue, 2012-10-30 at 10:19 -0500, Everett Toews wrote: Is http://wiki.openstack.org/Boson still up to date? Is there a blueprint for it? There is not a blueprint for it, since it's a brand-new project, and I'm just getting started on it. I can point you at the code repository I have for it, at https://github.com/klmitch/boson (I'll worry about going into incubation later, after we have something that kinda works, but I'm happy to accept pull requests…) I'll likely hold off on blueprints until it's time to start integrating it into the openstack projects… -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Quotas in folsom
On Mon, 2012-10-29 at 10:53 -0400, Mitchell Broome wrote: I'm running into quota problems trying to increase the number of security groups and rules within security groups per tenant. Setting quota_security_groups and quota_security_group_rules in nova.conf seem to have no effect. There also doesn't seem to be any way to change the quota limits for security groups through the nova client or horizon. The quotas system checks the database for quotas specific to the tenant, then for quotas for the tenant's quota class (if you're using quota classes). Only if it can't find any such quotas will it go to the values defined in nova.conf. You're right that these particular quotas are not among the quotas recognized by the nova shell command, but you can access them through the pythonic API; I'm guessing that the new quotas were added to nova itself during the folsom release cycle, but nobody remembered to update novaclient to recognize them. Could you log a bug against folsom for that, please? How do I go about changing these quotas or is there a way to disable all quotas all together? Check the database itself for quota records for your tenants; you can revert to defaults (drawn from nova.conf) by deleting any 'quotas' table rows for the resources you're interested in. If it still doesn't take the values you set in nova.conf, then there's likely some other bug that needs to be looked into… -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Quotas in folsom
On Mon, 2012-10-29 at 18:01 +, Gabriel Hurley wrote: It's also worth noting that we are now in territory where quotas are controlled by multiple projects: volumes and gigabytes have quotas in both Nova and Cinder; network quotas are in both Nova and Quantum... While I don't think it makes sense to try and centralize these things, I think the projects could coordinate more to understand who should be managing a given quota and to try and make the end-user experience less baffling. It's also worth noting that I've finally been able to start working on Boson, which may help with that… -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Keystone connection issue
On Wed, 2012-10-24 at 21:40 +, Bhandaru, Malini K wrote: I have an Ubuntu 12.10 install with devstack freshly downloaded. Does anybody have an issue where devstack/stack.sh script fails because keystone is unable to start, and consequently, none of the services start. .. 'one/keystone.conf --log-config /etc/keystone/logging.conf -d --debug + echo 'Waiting for keystone to start...' keystone endpoint-create: error: argument --service-id/--service_id: expected one argument Actually, it seems like I've seen that happen with our gate jobs, which run tests under a fresh devstack environment. You might try running it again and seeing if it runs the second time… -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Discussion / proposal: Ability to reset tenant's quotas to default
On Tue, 2012-10-09 at 12:17 -0400, Eoghan Glynn wrote:
I don't think a new nova command is needed for this use-case,
just add a simple custom script:
nova quota-update `nova quota-defaults $1 | tail -n +4 | tr '_' '-' | awk
'/|/ {printf( --%s %s, $2,$4)}'` $1
then call with the tenant ID as command line arg.
The problem with this approach is that if you then change the default
quotas, they are not reflected for the tenant. I've noticed the lack of
a DELETE handler in the quotas (and quota_classes) extension(s) and
often thought we needed to add one for just this case…
--
Kevin L. Mitchell kevin.mitch...@rackspace.com
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
Re: [Openstack] TC candidacy
On Mon, 2012-09-17 at 21:34 +, Chris Behrens wrote: I'd like to announce my candidacy for a seat on the OpenStack Technical Committee. +1 -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Cells Status
On Fri, 2012-09-14 at 11:07 +0530, balaji patnala wrote: We didnt find any information related to CELLS [which is planned to replace ZONES] in the latest Folsom pre-release. Can any body give us information on this. Unfortunately, cells was unable to make feature freeze. It should be in Grizzly. Sorry for the delay :/ -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] nova-manage db sync fails
On Tue, 2012-08-28 at 18:53 +0200, Afef MDHAFFAR wrote: I am trying to install openstack on an ubuntu server 12.04, with Xen as a virtualization technology. I unfortunately got a problem while trying to install the nova service. Actually, the nova-manage db sync fails and returns the following warnings: These are just warnings and can be safely ignored at this point. The next release of nova should not emit these warnings. -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [glance] legacy client removal and python-glanceclient
On Wed, 2012-08-01 at 18:37 +, Gabriel Hurley wrote: As a rule of thumb, we need to start doing proper deprecation on all public interfaces, whether that's a CLI, client method signatures, APIs, etc. It's a little late for this on the old vs. new glance client/CLI (unless Brian feels the work can be reasonably done to make them compatible) but it's something we need to be really mindful of going forward. As an example of how it can be done properly, check out https://review.openstack.org/#/c/10577/ (at least, I believe I did it correctly ;) -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Instance stuck in deleting state with error
On Wed, 2012-08-01 at 15:40 -0400, Lorin Hochstein wrote: From the python-novaclient tests, it looks like nova reset-state instance puts an instance into the error state or (with the --active) flag into the active state: That is correct. What's the use case for resetting an instance to the error state? Is the idea to do: nova reset-state instance nova delete instance Yes. At the time, it was not possible to delete an instance that had its task_state set to a non-None value. A subsequent patch I contributed fixed that behavior, however; the problem was that compute_api.delete() and compute_api.soft_delete() had divergent state requirements, where the former was allowed from any state (the desired behavior) and the latter only allowed from vm_state ACTIVE, ERROR, or one other (I forget what it was) and task_state None. The reset-state API is one of the admin_actions extension, by the way; the goal of that default configuration was to prevent gratuitous state changes while still allowing administrators to help users who were having problems deleting instances. -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [glance] legacy client removal and python-glanceclient
On Wed, 2012-08-01 at 19:50 +, Gabriel Hurley wrote: Personally I'd recommend using Python's built-in warnings module and the standard DeprecationWarning and PendingDeprecation warning classes: http://docs.python.org/library/warnings.html#warning-categories For an example of this in action (outside OpenStack) check out Django's usage here: https://github.com/django/django/blob/stable/1.4.x/django/core/management/sql.py#L99 Indeed. I even wrote a whole suite of deprecation decorators for marking functions and classes as deprecated, using the DeprecationWarning…but it got ripped out by someone because it wasn't used anywhere. -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Instance stuck in deleting state with error
On Tue, 2012-07-31 at 07:14 +0200, Wolfgang Hennerbichler wrote: On 07/30/2012 09:35 PM, Kevin L. Mitchell wrote: That said, be aware that there is a reset-state command to novaclient, so that you can do Chris's recommended reset without having to muck around with the database directly. where? nova help | grep reset yields nothing. What version of novaclient are you using? (For that matter, what version of nova are you using?) The reset-state subcommand exists in current trunk. I think this is one of openstack worst weaknesses, that if the status of an instance is in error-state and one has to wade through a couple of logfiles (scheduler, nova-network, nova-compute) in order to find out what really happened. I would be superior if the error itself would be reported back to the database. -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Instance stuck in deleting state with error
On Mon, 2012-07-30 at 13:03 -0400, Jonathan Proulx wrote: I have an instance that has been in this state for a couple days: | OS-EXT-STS:power_state | 0| | OS-EXT-STS:task_state | deleting| | OS-EXT-STS:vm_state | error | If you're using the Xen driver on trunk, I recently cleaned up a few bugs that might have lead to this problem. Make sure you're updated and try again… -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Instance stuck in deleting state with error
On Mon, 2012-07-30 at 14:25 -0500, Chris Behrens wrote: You may still have to reset the instance's task_state to NULL in the DB (instances table) to delete ones already in this state. No, I fixed the state problem with soft_delete(); as long as he updates, delete should work fine. That said, be aware that there is a reset-state command to novaclient, so that you can do Chris's recommended reset without having to muck around with the database directly. -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Keyring support in openstack
On Mon, 2012-07-30 at 13:50 -0700, Bhuvaneswaran A wrote: The wiki mentions the password being saved using keyring.backend.UncryptedFileKeyring. Does that mean the password is saved in cleartext? Is the file protected in some way besides filesystem permissions? As mentioned in wiki page, the password is stored in base64 format. Which means it's stored in cleartext. That is Not Good(tm) :) -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] quota question
On Fri, 2012-07-20 at 15:59 +0100, Kiall Mac Innes wrote: But - what about making quotas pluggable, like the scheduler? They are; see the quota_driver configuration option. However… This would allow for even more complex quotas, like limiting the number of SSD backed instances across the entire cloud per tenant, while still keeping the core implementation lean. As Eoghan points out, a lot more context would need to be provided than the current quota system uses, and you'd end up with something considerably more complex. (BTW, I'd like to point out the Boson proposal and thread…) -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Distributed quota manager concept
On Tue, 2012-07-17 at 16:08 -0600, Everett Toews wrote: Were you envisioning Boson going through the incubation process and becoming a core project in OpenStack? Yes, I could envision that. If that were to happen, would Boson become a required dependency for all of the other OpenStack projects (that require quotas)? No; my thought was to create a client that could then optionally be used by quota code in each project. Boson came from thinking about the quota refactoring I did in Nova, so I envisioned writing a Boson driver for the Nova quota code. Would it be possible to run OpenStack without Boson? Yes. The benefit of Boson really comes when you have to deal with the distributed quota problem, which is most apparent if you're using multi-cell Nova. Beyond that, it provides a unified interface to quotas for multiple projects, but deployers may prefer the simplicity of a non-Boson deploy. My main concern here is the cost of the complexity of adding another service to deploy and maintain. But, one way or another, obviously something needs to be done about quotas. Indeed. Speaking of deployment and maintenance complexity...the Data Storage section reads, It is also necessary to be able to search for a given Usage or Reservation based on some or all of the key/value pairs, so that usage information may be obtained and easily displayed to the user. This latter requirement may indicate that a NoSQL solution is the best for Boson's backend. Setting aside any SQL/NoSQL religious debate or even the best tool for the job argument, I think you'd find this to be a hard sell to the operations crowd. Nobody is going to want to have all of their OpenStack data in an SQL DB (which they may have already gone through the trouble to make HA) but then have just the quota data in a NoSQL DB. I would urge you to consider starting with SQL and then make NoSQL an option if there is demand for it. I wrote that because I couldn't see a simple way of doing what I wanted via SQL at the time. Now that I think about it, I think it is possible, and I'm not against using SQL at all. (I also think there was an aspect of I'd like to try working with NoSQL sometime when I wrote that, so… ;) -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] Distributed quota manager concept
I recently thought about and wrote up a concept for a distributed quota manager that I have dubbed Boson. Unfortunately, higher priorities at Rackspace have kept me from working on it, so I wanted to get the proposal out there for others to comment and cogitate on. The writeup is at http://wiki.openstack.org/Boson -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [keystone] Rate limit middleware
On Thu, 2012-07-12 at 18:26 +0200, Rafael Durán Castañeda wrote: Unless I'm missing something, nova_limits is not applicable to Keystone since it takes the tenant_id from 'nova.context', which obiously is not available for Keystone; thought adapt/extend it to keystone should be trivial and probably is the way to go. You are correct, you would not take nova_limits and use it with Keystone. What you likely would do is use nova_limits as a model to develop your own shim with the additional capabilities you need. I expect that you would not need much of what nova_limits does, by the way… -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Nova] resource free -vs- allocated utilization?
On Thu, 2012-07-12 at 12:31 -0400, Jonathan Proulx wrote: for posterity yes the info isn't hard to find in the database: mysql select id,vcpus,vcpus_used,memory_mb,memory_mb_used from compute_nodes; I'm not terribly keen on SQL as an interface, guess if it bothers me enough I'll implement a different interface... Check out the hypervisors extension and related novaclient addition, now in trunk; I made all the information from the compute_nodes table available via the API. -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [keystone] Rate limit middleware
On Wed, 2012-07-11 at 01:50 +0200, Rafael Durán Castañeda wrote: I'm working on a blueprint [1] and implementation [2] doing rate limit middleware for Keystone; after discussing it at keystone's meeting today I was suggested to ask for some feedback from the community. Have you taken a look at Turnstile and the related integration package, nova_limits? Unfortunately, trunk Turnstile doesn't support multiprocess, but I intend to address that as soon as job responsibilities permit. URLs: * http://pypi.python.org/pypi/turnstile * http://pypi.python.org/pypi/nova_limits * https://github.com/klmitch/turnstile * https://github.com/klmitch/nova_limits -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Please vote for the name of the G release
On Wed, 2012-07-11 at 16:38 -0400, Duncan McGreggor wrote: You just don't know what the Bear Revolt crew was ready to do to let Grizzly win :) *laughs* Not the BEAR REVOLT!!! Nobody expects the BEAR REVOLT! -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] PEP8 checks
On Mon, 2012-07-02 at 08:15 -0400, Monty Taylor wrote: It's not really expected, and I honestly don't understand why run_tests.sh -p would have problems running pep8. Although we do not use run_tests.sh for anything in jenkins, we have not done anything to disable or change what it's doing. I need to point out that run_tests.sh -p doesn't run straight-up pep8; it monkey-patches the pep8 tool to include several HACKING-compliance tests. Ever since tox stopped using this version of pep8, several HACKING-compliance issues have crept into the code base. -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] Jenkins and transient failures
One of the things that's really bugging me these days is transient failures, such as the inability to download a package, causing a gate job to fail. It seems to me that we can distinguish test failure from environment build failure easily enough, and automatically retry in the latter case. Is this possible in practice with our current CI infrastructure? -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Unit tests, individual vs. test suite
On Fri, 2012-06-29 at 10:34 -0500, Andrew Bogott wrote: $ /opt/stack/nova$ ./run_tests.sh test_virt_drivers AbstractDriverTestCase test_add_to_aggregateERROR 0.02 test_agent_updateERROR 0.02 etc. And yet, if I scroll up and look at the earlier run (where everything passed) I see it running AbstractDriverTestCase with all green. Try: ./run_tests.sh nova.tests.test_virt_drivers and see if there's any difference with the full path as compared to the abbreviated path? -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] RFC: Thoughts on improving OpenStack GIT commit practice/history
On Thu, 2012-06-28 at 09:24 -0700, Matt Joyce wrote: Can we set a location to the Authoritative HACKING.rst? There are fundamental and conflicting differences between the HACKING.rst in some of the projects. The HACKING.rst in each project is authoritative for that project. There are slight stylistic differences between the different project, and there is resistance to adopting an openstack-wide HACKING style guide. -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [OpenStack][Nova] No tests available in custom branch
On Tue, 2012-06-26 at 09:36 +0100, Leander Bessa Beernaert wrote: Any ideas? My gut instinct is that you have a syntax error as well, somewhere in diagnostics.py. Try running the python interpreter and manually trying that import; if it fails, you should have an error message that will help you track down what the problem is. -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [OpenStack][Nova] No tests available in custom branch
On Tue, 2012-06-26 at 15:50 +0100, Leander Bessa Beernaert wrote: I've successfully imported the diagnostics.py in the interpreter, so that can't be the problem. Then try importing the other file you modified. If that still doesn't help you find the problem, then I'm all out of ideas… -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [OpenStack][Nova] No tests available in custom branch
On Tue, 2012-06-26 at 16:05 +0100, Leander Bessa Beernaert wrote: The file, is right there in the same directory. I have double checked the names but it still keeps failing :s The first thing I'd try is clearing out all your *.pyc files. (run_tests.sh should do this for you, so I don't expect this to actually fix your problem, but it's a place to start…) Also, double-check the permissions on the diagnostics.py file. Finally, verify that your current directory is the top-level directory of the repository, not the nova subdirectory of the repository… -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [OpenStack][Nova] No tests available in custom branch
On Tue, 2012-06-26 at 16:21 +0100, Leander Bessa Beernaert wrote: It works from the top level, but fails if i try to import it directly from the same dir (nova/virt/libvirt). The other files there import just fine :/ Relative imports are iffy at best; imports should ideally always be absolute. This is why I suggest that you should always call run_tests.sh from the top-level of the repository, not from the nova subdirectory… -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [OpenStack][Nova] No tests available in custom branch
On Tue, 2012-06-26 at 16:33 +0100, Leander Bessa Beernaert wrote: I'm not calling run_tests.sh from the nova subdirectory. I'm saying that from the location where nova_tests.sh is located, i can import connection.py. However, when i try to import connection.py from with cd=nova/virt/libvirt, it fails. The funny thing is i can import all the modules int that directory perfectly, except connection.py. That's expected. Whenever you run Python, the current directory is added to the import path. connection.py has several absolute imports that it depends on being able to import; when you run from the top-level of the repo, things like nova.virt.libvirt.utils can be found, whereas importing from the nova/virt/libvirt directory means that nova.virt.libvirt makes no sense. -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [OpenStack][Nova] No tests available in custom branch
On Mon, 2012-06-25 at 16:56 +0100, Leander Bessa Beernaert wrote: Here's the diff http://paste.openstack.org/show/18756/ Change import diagnostics to from nova.virt.libvirt import diagnostics. (Also note that you may need to add a space between your name and your email address in Authors, and if you mean to submit this to the trunk, it'd be nice to put your entry in the alphabetically-appropriate place…) -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] glance_api_servers vs. glance_host vs. keystone?
On Fri, 2012-06-15 at 20:54 -0400, Lars Kellogg-Stedman wrote: Thanks for the reply, makes sense. Just to make sure I understand things, it sounds like Nova does not currently query Keystone for endpoints and continues to rely on explicit configuration (or to rephrase your answer, the reason these options have not gone away is because Nova does not yet have the necessary support for Keystone). Is that approximately correct? The problem with the Keystone endpoints is that you have to make a query to Keystone to get them. We want to reduce the number of hits we make on Keystone, not increase them—there are already too many as it is. Thus, I suspect that nova may not even use the Keystone endpoints. It *does* support image URLs, however. Thus, you use the options to configure the default glance endpoint, and if you want to hit another glance, you simply give a URL to the desired image rather than a simple identifier. (My comments about the support for endpoints in this email may differ from my previous comments; chalk that up to further reflection on the problem being solved…) -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] glance_api_servers vs. glance_host vs. keystone?
On Mon, 2012-06-18 at 10:18 -0400, Nathanael Burton wrote: What's the point of a service catalog (list of endpoints) if we don't want to use it?! Looking up endpoints should be a cacheable request and in the grand scheme of things -- low impact. We do use the service catalog, quite extensively—on the client side. From nova to glance, I suspect we don't use the service catalog, since nova just uses the delegated credentials from the user. Looking up the service catalog is indeed quite cacheable; however: I don't believe that such code has been added; it may be necessary to pierce abstraction boundaries to perform that caching; and the glance endpoint is likely to be pretty static anyway, and thus fine for setting by means of configuration. And again, it has been a while since I looked at that code path… -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] glance_api_servers vs. glance_host vs. keystone?
On Mon, 2012-06-18 at 10:41 -0400, Lars Kellogg-Stedman wrote: That sounds crazy to me, but I just got here. That is, why go to the effort to develop an endpoint registration service and then decide not to use it? Given the asynchronous, distributed nature of OpenStack, an endpoint directory seems like a good idea. Just out of question, what *does* use the endpoint registry in KeyStone (in the Essex release)? The clients. The endpoint registration system, so far as I understand, was primarily intended for use by the clients. It certainly would be useful for use by the servers, but there are subtleties, and I am not aware that it is currently used by nova-glance. But yet again, I have not looked at that code for a while; last time I was there, I was adding the initial support for nova to feed the user's credentials into glance; that was pre-Diablo, if I recall correctly. Nova, glance, keystone, etc. are all moving targets; there are tons of things that have only been added recently in the grand scheme of things, and there are many loose ends still to be tied. As an example, while I was rototilling the quotas system in nova, new quotas were added that changed the requirements I was working from, and since I was running up against deadlines, I had to leave some of those ends untied for now; there's no telling when I'll be able to get back to those loose ends and finally tie them up. I would not be surprised if something similar has happened WRT the endpoints system, since there are so many subtleties that need to be taken into account. -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Thoughts on client library releasing
On Mon, 2012-06-18 at 17:25 -0400, Doug Hellmann wrote: How do these plans fit with the idea of creating a unified client library (either as one package or several, based on a common core)? I am under the impression that there is not a desire, at present, to create a unified client library. There is work underway to create a unified client (command-line interface), but I believe it was intended to use the client libraries for each of the projects. -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] glance_api_servers vs. glance_host vs. keystone?
On Fri, 2012-06-15 at 16:26 -0400, Lars Kellogg-Stedman wrote: nova.conf appears to sport several configuration options related to glance, including: - glance_host - glance_port - glance_api_servers These seem suspiciously similar. Indeed. Do they do the same thing? Yes, they do. And shouldn't this information actually come from Keystone, in which there is an endpoint registered for the glance service? Yes, it should :) Now, a little history lesson: First came nova. Then, an index server was needed, and so the glance_host and glance_port options were added. Then, an enhancement: use of multiple glance hosts, and so glance_api_servers was added, with reasonable defaults drawn from glance_host and glance_port if it wasn't provided. Then, a centralized authentication service called Keystone was added, and as a benefit, it added the concept of endpoints. The reason these options have not gone away is probably a combination of supporting non-Keystone authentication and general programmer laziness… -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] The right way to deprecate things in nova?
On Tue, 2012-06-12 at 15:50 -0400, Sean Dague wrote: Here's my current suggested path forward, which I'd like comments on: * keep the existing nova.utils deprecation functions (don't remove them) As the author of nova.utils.deprecated, I approve :) * add the fatal config option, and associated unit tests to make sure it works correctly. This would be helpful for people to ensure they weren't depending on deprecated functions towards the end of a release. That makes sense to me. * possibly move them to nova.common as they might make for good openstack-common material down the road I created the @deprecated decorator just as openstack-common was getting started, and I always considered it a perfect candidate for openstack-common. I wonder if this is an interface that should skip the incubation state, though, and be used as a library? * use this instead of the direct LOG.error in get_connection This would have the side effect of making the message warning level, instead of error level, which I think is fine at this point. *nod* I'll take an opportunity to comment on the motivation behind adding @deprecated. I was doing some extensive changes to the openstack API infrastructure at the time, and the old calls were used all over the place. I wanted the old interface to continue to function, but to generate warnings that would be easy to find in the logs, so that I could change out one piece at a time without totally breaking everything. This seemed like something that others would also need to do, probably regularly. I also had considered the N/N+1 issue with releases: one of the disadvantages of a plugin-supporting system like nova is that the primary developers don't have control of all the code. We need to have a way to warn third party developers that the interfaces they use are about to go away, before they actually do. That unfortunately means we'll be looking at even more complex code in the future, to cover all the N/N+1 issues, but I don't really think we can avoid that if we want people to actually use nova. -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] gerrit reviews change?
For the past few days, I have noticed that I no longer get emails when new changes are pushed, when changes I've commented on have new patch sets pushed, or when changes I've commented on are finally merged. I do receive emails when comments are made on changes I've commented on, but the other emails are MIA. What's up? I depended on those emails to tell me when I needed to re-review a change or stop tracking a change because it merged… -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] How to let nova use localtime rather than UTC time?
On Wed, 2012-06-06 at 21:33 +0800, livemoon wrote: I found nova use utcnow to get time and write it to db. So the create_time of vm also show utc time rather than localtime. That is correct. Is there any flag in nova.conf to let nova use localtime . You really don't want to do this. Trust me. If you need to see the time in the local timezone, then convert it; there are tools in python to do this. -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Quotas... 1 of 1 instances? What's the deal?
On Mon, 2012-06-04 at 16:52 -0400, Jay Pipes wrote: In Horizon, my tenant/user clearly says that 10 instances is my quota, and yet trying to create a single server I'm getting this: jpipes@uberbox:~/repos/tempest$ nosetests -v --nologcapture == ERROR: test suite for class 'tempest.tests.compute.test_servers_negative.ServersNegativeTest' -- [snip] File /home/jpipes/repos/tempest/tempest/common/rest_client.py, line 205, in request raise exceptions.OverLimit(resp_body['overLimit']['message']) OverLimit: Quota exceeded Details: Quota exceeded: already used 1 of 1 instances But there are no instances at all on the box: One thing to check is the total quotas on memory and disk, relative to the size of the instance. The original code computes a maximum number of instances based on those values; my new code simply tries to emulate that computation. (I'm pretty sure I got it right, but honestly quotas needs further rototillings…) When I check the DB, though, I've seeing the following: mysql select project_id, in_use, reserved, until_refresh from quota_usages where resource = 'instances'; +--++--+---+ | project_id | in_use | reserved | until_refresh | +--++--+---+ | 287a92da0cf14a27a43c8737417b029d | 0 | 10 | NULL | | f0c72dea9fda459aac64de460300e1ec | 0 |2 | NULL | +--++--+---+ 2 rows in set (0.00 sec) Hmmm…when quiesced, you should only see reservations if instances are actively building. (In fact, reservations should be committed as soon as the instances are created in the database.) It's possible that I missed places where instances are created, but I thought I got them all… What's the deal here? Tempest needs to create and delete servers in rapid succession, and it seems the reservation system might not be able to keep up? Honestly, I thought the new quotas system was passing Tempest. When I originally pushed the patch, there were some problems with quota usages dropping negative, but I hacked around that by forcing usages to be refreshed if they would be set negative. This system is pretty complex, because of everything it has to deal with, and it's possible there are problems I haven't found yet :/ At a minimum, I think that the OverLimit: Quota exceeded: already used 1 of 1 instances message should be updated to not be so obviously wrong with regard to the value of the resource quota itself? If my theory above about computed maximum instances is correct, then we might be looking at an interface change to do such an update. That said, this probably should be done :) -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] Lossage in nova test suite?
Today I've noticed some significant problems with nova's test suite leaving literally hundreds of python processes out. I'm guessing that this has to do with the unit tests for the multiprocess patch, which was just approved. This could be causing problems with jenkins, too… Anybody have any other insights? -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] No JSON object could be decoded
On Thu, 2012-05-31 at 10:53 +0100, khabou imen wrote:
curl -d '{auth: {tenantName: service,
passwordCredentialsusername: swift, password: swiftpass}}}'
-H Content-type: application/json
http://192.168.1.68:35357/v2.0/tokens | python -mjson.tool
Your submitted JSON data is improperly formatted, unless that's a cp
error. You appear to have left out a ':' and a '{' after the
passwordCredentials dictionary key.
--
Kevin L. Mitchell kevin.mitch...@rackspace.com
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Identity API v3 - Why allow multi-tenant users?
On Tue, 2012-05-29 at 17:18 +, Caitlin Bestler wrote: One of the major complication I see in the API is that users can be associated with multiple tenants. What is the benefit of this? What functionality would be lost if a human user merely had to use a different account with each tenant? There are numerous issues with multi-tenant users. For example, if a user is associated with multiple tenants, who resets the user’s password? The use case that immediately springs to mind is that of a consultant. A consultant may be working for several clients that all happen to use one OpenStack-powered provider, and it would be handy for that consultant to only have to worry about a single set of login credentials, but still be able to access the relevant parts of all the tenants for which he or she is working. I could imagine several other somewhat similar scenarios, such as the value-added reseller; having multiple tenants allows them to ensure the proper client is billed the proper amount, while still being able to perform whatever their value-add is. -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Unused code in Nova [was Re: Quota classes]
On Thu, 2012-05-24 at 10:56 +0100, Mark McLoughlin wrote: So, I'm looking into the quotas code properly for the first time while reviewing one of the quota refactor patches and I come across the quota_class property on RequestContext My last 45 minutes have been: Where is quota_class being set? Nowhere in Nova, really? Let's double, triple check that. Maybe the auth token middleware is setting it? Nope. Yeah, I had always meant to revisit this and do something about it. The obvious thing to do would be to add the quota_class to Keystone, then have authtoken/keystonecontext use it when creating the RequestContext, which is why I added an argument to RequestContext to do that. Unfortunately, I have to move on to working on things other than quotas, now :/ -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] ERROR: Malformed request url (HTTP 400)
On Wed, 2012-05-09 at 15:32 -0500, Dolph Mathews wrote:
It also just occurred to me that perhaps you're using a *very* old
novaclient against a more recent version of keystone?
Actually, if you look a little more closely:
$ nova --debug image-list
connect: (192.168.1.71, 5000)
send: 'POST /v2.0/tokens HTTP/1.1\r\nHost:
192.168.1.71:5000\r\nContent-Length: 117\r
\ncontent-type:
application/json\r\naccept-encoding: gzip, deflate\r
\naccept:
application/json\r\nuser-agent: python-novaclient\r\n
\r\n{auth:
{tenantName: labSpaceDemo, passwordCredentials:
{username:
adminUser, password: lfplhfgthvf}}}'
The request body for Keystone is not, in fact, malformed. It would be
interesting to look at the nova-api logs for this request…
--
Kevin L. Mitchell kevin.mitch...@rackspace.com
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Swift][Keystone] Swift Quotas
I missed the first post(s) in this thread, but I should probably put out there that I'm currently working on refactoring quotas in Nova; see: * https://blueprints.launchpad.net/nova/+spec/quota-refactor * https://github.com/klmitch/nova/tree/quota-atomicity * https://review.openstack.org/#/c/6774 * https://review.openstack.org/#/c/7048 To get a sense of what I'm doing. I've also been thinking about the constraints of an external quota manager, but haven't gotten much further than some kind of RPC or REST-based API. (Note I haven't been strongly considering integrating this with Keystone for a couple of reasons: 1. I tend to prefer the UNIX paradigm of doing one thing well; 2. I want to ensure that this external quota manager is usable for those who choose to use something other than Keystone.) Feel free to ask me questions; I'm sure there's a lot of stuff I've thought of that may not be obvious from the above references, and your questions will probably help me articulate it better :) -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] proposal for Russell Bryant to be added to Nova Core
On Fri, 2012-04-27 at 11:09 -0400, Dan Prince wrote: I'd like to seem him Nova core so he can help out w/ reviews... definitely the RPC ones. +1 -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [OpenStack][Nova] Minimum required code coverage per file
On Thu, 2012-04-26 at 11:53 -0700, Joe Gordon wrote: It would nice to initially see the code coverage delta per merge proposal as a comment in gerrit (similar to SmokeStack), and not as a gating factor. +1 Kevin, should we start copying openstack-common tests to client projects? Or just make sure to not count openstack-common code in the code coverage numbers for client projects? That's a tough one. If we copy in the tests, they end up being somewhat redundant, but slow down the project unit tests, but on the other hand, we'd be able to easily demonstrate that that code works properly. I think I'd prefer if we just try to not count openstack-common code for code coverage numbers… (Personally, I would prefer if openstack-common was a library, rather than copying its code into the client project, but I am not familiar with the arguments for why it was decided to do the copy, and I'm not really involved in openstack-common development at the moment…) -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [OpenStack][Nova] Minimum required code coverage per file
On Tue, 2012-04-24 at 13:11 -0700, Joe Gordon wrote: nova/openstack/common/iniparser 40% nova/openstack/common/cfg 41% It's probably worth pointing out that, although openstack-common has comprehensive unit tests, apparently, those tests are not copied into client projects when the code is… -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Using Nova APIs from Javascript: possible?
On Wed, 2012-04-25 at 16:19 -0400, Adam Young wrote: Kerberos is designed to solve this problem. It has the benefit of being integrated into the browser. Where Kerberos fails is that: typically it only allows a single authentication provider (KDC in Kerberso speak) and it does not work well with Firewalls. Well, Kerberos uses UDP for its network communication, so that's the place it fails with firewalls. (Krb4 also embedded the IP in the ticket, but I don't believe this is required in Krb5; that said, it's been a long time since I looked at Kerberos.) Once you have the service credential (as opposed to the TGT, which you would use to get the service credential), firewalls are irrelevant to Kerberos. As for authentication provider…this obviously is not the place to discuss cross-realm authentication in Kerberos, but rest assured it works fine. It requires the realm administrators to set up a trust relationship, however, which is the disadvantage relative to certificates. As far as KDC availability goes—it is incredibly stable; the master KDC at MIT used to run on an old Ultrix machine, and it ran for so long that the Ultrix operating system uptime counter rolled over and crashed the machine. The KDC is also surprisingly low traffic—it uses UDP, so you avoid the overhead of TCP (at the expense of having to implement exponential backoff), and the KDC is only contacted when you need to get your initial TGT and later the first time you need to contact a particular service, thanks to the credentials cache. Finally, it is very easy to set up redundant slave KDCs. -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Monitoring / Billing Architecture proposed
On Sun, 2012-04-22 at 20:50 +0200, Luis Gervaso wrote: I want to share the architecture i am developing in order to perform the monitorig / billing OpenStack support: 1. AMQP Client which listen to RabbitMQ / QPid (this should be interchangeable) (Own Stuff or ServiceMix / Camel) 2. Events should be stored on a NoSQL document oriented database (I think mongodb is perfect, since we can query in a super easy fashion) Except for the use of MongoDB, the above seems to me to be almost identical to the notifications system already in Nova, which Yagi consumes. Have you looked at our existing notifications? Yagi? One or both might solve at least parts of your problem… -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] control user quota
On Thu, 2012-04-12 at 11:10 -0400, Eoghan Glynn wrote: Project-specific quotas may be set via the nova CLI, e.g. $ nova quota-update tenant_ID --instances=50 otherwise the configured default quota is inherited. Since you're still on diablo, the new quota classes mechanism would not be relevant. I should also point out that nova quota-update did not exist in the diablo release of python-novaclient… -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Image API v2 Draft 4
On Tue, 2012-04-10 at 10:05 -0700, Justin Santa Barbara wrote: I wasted a lot of time with nova's XML support; I'm sure the Java binding was the only project ever to try to use it; we'd have been able to proceed much faster if we'd just stuck with JSON - we now have a horrible hybrid, where JSON is used for some calls because the XML has/had bugs. Well, hopefully the XML support has been a little better since my templates stuff went in. Either way, though, if I had the choice, I'd rip all of nova's XML support out tomorrow… -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Nova] removing nova-direct-api
On Mon, 2012-04-09 at 11:58 -0700, Vishvananda Ishaya wrote: +1 to removal. I just tested to see if it still works, and due to our policy checking and loading objects before sending them into compute.api, it no longer functions. Probably wouldn't be too hard to fix it, but clearly no one is using it so lets axe it. Also +1 for removal. I discovered this thing when I was first trying to figure out how the API worked, and it confused me no end… -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Quota classes
On Fri, 2012-03-30 at 08:31 -0400, Eoghan Glynn wrote: A couple of quick questions on how this quota class mechanism is intended to work ... - how is the mapping between project and quota-class established? I was expecting a project_quota_class_association table or some-such in the nova DB. Is this association maintained by keystone instead? - is the quota_class attribute currently being set on the request context anywhere in the dispatch path? Is the idea that the auth middleware takes care of this? The basic answer is that there isn't anything in nova right now that does this, partly because it's a slightly difficult question to answer correctly for everyone. In my testing environment, for instance, I use a Turnstile preprocessor to set the quota_class attribute on the request context to be the same as the selected rate limit class. I envisioned that, ultimately, the quota_class would be set by the authentication processing middleware(s), but I'm not against adding an association to nova to manage that. -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Quota classes
On Fri, 2012-03-30 at 14:41 -0400, Eoghan Glynn wrote: I envisioned that, ultimately, the quota_class would be set by the authentication processing middleware(s), but I'm not against adding an association to nova to manage that. Presumably we'd also need some additional logic in the quota-classes API extension to allow tenant-to-quota-class mappings be established and torn down? Well, yeah :) -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Distributed rate-limiting
On Thu, 2012-03-29 at 22:58 +0100, Day, Phil wrote: - As you get the tenant id from the context I assume this module has to come after the authentication in the pipeline. Yes, I have made that assumption. It seems reasonable, given that the existing rate-limit middleware is right after authentication as well. Have you thought about using the tenant_id in the URL instead ? (I'm thinking of the case where you want rate limit requests into the authentication system as well as Nova itself). No, I haven't. I don't trust the user, which is where the tenant_id in the URL comes from. I do trust the auth system, which is why I want to use the tenant ID from the context. (And yes, you could argue that authz would prevent their access to other tenants anyway, but why make nova have to check authz if rate limiting would stop them in the first place?) As for rate limiting requests into the authentication system, I'd suggest using a Limit subclass which uses the remote IP address in place of a tenant ID, at least for the user endpoint. I don't think we want any rate limiting at all on the service side of Keystone; our current architecture means that Keystone is going to be hit a *lot*: at least once for each request that hits Nova, and more in certain cases (i.e., instance boot, where we'll have to hit quantum and glance as well). - Does this work for EC2 as well as OSAPI ? Actually, it didn't occur to me to test, given that I don't really use the EC2 API. I don't think there's anything in the basic architecture which would be incompatible with EC2; the only possible sticking point that occurs to me is the URL construction in nova_limits:NovaClassLimit.route(): if the URL specified is prefixed with '/v1.1/' or '/v2/', the version identifier is dropped (otherwise the route wouldn't match). That would be easy to work around; simply extend NovaClassLimit and override route() to do the appropriate transformation for EC2. Any EC2 experts want to weigh in? -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Caching strategies in Nova ...
On Fri, 2012-03-23 at 13:43 +, Gabe Westmaas wrote: However, I kind of expect that many users will still poll even if they know they won't get new data until X time. I wish there was some kind of way for us to issue push notifications to the client, i.e., have the client register some sort of callback and what piece of data / state change they're interested in, then nova would call that callback when the condition occurred. It probably wouldn't stop polling, but we could ratchet down rate limits to encourage users to use the callback mechanism. Of course, then there's the problem of, what if the user is behind a firewall or some sort of NAT... :/ -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Caching strategies in Nova ...
On Fri, 2012-03-23 at 08:55 -0300, Sandy Walsh wrote: I don't doubt for a second the db is the culprit for many of our woes. The thing I like about internal caching using established tools is that it works for db issues too without having to resort to custom tables. SQL query optimization, I'm sure, will go equally far. For that matter, I wouldn't be surprised if there were things we could do to nova's DB to speed things up. For instance, what if we supported non-SQL data stores? -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Being pedantic about pedanticism: HACKING styleguide
On Thu, 2012-03-22 at 11:22 -0500, Andrew Bogott wrote: Nova, this: A docstring ends with an empty line before the closing quotations. Huh? I thought I removed that... -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Being pedantic about pedanticism: HACKING styleguide
On Thu, 2012-03-22 at 13:47 -0400, Doug Hellmann wrote: Why are those sorts of instructions replicated in each project in the first place? Shouldn't they be in the wiki? Well, you're both right and wrong. Right in that they should be in a wiki somewhere. Wrong in that they should *also* be in the project in a prominent place—someone just getting started is likely to start with exactly one project, and it would be best to have the HACKING instructions where they can find them in that project. Besides, copying bits is cheap, right? :) -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Quota classes
On Sat, 2012-03-17 at 12:24 -0400, Jay Pipes wrote: On 03/16/2012 07:02 PM, Jesse Andrews wrote: There is the concept of limits that are very similar. Should we align quotas limits? Jesse: I'll point out that they are intimately related; they just have different names. You can see the quotas using the novaclient absolute-limits command. (Rate limits are different.) Oh, yes please! :) And make it configurable via a REST API, since editing config files ain't the most admin-friendly thang ;) Quotas are already configurable via an extension, but no CLI command was available for manipulating them in python-novaclient (although the API support was there). My quota classes patch to novaclient adds the CLI commands as well. -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Distributed rate-limiting
On Sat, 2012-03-17 at 12:31 -0400, Jay Pipes wrote: Kevin, you've really impressed me. Well documented, well thought-out code. Yeah, well…I got into the habit of documenting my code well when I wrote a very large project and discovered I was forgetting how to use its pieces :) I hope you won't mind if I contribute a REST-ful interface for configuration management and status reporting? Not at all; I designed the limit classes—specifically the limit attributes—to be easily introspectable for exactly that sort of purpose. -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] Distributed rate-limiting
Howdy, folks. I've been working on a replacement for nova's rate-limiting middleware that will handle the multiple-node case, and I've developed a fairly generic rate-limiting package, along with a second package that adapts it to nova. (This means you could also use this rate-limiting setup with, say, glance, or with any other project that uses Python middleware.) Here is some information: * Turnstile Turnstile is a piece of WSGI middleware that performs true distributed rate-limiting. System administrators can run an API on multiple nodes, then place this middleware in the pipeline prior to the application. Turnstile uses a Redis database to track the rate at which users are hitting the API, and can then apply configured rate limits, even if each request was made against a different API node. - https://github.com/klmitch/turnstile - http://pypi.python.org/pypi/turnstile * nova_limits This package provides the ``nova_limits`` Python module, which contains the ``nova_preprocess()`` preprocessor, the ``NovaClassLimit`` limit class, and the ``NovaTurnstileMiddleware`` replacement middleware class, all for use with Turnstile. These pieces work together to provide class-based rate limiting integration with nova. - https://github.com/klmitch/nova_limits - http://pypi.python.org/pypi/nova_limits Both packages should be fairly well documented (start with README.rst), and please feel free to log issues or make pull requests. -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] Quota classes
I wanted to let everyone know about a quota classes blueprint I've submitted; you can find the details here: * https://blueprints.launchpad.net/nova/+spec/quota-classes * http://wiki.openstack.org/QuotaClass I've already implemented this blueprint and pushed to Gerrit, but have it -2'd for right now since we haven't opened trunk yet for Folsom. If you'd like to have a look at it, the relevant changes are: * Nova: https://review.openstack.org/#change,5298 * Nova client: https://review.openstack.org/#change,5299 -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Removal of VSA Code
On Thu, 2012-03-15 at 09:02 -0700, Vladimir Popovski wrote: I was not aware of any issue with VSA code in diablo/stable (or at least major issues). I'll point out that the code we're concerned about is the code in trunk, not the code in diablo/stable. There have been substantial changes to the code since diablo was released, which has resulted in bitrot in the VSA code and the attendant breakages to which Vish is referring. -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] WebOb + DeprecationWarning
On Wed, 2012-03-07 at 22:40 -0800, Maru Newby wrote: I'm using a devstack-configured box with all the latest code and am running into DeprecationWarning wherever weob.Request.str_[GET,PUT,cookies,params] are accessed (they are being replaced by unicode equivalents). Since Python 2.7 does not ignore DeprecationWarning, and I am running on Python 2.6, the warnings are being thrown as exceptions. They're being thrown as exceptions? I thought the default in Python 2.6 was to report them, not to throw them. Did someone change the warnings settings to throw instead? I then realized that the nova api is similarly afflicted, and thought that some discussion might be warranted since so many projects were affected: 1. Should DeprecationWarning be ignored by OpenStack projects when using Python 2.7? I vote 'no' on development. I'd say it makes sense to ship final releases with deprecation warnings disabled, but they exist to warn us developers that some interface is going away, and we should pay attention to that. That said, unless you're specifically hunting deprecation warnings, I wouldn't set them up to throw exceptions… 2. If no to #1, should OpenStack projects be proactively surveyed for use of deprecated webob.Request properties, with an eye towards replacing such use immediately? Note that the string properties will not be removed until WebOb 1.2 and all projects are currently on 1.0.8. I say yes. 3. As a follow-on to #2, is there going to be any fallout from switching from string to unicode webob.Request properties? Web apps generally code defensively against non-ascii input, but being new to OpenStack I'm not sure how well this best-practice has been adhered to. I think the only way to really know is going to be to go there and exercise it, then make sure everything handles it OK. Maybe we should consider some unit tests? -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Keystone should to Apache HTTPD.
On Thu, 2012-03-01 at 14:05 -0500, Adam Young wrote: The traffic in an Openstack deployment to a Keystone server is going to be about two orders of magnitude less than any other traffic, and is highly unlikely to be the bottleneck. Not quite. I wrote this up, back in November: http://etherpad.openstack.org/keystone-scalability Since then, of course, Keystone has gone through some major cleanups that have improved its efficiency, but, as Vish pointed out in the other thread, every service still has to hit Keystone to verify a given token, which makes Keystone have the highest number of hits for any given operation…which in turn makes it *the* most likely bottleneck. -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Remove Zones code - FFE
On Sat, 2012-02-18 at 19:36 +, Ed Leafe wrote: I still prefer 'cell'. The parallel to single celled / multi-cellular life forms makes sense, and there is really no overloading of the word in the world of computers. I'll point out the concept of AFS cells. That said, +1 for cell… -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Remove Zones code - FFE
On Wed, 2012-02-15 at 00:00 +, Monsyne Dragon wrote: Other possibilities: * Container (not recommended, as it is overloaded with Solaris or Linux container virtualization) * ServerGroup * HostGroup * Group * Collection - Set - Cell - Huddle - Constellation - Herd/Flock//Pod/Animal metaphor of choice. - System - Realm - Universe - Galaxy - Kingdom - Nebula ... -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Scaling][Orchestration] Zone changes. WAS: [Question #185840]: Multi-Zone finally working on ESSEX but cant nova list (KeyError: 'uuid') + doubts
On Thu, 2012-01-26 at 10:13 -0600, Blake Yeager wrote: Does anyone have other thoughts about how we ensure we are all working toward building a massively scalable system? I recently discussed with both Sandy and Ziad a multi-realm extension to Keystone. I've documented my thoughts on it as the following blueprint: https://blueprints.launchpad.net/keystone/+spec/multi-realm The spec is at: http://wiki.openstack.org/MultiRealmKeystone And your email provides a perfect starting point for kicking off a discussion on the concept :) -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] Deprecations for you to be aware of
Greetings. Changes have recently been made to the nova API which deprecate the old RequestExtension and ActionExtension classes in favor of ControllerExtension. For now, the old-style extensions will work, but I am currently working on a patch that will remove them entirely (and, by extension, also remove ExtensionMiddleware, which will remove the need for LazySerializationMiddleware). Any third-party extensions should probably be updated to use the new ControllerExtension interface soon-ish. For examples of how this may be done, check out https://review.openstack.org/#change,3020 and https://review.openstack.org/#change,3049 (ActionExtension and RequestExtension conversion examples, respectively). (Just FYI, I'm leaving do-nothing deprecated versions of ExtensionMiddleware and LazySerializationMiddleware; this will give people breathing space to update their api-paste.ini files.) -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Proposal to limit decorator usage
On Tue, 2012-01-17 at 11:09 -0500, Lorin Hochstein wrote: Decorators -- A function or method should not have more than two decorators applied to it where it is defined. I'll point out that current discussion on that merge thread is favoring a different idea altogether: restrict decorators to only those that do not mess with the conceptual interface of a function. Mark provides two examples of decorators which would not fit that restriction, and Naveed objects to one of them because he feels that validation of input arguments is a legitimate use of decorators. -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Configure Rate limits on OS API
On Tue, 2012-01-10 at 16:06 -0600, Blake Yeager wrote: Am I correct in assuming that this will only work with setting the global limits? Is there anyway to specify different limits for different accounts or groups of accounts? You are correct that the 'limits = […]' syntax sets global limits. However, if you use 'user:username = […]', that should allow you to set specific limits for a given user. -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] openstack-common
On Tue, 2012-01-03 at 19:54 +, Ewan Mellor wrote: I'd love to see openstack-common get off the ground, so I'm all in favor of this. One question: why do you feel that you need such strong backwards compatibility? If someone makes a change in openstack-common and makes simultaneous changes in all OpenStack projects to match, isn’t that sufficient? No simultaneous change is ever actually simultaneous. We see this all the time with interop between keystone (in particular), nova, and glance. Once openstack-common gets into the picture, the interop problems stand to be significantly worse; if one tiny change is not backwards compatible, you break *everything* that uses openstack-common. The good thing, of course, is that it'll be noticed quickly; the bad thing is that all work gets significantly impeded until the fix(es) go in. Speaking from experience: it is possible to preserve N+2 backwards compatibility while still making major enhancements. It can be a pain in the butt sometimes, but it is doable, and, in cases like openstack-common, I think it is necessary. -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Problems with run_tests.sh on 11.10
On Fri, 2011-12-30 at 12:30 -0700, John Griffith wrote: Looking in .venv M2Crypto was NOT installed, I ran things on a clean 11.10 install last night and the results were the same. Repeat on a clean 11.04 and everything is fine. Looking at the venv setup script to see if I can figure out why this failed. Seems odd I've reproduced on multiple machines but nobody else has seen this? Maybe there's a step I'm missing still? I believe M2Crypto is a C extension, which means that it has to be compiled. One of its compilation dependencies is likely SSL, given the name of the undefined symbol. So, my best guess at the best way to resolve your problem is to ensure you have the openssl-dev package installed (or whatever name it really has on 11.10). -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Compute API Versioning
On Wed, 2011-12-21 at 11:41 -0600, Bryan Taylor wrote: I would suggest taking at least learning something from libtool. libtool does this stuff really well if you pay attention to the rules. They are as follows: Libtool is not a web service API. I don't see the analogy here. It's a fine tool for what it does - encapsulating shared software libraries. No, it isn't. But the point is that it does *API* versioning, not code versioning. The docs actually tell you that if you want to lock your API versioning to your code versioning, you're doing it wrong (though they also tell you how, and tell you what will break if you do). That said, it's hard for me to see how we could effectively communicate current and age to clients. (revision doesn't really have a place, except possibly advertising that certain bugs got fixed…) -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] using objects returned from DB layer
On Thu, 2011-12-15 at 07:10 +, Chris Behrens wrote: There's a mix of usage throughout the code, and I know some people are just matching the surrounding code. But, in a number of cases, I've asked for these to be corrected to the latter, on assumption that the DB layer will be returning dictionaries at some point vs the models. It also pushes the code towards consistent usage. But I might be the only Nova Core member looking at this and/or maybe my assumption is wrong. So, I ask here: Should Nova Core make an effort to reject patches with the former format? Or did I miss any DB layer plans where the former format is now preferred? I have two, diametrically opposed answers. 1. When doing reviews, I've generally tried to enforce the dict access format, because it's been my understanding that that is the direction we're going in. 2. However, I violently disagree with the idea that the DB layer must return dicts. It does not, even if you start talking about allowing use of other kinds of databases. We can, and should, wrap these things in objects, upon which we can call methods that do things—i.e., we should, you know, actually use object-oriented programming. -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] resize error (diablo)
On Thu, 2011-11-17 at 08:26 +, adrian_f_sm...@dell.com wrote: The resize operation requires you have at least two hosts. Try setting the flag “allow_resize_to_same_host=true”. Also be aware that, due apparently to bitrot, the code which is supposed to keep the scheduler from proposing the instance's current host for the resize target vanished. I'm in the process of fixing this up: https://review.openstack.org/#change,1593 -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] describing APIs for OpenStack consumers
On Thu, 2011-10-27 at 10:50 -0700, Nati Ueno wrote: I tried to generate WADL from nova code. I could get all resource URI and method from Routes object. However, I could not get input parameters from code. (The api method accesses body argument directly. This is also bad for input validation QA effort.) But If we use some annotations, it may be solved. Also, It looks possible to generate Resource definitions from model class. I've also considered that my templates code could be adapted to perform deserialization as well as serialization. You'd have to add a bit more data to its structure to pull that trick off, but it'd show you the exact structure of the XML input and output for automation tricks like this. -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] describing APIs for OpenStack consumers
On Wed, 2011-10-26 at 12:14 -0400, Jay Pipes wrote: That's fine for generating a WADL for existing APIs that are already implemented. Not so good for proposed APIs ;) Oh, certainly, but there the auto-generation could be used to verify that the code implements the proposed API :) -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] nova.conf changes for API extensions
Short form: --osapi_extensions_path is now gone; use --osapi_extension, which takes a different form of input. Long form: Extensions have been refactored in a couple of ways. For one, the get_name(), get_alias(), get_description(), get_namespace(), and get_updated() accessor methods are all gone; set name, alias, namespace, and updated attributes instead, and use the docstring for the description. The more important change, for the purposes of nova.conf, is the replacement of --osapi_extensions_path. Before, this took a directory name, and all extensions from that directory were loaded. Now, you use one or more instances of --osapi_extension to name a callable located within the Python path. For instance, if you extension was Foxinsocks, located in foxinsocks.py, you would now use something like package.path.to.foxinsocks.Foxinsocks. Since this now loads only a single extension, you are now able to use --osapi_extension multiple times. Note that it is not necessary to name every single extension that's distributed with nova in nova.api.openstack.contrib; extensions in that directory will be automatically loaded as long as they comply with the previous naming convention (class needs the exact same name as the module, with the first character upper-cased). Further details: The argument to --osapi_extension must be a callable taking one argument--the extension manager. It must arrange to call the register() method of the extension manager, passing it an extension object (not class). The ExtensionDescriptor object has been extended with an __init__() method that does this, so all extensions inheriting from it are automatically covered unless they override __init__() and don't pass the argument to the superclass constructor. To auto-load the extensions in nova.api.openstack.contrib, the nova/api/openstack/contrib/__init__.py module now has a standard_extensions() callable which walks the directory tree rooted there and loads all other modules it finds, using the previously established conventions. This standard_extensions() function is simply added to the default list of extensions to load. -- Kevin L. Mitchell kevin.mitch...@rackspace.com This email may include confidential information. If you received it in error, please delete it. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
