Re: [openstack-dev] [nova][limits] Does ANYONE at all use the quota class functionality in Nova?
> On 10/24/18 10:10, Jay Pipes wrote: > > Nova's API has the ability to create "quota classes", which are > > basically limits for a set of resource types. There is something called > > the "default quota class" which corresponds to the limits in the > > CONF.quota section. Quota classes are basically templates of limits to > > be applied if the calling project doesn't have any stored > > project-specific limits. For the record, my original concept in creating quota classes is that you'd be able to set quotas per tier of user and easily be able to move users from one tier to another. This was just a neat idea I had, and AFAIK, Rackspace never used it, so you can call it YAGNI as far as I'm concerned :) > > Has anyone ever created a quota class that is different from "default"? > > > > I'd like to propose deprecating this API and getting rid of this > > functionality since it conflicts with the new Keystone /limits endpoint, > > is highly coupled with RAX's turnstile middleware I didn't intend it to be highly coupled, but it's been a while since I wrote it, and of course I've matured as a developer since then, so *shrug*. I also don't think Rackspace has ever used turnstile. > > and I can't seem to > > find anyone who has ever used it. Deprecating this API and functionality > > would make the transition to a saner quota management system much easier > > and straightforward. I'm fine with that plan, speaking as the original developer; as I say, I don't think Rackspace ever utilized the functionality anyway, and if no one else pipes up saying that they're using it, I'd be all over deprecating the quota classes in favor of the new hotness. -- Kevin L. Mitchell signature.asc Description: This is a digitally signed message part __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] pep8 failures on master
On Wed, 2018-04-04 at 07:54 +1000, Michael Still wrote: > Thanks to jichenjc for fixing the pep8 failures I was seeing on > master. I'd decided they were specific to my local dev environment > given no one else was seeing them. > > As I said in the patch that fixed the issue [1], I think its worth > exploring how these got through the gate in the first place. There is > nothing in the patch which stops us from ending up here again, and no > real explanation for what caused the issue in the first place. While there was no discussion in the patch, the topic of the patch hints at the cause: "fix_pep8_py3". These were probably pep8 errors that would only occur if pep8 was running under Python 3 and not Python 2. The first error was fixed by removing a debugging print that was formatted as "print (…)", which would satisfy pep8 under Python 2—since 'print' is a statement—but not under Python 3, where it's a function. The second error was in a clause protected by six.PY2, and was caused by "unicode" being missing in Python 3; the solution jichenjc chose there was to disable the pep8 check for that line. The only way I can imagine stopping these errors in the future would be to double-up on the pep8 check: have the gate run pep8 under both Python 2 and Python 3. -- Kevin L. Mitchell signature.asc Description: This is a digitally signed message part __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Proposing Balazs Gibizer for nova-core
On Tue, 2017-08-22 at 20:18 -0500, Matt Riedemann wrote: > I'm proposing that we add gibi to the nova core team. He's been around > for awhile now and has shown persistence and leadership in the > multi-release versioned notifications effort, which also included > helping new contributors to Nova get involved which helps grow our > contributor base. +1 from me -- Kevin L. Mitchell signature.asc Description: This is a digitally signed message part __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [keystone][api] Backwards incompatible changes based on config
On Fri, 2017-08-04 at 14:52 -0700, Morgan Fainberg wrote: > > Maybe not, but please do recall that there are many deployers out > > there > > that track master, not fixed releases, so we need to take that > > level of > > compatibility into account. > > > > I am going to go out on a limb and say that we should not assume that > if code merges ever it is a contract because someone might be > following master. The contract should be for releases. We should do > everything we can to avoid breaking people, but in the case of an API > contract (behavior) that was never part of a final release, it should > be understood this may change if needed until it is released. > > This is just my $0.002 as this leads rapidly to "why bother having > real releases" if everything is a contract, let someone take a > snapshot where they're happy with the code to run. You're devaluing > the actual releases. In my view, following master imposes risks that deployers should understand and be prepared to mitigate; but I believe that it is our responsibility to acknowledge that they're doing it, and make a reasonable effort to not break them. There are, of course, times when no reasonable effort will avoid breaking them, and in those cases, I feel that the reasonable course of action is to try to notify them of the upcoming breakage. That's why then I went on to suggest that fixing this problem in keystone shouldn't require a version bump in this case: it _is_ a breakage that's being fixed. -- Kevin L. Mitchell signature.asc Description: This is a digitally signed message part __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [keystone][api] Backwards incompatible changes based on config
On Fri, 2017-08-04 at 16:45 -0400, Kristi Nikolla wrote: > Is this the case even if we haven’t made any final release with the change > that introduced this issue? [0] > > It was only included in the Pike milestones and betas so far, and was not > part of the Ocata release. Maybe not, but please do recall that there are many deployers out there that track master, not fixed releases, so we need to take that level of compatibility into account. > Therefore the call which now returns a 403 in master, returned a 2xx in > Ocata. So we would be fixing something which is broken on master rather > than changing a ‘contract’. > > 0. > https://github.com/openstack/keystone/commit/51d5597df729158d15b71e2ba80ab103df5d55f8 I would be inclined to accept this specific change as a bug fix not requiring a version bump, because it is a corner case that I believe a deployer would view as a bug, if they encountered it, and because it was introduced prior to a named final release. -- Kevin L. Mitchell signature.asc Description: This is a digitally signed message part __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [python-openstacksdk] Status of python-openstacksdk project
On Fri, 2017-08-04 at 12:26 -0700, Boris Pavlovic wrote: > By the way stevedore is really providing very bad plugin experience > and should not be used definitely. Perhaps entrypointer[1]? ;) [1] https://pypi.python.org/pypi/entrypointer -- Kevin L. Mitchell signature.asc Description: This is a digitally signed message part __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [ironic] Translations removal
On Wed, 2017-03-22 at 18:44 +, Taryma, Joanna wrote:
> Thanks for pointing out that 2 and 3 won’t actually work, I apologize
> for the confusion it could’ve created.
>
>
>
> I don’t like the option 6, because making user-messages friendlier was
> the whole purpose of translation. Mixing languages in exception would
> be even worse than doing it in logs, IMHO. What is more – if there’s a
> custom message passed to exception (e.g. MyException(“My message” %
> {k: v}), it overwrites the default one, so it would end up with
> English-only message.
>
>
>
> Option 5 looks nice (and easy), but I don’t think that it will be very
> good if all other components will allow showing translated messages
> and Ironic won’t.
>
> Seems like *if* we want to translate entire exception messages, we’re
> left with option 1 only, right?
It occurred to me that i18n may provide a means of handling this
directly; I don't know for sure, but one of the library developers could
probably comment. IIRC, i18n uses (or can use) "lazy translation",
where it keeps around the original message but only translates it on
output. If that's the case, that may help provide a solution to
translate user-visible messages while leaving logs untranslated.
--
Kevin L. Mitchell
signature.asc
Description: This is a digitally signed message part
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [ironic] Translations removal
On Tue, 2017-03-21 at 22:10 +, Taryma, Joanna wrote:
> However, pep8 does not accept passing variable to translation
> functions, so this results in ‘H701 Empty localization string’ error.
>
> Possible options to handle that:
>
> 1) Duplicate messages:
>
> LOG.error(“”, {: })
>
> raise Exception(_(“”) % {: })
>
> 2) Ignore this error
>
> 3) Talk to hacking people about possible upgrade of this check
>
> 4) Pass translated text to LOG in such cases
>
>
>
> I’d personally vote for 2. What are your thoughts?
When the translators go to translate, they generally only get to see
what's inside _(), so #2 is a no-go for translations, and #3 also is a
no-go.
--
Kevin L. Mitchell
signature.asc
Description: This is a digitally signed message part
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [All] IRC Mishaps
On Thu, 2017-02-09 at 21:14 -0600, Sean McGinnis wrote: > Oh, there was the time I accidentally banned everyone from the Cinder > channel. That was fun. :) Reminds me of a time that a server operator on another IRC network accidentally managed to ban everyone from that network…that happened to be the #3 largest IRC network at the time. Fixing *that* was fun :) -- Kevin L. Mitchell signature.asc Description: This is a digitally signed message part __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Nominating Stephen Finucane for nova-core
On Fri, 2016-12-02 at 09:22 -0600, Matt Riedemann wrote: > I'm proposing that we add Stephen Finucane to the nova-core team. +1 from me. -- Kevin L. Mitchell signature.asc Description: This is a digitally signed message part __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all][API WG] How does one best express a time interval as an API filtering query?
On Thu, 2016-10-06 at 10:59 -0400, Jay Pipes wrote: > > Currently, we propose: > > > > ?finished_at=/&finished_at=/&... > > The issue I have with the is that it uses a / character to delimit the a > and b dates. > > In keeping with the style of the API-WG filter operators, I would > propose something like this: > > ?finished_at=between:ISO_DATE_A:ISO_DATE_B&finished_at=between:ISO_DATE_C:ISO_DATE_D Problem with that is that ':' is a valid character within an ISO date, though I do like the 'between' prefix. Now, '/' can be used if it's URL encoded, but I agree that that is non-ideal. How about a syntax something like: ?finished_at=between:ISO_DATE_A@ISO_DATE_B&finished_at=between:ISO_DATE_C@ISO_DATE_D -- Kevin L. Mitchell signature.asc Description: This is a digitally signed message part __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all][oslo] Parse ISO8601 (open) time intervals
On Thu, 2016-09-29 at 12:07 -0400, Doug Hellmann wrote: > Excerpts from milanisko k's message of 2016-09-29 09:48:40 +: [snip] > > > Doug, I'm afraid that dateutil.parser.parse doesn't support intervals > > either: http://paste.openstack.org/show/583452/ > > Is there any interest in oslo_utils.timeutils parsing ISO8601 time > > intervals as in [2]? > > What do OS projects use instead especially w/r http api queries? > > Have you talked with the dateutils maintainers about adding support > there? It would still be better to contribute to dateutil than to > put the function in a library not meant for anyone else outside of > OpenStack to use. You might also want to look at the timestring library… -- Kevin L. Mitchell signature.asc Description: This is a digitally signed message part __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Let's kill quota classes (again)
The original concept of quota classes was to allow the default quotas applied to a tenant to be a function of the type of tenant. That is, say you have a tiered setup, where you have gold-, silver-, and bronze-level customers, with gold having lots of free quota and bronze having a small amount of quota. Rather than having to set the quotas individually for each tenant you created, the idea is that you set the _class_ of the tenant, and have quotas associated with the classes. This also has the advantage that, if someone levels up (or down) to another class of service, all you do is change the tenant's class, and the new quotas apply immediately. (By the way, the turnstile integration was not part of turnstile itself; there's a turnstile plugin to allow it to integrate with nova that's quota_class-aware, so you could also apply rate limits this way.) Personally, it wouldn't break my heart if quota classes went away; I think this level of functionality, if it seems reasonable to include, should become part of a more unified quota API (which we're still struggling to come up with anyway) so that everyone gets the benefit…or perhaps shares the pain? ;) Anyway, I'm not aware of anyone using this functionality, though it might be worth asking about on the operators list—for curiosity's sake, if nothing else. It would be interesting to see if anyone would be interested in the original idea, even if the current implementation doesn't make sense :) On Wed, 2016-07-13 at 14:47 -0500, Matt Riedemann wrote: > We got a bug that the os-quota-class-sets API isn't documented: > > https://bugs.launchpad.net/nova/+bug/1602400 > > That's probably because we hate it and no one understands it. > > See this previous thread about trying to sort this out from the long > long ago: > > https://lists.launchpad.net/openstack/msg12200.html > > We tried killing it before, but it turns out, it's actually used by > something! > > http://lists.openstack.org/pipermail/openstack-dev/2014-May/036031.html > > But we didn't have integration testing in Tempest for default quotas at > that time (we added those tests in when we reverted the delete of the > API back in Juno). > > I got looking at this because of the quota_class attribute in the nova > RequestContext: > > https://github.com/openstack/nova/blob/93cc5e3ffd2867bdb39a707a230c1efc6ed2f5f4/nova/context.py#L138-L141 > > That led me to markmc's thread about that only being there for the > turnstile project and some old API rate limiting stuff that Rackspace > was doing out of tree (it appears to set a type of middleware for a > quota class for rate limiting). > > Anyway, super duper out of tree stuff that is probably not even used > anymore (Vek - if you're reading, please speak up). > > I'll also point out that API rate limiting as a paste config was only in > the v2 API and that code was all dropped and the API rate limiting stuff > wasn't carried over for the v2.1 API, for good reason, see: > > http://lists.openstack.org/pipermail/openstack-operators/2016-June/010692.html > > You can still create unique quota classes via the os-quota-class-sets > API (it does a create if the update operation fails), but as far as I > can tell you can't really use those in any meaningful way. > > We really just have the 'default' quota class with a buttload of code > and plumbing to use that, which sucks, because it's all very complicated. > > So I think I'm going to start a pet project of rooting this stuff out > again, starting with nova.context.RequestContext.quota_class, unless > anyone has a good reason we should keep this in tree. > > I think we should also add a microversion to the API in Ocata to disable > the ability to create new quota classes, so that update is only update, > and a 404 for anything else. > -- Kevin L. Mitchell signature.asc Description: This is a digitally signed message part __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] disabling deprecated APIs by config?
On Wed, 2016-05-18 at 10:56 -0500, Dean Troyer wrote: > With my OSC hat on, I need to still support those APIs for a couple of > years at least, so I would love to have a way to discover that those > APIs have been disabled without requiring a round trip to get a 404. > This seems like a useful thing for the '/' route version information > when no '/capabilities'-like endpoint is available. This thread started me thinking about ways to indicate a deprecation to consumers of the API. Given that most of our APIs return a top-level dictionary with a single key, would it make any sense at all to add additional keys? We could add a 'warnings' key in that top-level dictionary that could document deprecations, among other things. (That said, /capabilities or equivalent would probably be superior for this particular case…) -- Kevin L. Mitchell signature.asc Description: This is a digitally signed message part __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [neutron][nova][oslo] Common backoff & timeout utils
On Wed, 2016-04-20 at 14:10 -0600, Boden Russell wrote: > Anyone adverse to me crafting an initial oslo patch to kick-off the > details on this one? Have you evaluated any existing solutions in this space? A quick search on PyPi turns up "backoff", which seems to provide several backoff strategies. Of course, it's a decorator-based backoff implementation, whereas I tend to implement iterator-based solutions, but there may already be a solution in that space as well… -- Kevin L. Mitchell signature.asc Description: This is a digitally signed message part __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all][infra] revert new gerrit
On Fri, 2016-03-18 at 20:58 +, Jeremy Stanley wrote: > Yes, we had a session on it several summits ago, a group of > contributors said they were going to work on developing it, pushed > up a skeleton repo, and then we never heard back from them after > that. Unfortunate. Yeah, unfortunately, we weren't able to get priority for the project, and I'm afraid it's probably not going to go anywhere now :/ -- Kevin L. Mitchell Rackspace __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all][tc] Proposal: Separate design summits from OpenStack conferences
On Mon, 2016-02-08 at 10:49 -0500, Jay Pipes wrote: > 5) Dealing with schwag, giveaways, parties, and other superfluous > stuff As a confirmed introvert, I have to say that I rarely attend parties, for a variety of reasons. However, I don't think our hypothetical design-only meeting should completely eliminate parties, though we can back off from some of the more extravagant ones. If we maintain at least one party, I think that would satisfy the social needs of the community without distracting too much from the main purpose of the event. Of course, I agree with eliminating the other distracting elements, such as schwag and giveaways… -- Kevin L. Mitchell Rackspace __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [nova][novaclient] Failing functional tests
I've been noticing that the functional tests on novaclient have been consistently failing, and the test failures appear to be related to keystone. I'm seeing tenant-create (using "admin" credentials) failing with 404, and a tenant-get on "admin" fails with "No tenant with a name or ID of 'admin' exists." Anyone have any insights into why "admin" would be missing from novaclient's functional test suite? -- Kevin L. Mitchell Rackspace __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [api] service type vs. project name for use in headers
On Thu, 2016-01-28 at 11:06 +, Chris Dent wrote: > I think it is high time we resolve the question of whether the > api-wg guidelines are evaluating existing behaviors in OpenStack and > blessing the best or providing aspirational guidelines of practices > which are considered best at a more universal level. From my historical perspective, the API WG had essentially two phases, with only the first phase getting general support at the time: 1. trying to document existing practices and recommend best practices; 2. establishing rules that all OpenStack APIs must adhere to. I think the first phase is essentially complete at this point, but I think Chris is right that it's high time to decide whether the guidelines are normative or informative…and my vote would be for normative, and with a focus on the API consumer. After all, an API is useless if it's a pain to use :) -- Kevin L. Mitchell Rackspace __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova][Glance] Xenplugin + Glance_v2 = Hate
On Thu, 2015-12-24 at 16:51 +0300, Mikhail Fedosin wrote: > Hello! As you may know there is a big initiative to adopt glance v2 > api in Nova and the important part is making related changes in > xenglugin. > Unfortunately xenplugin doesn't use neither nova.image api nor > glanceclient. Instead of this it has own http client implementation > and bunch of hardcoded 'v1' urls (example, > https://github.com/openstack/nova/blob/master/plugins/xenserver/xenapi/etc/xapi.d/plugins/glance#L130). > It leads to the fact, that it will be really hard to switch it on v2 api > right. > > Personally I see 2 solutions: > 1. Make xenplugin to adopt nova.image api, which will make it > version-agnostic. Here it's not easy to implement and we won't be able > to keep backward compatibility with the existing lowlevel code. > 2. Also hardcode v2 urls on par with v1 and do the same thing as in > nova.image - to determine current glance api version before request > and then use appropriate urls in methods. > > IMHO, the second solution is more preferable, because I understand how > to do it and the v1/v2 compatibility will be 100%. It guarantees that > we won't break any of existing deployments and it will allow to merge > glance_v2 code in nova.image much quicker. > > All opinions and advice will be very helpful. Thanks in advance! Unfortunately, the big problem is that, last I knew, Xenserver comes with Python 2.4 installed. This forces all sorts of constraints on the plugin code, including the inability to use interfaces like glanceclient, due to the unsupported Python version; this is why the plugin uses its own HTTP client implementation. If we're going to update the plugins to use more recent versions of Python, we're going to have to warn the operator community about the consequences well in advance, so that all the issues can be worked out… -- Kevin L. Mitchell Rackspace __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] no user/project validation in "nova quota-show"?
On Mon, 2015-12-07 at 18:21 -0600, Chris Friesen wrote: > Can someone explain why nova doesn't seem to be doing any validation in the > "nova quota-show" command? (At least in kilo/stable.) > > If I run: > nova --debug quota-show --tenant aprojectdoesnotexist --user nosuchuser > > the debug info shows: > > DEBUG (connectionpool:383) "GET > /v2/ceddf233621f4772a8b4f17de3d45e31/os-quota-sets/aprojectdoesnotexist?user_id=nosuchuser > > HTTP/1.1" 200 359 > > and it returns a reasonable-looking set of quota information. > > > > Shouldn't nova be complaining that the specified tenant/user don't actually > exist? 1. Nova doesn't know what tenants and users exist; that's something only Keystone knows. 2. There are defaults for quotas, which is how nova determines what quotas to apply to a tenant when there's no specific quota for that tenant in its database. That's why you're getting a reasonable-looking set of quota information. -- Kevin L. Mitchell Rackspace __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] [python-novaclient] microversions support
On Fri, 2015-12-04 at 18:58 +0200, Andrey Kurilin wrote: > This week I added 5 patches to enable 2.7-2.11 microversions in > novaclient[1][2][3][4][5]. I'm not bragging. Just want to ask everyone > who are working on new microversions: Please, do not forget to add > support of your microversion to official Nova client. Perhaps this is something we should add to the review guidelines—no API change can be merged to nova unless there is a pending change to novaclient to add support? We already more or less enforce the criteria that no addition to novaclient can be added unless the corresponding nova change has been merged… -- Kevin L. Mitchell Rackspace __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova]New Quota Subteam on Nova
On Thu, 2015-12-03 at 12:33 -0800, melanie witt wrote: > On the parallel request bug [2], if you check out alaski's comment #3, > he mentions the keypairs and security groups apis check quota without > using the reservation system. We should investigate why they don't and > whether we can convert them to use the reservation system. I think > that would provide a lot of improvement. So, there are three types of resources defined by the quota system: ReservableResource, which can be used with reservations; AbsoluteResource, which is used for things like checking quotas on the number of injected files (not even any sense *trying* to track that in the DB :); and a subclass of AbsoluteResource, CountableResource. Keypairs are a CountableResource, and the __init__() docstring says: Countable resources are those resources which directly correspond to objects in the database, i.e., instances, cores, etc., but for which a count by project ID is inappropriate. A CountableResource must be constructed with a counting function, which will be called to determine the current counts of the resource. And sure enough, nova.compute.api.KeypairAPI._validate_new_key_pair() uses the counting interface. I believe this came about from a basic limitation with the architecture of the quotas subsystem, which initially assumed that every resource would be accounted only to a tenant; this is a mismatch for something like keypairs, which are associated solely with specific users, or security group rules, which are associated solely with specific security groups. There are exactly three currently defined CountableResource quotas: security_group_rules, key_pairs, and server_group_members. I would guess that all three are prone to race conditions that are not a problem for other quota types. (For reference, the AbsoluteResource quotas are metadata_items, injected_files, injected_file_content_bytes, and injected_file_path_bytes; everything else is a ReservableResource.) -- Kevin L. Mitchell Rackspace __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova]New Quota Subteam on Nova
On Fri, 2015-11-20 at 16:13 +, Raildo Mascena wrote: > Me and other guys are working in the nested quota driver > (https://review.openstack.org/#/q/status:open+project:openstack/nova > +branch:master+topic:bp/nested-quota-driver-api,n,z) on Nova. > > > in addition, We want discuss the re-design of the quota implementation > on nova and in other projects, like cinder and neutron and we already > have a base spec for this here: > https://review.openstack.org/#/c/182445/4/specs/backlog/approved/quotas-reimagined.rst > > > So was I thinking on create a subteam on Nova to speed up the code > review in the nested quota implementation and discuss this re-design > of quotas. Someone have interest on be part of this subteam or > suggestions? Given my history of working on the nova quota code, I'm potentially interested in participating. However, I'd like to start off by asking if "quota" is the correct concept? It might be worthwhile to make a detour and consider what the problem is we're actually trying to solve. I think in the end it'll still end up being recognizably "quota", but I think it would be a worthwhile exercise and may provide better insights… -- Kevin L. Mitchell Rackspace __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Versioned notifications... who cares about the version?
On Mon, 2015-11-23 at 10:01 +, Alexis Lee wrote: > gord chung said on Fri, Nov 20, 2015 at 01:32:02PM -0500: > > On 20/11/15 11:33 AM, Alexis Lee wrote: > > >why would a producer spit out non-useful datapoints? If no-one cares > > >or will ever care, it simply shouldn't be included. > > > > ... right now the > > producer is just sending out a grab bag of data that it thinks is > > important but doesn't define who the audience is. ... > > > > ... whoever the producer of notifications is, it should know it's > > audience. > > Here I, with cdent, have to disagree. The criterion has to be > "potentially useful to someone", rather than tailoring production to the > known audience. > > > >The problem is knowing what each consumer thinks is interesting and that > > >isn't something that can be handled by the producer. > > ^ this is important. I think there is a minor disconnect here; when Gord talks about tailoring to a specific audience, there seems to be a jump to the conclusion that the audience must be ceilometer. I think Gord is instead suggesting what you could think of as an audience *class*—this notification is intended for metrics consumers, this one is intended for log monitors, etc. That kind of data could be included in the actual notification, so you still have a single notification stream. (I could envision later refinements allowing filtering, maybe by using multiple queues…) -- Kevin L. Mitchell Rackspace __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [oslo][all] Oslo libraries dropping python 2.6 compatability
On Wed, 2015-11-11 at 14:14 -0500, Davanum Srinivas wrote: > Any concerns? please chime in: > https://review.openstack.org/244275 Commented on the review, but I have to point out that python-novaclient, which, to my knowledge, still supports Python 2.6, also happens to depend on oslo.i18n, oslo.serialization, and oslo.utils. If we drop Python 2.6 compatibility on any of those three, we would also have to drop it from novaclient (and potentially other clients). Perhaps we need to have a discussion about whether the clients still need to support Python 2.6? -- Kevin L. Mitchell Rackspace __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] novaclient functional test failures
On Mon, 2015-10-12 at 12:16 -0400, Sean Dague wrote: > On 10/12/2015 11:54 AM, Kevin L. Mitchell wrote: > > Functional tests on novaclient (gate-novaclient-dsvm-functional) have > > started failing consistently. The test failures all seem to be for an > > HTTP 300, which leads me to suspect a problem with the test environment, > > rather than with the tests in novaclient. Anyone have any insights as > > to how to address the problem? > > Do you have a link to failed jobs? Or a bug to start accumulating that in? And I thought about including such a link right after I hit "Send". Figures :) http://logs.openstack.org/77/232677/1/check/gate-novaclient-dsvm-functional/2de31bc/ (For review https://review.openstack.org/232677) http://logs.openstack.org/99/232899/1/check/gate-novaclient-dsvm-functional/6d6dd1d/ (For review https://review.openstack.org/232899) The first review does some stuff with functional tests, but the second is a simple global-requirements update, and both have the same failure signature. -- Kevin L. Mitchell Rackspace __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [nova] novaclient functional test failures
Functional tests on novaclient (gate-novaclient-dsvm-functional) have started failing consistently. The test failures all seem to be for an HTTP 300, which leads me to suspect a problem with the test environment, rather than with the tests in novaclient. Anyone have any insights as to how to address the problem? -- Kevin L. Mitchell Rackspace __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Scheduler proposal
On Wed, 2015-10-07 at 23:17 -0600, Chris Friesen wrote: > Why is it inevitable? Well, I would say that this is probably a consequence of the CAP[1] theorem. > Theoretically if the DB knew about what resources were originally available > and > what resources have been consumed, then it should be able to allocate > resources > race-free (possibly with some retries involved if racing against other > schedulers updating the DB, but that would be internal to the scheduler > itself). The problem is, it can't. The scheduler may be making the decision at the same time that an update from a compute node is in flight, meaning that the scheduler is missing (at least) one piece of information. When you include a database, that just makes the possibility of missing an in-flight update worse, because you also have to factor in the latency of the database update as well. Also, we have to factor in the possibility that there are multiple schedulers in play, which further worsens the possibility of in-flight information critical to the scheduling decision. If you employ some sort of locking to try to mitigate all this, you've just effectively thrown away the scalability that deploying multiple schedulers was supposed to buy you. [1] https://en.wikipedia.org/wiki/CAP_theorem -- Kevin L. Mitchell Rackspace __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] Doc dependencies
On Fri, 2015-10-02 at 11:00 -0400, Sean Dague wrote: > On 10/02/2015 10:53 AM, Kevin L. Mitchell wrote: > > The Pillow-breaking-gate issue was related to having doc dependencies > > listed in the test-requirements.txt; however, those dependencies are not > > needed for testing, except for doc testing and generation. What do > > people think about creating a new "doc-requirements.txt" file that would > > contain only the doc dependencies? The appropriate doc environments in > > tox.ini would then need to be extended to pull in that file, and of > > course the global requirements tooling would have to be enhanced to > > recognize the new file as well. > > It would still have broken the gate, we test if docs build on every commit. Yes—this is why I characterized it as "related"—but the test failure would have at least been isolated to the docs test. That would have allowed reviews to get useful test results, instead of having a dependency installation issue. It's ultimately just a thought; it feels wrong for us to install dependencies not directly related to testing, even if we also test the doc generation. > I think the bigger question is if we believe that a c compiler should be > required to build docs on a python project. Which seems really weird to > me. And a substantially higher barrier to entry than I think we want for > docs contributions. Well, Pillow isn't the only transitive dependency that's going to require a C compiler; there are many other dependencies we have that are actually extensions. > There was only 1 use of this in all of Nova, and I think we're better > off removing it and coming up with other ways of addressing it. Perhaps, but having diagrams in the documentation is still going to be incredibly useful, and Nova isn't the only component we have to consider (which is why I removed the [nova] tag from the subject line). That said, I have no objection if we find or create a tool that generates images without depending on Pillow, but…*shrug* -- Kevin L. Mitchell Rackspace __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [all] Doc dependencies (was: Re: [infra][nova][all] Pillow breaking gate?)
The Pillow-breaking-gate issue was related to having doc dependencies listed in the test-requirements.txt; however, those dependencies are not needed for testing, except for doc testing and generation. What do people think about creating a new "doc-requirements.txt" file that would contain only the doc dependencies? The appropriate doc environments in tox.ini would then need to be extended to pull in that file, and of course the global requirements tooling would have to be enhanced to recognize the new file as well. -- Kevin L. Mitchell Rackspace __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [infra][nova][all] Pillow breaking gate?
On Thu, 2015-10-01 at 15:53 -0700, Clark Boylan wrote: > On Thu, Oct 1, 2015, at 03:48 PM, Kevin L. Mitchell wrote: > > It looks like Pillow (pulled in by blockdiag, pulled in by > > sphinxcontrib-seqdiag, in test-requirements.txt of nova and probably > > others) had a 3.0.0 release today, and now the gate is breaking because > > libjpeg isn't available in the image…thoughts on how best to address > > this problem? > Two changes are already in flight to address this. > > The first updates global requirements to require an older version of > Pillow: > https://review.openstack.org/#/c/230167/ Pillow is not explicitly listed in nova's requirements; would this still be sufficient to unwedge the gate? > The second updates the images for Jenkins slaves that run things like > unittests and doc builds: > https://review.openstack.org/#/c/230175/ This one will take more time to > get through as it requires new images be built. > > It is also a possibility that devstack will need to install zlib and > libjpeg headers depending on whether or not sphinxcontrib is needed in > devstack (I think it is but protected now by constraints). > > A completely different approach would be to not use blockdiag, but these > changes should get us moving along again and we can worry about making > it more correct later. -- Kevin L. Mitchell Rackspace __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [infra][nova][all] Pillow breaking gate?
It looks like Pillow (pulled in by blockdiag, pulled in by sphinxcontrib-seqdiag, in test-requirements.txt of nova and probably others) had a 3.0.0 release today, and now the gate is breaking because libjpeg isn't available in the image…thoughts on how best to address this problem? -- Kevin L. Mitchell Rackspace __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] [api] Nova currently handles list with limit=0 quite different for different objects.
On Fri, 2015-09-11 at 15:41 +0800, Zhenyu Zheng wrote: > Hi, I found out that nova currently handles list with limit=0 quite > different for different objects. > > Especially when list servers: > > According to the code: > http://git.openstack.org/cgit/openstack/nova/tree/nova/api/openstack/common.py#n206 > > when limit = 0, it should apply as max_limit, but currently, in: > http://git.openstack.org/cgit/openstack/nova/tree/nova/db/sqlalchemy/api.py#n1930 > > we directly return [], this is quite different with comment in the api > code. > > > I checked other objects: > > when list security groups and server groups, it will return as no > limit has been set. And for flavors it returns []. I will continue to > try out other APIs if needed. > > I think maybe we should make a rule for all objects, at least fix the > servers to make it same in api and db code. > > I have reported a bug in launchpad: > > https://bugs.launchpad.net/nova/+bug/1494617 > > > Any suggestions? After seeing the test failures that showed up on your proposed fix, I'm thinking that the proposed change reads like an API change, requiring a microversion bump. That said, I approve of increased consistency across the API, and perhaps the behavior on limit=0 is something the API group needs to discuss a guideline for? -- Kevin L. Mitchell Rackspace __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [hacking] [style] multi-line imports PEP 0328
On Mon, 2015-08-24 at 22:53 -0700, Clay Gerrard wrote: > So, I know that hacking has H301 (one import per line) - but say maybe > you wanted to import *more* that one thing on a line (there's some > exceptions right? sqlalchemy migrations or something?) There's never a need to import more than one thing per line given the rule to only import modules, not objects. While that is not currently enforced by hacking, it is a strong style guideline. (Exceptions for things like sqlalchemy do exist, of course.) -- Kevin L. Mitchell Rackspace __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] contextlib.nested and Python3 failing
On Wed, 2015-08-19 at 16:51 -0700, Sylvain Bauza wrote: > I was writing some tests so I added a contextlib.nested to a checked > TestCase [1]. Unfortunately, contextlib.nested is no longer available in > Python3 and there is no clear solution on how to provide a compatible > import for both python2 and python3: > - either providing a python3 compatible behaviour by using > contextlib.ExitStack but that class is not available in Python 2 > - or provide contextlib2 for python2 (and thus adding it to the > requirements) Actually, there should no longer be a need to use contextlib.nested. We've explicitly dropped Python 2.6 compatibility, which means we're expecting compatibility with Python 2.7+ only, and as of Python 2.7, the 'with' statement supports accepting multiple 'as' clauses. The contextlib.nested tool was really only necessary to work around that functionality being missing in Python 2.6, and has been deprecated as of Python 2.7 because it's no longer necessary. -- Kevin L. Mitchell Rackspace __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Quick question..
On Thu, 2015-07-30 at 15:27 -0700, Adam Lawson wrote: > How many "up" votes are needed for code to be accepted into the trunk > (assuming it passes testing etc)? Which project? Different projects may have different criteria. -- Kevin L. Mitchell Rackspace __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [openstack][nova] Streamlining of config options in nova
On Thu, 2015-07-23 at 13:50 -0500, Sean Dague wrote: > Maybe a directory is fine, especially if module mapped to [subsection]. > > nova/config/ >default.py >glance.py >... > > which makes it reasonably discoverable and mappable back and forth from > config file to files. I like that division based on subsection scheme; that seems perfect to me. -- Kevin L. Mitchell Rackspace signature.asc Description: This is a digitally signed message part __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [openstack][nova] Streamlining of config options in nova
On Thu, 2015-07-23 at 11:41 -0500, Sean Dague wrote: > On 07/23/2015 11:23 AM, Roman Podoliaka wrote: > > Hi all, > > > > FWIW, this is exactly what we have in oslo libs, e.g. in oslo.db [0] > > > > Putting all Nova options into one big file is probably not a good > > idea, still we could consider storing those per-package (per backend, > > per driver, etc), rather than per Python module to reduce the number > > of possible circular imports when using import_opt() helper. > > > > Thanks, > > Roman > > > > [0] https://github.com/openstack/oslo.db/blob/master/oslo_db/options.py > > Actually, we just happened to be discussing this at the Nova meetup this > morning. And the general consensus was the opposite. It would be better > to collect all the config options in one file, especially if we are > going to expand the help (which we would like to do). Exceptions are > done that way in Nova. > > Michael Still is going to propose some initial patches to get this > rolling. We don't expect this to be a big bang, but in chunks as the > help is being improved. I'm concerned because of how big the single configuration file will become. It is a true pain in the butt to review anything that touches the db API (or tests) because of how huge that one file is (traditional view in Gerrit performs like a dog on large files), and I'm worried that the same will become true of a single nova/flags.py if we go that route. I do like Roman's idea of centralizing them into a smattering of files, though. What if we were to create a new top-level package and split the config options up into a small number of individual files there? Would that be a reasonable compromise between Roman's suggestion and the one-file-of-config route? -- Kevin L. Mitchell Rackspace __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [openstack][nova] Streamlining of config options in nova
On Thu, 2015-07-23 at 17:55 +0300, mhorban wrote: > During development process in nova I faced with an issue related with config > options. Now we have lists of config options and registering options mixed > with source code in regular files. > From one side it can be convenient: to have module-encapsulated config > options. But problems appear when we need to use some config option in > different modules/packages. > > If some option is registered in module X and module X imports module Y for > some reasons... > and in one day we need to import this option in module Y we will get > exception > NoSuchOptError on import_opt in module Y. > Because of circular dependency. > To resolve it we can move registering of this option in Y module(in the > inappropriate place) or use other tricks. Isn't this use case what the import_opt() method of CONF is for? The description given in the docstring is: Import a module and check that a given option is registered. This is intended for use with global configuration objects like cfg.CONF where modules commonly register options with CONF at module load time. If one module requires an option defined by another module it can use this method to explicitly declare the dependency. It's used all over the place in nova for this purpose, as far as I can see. > I offer to create file options.py in each package and move all package's > config options and registration code there. > Such approach allows us to import any option in any place of nova without > problems. The problem with this reorganization is that it moves the options from the place where they're primarily intended to be used. This could make it harder to maintain, such as ensuring the help text is updated when the code is. If nova were a smaller code base, I think it would make sense to reorganize in this fashion, but given how large nova actually is… -- Kevin L. Mitchell Rackspace __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][ceilometer] proposal to send bulk hypervisor stats data in periodic notifications
On Mon, 2015-06-22 at 18:10 +0100, Daniel P. Berrange wrote: > On Mon, Jun 22, 2015 at 11:10:40AM -0500, Kevin L. Mitchell wrote: > > On Sat, 2015-06-20 at 21:35 +0100, Daniel P. Berrange wrote: > > > In general I would say that is an unsupported deployment scenario to > > > have other random virt guests running on a nova compute node. > > > > On the other hand, this is exactly how compute nodes themselves are > > often deployed—a random guest on the hypervisor node… > > In a devstack env maybe, but in production we expect the hypervisor to > be exclusively used by Nova, or things Nova uses. In production, at least in some Xen deployments, the hypervisor does not run the nova-compute service; instead, we have a VM specifically set aside for running the nova-compute. This is done because the hypervisor (in older Xen deployments, at least) has an old version of Python, and because some operations are most conveniently done by a co-located compute node. -- Kevin L. Mitchell Rackspace __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][ceilometer] proposal to send bulk hypervisor stats data in periodic notifications
On Sat, 2015-06-20 at 21:35 +0100, Daniel P. Berrange wrote: > In general I would say that is an unsupported deployment scenario to > have other random virt guests running on a nova compute node. On the other hand, this is exactly how compute nodes themselves are often deployed—a random guest on the hypervisor node… -- Kevin L. Mitchell Rackspace __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [api] [all] To changes-since or not to changes-since
On Fri, 2015-06-19 at 10:07 +0100, Chris Dent wrote: > * Are there additional relevant pros and cons for the two proposals? > * Are there additional proposals which can address the shortcomings >in either? On the latter question, would using the If-Modified-Since header[1] make any sense as a possible solution? That at least would be a standard HTTP convention for this sort of thing, and tends to match up with the semantics of a changes-since query parameter. [1] http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.25 -- Kevin L. Mitchell Rackspace __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [api][nova][ironic] Microversion API HTTP header
On Mon, 2015-06-15 at 13:07 -0400, Jay Pipes wrote: > The original spec said that the HTTP header should contain the name of > the service type returned by the Keystone service catalog (which is also > the official name of the REST API). I don't understand why the spec was > changed retroactively and why Nova has been changed to return > X-OpenStack-Nova-API-Version instead of X-OpenStack-Compute-API-Version > HTTP headers [4]. Given the disagreement evinced by the responses to this thread, let me ask a question: Would there be any particular problem with using "X-OpenStack-API-Version"? -- Kevin L. Mitchell Rackspace __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Looking for help getting git-review to work over https
On Fri, 2015-06-12 at 17:08 +, KARR, DAVID wrote: > Thanks. I already tried that. It's not even clear this is failing to > connect. I don't know what this is telling me. > -- > # pip install --proxy http://one.proxy.att.com:8080 . > Processing /home/dk068x/work/git-review > Complete output from command python setup.py egg_info: > Download error on https://pypi.python.org/simple/pbr/: [Errno 1] > _ssl.c:504: error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown > protocol -- Some packages may not be found! > Couldn't find index page for 'pbr' (maybe misspelled?) > Download error on https://pypi.python.org/simple/: [Errno 1] > _ssl.c:504: error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown > protocol -- Some packages may not be found! > No local packages or download links found for pbr > Traceback (most recent call last): > File "", line 20, in > File "/tmp/pip-MStPEo-build/setup.py", line 20, in > setuptools.setup(setup_requires=['pbr'], pbr=True) Have you confirmed that your proxy at 8080 is capable of SSL? Usually, people use port 8080 for plain old HTTP servers or proxies, and trying to talk SSL to a plain HTTP proxy would probably result in that error. (Also noticed that your proxy URL is specified as "http://";; if you know that proxy works for SSL, try "https://"…) -- Kevin L. Mitchell Rackspace __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Looking for help getting git-review to work over https
On Thu, 2015-06-11 at 16:42 +, KARR, DAVID wrote: > Concerning the question ‘Do you have "gerrit" remote already > configured?’, I guess I’d have to say I don’t know. I’ve followed > instructions for setting up my pub key, but I’m not sure exactly what > is entailed in “gerrit remote”. The "git review" command does its magic, in part, through configuring a "git remote" on the repository. Go to the repository and do a "git remote -v" and look for any lines beginning with "gerrit"; they probably have the SSH URL instead of the https URL. You should be able to use "git remote remove gerrit" and re-run the "git review -s" to get that all fixed up. (Could also use "git remote set-url", FYI, but I figured starting from scratch may be easier for you…) -- Kevin L. Mitchell Rackspace __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [keystone] [nova] [oslo] oslo.policy requests from the Nova team
On Tue, 2015-06-02 at 16:16 -0600, David Lyle wrote: > The Horizon project also uses the nova policy.json file to do role > based access control (RBAC) on the actions a user can perform. If the > defaults are hidden in the code, that makes those checks a lot more > difficult to perform. Horizon will then get to duplicate all the hard > coded defaults in our code base. Fully understanding UI is not > everyone's primary concern, I will just point out that it's a terrible > user experience to have 10 actions listed on an instance that will > only fail when actually attempted by making the API call. For the record, the discussion at the summit also touched on the discoverability of the policy affecting a given user/API. I don't believe we considered the ordering between that and the defaults feature we suggested, but I believe we can code a defaults mechanism to dynamically generate an output file in the interim (as is done for configuration now), which may improve the situation from Horizon's standpoint, until the discoverability piece is in place. -- Kevin L. Mitchell Rackspace __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [OpenStack-Infra] Gerrit downtime and upgrade on Saturday 2015-05-09 at 1600 UTC
On Mon, 2015-05-11 at 15:56 +, Jeremy Stanley wrote: > On 2015-05-11 01:45:30 + (+), Jeremy Stanley wrote: > > For what it's worth, I tried changing Gerrit's "canonicalweburl" > > setting to not include a trailing slash, but it doesn't help. I have > > a feeling this is not a misconfiguration, but something intrinsic to > > the OpenID implementation in Gerrit which has changed since 2.8. > > I've tested https://review.openstack.org/181949 on review-dev, and > it will solve those 404 hyperlinks once it merges. We still need to > track down what's causing the OpenID callback URL to end up with a > second trailing slash however. As a point of information, I logged in to review a python-novaclient review, and now I find I can't load any nova reviews at all; I get a page with the top bar, but below that is a "Toggle CI" button and nothing else. Reloading the page has no effect, and the same behavior applies to all nova reviews I've tried to load. Restarting by browser did not have any effect. -- Kevin L. Mitchell Rackspace __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][api] Allow passing body for more API methods
(Added [api] to subject to bring in the attention of the API team.) On Mon, 2015-05-11 at 14:48 +, Rosa, Andrea (HP Cloud Services) wrote: > I noticed that in the nova API we allow to specify body just for the > PUT and POST requests [0], for all the other methods, if a body is > specified, it gets ignored. > > I had a look at the RFC 7231 [1] and I noticed that just the TRACE > must not have a body, for all the other request a body can be passed > and managing or ignoring it depends on the semantic of the request. > > For that reason my proposal is to allow at WSGI layer to define a body > for all the requests but for the TRACE then it is up to the specific > controller to ignore or deal with the body in the request. > > I put a WIP to implement that change [3]. > > The rationale behind it is double: > > - Be more in compliance with the RFC > > - Having more flexibility in our framework. I have a valid > (at least for me) use case here [4]: at the moment a volume detach is > implemented using a HTTP DELETE, I’d like to add the option for > calling from nova, the cinder –force-delete > > My idea to implement it is to add a parameter in the body of the > DELETE call, but at the moment the only valid option is to create a > new API using a POST method. I have worked with client frameworks which raise exceptions if you attempt to pass a body using the DELETE method, and would presumably also prohibit a body with GET and HEAD, so I'm -1 on this: we should actively discourage service developers from requiring bodies on HTTP methods that a client framework may prohibit sending bodies with. -- Kevin L. Mitchell Rackspace __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Which error code should we return when OverQuota
On Tue, 2015-05-05 at 21:30 +0200, Chen CH Ji wrote:
> In doing patch [1], A suggestion is submitted that we should return
> 400 (bad Request) instead of 403 (Forbidden)
> I take a look at [2], seems 400 is not a good candidate because
> 'The request could not be understood by the server due to malformed
> syntax. The client SHOULD NOT repeat the request without
> modifications. '
>
> may be a 409 (conflict) error if we really don't think 403 is a good
> one?
> Thanks
>
>
> [1] https://review.openstack.org/#/c/173985/
> [2] http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html
Looking through the HTTP spec you reference in [2], there isn't really a
good match. You're right that 400 doesn't really make sense, but I
don't really think that 409 is a good fit either. The only ones that
would make sense to me would be 413 ("Request Entity Too Large") and
403, the definition of which is wide enough to encompass the over-quota
condition; and of those, 403 makes the most sense. (Note that I believe
we use 413 for rate limiting, because of the Retry-After header…)
--
Kevin L. Mitchell
Rackspace
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Proposal to add Melanie Witt to nova-core
On Thu, 2015-04-30 at 12:30 +0100, John Garbutt wrote: > I propose we add Melanie to nova-core. > > She has been consistently doing great quality code reviews[1], > alongside a wide array of other really valuable contributions to the > Nova project. > > Please respond with comments, +1s, or objections within one week. Definitely a +1 from me… -- Kevin L. Mitchell Rackspace __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Navigating the ever changing OpenStack codebase
On Tue, 2015-04-28 at 16:08 -0400, Jay Pipes wrote: > Honestly, I see no problem with some helper bash scripts that simplify > life for new contributors. The bash scripts do wonders for developers > new to OpenStack or Python coding by having a pretty easy and readable > way of determining what CLI commands are used to execute tests. Hell, > devstack [1] itself was written originally in the way it was to > well-document the deployment process for OpenStack. Many packagers and > configuration management script authors have looked at devstack's Bash > scripts for inspiration and instruction in this way. > > The point Ronald was making that nobody seems to have addressed is the > very valid observation that as a new contributor, it can be very > confusing to go from one project to another and see different ways of > running tests. Some projects have run_tests.sh and still actively > promote it in the devref docs. Others don't > > While Ronald seems to have been the victim of unfortunate timing (he > started toying around with python-openstackclient and within a week, > they removed the script he was using to run tests), that doesn't make > his point about our inconsistency moot. Completely agreed, actually; I was only responding to the comment suggesting the complete removal of run_tests.sh. I personally think we should promote only tox in the various doc files, and reference run_tests.sh only as a legacy thing we can't fully get rid of quite yet. (Incidentally, for my testing purposes, I don't care where it is, as long as it's somewhere; so we could also move it to, say, "tools". I don't even care what it outputs, as long as it gives a reasonable return value; so we could have it print out a scary-looking warning about it being legacy… :) -- Kevin L. Mitchell Rackspace __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Navigating the ever changing OpenStack codebase
On Tue, 2015-04-28 at 10:40 -0700, Clint Byrum wrote: > > No dice. I don't want to have to parse the tox.ini directly. We're > > talking about automated tests here, by the way. > > Why not? It's an ini file, with a stable interface. Yes, but it's tox's perview. Also, it's not just the command; I may be building my own virtual environment to use, but there are other environment variables typically set up by the test environment (e.g., PYTHONHASHSEED). Also, there could be multiple tests to run, e.g., I might need to run py27 and pep8. With all of that, getting the proper test setup programmatically becomes almost a bastardized reimplementation of tox, and I'd *really* rather not do that. As I say, I'd love to just run tox; I just need it to use my virtualenv, not build another. > I'm sure you've thought more about this than me, so I apologize for > sounding dense. However, I'm struggling to see where having to maintain > two test harnesses is less complicated than the code above. Well, we don't typically make all that many changes to the test harness to begin with, so while it's not exactly the ideal situation, it's not really all that hard to try to maintain both (though admittedly a bit more error prone). Also, run_tests.sh typically doesn't run all the tests that tox is capable of running, and that's fine; as long as it runs basic acceptance-type tests (e.g., unit tests and pep8), I'm happy. I raised this issue before on the list some months ago…IIRC, it was when someone had proposed removing run_tests.sh from nova. I explained why I needed it and pointed to the tox issue I had open. As I recall, that thread included some comments from some other operators that also wanted to use run_tests.sh for basically the same reason we were: acceptance tests on the virtual environment. The result of that thread was to keep run_tests.sh around for now. I can say that, if the issue I reported ever gets addressed, I'll happily drop run_tests.sh like a hot potato. (Well, there's some other minor issue, having to do with odd characters in path names, but I can just find a different way to encode the path name to deal with that problem :) -- Kevin L. Mitchell Rackspace __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Navigating the ever changing OpenStack codebase
On Mon, 2015-04-27 at 15:54 -0700, Clint Byrum wrote: > Excerpts from Kevin L. Mitchell's message of 2015-04-27 15:38:25 -0700: > > On Mon, 2015-04-27 at 21:42 +, Jeremy Stanley wrote: > > > I consider it an unfortunate oversight that those files weren't > > > deleted a very, very long time ago. > > > > Unfortunately, there's one problem with that: you can't tell tox to use > > a virtualenv that you've built. We need this capability at present, so > > we have to run tests using run_tests.sh instead of tox :( I have an > > issue open on tox to address this need, but haven't seen any movement on > > that; so until then, I have to oppose the removal of run_tests.sh… > > despite how much *I'd* like to see it bite the dust! > > Err.. you can just run the commands in tox.ini in the venv of your > choice. You don't need run_tests.sh for that. No dice. I don't want to have to parse the tox.ini directly. We're talking about automated tests here, by the way. -- Kevin L. Mitchell Rackspace __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Navigating the ever changing OpenStack codebase
On Mon, 2015-04-27 at 21:42 +, Jeremy Stanley wrote: > I consider it an unfortunate oversight that those files weren't > deleted a very, very long time ago. Unfortunately, there's one problem with that: you can't tell tox to use a virtualenv that you've built. We need this capability at present, so we have to run tests using run_tests.sh instead of tox :( I have an issue open on tox to address this need, but haven't seen any movement on that; so until then, I have to oppose the removal of run_tests.sh… despite how much *I'd* like to see it bite the dust! -- Kevin L. Mitchell Rackspace __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Port Nova to Python 3
On Fri, 2015-04-24 at 16:07 -0400, Sean Toner wrote: > What I meant by the worst of both worlds is that you don't get the nice > new features of python3, while simultaneously dealing with the headaches > of making code run under both python versions. You'll have to do weird > things with imports (for example urllib) and deal with the > inconsistencies between some functions that return strings and some that > return unicode, and some that return bytes. > > It's not impossible, but you have to add that extra work while also > depriving yourself of the goodness of python3 only features :) This is why the 'six' library is such a godsend; this stuff is still not easy, but the hardest parts, like the imports problem, are already taken care of by six…and maintaining the bytes/strings/unicode distinction is actually just as useful in Python 2, it just doesn't have the machinery to really detect the mixing :) -- Kevin L. Mitchell Rackspace __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [oslo][policy][neutron] oslo.policy API is not powerful enough to switch Neutron to it
On Wed, 2015-04-22 at 08:49 -0400, Doug Hellmann wrote: > That feature sounds like it could be useful outside of neutron, so let's > see if we can come up with a new syntax to make it portable. Bonus > points if the new syntax results in a proper DSL. I have been thinking that I should point people to the "policies" package[1] I built, and this DSL comment has finally pushed me over the edge :) It's a complete clean-room implementation with a completely different syntax than oslo.policy, so I don't know how interested anyone here would be in using it, but there it is. (I have it licensed under the GPL right now, but if anyone wants to use it, I'd be happy to relicense under Apache…) [1] https://pypi.python.org/pypi/policies -- Kevin L. Mitchell Rackspace __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all][releases] upcoming library releases to unfreeze requirements in master
On Tue, 2015-04-21 at 15:44 -0400, Doug Hellmann wrote: > OK, this is all done. I have verified that all of the libraries are on > PyPI and I have sent the release notes (sometimes several copies, at no > extra charge to you -- seriously, sorry about the noise). Just out of curiosity, how many entries are in whatever array from which the script selects randomly? That's the best part of the release script, IMO :) -- Kevin L. Mitchell Rackspace __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [api] Minor changes to API
On Mon, 2015-04-20 at 13:57 -0600, Chris Friesen wrote: > > However, minor changes like that could still possibly break clients that > > are not > > expecting them. For example, a client that uses the json response as > > arguments > > to a method via **kwargs would start seeing TypeErrors for unexpected > > arguments. > > Isn't this what microversions were intended to solve? Yes. > I'm relatively recent with OpenStack, so I don't have the history. Did > anyone > ever consider explicitly allowing new attributes to be added to responses? The problem is advertising that this information is available. That's why, in the past, nova required a new extension even if all you were doing was adding an attribute, and that's why we want a new microversion nowadays. > That > would put the onus on clients to explicitly grab the attributes that they're > looking for, but that doesn't seem too crazy and it would simplify adding new > attributes on the server side. You'd still need versioning to handle > deleting > attributes or changing their meaning, of course. -- Kevin L. Mitchell Rackspace __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Beyond IRC (was Re: Cinder Third-Party CI: what next?)
On Tue, 2015-03-24 at 21:21 -0400, Doug Hellmann wrote: > > > ·A separate mailing list for project review requests > > > > I'm skeptical about this being effective: just another source of > > incoming email that needs to be filtered out (at which point a mailman > > topic would have the same effect). > > Yes, email notifications scale poorly for really active reviewers. > Between gerrit and launchpad, there is already a lot of notification > email being filtered out of inboxes. I actually do rely on email notifications for review duties. I use filtering that moves the messages into dedicated folders per repository, and I use threading in those folders. When I see a review merge (or when I see it abandoned), I select the whole thread and delete it; otherwise I follow changes made to a review and determine whether I need to re-review it based on whether votes to that point have been +'s or -'s. I even subscribe to a couple of projects and check out new reviews submitted there. This is actually partly to blame for me being such a prolific reviewer in nova and novaclient :) -- Kevin L. Mitchell Rackspace __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [stable][all] Revisiting the 6 month release cycle [metrics]
On Thu, 2015-02-26 at 11:45 -0800, Stefano Maffulli wrote: > The interesting bit of those charts is that overall for OpenStack > projects, it seems that the reviews (comments to patchsets) are arriving > quite quickly but the new patchsets take a lot more to be submitted. > > Too much debating and commenting over each patch? Or are the > authors/owners of the changeset slow to respond with new patches? I > don't have an answer. I'd be happy to look at the data with other > people. One thing that comes to mind is that there are a lot of reviews that appear to have been abandoned; I just cleared several from the novaclient review queue (or commented on them to see if they were still alive). I also know of a few novaclient changes that are waiting for corresponding nova changes before they can be merged. Could these be introducing a skew factor? -- Kevin L. Mitchell Rackspace __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] The API WG mission statement
On Tue, 2015-02-03 at 18:49 +, Everett Toews wrote: > I’ll echo Mike’s sentiment that we should be very mindful of the idea > that these are guidelines not hard standards. H…even that might be > a bit restrictive. In the Openstack HTTP error codes [1] discussion > I’m getting the impression that there is a desire to make this a > standard. Perhaps we need to leave the door open to setting standards > in certain cases? I tend to be in the "guideline for now" camp, but I see us slowly shifting over to establishing standards where it makes sense. Once the error codes discussion truly starts to converge toward consensus (something I feel it's close to, but not quite there yet), it seems reasonable to make it a guideline. As far as standards go—if we go with the idea of header addition to tell the client about the codes, it becomes something that can easily be added to all OpenStack APIs, and once that happens, then I can foresee it becoming a formal standard recommended by the API WG… -- Kevin L. Mitchell Rackspace __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Which repo should the API WG use?
On Fri, 2015-01-30 at 22:33 +, Everett Toews wrote: > It was suggested that the API WG use the openstack-specs [1] and/or > the api-wg [2] repo to publish its guidelines. We’ve already arrived > at the consensus that we should only use 1 repo [3]. So the purpose of > this thread is to decide... > > Should the API WG use the openstack-specs repo or the api-wg repo? > > Let’s discuss. Well, the guidelines are just that: guidelines. They don't implicitly propose changes to any OpenStack projects, just provide guidance for future API changes. Thus, I think they should go in a repo separate from any of our *-specs repos; to me, a spec provides documentation of a change, and is thus independent of the guidelines. -- Kevin L. Mitchell Rackspace __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Openstack-operators] [all][log] Openstack HTTP error codes
On Fri, 2015-01-30 at 21:08 +, Everett Toews wrote:
> Project: A client dealing with the API already knows what project
> (service) they’re dealing with. Including this in an API error message
> would be redundant. That’s not necessarily so bad and it could
> actually be convenient for client logging purposes to have this there.
Do they? We boot a server and interact with Cinder and Neutron, right?
What if the nova API is simply forwarding an error that originally came
from Cinder?
> Vendor/Component: Including any vendor information at all would be
> leaking implementation details. This absolutely cannot be exposed in
> an API error message. Even including the component would be leaking
> too much.
While I agree with you from a security standpoint, this is probably
coming in due to a desire to namespace the errors. Ideally, we'd have a
set of common error codes to cover conditions that the API user could
rectify ("You picked a nic type we don't support" or something like
that), but I fear there may always be errors that are things the API
user could rectify but which don't fit into any of those buckets…
> Error Catalog Number: If there could be alignment around this, that
> would be great.
[snip]
> Criticality: This might be useful to clients? I don’t know. I don’t
> feel too strongly about it.
I feel this part of the code needs more thought to properly round out.
Is it intended to convey information similar to the distinction between
4xx and 5xx errors in HTTP? ("You made an error" vs. "The server messed
up".) Is it intended to convey a retryable condition? ("If you retry
this, it may succeed.") If it's intended to convey that the server
messed up spectacularly and that everything's broken now, well… :)
--
Kevin L. Mitchell
Rackspace
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] design question : green thread model
On Wed, 2015-01-28 at 22:48 +0530, murali reddy wrote: > On hosts with multi-core processors, it does not seem optimal to run a > single service instance with just green thread. I understand that on > controller node, we can run one or more nova services but still it > does not seem to utilize multi-core processors. It's also worth pointing out that Python itself isn't all that parallelizable, even using the core threading packages. (See [1] for a write-up about the main block for multiple core utilization in Python, namely the GIL.) The upshot is that this is why some components like nova-api have the capability to fork; but the other components that use green threads are largely IO-bound, so threading based on IO call blocking makes sense there. [1] https://wiki.python.org/moin/GlobalInterpreterLock -- Kevin L. Mitchell Rackspace __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Nominating Melanie Witt for python-novaclient-core
On Tue, 2015-01-27 at 14:41 -0800, Michael Still wrote: > I would like to nominate Melanie Witt for the python-novaclient-core > team. I +1 this… -- Kevin L. Mitchell Rackspace __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][gate][stable] How eventlet 0.16.1 broke the gate
On Wed, 2015-01-21 at 18:48 -0800, Joshua Harlow wrote: > Another thing that I just started whipping together: > > https://gist.github.com/harlowja/5e39ec5ca9e3f0d9a21f One problem, though, is that parse_requirements() now requires the session keyword argument. In version 6.0.6, parse_requirements() begins with: def parse_requirements(filename, finder=None, comes_from=None, options=None, session=None): if session is None: raise TypeError( "parse_requirements() missing 1 required keyword argument: " "'session'" ) -- Kevin L. Mitchell Rackspace __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [api] Optional Properties in an Entity
On Mon, 2015-01-19 at 19:55 +, Douglas Mendizabal wrote: > I’m curious about something that came up during a bug discussion in > one of the Barbican weekly meetings. The question is about optional > properties in an entity. e.g. We have a Secret entity that has some > properties that are optional, such as the Secret’s name. We were > split on what the best approach for returning the secret > representation would be when an optional property is not set. > > In one camp, some developers would like to see the properties returned > no matter what. That is to say, the Secret dictionary would include a > key for “name” set to null every single time. i.e. [snip] > On the other camp, some developers would like to see optional > properties omitted if they were not set by the user. > > The advantage of always returning the property is that the response is > easier to parse, since you don’t have to check for the existence of > the optional keys. The argument against it is that it makes the API > more rigid, and clients more fragile. I keep trying to come up with technical arguments for or against, and the only one I can come up with that has any true meaning is that omitting properties reduces bandwidth usage a little…but I don't really think that's something we've particularly concerned about. Thus, from a technical perspective, either way is perfectly fine, and we just have to answer consistency. When we look at consistency, we look at everything else in OpenStack. From the standpoint of the nova API (with which I am the most familiar), I am not aware of any property that is ever omitted from any payload without versioning coming in to the picture, even if its value is null. Thus, I would argue that we should encourage the first situation, where all properties are included, even if their value is null. -- Kevin L. Mitchell Rackspace __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [oslo] Test failures due to oslo_concurrency?
Noticed that https://review.openstack.org/#/c/145626/ failed tests. That's kind of curious, as all it does is alter HACKING.rst. Investigation of the test failures[1] showed some UTF-8 decoding errors that appeared to be coming from oslo_concurrency/processutils.py. Any ideas? [1] http://logs.openstack.org/26/145626/2/check/gate-nova-python27/b350bbc/console.html -- Kevin L. Mitchell Rackspace __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [api] [sdk] Proposal to achieve consistency in client side sorting
On Tue, 2015-01-06 at 12:19 -0600, Anne Gentle wrote: > I'm all for consistency. Sounds like a great case for the API Working > Group to document. You can propose a patch describing the way we want > sorting to work. > > > See https://review.openstack.org/#/q/project:openstack/api-wg,n,z I really think that the API WG should be responsible for the REST API only, TBH, and maybe for the Pythonic APIs. Once we start talking about CLI options, I think that's outside the API WG's perview, and we probably should have that be up to CLI authors. My thinking is that a REST API and a Python API are both used by developers, where we have one set of conventions; but when you start talking about CLI, you're really talking about UX, and the rules there can be vastly different. Now, my personal preference on the CLI side is that "--sort key[:dir]" is the way to go, but I helped hash that out in the novaclient implementation as a reviewer. I also had suggestions about how it should be transmitted in the REST API, but that had already been settled at the blueprint phase; that may be something the API WG will want to take up and review and provide guidance on now, though… -- Kevin L. Mitchell Rackspace ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] should 'ip address' be retrived when decribe host?
On Wed, 2014-12-31 at 20:56 +0100, Chen CH Ji wrote: > Sorry If I didn't understand clearly about it , looks to > me the hypervisor itself hosts the instances and it should have a IP > with it (like Linux host KVM instances, Linux is the hypervisor, the > PC is the host) > while the host is physical node and only to be used by > 'hypervisor' concept ,so I think maybe we don't need ip for the > 'host' ? thanks a lot The hypervisor hosts the VMs, yes, but the component that sits between the hypervisor and the rest of nova—that is, nova-compute—does not necessarily reside on the hypervisor. It is the nova-compute node (which may be either a VM or a physical host) that is referred to by the nova term "host". For KVM, I believe the host is often the same as the hypervisor, meaning that nova-compute runs directly on the hypervisor… but this is not necessarily the case for all virt drivers. For example, the host for Xen-based installations is often a separate VM on the same hypervisor, which would have its own distinct IP address. -- Kevin L. Mitchell Rackspace ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] should 'ip address' be retrived when decribe host?
On Tue, 2014-12-30 at 14:52 +0800, Lingxian Kong wrote: > Just as what Jay Lau said, 'nova hypervisor-show ' > indeed returns host ip address, and there are more other information > included than 'nova host-describe '. I feel a little > confused about the 'host' and 'hypervisor', what's the difference > between them? For cloud operator, maybe 'host' is more usefull and > intuitive for management than 'hypervisor'. From the implementation > perspective, both 'compute_nodes' and 'services' database tables are > used for them. Should them be combined for more common use cases? Well, the host and the hypervisor are conceptually distinct objects. The hypervisor is, obviously, the thing on which all the VMs run. The host, though, is the node running the corresponding nova-compute service, which may be separate from the hypervisor. For instance, on Xen-based setups, the host runs in a VM on the hypervisor. There has also been discussion of allowing one host to be responsible for multiple hypervisors, which would be useful for providers with large numbers of hypervisors. -- Kevin L. Mitchell Rackspace ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] ask for usage of quota reserve
On Thu, 2014-12-18 at 15:34 +0800, Eli Qiao(Li Yong Qiao) wrote: > can anyone tell if we call quotas.reserve() but never call > quotas.commit() or quotas.rollback(). > what will happen? A reservation is always created with an expiration time; by default, this expiration time is 86400 seconds (1 day) after the time at which the reservation is created. Expired reservations are deleted by the _expire_reservations() periodic task, which is defined on the scheduler. Thus, if a resource is reserved, but never committed or rolled back, it should continue to affect quota requests for approximately one day, then be automatically rolled back by the scheduler. -- Kevin L. Mitchell Rackspace ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [hacking] proposed rules drop for 1.0
On Tue, 2014-12-09 at 13:46 -0500, Sean Dague wrote: > Yes, the following fails H305 and H306. > > nova/tests/fixtures.py > > """Fixtures for Nova tests.""" > from __future__ import absolute_import > > import gettext > import logging > import os > import uuid > > import fixtures > from oslo.config import cfg > > from nova.db import migration > from nova.db.sqlalchemy import api as session > from nova import service > > > Because name normalization is hard (fixtures is normalized to > nova.tests.fixtures so H305 thinks it should be in group 3, and H306 > thinks it should be after oslo.config import cfg). > > To sort things you have to normalize them. I agree you have to normalize imports to sort them, but to my mind the appropriate normalization here is purely textual; we shouldn't be expecting any relative imports (and should raise an error if there are any). Still, that does show that some work needs to be done to the simpler H306 test (probably involving changes to the core import normalization)… -- Kevin L. Mitchell Rackspace ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [hacking] proposed rules drop for 1.0
On Tue, 2014-12-09 at 12:05 -0500, Sean Dague wrote: > > I agree that dropping H302 and the grouping checks makes sense. I > think > > we should keep the H301, H303, H304, and the basic ordering checks, > > however; it doesn't seem to me that these would be that difficult to > > implement or maintain. > > Well, be careful what you think is easy - > https://github.com/openstack-dev/hacking/blob/master/hacking/checks/imports.py > :) So, hacking_import_rules() is very complex. However, it implements H302 as well as H301, H303, and H304. I feel it can be simplified to just a textual match rule if we remove the H302 implementation: H301 just needs to exclude imports with ',', H303 needs to exclude imports with '*', and H304 is already implemented as a regular expression match. It looks like the basic ordering check I was referring to is H306, which isn't all that complicated. It seems like the rest of the code is related to the checks which I just agreed should be dropped :) Am I missing anything? -- Kevin L. Mitchell Rackspace ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [hacking] proposed rules drop for 1.0
On Tue, 2014-12-09 at 10:05 -0500, Sean Dague wrote: > Sure, the H8* group is git commit messages. It's checking for line > length in the commit message. I agree the H8* group should be dropped. It would be appropriate to create a new gate check job that validated that, but it should not be part of hacking. > H3* are all the module import rules: > > Imports > --- > - [H302] Do not import objects, only modules (*) > - [H301] Do not import more than one module per line (*) > - [H303] Do not use wildcard ``*`` import (*) > - [H304] Do not make relative imports > - Order your imports by the full module path > - [H305 H306 H307] Organize your imports according to the `Import order > template`_ and `Real-world Import Order Examples`_ below. > > I think these remain reasonable guidelines, but H302 is exceptionally > tricky to get right, and we keep not getting it right. > > H305-307 are actually impossible to get right. Things come in and out of > stdlib in python all the time. > > > I think it's time to just decide to be reasonable Humans and that these > are guidelines. > > The H3* set of rules is also why you have to install *all* of > requirements.txt and test-requirements.txt in your pep8 tox target, > because H302 actually inspects the sys.modules to attempt to figure out > if things are correct. I agree that dropping H302 and the grouping checks makes sense. I think we should keep the H301, H303, H304, and the basic ordering checks, however; it doesn't seem to me that these would be that difficult to implement or maintain. -- Kevin L. Mitchell Rackspace ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [api] Using query string or request body to pass parameter
On Tue, 2014-12-09 at 07:38 +0800, Alex Xu wrote: > Not sure all, nova is limited > at > https://github.com/openstack/nova/blob/master/nova/api/openstack/wsgi.py#L79 > That under our control. It is, but the client frameworks aren't, and some of them prohibit sending a body with a DELETE request. Further, RFC7231 has this to say about DELETE request bodies: A payload within a DELETE request message has no defined semantics; sending a payload body on a DELETE request might cause some existing implementations to reject the request. (§4.3.5) I think we have to conclude that, if we need a request body, we cannot use the DELETE method. We can modify the operation, such as setting a "force" flag, with a query parameter on the URI, but a request body should be considered out of bounds with respect to DELETE. > Maybe not just ask question for delete, also for other method. > > 2014-12-09 1:11 GMT+08:00 Kevin L. Mitchell : > On Mon, 2014-12-08 at 14:07 +0800, Eli Qiao wrote: > > I wonder if we can use body in delete, currently , there isn't any > > case used in v2/v3 api. > > No, many frameworks raise an error if you try to include a body with a > DELETE request. > -- > Kevin L. Mitchell > Rackspace -- Kevin L. Mitchell Rackspace ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [api] Using query string or request body to pass parameter
On Mon, 2014-12-08 at 14:07 +0800, Eli Qiao wrote: > I wonder if we can use body in delete, currently , there isn't any > case used in v2/v3 api. No, many frameworks raise an error if you try to include a body with a DELETE request. -- Kevin L. Mitchell Rackspace ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Handling soft delete for instance rows in a new cells database
On Tue, 2014-11-25 at 09:20 +1100, Michael Still wrote: > - we do what we do in the current nova database -- we have a deleted > column, and we set it to true when we delete the instance. Actually, current nova uses the oslo.db.sqlalchemy.models.SoftDeleteMixin class, which defines the columns 'deleted_at' (a DateTime) and 'deleted' (an *integer*). It also defines a 'soft_delete()' method, which sets the 'deleted' column to the row 'id'. As I understand it, this is to keep from breaking uniqueness constraints; you factor in 'deleted' in your uniqueness constraint, and you can have as many identical deleted records as you want… > - we have shadow tables and we move delete rows to a shadow table. > > - something else super clever I haven't thought of. Well, one thought might be to create a single 'audit' table with a couple of columns—a timestamp, say, and some sort of description of the change, perhaps as a JSON object. On a 'delete' operation, you could store the values of the row into this audit table. From an operator's standpoint, this could provide the required auditing and perhaps even a limited DR solution, while centralizing the data you need to monitor in a single location, which makes it easier to trim the data at intervals as needed. While I've proposed this as a soft-delete solution, it would also provide the ability to record other changes to objects; one could even include a column to record who performed the change. And of course I've suggested this as a DB table, but we could also consider the merits of ditching the table and doing the same thing as some sort of notification through the notifications system… -- Kevin L. Mitchell Rackspace ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [api] Counting resources
On Thu, 2014-11-20 at 08:28 -0800, Morgan Fainberg wrote: > The only thing I want to caution against is making a SQL-specific > choice. In the case of some other backends, it may not be possible > (for an extremely large dataset) to get a full count, where SQL does > this fairly elegantly. For example, LDAP (in some cases) may have an > administrative limit that will say that no more than 10,000 entries > would be returned; likely you’re going to have an issue, since you > need to issue the query and see how many things match, if you hit the > overall limit you’ll get the same count every time (but possibly a > different dataset). Hmmm…interesting limitation. > I want to be very careful that we’re not recommending functionality as > a baseline that should be used as a pattern across all similar APIs, > especially since we have some backends/storage systems that can’t > elegantly always support it. > > > Personally, I like Gerrit’s model (as Sean described) - with the above > caveat that not all backends support this type of count. How about if we include some sentinel that can be used to indicate that count is unsupported, to cover cases such as an LDAP backend? That could be as simple as allowing a null value. -- Kevin L. Mitchell Rackspace ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Quota management and enforcement across projects
On Thu, 2014-11-20 at 10:16 +1100, Blair Bethwaite wrote: > For actions initiated directly through core OpenStack service APIs > (Nova, Cinder, Neutron, etc - anything using Keystone policy), > shouldn't quota-enforcement be handled by Keystone? To me this is just > a subset of authz, and OpenStack already has a well established > service for such decisions. If you look a little earlier in the thread, you will find a post from me where I point out just how complicated quota management actually is. I suggest that it should be developed as a proof-of-concept as a separate service; from there, we can see whether it makes sense to roll it into Keystone or maintain it as a separate thing. -- Kevin L. Mitchell Rackspace ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Quota management and enforcement across projects
On Mon, 2014-11-17 at 18:48 -0500, Doug Hellmann wrote: > I’ve spent a bit of time thinking about the resource ownership issue. > The challenge there is we don’t currently have any libraries that > define tables in the schema of an application. I think that’s a good > pattern to maintain, since it avoids introducing a lot of tricky > issues like how to manage migrations for the library, how to ensure > they are run by the application, etc. The fact that this common quota > thing needs to store some data in a schema that it controls says to me > that it is really an app and not a library. Making the quota manager > an app solves the API definition issue, too, since we can describe a > generic way to configure quotas and other applications can then use > that API to define specific rules using the quota manager’s API. > > I don’t know if we need a new application or if it would make sense > to, as with policy, add quota management features to keystone. A > single well-defined app has some appeal, but there’s also a certain > amount of extra ramp-up time needed to go that route that we wouldn’t > need if we added the features directly to keystone. I'll also point out that it was largely because of the storage needs that I chose to propose Boson[1] as a separate app, rather than as a library. Further, the dimensions over which quota-covered resources needed to be tracked seemed to me to be complicated enough that it would be better to define a new app and make it support that one domain well, which is why I didn't propose it as something to add to Keystone. Consider: nova has quotas that are applied by user, other quotas that are applied by tenant, and even some quotas on what could be considered sub-resources—a limit on the number of security group rules per security group, for instance. My current feeling is that, if we can figure out a way to make the quota problem into an acceptable library, that will work; it would probably have to maintain its own database separate from the client app and have features for automatically managing the schema, since we couldn't necessarily rely on the client app to invoke the proper juju there. If, on the other hand, that ends up failing, then the best route is probably to begin by developing a separate app, like Boson, as a PoC; then, after we have some idea of just how difficult it is to actually solve the problem, we can evaluate whether it makes sense to actually fold it into a service like Keystone, or whether it should stand on its own. (Personally, I think Boson should be created and should stand on its own, but I also envision using it for purposes outside of OpenStack…) Just my $.02… [1] https://wiki.openstack.org/wiki/Boson -- Kevin L. Mitchell Rackspace ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [api] Requesting opinion/guideline on IDs vs hrefs
On Mon, 2014-11-17 at 19:47 +, Shaunak Kashyap wrote:
> Poppy APIs refer to a flavor in their request and/or response
> representations. Some representations do this by using the flavor ID
> (e.g. 12345) while others use the flavor resource URI (e.g.
> {base}/flavors/12345).
>
> In this context, I created a bug[2] to settle on one way of referring
> to a flavor across all API representations in Poppy. So the question
> to the API working group is this:
>
> How should flavors be referred to in Poppy API representations? Some
> options to consider:
>
> a) Using a “flavor_id” (or similar) property whose value is the flavor
> ID (e.g. 12345),
> b) Using a “flavor_ref” (or similar) property whose value is the
> flavor resource URI (e.g. {base}/flavors/12345),
> c) Using a “links” property whose value is an array of links, one of
> which has an object like { “rel”: “flavor”, “href”:
> “{base}/flavors/12345” },
> d) Similar to c) but using HAL[3] instead,
> e) A combination of a) and c),
> f) A combination of a) and d),
> g) Something else?
I can't think of any API in OpenStack that uses a URI for referencing
anything. We may have *links* in the payloads, but those are provided
for convenience; anytime nova refers to a flavor, it returns the flavor
UUID.
I would, by the way, suggest using UUIDs rather than plain IDs, for
consistency with the rest of the APIs…
--
Kevin L. Mitchell
Rackspace
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Process for program without lead
On Mon, 2014-11-17 at 17:25 +, Hayes, Graham wrote: > Quite often people will come forward in a vacuum - people who thought > they were not right for the job, or felt that someone else would suit > the role better can come forward in a by-election. (I only have > anecdotal evidence for this, but it is first hand, based on other > voluntary, self organising groups I have been part of, and run elections > for over the years). > > I would suggest when nominations close with no candidates, they re-open > immediately for one week, at with point, if there is no candidates I > goes to the TC. While I think the point is valid, an alternate process would be for the election coordinator(s) to point out the lack of candidates and issue a reminder for the procedure a certain amount of time prior to the end of the nomination period. Say, if no candidates have been put forward with 3 days left in the nomination period, then the election coordinator would send out the appropriate reminder email. I think this would have the same effect as the one week re-open period without delaying the election process. -- Kevin L. Mitchell Rackspace ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [oslo] python 2.6 support for oslo libraries
On Thu, 2014-10-23 at 17:19 -0400, Doug Hellmann wrote: > I’m not aware of any Oslo code that presents a problem for those > plugins. We wouldn’t want to cause a problem, but as you say, we don’t > have anywhere to test 2.4 code. Do you know if the Xen driver uses any > of the Oslo code? I missed the [oslo] tag in the subject line and was thinking generally; so no, none of the Xen plugins use anything from oslo, because of the need to support 2.4. -- Kevin L. Mitchell Rackspace ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [oslo] python 2.6 support for oslo libraries
On Thu, 2014-10-23 at 18:56 +0300, Andrey Kurilin wrote: > Just a joke: Can we drop supporting Python 2.6, when several project > still have hooks for Python 2.4? > > https://github.com/openstack/python-novaclient/blob/master/novaclient/exceptions.py#L195-L203 > https://github.com/openstack/python-cinderclient/blob/master/cinderclient/exceptions.py#L147-L155 It may have been intended as a joke, but it's worth pointing out that the Xen plugins for nova (at least) have to be compatible with Python 2.4, because they run on the Xenserver, which has an antiquated Python installed :) As for the clients, we could probably drop that segment now; it's not like we *test* against 2.4, right? :) -- Kevin L. Mitchell Rackspace ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [api] API recommendation
On Wed, 2014-10-15 at 12:39 -0400, Andrew Laski wrote: > On 10/15/2014 11:49 AM, Kevin L. Mitchell wrote: > > Now that we have an API working group forming, I'd like to kick off some > > discussion over one point I'd really like to see our APIs using (and > > I'll probably drop it in to the repo once that gets fully set up): the > > difference between synchronous and asynchronous operations. Using nova > > as an example—right now, if you kick off a long-running operation, such > > as a server create or a reboot, you watch the resource itself to > > determine the status of the operation. What I'd like to propose is that > > future APIs use a separate "operation" resource to track status > > information on the particular operation. For instance, if we were to > > rebuild the nova API with this idea in mind, booting a new server would > > give you a server handle and an operation handle; querying the server > > resource would give you summary information about the state of the > > server (running, not running) and pending operations, while querying the > > operation would give you detailed information about the status of the > > operation. As another example, issuing a reboot would give you the > > operation handle; you'd see the operation in a queue on the server > > resource, but the actual state of the operation itself would be listed > > on that operation. As a side effect, this would allow us (not require, > > though) to queue up operations on a resource, and allow us to cancel an > > operation that has not yet been started. > > > > Thoughts? > > Something like https://review.openstack.org/#/c/86938/ ? > > I know that Jay has proposed a similar thing before as well. I would > love to get some feedback from others on this as it's something I'm > going to propose for Nova in Kilo. Yep, something very much like that :) But the idea behind my proposal is to make that a codified API guideline, rather than just an addition to Nova. -- Kevin L. Mitchell Rackspace ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [api] API recommendation
Now that we have an API working group forming, I'd like to kick off some discussion over one point I'd really like to see our APIs using (and I'll probably drop it in to the repo once that gets fully set up): the difference between synchronous and asynchronous operations. Using nova as an example—right now, if you kick off a long-running operation, such as a server create or a reboot, you watch the resource itself to determine the status of the operation. What I'd like to propose is that future APIs use a separate "operation" resource to track status information on the particular operation. For instance, if we were to rebuild the nova API with this idea in mind, booting a new server would give you a server handle and an operation handle; querying the server resource would give you summary information about the state of the server (running, not running) and pending operations, while querying the operation would give you detailed information about the status of the operation. As another example, issuing a reboot would give you the operation handle; you'd see the operation in a queue on the server resource, but the actual state of the operation itself would be listed on that operation. As a side effect, this would allow us (not require, though) to queue up operations on a resource, and allow us to cancel an operation that has not yet been started. Thoughts? -- Kevin L. Mitchell Rackspace ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] [All] API standards working group
On Tue, 2014-09-23 at 18:18 -0400, Jay Pipes wrote: > Yes, I'd be willing to head up the working group... or at least > participate in it. I'll raise my hand to participate… -- Kevin L. Mitchell Rackspace ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Some ideas for micro-version implementation
On Tue, 2014-09-23 at 18:39 +0100, Louis Taylor wrote: > On Tue, Sep 23, 2014 at 01:32:50PM -0400, Andrew Laski wrote: > > I've been thinking along very similar lines, but I don't think each current > > API needs to be replaced. I would very much like to see a unified API > > project that could be responsible for managing requests to "boot an instance > > with this network and that volume" which would make requests to > > Nova/Neutron/Cinder on the users behalf. > > Isn't this what openstacksdk [0] is? > > [0] https://wiki.openstack.org/wiki/SDK-Development/PythonOpenStackSDK Well, openstacksdk is a client, and we're talking about a server. A server, in this instance, has some advantages over a client, including making it easier to create that client in the first place :) -- Kevin L. Mitchell Rackspace ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Some ideas for micro-version implementation
On Tue, 2014-09-23 at 12:09 -0400, Jay Pipes wrote: > I'd like to say finally that I think there should be an OpenStack API > working group whose job it is to both pull together a set of OpenStack > API practices as well as evaluate new REST APIs proposed in the > OpenStack ecosystem to provide guidance to new projects or new > subprojects wishing to add resources to an existing REST API. One of the things that's been bothering me about OpenStack for a while now is the fact that we have all these different APIs on different endpoints. What I've been wondering about is if we should create a unified ReST API service to replace the service from all of the individual projects. Then, users can just hit that one service to handle all their different interactions. -- Kevin L. Mitchell Rackspace ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [neutron] [infra] Python 2.6 tests can't possibly be passing in neutron
On Mon, 2014-09-22 at 10:32 -0700, Armando M. wrote: > What about: > > https://github.com/openstack/neutron/blob/master/test-requirements.txt#L12 Pulling in ordereddict doesn't do anything if your code doesn't use it when OrderedDict isn't in collections, which is the case here. Further, there's no reason that _get_collection_kwargs() needs to use an OrderedDict: it's initialized in an arbitrary order (generator comprehension over a set), then later passed to functions with **, which converts it to a plain old dict. -- Kevin L. Mitchell Rackspace ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [neutron] [infra] Python 2.6 tests can't possibly be passing in neutron
My team just ran into an issue where neutron was not passing unit tests when run under Python 2.6. We tracked this down to a test support function using collections.OrderedDict. This was in locally forked code, but when I compared it to upstream code, I found that the code in upstream neutron is identical…meaning that upstream neutron cannot possibly be passing unit tests under Python 2.6. Yet, somehow, the neutron reviews I've looked at are passing the Python 2.6 gate! Any ideas as to how this could be happening? For the record, the problem is in neutron/tests/unit/test_api_v2.py:148, in the function _get_collection_kwargs(), which uses collections.OrderedDict. As there's no reason to use OrderedDict here that I can see—there's no definite order on the initialization, and all consumers pass it to an assert_called_once_with() method with the '**' operator—I have proposed a review[1] to replace it with a simple dict. [1] https://review.openstack.org/#/c/123189/ -- Kevin L. Mitchell Rackspace ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [solum] pep8 - splitting expressions
On Thu, 2014-09-11 at 18:35 -0500, Ed Leafe wrote: > On Sep 11, 2014, at 5:05 PM, Kevin L. Mitchell > wrote: > > > I'd suggest trying: > > > >res = amodel.Assemblies( > >uri=common.ASSEM_URI_STR % pecan.request.host_url, > >name='Solum_CAMP_assemblies', > >type='assemblies', > >description=common.ASSEM_DESC_STR, > >assembly_links=a_links, > >parameter_definitions_uri=common.ASSEM_PARAM_STR % > >pecan.request.host_url) > > > > By moving the first argument to a line by itself, pep8 can be satisfied > > by indenting the following lines by 4 spaces. > > When not using visual indentation, the standard is to indent two > levels (i.e., 8 spaces), to distinguish it from typical blocks. But > yes, I much prefer this than creating temporary names to accommodate > the visual indentation style. Using an 8 space indent here would result in an E126 pep8 error, which is why I use 4 spaces for this case. Of course, I don't know if Solum enforces E126. -- Kevin L. Mitchell Rackspace ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [solum] pep8 - splitting expressions
On Tue, 2014-09-09 at 12:05 -0700, Gilbert Pilz wrote: > I have a question with regards to splitting expressions in order to > conform to the pep8 line-length restriction. I have the following bit > of code: > > > res = amodel.Assemblies(uri=common.ASSEM_URI_STR % > pecan.request.host_url, > name='Solum_CAMP_assemblies', > type='assemblies', > description=common.ASSEM_DESC_STR, > assembly_links=a_links, > > parameter_definitions_uri=common.ASSEM_PARAM_STR % > pecan.request.host_url) I'd suggest trying: res = amodel.Assemblies( uri=common.ASSEM_URI_STR % pecan.request.host_url, name='Solum_CAMP_assemblies', type='assemblies', description=common.ASSEM_DESC_STR, assembly_links=a_links, parameter_definitions_uri=common.ASSEM_PARAM_STR % pecan.request.host_url) By moving the first argument to a line by itself, pep8 can be satisfied by indenting the following lines by 4 spaces. -- Kevin L. Mitchell Rackspace ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [neutron] [nova] non-deterministic gate failures due to unclosed eventlet Timeouts
On Mon, 2014-09-08 at 17:25 -0400, Jay Pipes wrote: > > Thanks, that might be what's causing this timeout/gate failure in the > > nova unit tests. [1] > > > > [1] https://bugs.launchpad.net/nova/+bug/1357578 > > Indeed, there are a couple places where eventlet.timeout.Timeout() seems > to be used in the test suite without a context manager or calling > close() explicitly: > > tests/virt/libvirt/test_driver.py > 8925:raise eventlet.timeout.Timeout() > > tests/virt/hyperv/test_vmops.py > 196:mock_with_timeout.side_effect = etimeout.Timeout() I looked into that too, but the docs for Timeout indicate that it's an Exception subclass, and passing it no args doesn't seem to start the timer running. I think you have to explicitly pass a duration value for Timeout to enable its timeout behavior, but that's just a guess on my part at this point… -- Kevin L. Mitchell Rackspace ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] doubling our core review bandwidth
> tl;dr I think we should drop our 'needs 2x+2 to land' rule and instead > use 'needs 1x+2'. We can ease up a large chunk of pressure on our > review bottleneck, with the only significant negative being that core > reviewers may see less of the code going into the system - but they > can always read more to stay in shape if thats an issue :) I'm going to make a tangential but somewhat related suggestion. Instead of reducing the number of +2s required, what would happen if we gave our more trusted but non-core reviewers the ability to +2, but leave +A in the hands of the existing cores? That way, their reviews can be counted by the core reviewers. With this change in policy, you still need two +2s, but you have more people that can +2, and you only need one of our limited number of cores to review. -- Kevin L. Mitchell Rackspace ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Averting the Nova crisis by splitting out virt drivers
On Fri, 2014-09-05 at 10:26 +0100, Daniel P. Berrange wrote: > > 2. Removal of drivers other than the reference implementation for each > > project could be the healthiest option > > a. Requires transparent, public, automated 3'rd party CI > > b. Requires a TRUE plugin architecture and mentality > > c. Requires a stable and well defined API > > As mentioned in the original mail I don't want to see a situation where > we end up with some drivers in tree and others out of tree as it sets up > bad dynamics within the project. Those out of tree will always have the > impression of being second class citizens and thus there will be constant > pressure to accept drivers back into tree. The so called 'reference' > driver that stayed in tree would also continue to be penalized in the > way it is today, and so its development would be disadvantaged compared > to the out of tree drivers. I have one quibble with the notion of "not even one" driver in core: I think it is probably useful to include a dummy, do-nothing driver that can be used for in-tree functional tests and as an example to point those interested in writing a driver. Then, the "second-class citizen" is the one actually in the tree :) Beyond that, I agree with this proposal: it has never made sense to me that *all* drivers live in the tree, and it actually offends my sense of organization to have the tree so cluttered; we split functions when they get too big, we split modules when they get too big, and we create subdirectories when packages get too big, so why not split repos when they get too big? -- Kevin L. Mitchell Rackspace ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][oslo] oslo.config and import chains
On Thu, 2014-08-07 at 17:41 +0100, Matthew Booth wrote: > ... or arg is an object which defines __nonzero__(), or defines > __getattr__() and then explodes because of the unexpected lookup of a > __nonzero__ attribute. Or it's False (no quotes when printed by the > debugger), but has a unicode type and therefore evaluates to True[1]. If you're passing such exotic objects as parameters that could potentially be drawn from configuration instead, maybe that code needs to be refactored a bit :) > However, if you want to compare a value with None and write 'foo is > None' it will always do exactly what you expect, regardless what you > pass to it. I think it's also nicer to the reviewer and the > maintainer, > who then don't need to go looking for context to check if anything > invalid might be passed in. In the vast majority of cases, however, we use a value that evaluates to False to indicate "use the default", where "default" may be drawn from configuration. Yes, there are cases where we must treat, say, 0 as distinct from None, but when we don't need to, we should keep the code as simple as possible. After all, I doubt anyone would seriously suggest that we must always use something like the "_unset" sentinel, even when None has no special meaning… -- Kevin L. Mitchell Rackspace ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][oslo] oslo.config and import chains
On Thu, 2014-08-07 at 17:46 +0100, Matthew Booth wrote: > > In any case, the operative point is that CONF. must > always be > > evaluated inside run-time code, never at module load time. > > ...unless you call register_opts() safely, which is what I'm > proposing. No, calling register_opts() at a different point only fixes the import issue you originally complained about; it does not fix the problem that the configuration option is evaluated at the wrong time. The example code you included in your original email evaluates the configuration option at module load time, BEFORE the configuration has been loaded, which means that the argument default will be the default of the configuration option, rather than the configured value of the configuration option. Configuration options must be evaluated at RUN-TIME, after configuration is loaded; they must not be evaluated at LOAD-TIME, which is what your original code does. -- Kevin L. Mitchell Rackspace ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
