Re: [openstack-dev] [nova][libvirt] Non-readonly connection to libvirt in unit tests
FYI, the context of this is that I would like to be able to test some of the libvirt storage pool code against a live file system, as we currently test the storage pool code. To do this, we need at least to be able to get a proper connection to a session daemon. IMHO, since these calls aren't expensive, so to speak, it should be fine to have them run against a real libvirt. So If we require libvirt-python for tests and that requires libvirt-bin, what's stopping us from just removing fakelibvirt since it's kind of useless now anyway, right? The thing about fakelibvirt is that it allows us to operate against against a libvirt API without actually doing libvirt-y things like launching VMs. Now, libvirt does have a test:///default URI that IIRC has similar functionality, so we could start to phase out fakelibvirt in favor of that. However, there are probably still some spots where we'll want to use fakelibvirt. Best Regards, Solly - Original Message - From: Matt Riedemann mrie...@linux.vnet.ibm.com To: openstack-dev@lists.openstack.org Sent: Wednesday, August 20, 2014 8:37:39 PM Subject: Re: [openstack-dev] [nova][libvirt] Non-readonly connection to libvirt in unit tests On 8/11/2014 4:42 AM, Daniel P. Berrange wrote: On Mon, Aug 04, 2014 at 06:46:13PM -0400, Solly Ross wrote: Hi, I was wondering if there was a way to get a non-readonly connection to libvirt when running the unit tests on the CI. If I call `LibvirtDriver._connect(LibvirtDriver.uri())`, it works fine locally, but the upstream CI barfs with libvirtError: internal error Unable to locate libvirtd daemon in /usr/sbin (to override, set $LIBVIRTD_PATH to the name of the libvirtd binary). If I try to connect by calling libvirt.open(None), it also barfs, saying I don't have permission to connect. I could just set it to always use fakelibvirt, but it would be nice to be able to run some of the tests against a real target. The tests in question are part of https://review.openstack.org/#/c/111459/, and involve manipulating directory-based libvirt storage pools. Nothing in the unit tests should rely on being able to connect to the libvirt daemon, as the unit tests should still be able to pass when the daemon is not running at all. We should be either using fakelibvirt or mocking any libvirt APIs that need to be tested Regards, Daniel Also, doesn't this kind of break with the test requirement on libvirt-python now? Before I was on trusty and trying to install that it was failing because I didn't have a new enough version of libvirt-bin installed. So if we require libvirt-python for tests and that requires libvirt-bin, what's stopping us from just removing fakelibvirt since it's kind of useless now anyway, right? -- Thanks, Matt Riedemann ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][libvirt] Non-readonly connection to libvirt in unit tests
(response inline) - Original Message - From: Daniel P. Berrange berra...@redhat.com To: Solly Ross sr...@redhat.com Cc: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Sent: Thursday, August 21, 2014 11:23:17 AM Subject: Re: [openstack-dev] [nova][libvirt] Non-readonly connection to libvirt in unit tests On Thu, Aug 21, 2014 at 11:14:33AM -0400, Solly Ross wrote: (reply inline) - Original Message - From: Daniel P. Berrange berra...@redhat.com To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Sent: Thursday, August 21, 2014 11:05:18 AM Subject: Re: [openstack-dev] [nova][libvirt] Non-readonly connection to libvirt in unit tests On Thu, Aug 21, 2014 at 10:52:42AM -0400, Solly Ross wrote: FYI, the context of this is that I would like to be able to test some of the libvirt storage pool code against a live file system, as we currently test the storage pool code. To do this, we need at least to be able to get a proper connection to a session daemon. IMHO, since these calls aren't expensive, so to speak, it should be fine to have them run against a real libvirt. No it really isn't OK to run against the real libvirt host system when in the unit tests. Unit tests must *not* rely on external system state in this way because it will lead to greater instability and unreliability of our unit tests. If you want to test stuff against the real libvirt storage pools then that becomes a functional / integration test suite which is pretty much what tempest is targetting. That's all well and good, but we *currently* manipulates the actual file system manually in tests. Should we then say that we should never manipulate the actual file system either? In that case, there are some tests which need to be refactored. Places where the tests manipulate the filesystem though should be doing so in an isolated playpen directory, not in the live location where a deployed nova runs, so that's not the same thing. Ah, but in the case I mentioned before, we're dealing with storage pools, which can just be created in the playpen directory. In that case, libvirt is simply acting as a library for filesystem access. To further ensure isolation, you could also connect to a session daemon instead of a system daemon. I'm of the opinion that requiring some form of libvirt to be installed to run the *libvirt* unit tests isn't actually that big of a deal, since you can build libvirt without extra stuff and get a libvirt that has just enough for you to test against. Generally it's the developers that will be running the unit tests (and the CI), and if a developer is running the libvirt unit tests, he or she is probably developing for the libvirt driver, and thus should probably have libvirt installed in some form. So If we require libvirt-python for tests and that requires libvirt-bin, what's stopping us from just removing fakelibvirt since it's kind of useless now anyway, right? The thing about fakelibvirt is that it allows us to operate against against a libvirt API without actually doing libvirt-y things like launching VMs. Now, libvirt does have a test:///default URI that IIRC has similar functionality, so we could start to phase out fake libvirt in favor of that. However, there are probably still some spots where we'll want to use fakelibvirt. I'm actually increasingly of the opinion that we should not in fact be trying to use the real libvirt library in the unit tests at all as it is not really adding any value. We typically nmock out all the actual API calls we exercise so despite using libvirt-python we are not in fact exercising its code or even validating that we're passing the correct numbers of parameters to API calls. Pretty much all we really relying on is the existance of the various global constants that are defined, and that has been nothing but trouble because the constants may or may not be defined depending on the version. Isn't that what 'test:///default' is supposed to be? A version of libvirt with libvirt not actually touching the rest of the system? Yes, that is what it allows for, however, even if we used that URI we still wouldn't be actually exercising any of the libvirt code in any meaningful way because our unit tests mock out all the API calls that get touched. So using libvirt-python + test:///default URI doesn't really seem to buy us anything, but it does still mean that developers need to have libvirt installed in order to run the unit tests. I'm not convinced that is a beneficial tradeoff. I think it would make writing unit tests easier, because you don't have to worry about making sure that the fakelibvirt implementation matches the real libvirt implementation, and you don't have
Re: [openstack-dev] [Nova] Feature Freeze Exception process for Juno
I will follow up with a more detailed email about what I believe we are missing, once the FF settles and I have applied some soothing creme to my burnout wounds, but currently my sentiment is: Contributing features to Nova nowadays SUCKS!!1 (even as a core reviewer) We _have_ to change that! I think this is *very* important. rant For instance, I have/had two patch series up. One is of length 2 and is relatively small. It's basically sitting there with one +2 on each patch. I will now most likely have to apply for a FFE to get it merged, not because there's more changes to be made before it can get merged (there was one small nit posted yesterday) or because it's a huge patch that needs a lot of time to review, but because it just took a while to get reviewed by cores, and still only appears to have been looked at by one core. For the other patch series (which is admittedly much bigger), it was hard just to get reviews (and it was something where I actually *really* wanted several opinions, because the patch series touched a couple of things in a very significant way). Now, this is not my first contribution to OpenStack, or to Nova, for that matter. I know things don't always get in. It's frustrating, however, when it seems like the reason something didn't get in wasn't because it was fundamentally flawed, but instead because it didn't get reviews until it was too late to actually take that feedback into account, or because it just didn't get much attention review-wise at all. If I were a new contributor to Nova who had successfully gotten a major blueprint approved and the implemented, only to see it get rejected like this, I might get turned off of Nova, and go to work on one of the other OpenStack projects that seemed to move a bit faster. /rant So, it's silly to rant without actually providing any ideas on how to fix it. One suggestion would be, for each approved blueprint, to have one or two cores explicitly marked as being responsible for providing at least some feedback on that patch. This proposal has issues, since we have a lot of blueprints and only twenty cores, who also have their own stuff to work on. However, I think the general idea of having guaranteed reviewers is not unsound by itself. Perhaps we should have a loose tier of reviewers between core and everybody else. These reviewers would be known good reviewers who would follow the implementation of particular blueprints if a core did not have the time. Then, when those reviewers gave the +1 to all the patches in a series, they could ping a core, who could feel more comfortable giving a +2 without doing a deep inspection of the code. That's just one suggestion, though. Whatever the solution may be, this is a problem that we need to fix. While I enjoyed going through the blueprint process this cycle (not sarcastic -- I actually enjoyed the whole structured feedback thing), the follow up to that was not the most pleasant. One final note: the specs referenced above didn't get approved until Spec Freeze, which seemed to leave me with less time to implement things. In fact, it seemed that a lot of specs didn't get approved until spec freeze. Perhaps if we had more staggered approval of specs, we'd have more staggered submission of patches, and thus less of a sudden influx of patches in the couple weeks before feature proposal freeze. Best Regards, Solly Ross - Original Message - From: Nikola Đipanov ndipa...@redhat.com To: openstack-dev@lists.openstack.org Sent: Wednesday, September 3, 2014 5:50:09 AM Subject: Re: [openstack-dev] [Nova] Feature Freeze Exception process for Juno On 09/02/2014 09:23 PM, Michael Still wrote: On Tue, Sep 2, 2014 at 1:40 PM, Nikola Đipanov ndipa...@redhat.com wrote: On 09/02/2014 08:16 PM, Michael Still wrote: Hi. We're soon to hit feature freeze, as discussed in Thierry's recent email. I'd like to outline the process for requesting a freeze exception: * your code must already be up for review * your blueprint must have an approved spec * you need three (3) sponsoring cores for an exception to be granted Can core reviewers who have features up for review have this number lowered to two (2) sponsoring cores, as they in reality then need four (4) cores (since they themselves are one (1) core but cannot really vote) making it an order of magnitude more difficult for them to hit this checkbox? That's a lot of numbers in that there paragraph. Let me re-phrase your question... Can a core sponsor an exception they themselves propose? I don't have a problem with someone doing that, but you need to remember that does reduce the number of people who have agreed to review the code for that exception. Michael has correctly picked up on a hint of snark in my email, so let me explain where I was going with that: The reason many features including my own may not make the FF is not because
Re: [openstack-dev] [nova] Averting the Nova crisis by splitting out virt drivers
My only question is about the need to separate out each virt driver into a separate project, wouldn't you accomplish a lot of the benefit by creating a single virt project that includes all of the drivers? I don't think there's particularly a *point* to having all drivers in one repo. Part of code review is looking for code gotchas, but part of code review is looking for subtle issues that are caused by the very nature of the driver. A HyperV core reviewing a libvirt change should certainly be able to provide the former, but most likely cannot provide the latter to a sufficient degree (if he or she can, then he or she should be a libvirt core as well). A strong +1 to Dan's proposal. I think this would also make it easier for non-core reviewers to get started reviewing, without having a specialized tool setup. Best Regards, Solly Ross P.S. This is a crisis. A large crisis. In fact, if you got a moment, it's a twelve-storey crisis with a magnificent entrance hall, carpeting throughout, 24-hour portage, and an enormous sign on the roof, saying 'This Is a Large Crisis'. A large crisis requires a large plan. Ha! - Original Message - From: Donald D Dugger donald.d.dug...@intel.com To: Daniel P. Berrange berra...@redhat.com, OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Sent: Thursday, September 4, 2014 10:33:27 AM Subject: Re: [openstack-dev] [nova] Averting the Nova crisis by splitting out virt drivers Basically +1 with what Daniel is saying (note that, as mentioned, a side effect of our effort to split out the scheduler will help but not solve this problem). My only question is about the need to separate out each virt driver into a separate project, wouldn't you accomplish a lot of the benefit by creating a single virt project that includes all of the drivers? I wouldn't necessarily expect a VMware guy to understand the specifics of the HyperV implementation but both people should understand what a virt driver does, how it interfaces to Nova and they should be able to intelligently review each other's code. -- Don Dugger Censeo Toto nos in Kansa esse decisse. - D. Gale Ph: 303/443-3786 -Original Message- From: Daniel P. Berrange [mailto:berra...@redhat.com] Sent: Thursday, September 4, 2014 4:24 AM To: OpenStack Development Subject: [openstack-dev] [nova] Averting the Nova crisis by splitting out virt drivers Position statement == Over the past year I've increasingly come to the conclusion that Nova is heading for (or probably already at) a major crisis. If steps are not taken to avert this, the project is likely to loose a non-trivial amount of talent, both regular code contributors and core team members. That includes myself. This is not good for Nova's long term health and so should be of concern to anyone involved in Nova and OpenStack. For those who don't want to read the whole mail, the executive summary is that the nova-core team is an unfixable bottleneck in our development process with our current project structure. The only way I see to remove the bottleneck is to split the virt drivers out of tree and let them all have their own core teams in their area of code, leaving current nova core to focus on all the common code outside the virt driver impls. I, now, none the less urge people to read the whole mail. Background information == I see many factors coming together to form the crisis - Burn out of core team members from over work - Difficulty bringing new talent into the core team - Long delay in getting code reviewed merged - Marginalization of code areas which aren't popular - Increasing size of nova code through new drivers - Exclusion of developers without corporate backing Each item on their own may not seem too bad, but combined they add up to a big problem. Core team burn out -- Having been involved in Nova for several dev cycles now, it is clear that the backlog of code up for review never goes away. Even intensive code review efforts at various points in the dev cycle makes only a small impact on the backlog. This has a pretty significant impact on core team members, as their work is never done. At best, the dial is sometimes set to 10, instead of 11. Many people, myself included, have built tools to help deal with the reviews in a more efficient manner than plain gerrit allows for. These certainly help, but they can't ever solve the problem on their own - just make it slightly more bearable. And this is not even considering that core team members might have useful contributions to make in ways beyond just code review. Ultimately the workload is just too high to sustain the levels of review required, so core team members will eventually burn out (as they have done many times already). Even if one person attempts to take the initiative
[openstack-dev] [nova][FFE] websocket-proxy-to-host-security
I would like to request a feature freeze exception for the Websocket Proxy to Host Security. The spec [1] was approved for Nova, and the patches [2] are currently sitting there with one +2 (courtesy of @danpb), with a +1 from Jenkins. For a TL;DR on the spec, essentially this patch series implements a framework for authenticating and encrypting communication between the VNC proxy and the actual compute nodes, and then implements a TLS/x509 driver using that framework. The patches don't touch much, and are enabled optionally, so they should be relatively safe. Best Regards, Solly Ross [1] https://review.openstack.org/#/c/115483/ [2] https://review.openstack.org/#/c/115483/, https://review.openstack.org/#/c/115484/ ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Feature Freeze Exception process for Juno
(see comments inline) - Original Message - From: Jay Pipes jaypi...@gmail.com To: openstack-dev@lists.openstack.org Sent: Thursday, September 4, 2014 2:34:28 PM Subject: Re: [openstack-dev] [Nova] Feature Freeze Exception process for Juno On 09/03/2014 11:57 AM, Solly Ross wrote: I will follow up with a more detailed email about what I believe we are missing, once the FF settles and I have applied some soothing creme to my burnout wounds, but currently my sentiment is: Contributing features to Nova nowadays SUCKS!!1 (even as a core reviewer) We _have_ to change that! I think this is *very* important. rant For instance, I have/had two patch series up. One is of length 2 and is relatively small. It's basically sitting there with one +2 on each patch. I will now most likely have to apply for a FFE to get it merged, not because there's more changes to be made before it can get merged (there was one small nit posted yesterday) or because it's a huge patch that needs a lot of time to review, but because it just took a while to get reviewed by cores, and still only appears to have been looked at by one core. For the other patch series (which is admittedly much bigger), it was hard just to get reviews (and it was something where I actually *really* wanted several opinions, because the patch series touched a couple of things in a very significant way). Now, this is not my first contribution to OpenStack, or to Nova, for that matter. I know things don't always get in. It's frustrating, however, when it seems like the reason something didn't get in wasn't because it was fundamentally flawed, but instead because it didn't get reviews until it was too late to actually take that feedback into account, or because it just didn't get much attention review-wise at all. If I were a new contributor to Nova who had successfully gotten a major blueprint approved and the implemented, only to see it get rejected like this, I might get turned off of Nova, and go to work on one of the other OpenStack projects that seemed to move a bit faster. /rant I see that you have done 7 reviews in the past 90 days. Doing reviews on other people's code goes a long way towards getting some core karma. So, I've tried to get in to reviewing, but my experience tends to be that, even with tools, I spend most of my time trying to find something to review. I've found a few tags that I feel comfortable reviewing (because I know the code paths well enough to give good feedback and not just nits). However, I often find myself on a thread where a couple of cores have already commented, in which case I likely have very little to say. This is not particularly an excuse, just an explanation. The other thing, of course, is that I get/got zoned in writing a large patch series, and didn't stop much to review. For me at least, I would love to be able to put my name into some sort of pool, and said ping me if you need a review of X subjects. If someone actually asked me for a review, I would gladly put down my work for a bit and do a review, but I need to get better and stopping on my own and proactively seeking out reviews. Anyway, I think it would be useful to have some sort of page where people could say I'm an SME in X, ask me for reviews and then patch submitters could go and say, oh, I need an someone to review my patch about storage backends, let me ask sross. So, it's silly to rant without actually providing any ideas on how to fix it. One suggestion would be, for each approved blueprint, to have one or two cores explicitly marked as being responsible for providing at least some feedback on that patch. This proposal has issues, since we have a lot of blueprints and only twenty cores, who also have their own stuff to work on. However, I think the general idea of having guaranteed reviewers is not unsound by itself. Perhaps we should have a loose tier of reviewers between core and everybody else. These reviewers would be known good reviewers who would follow the implementation of particular blueprints if a core did not have the time. Then, when those reviewers gave the +1 to all the patches in a series, they could ping a core, who could feel more comfortable giving a +2 without doing a deep inspection of the code. This is exactly the reason I believe there are so many places in Nova that we have such an astonishing amount of technical debt. Core reviewers are so swamped with the review load that they see +1s from folks that they perceive to be SMEs on certain parts of the code and give a less-than-needed-quality review of the patch and end up +2'ing it too early. It is *precisely* the holistic understanding of multiple parts of the Nova codebase that is the critical piece that having 2 core reviewers sign-off on the patch. Cores are supposed to have a good grasp
Re: [openstack-dev] [Nova] Feature Freeze Exception process for Juno
Well, I'm definitely down to do some work on something like that (especially since the original quote was from me ;-)). Perhaps we should split this off into a separate thread and have some design/feature discussions once the mad rush to get Juno out the door dies down? Best Regards, Solly - Original Message - From: Joe Gordon joe.gord...@gmail.com To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Sent: Friday, September 5, 2014 4:30:57 PM Subject: Re: [openstack-dev] [Nova] Feature Freeze Exception process for Juno On Fri, Sep 5, 2014 at 4:05 AM, Nikola Đipanov ndipa...@redhat.com wrote: On 09/04/2014 10:25 PM, Solly Ross wrote: Anyway, I think it would be useful to have some sort of page where people could say I'm an SME in X, ask me for reviews and then patch submitters could go and say, oh, I need an someone to review my patch about storage backends, let me ask sross. This is a good point - I've been thinking along similar lines that we really could have a huge win in terms of the review experience by building a tool (maybe a social network looking one :)) that relates reviews to people being able to do them, visualizes reviewer karma and other things that can help make the code submissions and reviews more human friendly. Dan seems to dismiss the idea of improved tooling as something that can get us only thus far, but I am not convinced. However - this will require even more manpower and we are already ridiculously short on that so... I have previously toyed with idea of making such a tool, and if someone else wants to work on it I would be happy to help. N. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [nova][FFE] Use Libvirt Storage Pools (Partial)
Hi, I would like to request a feature freeze exception for the first part of the use-libvirt-storage-pools blueprint [1]. Overview The patches in question [2] would entail adding support for libvirt storage pools, but neither making them default nor adding in automatic transitioning [3] from the legacy backends to the storage pool backends. This would allow new deployments and/or adventurous existing deployments to switch to libvirt storage pools, while adding an extra release before the automated transitioning and switch to default (as per the blueprint) is made. Risk While the risk is not negligible, it is relatively minor, since the major changes are behind a configuration option ([libvirt] use_storage_pools). Additionally, since the major changes are enabled by default [4], there is little danger of breakage in existing deployments Benefits The plan according to the original blueprint was to enable automatic transitioning of legacy backends to the new backend, and setting the storage pools to default to enabled. However, this would likely not be a good idea for feature freeze, as things like that can cause complicated and odd bugs (although we all hope our code has no bugs, it's more or less inevitable that bugs pop up ;-)). However, adding support for using the storage pool backend without enabling it by default or adding in automatic transitioning in Juno would give an extra cycle to ensure that any bugs get reported before the storage pool backend is made default. Then, the rest of the spec could proceed as planned, just moved up a relase (i.e. deprecate the legacy backends in Kilo, and remove in L, assuming, of course, that the spec is re-approved). [1] https://review.openstack.org/#/c/86947/7 [2] https://review.openstack.org/#/c/113058/, https://review.openstack.org/#/c/113059/, https://review.openstack.org/#/c/113060/ [3] The patch for enabling automatic transitioning is up on Gerrit. However, there are a couple of flaws in its implementation (not bugs, per se, but things I would like to fix none-the-less). However, this patch is not needed for the previous patches to function properly, as they stand on their own. [4] The only significant change not behind a configuration option is the change that makes the image cache use a file-based storage pool to manage its images. Since the pool is file-based and automatically created in the same location as the current image cache, there is little risk associated. Best Regards, Solly Ross ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] global-reqs on tooz pulls in worrisome transitive dep
For the future == IMHO, we shouldn't be pulling in duplicate dependencies when we can control it. Since tooz is part of stackforge, it's somewhat part of OpenStack. We should strive to make all OpenStack projects use one memcached client. That being said, a quick Google search indicates that pymemcache has some benefits over python-memcache, the latter not being python 3 compatible. Additionally, pymemcache was written by the Pinterest people, so I'd imaging it stands up fairly well under stress. Perhaps we should consider porting the existing OpenStack projects from python-memcache to pymemcache. In the future, though, we need to make sure to avoid getting into a situation like this. For the present === Probably, we'll just have to get pymemcache packaged for Fedora in some form. Like you said, it can be bundled in RDO. If you're not using RDO I think it's safe to just tell people to install it from other sources (pip install) until we can get the package into Fedora. Best Regards, Solly Ross - Original Message - From: Matt Riedemann mrie...@linux.vnet.ibm.com To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Sent: Tuesday, September 9, 2014 4:30:02 PM Subject: [openstack-dev] global-reqs on tooz pulls in worrisome transitive dep It took me a while to untangle this so prepare for links. :) I noticed this change [1] today for global-requirements to require tooz [2] for a ceilometer blueprint [3]. The sad part is that tooz requires pymemcache [4] which is, from what I can tell, a memcached client that is not the same as python-memcached [5]. Note that python-memcached is listed in global-requirements already [6]. The problem I have with this is it doesn't appear that RHEL/Fedora package pymemcache (they do package python-memcached). I see that openSUSE builds separate packages for each. It looks like Ubuntu also has separate packages. My question is, is this a problem? I'm assuming RDO will just have to package python-pymemcache themselves but what about people not using RDO (SOL? Don't care? Other?). Reverting the requirements change would probably mean reverting the ceilometer blueprint (or getting a version of tooz out that works with python-memcached which is probably too late for that right now). Given the point in the schedule that seems pretty drastic. Maybe I'm making more of this than it's worth but wanted to bring it up in case anyone else has concerns. [1] https://review.openstack.org/#/c/93443/ [2] https://github.com/stackforge/tooz/blob/master/requirements.txt#L6 [3] http://specs.openstack.org/openstack/ceilometer-specs/specs/juno/central-agent-partitioning.html [4] https://pypi.python.org/pypi/pymemcache [5] https://pypi.python.org/pypi/python-memcached/ [6] https://github.com/openstack/requirements/blob/master/global-requirements.txt#L108 -- Thanks, Matt Riedemann ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] On an API proxy from baremetal to ironic
With my admittedly limited knowledge of the whole Ironic process, the question seems to me to be: If we don't implement a proxy, which people are going to have a serious problem? Do we have an data on which users/operators are making use of the baremetal API in any extensive fashion? If nobody's using it, or the people using it aren't using in an extensive fashion, I think we don't need to make a proxy for it. Strengthening this argument is the fact that we would only be proxying the first two calls, so it wouldn't be a drop-in replacement anyway. Best Regards, Solly Ross - Original Message - From: Michael Still mi...@stillhq.com To: OpenStack Development Mailing List openstack-dev@lists.openstack.org Sent: Tuesday, September 9, 2014 5:24:11 PM Subject: [openstack-dev] On an API proxy from baremetal to ironic Hi. One of the last things blocking Ironic from graduating is deciding whether or not we need a Nova API proxy for the old baremetal extension to new fangled Ironic API. The TC has asked that we discuss whether we think this functionality is actually necessary. It should be noted that we're _not_ talking about migration of deployed instances from baremetal to Ironic. That is already implemented. What we are talking about is if users post-migration should be able to expect their previous baremetal Nova API extension to continue to function, or if they should use the Ironic APIs from that point onwards. Nova had previously thought this was required, but it hasn't made it in time for Juno unless we do a FFE, and it has been suggested that perhaps its not needed at all because it is an admin extension. To be super specific, we're talking about the baremetal nodes admin extension here. This extension has the ability to: - list nodes running baremetal - show detail of one of those nodes - create a new baremetal node - delete a baremetal node Only the first two of those would be supported if we implemented a proxy. So, discuss. Thanks, Michael -- Rackspace Australia ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] On an API proxy from baremetal to ironic
As far as I understand it, though, that's a patch for a read-only mode. It seems bizzare, and possibly dangerous, to proxy read commands, but not write commands. It gives the impression that everything's fine until it's not fine (because someone tried to use an existing script to do a create command). IMHO, it would be better to just tell people up front Update your scripts to use Ironic, because they won't work at all instead of leading people (through empirical evidence) to believe that their scripts will work, and then having them discover later that something broke because they tried to create a node. Best Regards, Solly Ross - Original Message - From: Sean Dague s...@dague.net To: openstack-dev@lists.openstack.org Sent: Wednesday, September 10, 2014 10:33:05 AM Subject: Re: [openstack-dev] On an API proxy from baremetal to ironic On 09/09/2014 11:22 PM, Russell Bryant wrote: On 09/09/2014 05:24 PM, Michael Still wrote: Hi. One of the last things blocking Ironic from graduating is deciding whether or not we need a Nova API proxy for the old baremetal extension to new fangled Ironic API. The TC has asked that we discuss whether we think this functionality is actually necessary. It should be noted that we're _not_ talking about migration of deployed instances from baremetal to Ironic. That is already implemented. What we are talking about is if users post-migration should be able to expect their previous baremetal Nova API extension to continue to function, or if they should use the Ironic APIs from that point onwards. Nova had previously thought this was required, but it hasn't made it in time for Juno unless we do a FFE, and it has been suggested that perhaps its not needed at all because it is an admin extension. To be super specific, we're talking about the baremetal nodes admin extension here. This extension has the ability to: - list nodes running baremetal - show detail of one of those nodes - create a new baremetal node - delete a baremetal node Only the first two of those would be supported if we implemented a proxy. So, discuss. I'm in favor of proceeding with deprecation without requiring the API proxy. In the case of user facing APIs, the administrators in charge of upgrading the cloud do not have full control over all of the apps using the APIs. In this particular case, I would expect that the cloud administrators have *complete* control over the use of these APIs. Assuming we have one overlap release (Juno) to allow the migration to occur and given proper documentation of the migration plan and release notes stating the fact that the old APIs are going away, we should be fine. In summary, +1 to moving forward without the API proxy requirement. The thing is, we have the patch - https://review.openstack.org/#/c/120433/, so it's not like there is a zomg run around to get the patch. I think we should FFE this patch as it provides a smoother transition from baremetal to ironic. The patch is extremely low risk to the rest of Nova, as it's a surface proxy feature, so lets just land it and move forward. -Sean -- Sean Dague http://dague.net ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Horizon] Licensing issue with using JSHint in build
JSHint *isn't* Douglas Crockford. It was written by someone who (understandably) thought Douglas Crockford had some good ideas, but was overzealous. It does mostly the same things, but is more lenient with regards to style elements. The license is as such: https://github.com/jshint/jshint/blob/master/LICENSE You are thinking of JSLint, which is written by Douglas Crockford. - Original Message - From: Aaron Sahlin asah...@linux.vnet.ibm.com To: openstack-dev@lists.openstack.org Sent: Wednesday, September 10, 2014 11:56:37 AM Subject: [openstack-dev] [Horizon] Licensing issue with using JSHint in build I noticed that the build is using JSHint now, and before I consider syncing it with the proposed options from the JavaScript best practices (https://review.openstack.org/#/c/117595/), I wanted to double check and be sure Horizon got past the legal problem with the good/evil licensing. Some background for those who are not aware. JSHint was authored by Doug Crockford, and he added an extra line in the licensing, The software shall be used for good, not evil. The issue is in the definition of what is good and what is evil. It is too subjective, what is evil differs from person to person therefore ends up being a liability and leaving users open to frivolous lawsuits. Did Horizon get permission or find some way around the licensing issue? ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Horizon] Licensing issue with using JSHint in build
- Original Message - From: Jeremy Stanley fu...@yuggoth.org To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Sent: Wednesday, September 10, 2014 1:10:18 PM Subject: Re: [openstack-dev] [Horizon] Licensing issue with using JSHint in build On 2014-09-10 13:00:29 -0400 (-0400), Solly Ross wrote: JSHint *isn't* Douglas Crockford. It was written by someone who (understandably) thought Douglas Crockford had some good ideas, but was overzealous. [...] Overzealous enough to copy his code. ?? This sentence doesn't make much sense. I meant to say that Douglas Crockford was overzealous (which he is, IMO). The license is as such: https://github.com/jshint/jshint/blob/master/LICENSE Ahem. https://github.com/jshint/jshint/blob/master/src/jshint.js#L19 Fair enough. I stand corrected. I didn't catch that. The general license, however, is as stated. You are thinking of JSLint, which is written by Douglas Crockford. JSHint is a derivative project of JSLint. Sorry to burst your bubble. To be fair, it's been undergoing *major* revisions lately, making it resemble JSHint less and less in terms of what it checks for. Having used it in the past, functionality wise it's very different. While it maintains some backwards compatibility, it has added in new checks, doesn't complain about nearly the number of things that JSLint complains about (for good reasons). -- Jeremy Stanley ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Horizon] Licensing issue with using JSHint in build
I want to apologize for my rapid response, I was incorrect about the license because of the file you pointed out. I did not intend to sound snarky or anything like that in either the original email or the reply. Anyway, for future reference, I believe the last thread where this was discussed was here: http://lists.openstack.org/pipermail/openstack-dev/2014-April/031689.html, which basically reiterates what David says above (it's good to have links to the past discussions, IMO). Best Regards, Solly Ross P.S. Here's hoping that the JSHint devs eventually find a way to remove that line from the file -- according to https://github.com/jshint/jshint/issues/1234, not much of the original remains. - Original Message - From: Aaron Sahlin asah...@linux.vnet.ibm.com To: openstack-dev@lists.openstack.org Sent: Wednesday, September 10, 2014 1:35:48 PM Subject: Re: [openstack-dev] [Horizon] Licensing issue with using JSHint in build What you are finding is the same as I found, which raised my concern. Thanks for the pointer to legal-disc...@lists.openstack.org, I will post the question there (let the lawyers figure it out). On 9/10/2014 12:16 PM, Solly Ross wrote: - Original Message - From: Jeremy Stanley fu...@yuggoth.org To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Sent: Wednesday, September 10, 2014 1:10:18 PM Subject: Re: [openstack-dev] [Horizon] Licensing issue with using JSHint in build On 2014-09-10 13:00:29 -0400 (-0400), Solly Ross wrote: JSHint *isn't* Douglas Crockford. It was written by someone who (understandably) thought Douglas Crockford had some good ideas, but was overzealous. [...] Overzealous enough to copy his code. ?? This sentence doesn't make much sense. I meant to say that Douglas Crockford was overzealous (which he is, IMO). The license is as such: https://github.com/jshint/jshint/blob/master/LICENSE Ahem. https://github.com/jshint/jshint/blob/master/src/jshint.js#L19 Fair enough. I stand corrected. I didn't catch that. The general license, however, is as stated. You are thinking of JSLint, which is written by Douglas Crockford. JSHint is a derivative project of JSLint. Sorry to burst your bubble. To be fair, it's been undergoing *major* revisions lately, making it resemble JSHint less and less in terms of what it checks for. Having used it in the past, functionality wise it's very different. While it maintains some backwards compatibility, it has added in new checks, doesn't complain about nearly the number of things that JSLint complains about (for good reasons). -- Jeremy Stanley ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Horizon] Licensing issue with using JSHint in build
Thanks! ESLint looks interesting. I'm curious to see what it says about the Horizon source. I'll keep it in mind for future personal projects and the like. Best Regards, Solly Ross - Original Message - From: Martin Geisler mar...@geisler.net To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Sent: Thursday, September 11, 2014 3:20:56 AM Subject: Re: [openstack-dev] [Horizon] Licensing issue with using JSHint in build Solly Ross sr...@redhat.com writes: Hi, I recently began using using ESLint for all my JavaScript linting: http://eslint.org/ It has nice documentation, a normal license, and you can easily write new rules for it. P.S. Here's hoping that the JSHint devs eventually find a way to remove that line from the file -- according to https://github.com/jshint/jshint/issues/1234, not much of the original remains. I don't think it matters how much of the original code remains -- what matters is that any rewrite is a derived work. Otherwise Debian and others could have made the license pure MIT long ago. -- Martin Geisler http://google.com/+MartinGeisler ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [neutron] [infra] Python 2.6 tests can't possibly be passing in neutron
I'm in favor of killing Python 2.6 with fire. Honestly, I think it's hurting code readability and productivity -- You have to constantly think about whether or not some feature that the rest of the universe is already using is supported in Python 2.6 whenever you write code. As for readability, things like 'contextlib.nested' can go away if we can kill Python 2.6 (Python 2.7 supports nested context managers OOTB, in a much more readable way). Best Regards, Solly - Original Message - From: Joshua Harlow harlo...@outlook.com To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Sent: Monday, September 22, 2014 2:33:16 PM Subject: Re: [openstack-dev] [neutron] [infra] Python 2.6 tests can't possibly be passing in neutron Just as an update to what exactly is RHEL python 2.6... This is the expanded source rpm: http://paste.ubuntu.com/8405074/ The main one here appears to be: - python-2.6.6-ordereddict-backport.patch Full changelog @ http://paste.ubuntu.com/8405082/ Overall I'd personally like to get rid of python 2.6, and move on, but then I'd also like to get rid of 2.7 and move on also ;) - Josh On Sep 22, 2014, at 11:17 AM, Monty Taylor mord...@inaugust.com wrote: On 09/22/2014 10:58 AM, Kevin L. Mitchell wrote: On Mon, 2014-09-22 at 10:32 -0700, Armando M. wrote: What about: https://github.com/openstack/neutron/blob/master/test-requirements.txt#L12 Pulling in ordereddict doesn't do anything if your code doesn't use it when OrderedDict isn't in collections, which is the case here. Further, there's no reason that _get_collection_kwargs() needs to use an OrderedDict: it's initialized in an arbitrary order (generator comprehension over a set), then later passed to functions with **, which converts it to a plain old dict. So - as an update to this, this is due to RedHat once again choosing to backport features from 2.7 into a thing they have labeled 2.6. We test 2.6 on Centos6 - which means we get RedHat's patched version of Python2.6 - which, it turns out, isn't really 2.6 - so while you might want to assume that we're testing 2.6 - we're not - we're testing 2.6-as-it-appears-in-RHEL. This brings up a question - in what direction do we care/what's the point in the first place? Some points to ponder: - 2.6 is end of life - so the fact that this is coming up is silly, we should have stopped caring about it in OpenStack 2 years ago at least - Maybe we ACTUALLY only care about 2.6-on-RHEL - since that was the point of supporting it at all - Maybe we ACTUALLY care about 2.6 support across the board, in which case we should STOP testing using Centos6 which is not actually 2.6 I vote for just amending our policy right now and killing 2.6 with prejudice. (also, I have heard a rumor that there are people running in to problems due to the fact that they are deploying onto a two-release-old version of Debian. No offense - but there is no way we're supporting that) ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [requirements] [nova] requirements freeze exception for websockify
The reason it was bounded was because we (the websockify upstream mantainers) made a backwards-incompatible change (for good reasons -- it brought websockify more inline with the Python standard library interfaces). However, OpenStack had subclassed the WebSocketProxy code, and so the change would have broken OpenStack. I did a commit a while ago that made it possible to use Nova with both the newest version and the older versions, but we never bumped the max version for OpenStack, even though we could. Best Regards, Solly - Original Message - From: Doug Hellmann d...@doughellmann.com To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Sent: Friday, September 19, 2014 1:54:08 PM Subject: Re: [openstack-dev] [requirements] [nova] requirements freeze exception for websockify On Sep 19, 2014, at 11:22 AM, Sean Dague s...@dague.net wrote: I'd like to request a requirements freeze exception for websockify - https://review.openstack.org/#/c/122702/ The rationale for this is that websockify version bump fixes a Nova bug about zombie processes - https://bugs.launchpad.net/nova/+bug/1048703. It also sets g-r to the value we've been testing against for the entire last cycle. I don't believe it has any impacts on other projects, so should be a very safe change. Gantt, Ironic, and Nova all use websockify. I’m +1 on updating the minimum based on the fact that our current version spec is causing us to test with this version anyway. However, the proposed change also removes the upper bound. Do we know why that was bounded before? Have we had issues with API changes in that project? Is it safe to remove the cap? Doug -Sean -- Sean Dague http://dague.net ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] usability anti-pattern
Monty, your messages are always super-entertaining to read. They also generally have very good points, which is an added bonus :-P - Original Message - From: Monty Taylor mord...@inaugust.com To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Sent: Friday, September 19, 2014 9:52:46 PM Subject: [openstack-dev] usability anti-pattern Hey, Not to name names, but some of our client libs do this: client.Client(API_VERSION, os_username, ... ) I'm pretty sure they got the idea from python-glanceclient, so I blame Brian Waldon, since he left us for CoreOS. PLEASE STOP DOING THIS - IT CAUSES BABIES TO CRY. MORE. As a developer, I have no way of knowing what to put here. Also, imagine I'm writing a script that wants to talk to more than one cloud to do things - like, say, nodepool for Infra, or an ansible openstack inventory module. NOW WHAT? What do I put??? How do I discover that? Let me make a suggestion... Default it to something. Make it an optional parameter for experts. THEN - when the client lib talks to keystone, check the service catalog for the API version. What's this you say? Sometimes your service doesn't expose a version in the keystone catalog? PLEASE STOP DOING THIS - IT CAUSES DOLPHINS TO WEEP If you have versioned APIs, put the version in keystone. Because otherwise, as as a developer have absolutely zero way to figure it out. Well, except for the algorithm jeblair suggested: just start with 11 and count backwards until a number works This message brought to you by frustrated humans trying to use the cloud. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Reverting recent refactorings of RBD support for config drives
Overall, this looks good. Your alternate implementation misses LVM support, and leaves an unused file behind in the instance directory. Other than that, it's acceptable for a late-stage change, IMO. Best Regards, Solly - Original Message - From: Michael Still mi...@stillhq.com To: OpenStack Development Mailing List openstack-dev@lists.openstack.org Sent: Monday, September 22, 2014 5:07:58 AM Subject: [openstack-dev] Reverting recent refactorings of RBD support for config drives Hi. Today I encountered bug 1369627 [1] as I trolled the status of release critical bugs, which appears to be fall out from the decision to implement adding support for config drives stored in RBD. While I have no problem with that being at thing we do, I'm concerned by the way it was implemented -- the image caching code for libvirt was being used to cache the config drive, and then upload it to ceph as a side effect of the image caching mechanism. I'd prefer we don't to it that way, and given its introduced as security bug, I have proposed the following reverts: - https://review.openstack.org/#/c/123070/ - https://review.openstack.org/#/c/123071/ - https://review.openstack.org/#/c/123072/ Now, because I want to move us forward, I've also proposed an alternate implementation which achieves the same thing without using the caching code: - https://review.openstack.org/#/c/123073/ The new implementation only supports RBD, but that's mostly because its the only image storage backend in the libvirt driver where it makes immediate sense to do this sort of thing. I think this code could do with a refactor, but I was attempting to produce the minimum functional implementation given where we are in the release cycle. Persuant to our revert policy [2], I am asking cores to take a look at these patches as soon as possible. Thanks, Michael 1: https://bugs.launchpad.net/nova/+bug/1369627 2: https://github.com/openstack/nova/blob/master/doc/source/devref/policies.rst -- Rackspace Australia ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [requirements] [nova] requirements freeze exception for websockify
FYI, we roughly do semantic versioning right-shifted by one -- for a release 0.X.Y, incrementing X is supposed to be backwards-incompatible, while incrementing Y is supposed to be backwards-compatible [1]. Thus, you can safely say anything less than 0.7 should work fine for Nova. Best Regards, Solly [1] https://github.com/kanaka/websockify/issues/122 - Original Message - From: Sean Dague s...@dague.net To: openstack-dev@lists.openstack.org Sent: Monday, September 22, 2014 3:22:54 PM Subject: Re: [openstack-dev] [requirements] [nova] requirements freeze exception for websockify On 09/22/2014 02:58 PM, Solly Ross wrote: The reason it was bounded was because we (the websockify upstream mantainers) made a backwards-incompatible change (for good reasons -- it brought websockify more inline with the Python standard library interfaces). However, OpenStack had subclassed the WebSocketProxy code, and so the change would have broken OpenStack. I did a commit a while ago that made it possible to use Nova with both the newest version and the older versions, but we never bumped the max version for OpenStack, even though we could. Actually, the max version is bumped. We're testing with 0.6.0 in the gate because of it. -Sean Best Regards, Solly - Original Message - From: Doug Hellmann d...@doughellmann.com To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Sent: Friday, September 19, 2014 1:54:08 PM Subject: Re: [openstack-dev] [requirements] [nova] requirements freeze exception for websockify On Sep 19, 2014, at 11:22 AM, Sean Dague s...@dague.net wrote: I'd like to request a requirements freeze exception for websockify - https://review.openstack.org/#/c/122702/ The rationale for this is that websockify version bump fixes a Nova bug about zombie processes - https://bugs.launchpad.net/nova/+bug/1048703. It also sets g-r to the value we've been testing against for the entire last cycle. I don't believe it has any impacts on other projects, so should be a very safe change. Gantt, Ironic, and Nova all use websockify. I’m +1 on updating the minimum based on the fact that our current version spec is causing us to test with this version anyway. However, the proposed change also removes the upper bound. Do we know why that was bounded before? Have we had issues with API changes in that project? Is it safe to remove the cap? Doug -Sean -- Sean Dague http://dague.net ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Sean Dague http://dague.net ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] New Websockify Release
Greetings! We (the websockify/noVNC team) have released a new version of websockify (0.6.0). It contains several fixes and features relating to OpenStack (a couple of bugs were fixed, and native support for the `logging` module was added). Unfortunately, to integrate it into OpenStack, a patch is needed to the websocketproxy code in Nova (https://gist.github.com/DirectXMan12/9233369) due to a refactoring of the websockify API. My concern is that the various distos most likely have not had time to update the package in their package repositories. What is the appropriate timescale for updating Nova to work with the new version? Best Regards, Solly Ross ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] New Websockify Release
Whoops! In case anyone was wondering, I wasn't telling myself that I had a good idea :-P. I just clicked reply to the wrong email. The correct email to which this was a reply is below. Good idea -- here's the blueprint: https://blueprints.launchpad.net/nova/+spec/update-to-latest-websockify - Original Message - From: Abhishek Kekane abhishek.kek...@nttdata.com To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Sent: Friday, February 28, 2014 1:00:59 AM Subject: Re: [openstack-dev] [nova] New Websockify Release Hi, Are you going to file new blueprint or log a bug in Launchpad to encounter this change. Thanks, Abhishek -Original Message- From: Nikola Đipanov [mailto:ndipa...@redhat.com] Sent: Thursday, February 27, 2014 4:17 PM To: openstack-dev@lists.openstack.org Subject: Re: [openstack-dev] [nova] New Websockify Release On 02/27/2014 11:22 AM, Thierry Carrez wrote: Solly Ross wrote: We (the websockify/noVNC team) have released a new version of websockify (0.6.0). It contains several fixes and features relating to OpenStack (a couple of bugs were fixed, and native support for the `logging` module was added). Unfortunately, to integrate it into OpenStack, a patch is needed to the websocketproxy code in Nova (https://gist.github.com/DirectXMan12/9233369) due to a refactoring of the websockify API. My concern is that the various distos most likely have not had time to update the package in their package repositories. What is the appropriate timescale for updating Nova to work with the new version? Thanks for reaching out ! I'll let the Nova devs speak, but in that specific case it might make sense to patch the Nova code to support both API versions. That would facilitate the migration to 0.6.0-style code. At some point in the future (when 0.6.0 is everywhere) we could bump the dep to =0.6.0 and remove the compatibility code. Yes - fully agreed with Thierry. I will try to put up a patch for this, but if someone gets there before me - even better :). Thanks, ndipanov ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ Disclaimer:This email and any attachments are sent in strictest confidence for the sole use of the addressee and may contain legally privileged, confidential, and proprietary data. If you are not the intended recipient, please advise the sender by replying promptly to this email and then delete and destroy this email and any attachments without any further use, copying or forwarding ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [neutron][rootwrap] Performance considerations, sudo?
Has anyone tried compiling rootwrap under Cython? Even with non-optimized
libraries,
Cython sometimes sees speedups.
Best Regards,
Solly Ross
- Original Message -
From: Vishvananda Ishaya vishvana...@gmail.com
To: OpenStack Development Mailing List (not for usage questions)
openstack-dev@lists.openstack.org
Sent: Wednesday, March 5, 2014 1:13:33 PM
Subject: Re: [openstack-dev] [neutron][rootwrap] Performance considerations,
sudo?
On Mar 5, 2014, at 6:42 AM, Miguel Angel Ajo majop...@redhat.com wrote:
Hello,
Recently, I found a serious issue about network-nodes startup time,
neutron-rootwrap eats a lot of cpu cycles, much more than the processes it's
wrapping itself.
On a database with 1 public network, 192 private networks, 192 routers,
and 192 nano VMs, with OVS plugin:
Network node setup time (rootwrap): 24 minutes
Network node setup time (sudo): 10 minutes
That's the time since you reboot a network node, until all namespaces
and services are restored.
If you see appendix 1, this extra 14min overhead, matches with the fact
that rootwrap needs 0.3s to start, and launch a system command (once
filtered).
14minutes = 840 s.
(840s. / 192 resources)/0.3s ~= 15 operations / resource(qdhcp+qrouter)
(iptables, ovs port creation tagging, starting child processes, etc..)
The overhead comes from python startup time + rootwrap loading.
I suppose that rootwrap was designed for lower amount of system calls
(nova?).
And, I understand what rootwrap provides, a level of filtering that sudo
cannot offer. But it raises some question:
1) It's actually someone using rootwrap in production?
2) What alternatives can we think about to improve this situation.
0) already being done: coalescing system calls. But I'm unsure that's
enough. (if we coalesce 15 calls to 3 on this system we get: 192*3*0.3/60 ~=3
minutes overhead on a 10min operation).
a) Rewriting rules into sudo (to the extent that it's possible), and live
with that.
b) How secure is neutron about command injection to that point? How much is
user input filtered on the API calls?
c) Even if b is ok , I suppose that if the DB gets compromised, that
could lead to command injection.
d) Re-writing rootwrap into C (it's 600 python LOCs now).
This seems like the best choice to me. It shouldn’t be that much work for a
proficient C coder. Obviously it will need to be audited for buffer overflow
issues etc, but the code should be small enough to make this doable with high
confidence.
Vish
e) Doing the command filtering at neutron-side, as a library and live with
sudo with simple filtering. (we kill the python/rootwrap startup overhead).
3) I also find 10 minutes a long time to setup 192 networks/basic tenant
structures, I wonder if that time could be reduced by conversion
of system process calls into system library calls (I know we don't have
libraries for iproute, iptables?, and many other things... but it's a
problem that's probably worth looking at.)
Best,
Miguel Ángel Ajo.
Appendix:
[1] Analyzing overhead:
[root@rhos4-neutron2 ~]# echo int main() { return 0; } test.c
[root@rhos4-neutron2 ~]# gcc test.c -o test
[root@rhos4-neutron2 ~]# time test # to time process invocation on this
machine
real0m0.000s
user0m0.000s
sys0m0.000s
[root@rhos4-neutron2 ~]# time sudo bash -c 'exit 0'
real0m0.032s
user0m0.010s
sys0m0.019s
[root@rhos4-neutron2 ~]# time python -c'import sys;sys.exit(0)'
real0m0.057s
user0m0.016s
sys0m0.011s
[root@rhos4-neutron2 ~]# time neutron-rootwrap --help
/usr/bin/neutron-rootwrap: No command specified
real0m0.309s
user0m0.128s
sys0m0.037s
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [OpenStack-Infra] Intermittent failures cloning noVNC from github.com/kanaka
@bnemec: I don't think that's been considered. I'm actually one of the upstream maintainers for noVNC. The only concern that I'd have with OpenStack adopting noVNC (there are other maintainers, as well as the author, so I'd have to check with them as well) is that there are a few other projects that use noVNC, so we'd need to make sure that no OpenStack-specific code gets merged into noVNC if we adopt it. Other that that, though, adopting noVNC doesn't sound like a horrible idea. Best Regards, Solly Ross - Original Message - From: Ben Nemec openst...@nemebean.com To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Cc: openstack-in...@lists.openstack.org Sent: Wednesday, March 12, 2014 3:38:19 PM Subject: Re: [openstack-dev] [OpenStack-Infra] Intermittent failures cloning noVNC from github.com/kanaka On 2014-03-11 20:34, Joshua Harlow wrote: https://status.github.com/messages * 'GitHub.com is operating normally, despite an ongoing DDoS attack. The mitigations we have in place are proving effective in protecting us and we're hopeful that we've got this one resolved.' If you were cloning from github.org and not http://git.openstack.org then you were likely seeing some of the DDoS attack in action. Unfortunately I don't think novnc is in git.openstack.org because it's not an OpenStack project. I wonder if we should investigate adopting it (if the author(s) are amenable to that) since we're using the git version of it. Maybe that's already been considered and I just don't know about it. :-) -Ben From: Sukhdev Kapur sukhdevka...@gmail.com Reply-To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Date: Tuesday, March 11, 2014 at 4:08 PM To: Dane Leblanc (leblancd) lebla...@cisco.com Cc: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org , openstack-in...@lists.openstack.org openstack-in...@lists.openstack.org Subject: Re: [openstack-dev] [OpenStack-Infra] Intermittent failures cloning noVNC from github.com/kanaka I have noticed that even clone of devstack has failed few times within last couple of hours - it was running fairly smooth so far. -Sukhdev On Tue, Mar 11, 2014 at 5:05 PM, Sukhdev Kapur sukhdevka...@gmail.com wrote: [adding openstack-dev list as well ] I have noticed that this has stated hitting my builds within last few hours. I have noticed exact same failures on almost 10 builds. Looks like something has happened within last few hours - perhaps the load? -Sukhdev On Tue, Mar 11, 2014 at 4:28 PM, Dane Leblanc (leblancd) lebla...@cisco.com wrote: Apologies if this is the wrong audience for this question... I'm seeing intermittent failures running stack.sh whereby 'git clone https://github.com/kanaka/noVNC.git /opt/stack/noVNC' is returning various errors. Below are 2 examples. Is this a known issue? Are there any localrc settings which might help here? Example 1: 2014-03-11 15:00:33.779 | + is_service_enabled n-novnc 2014-03-11 15:00:33.780 | + return 0 2014-03-11 15:00:33.781 | ++ trueorfalse False 2014-03-11 15:00:33.782 | + NOVNC_FROM_PACKAGE=False 2014-03-11 15:00:33.783 | + '[' False = True ']' 2014-03-11 15:00:33.784 | + NOVNC_WEB_DIR=/opt/stack/noVNC 2014-03-11 15:00:33.785 | + git_clone https://github.com/kanaka/noVNC.git /opt/stack/noVNC master 2014-03-11 15:00:33.786 | + GIT_REMOTE= https://github.com/kanaka/noVNC.git 2014-03-11 15:00:33.788 | + GIT_DEST=/opt/stack/noVNC 2014-03-11 15:00:33.789 | + GIT_REF=master 2014-03-11 15:00:33.790 | ++ trueorfalse False False 2014-03-11 15:00:33.791 | + RECLONE=False 2014-03-11 15:00:33.792 | + [[ False = \T\r\u\e ]] 2014-03-11 15:00:33.793 | + echo master 2014-03-11 15:00:33.794 | + egrep -q '^refs' 2014-03-11 15:00:33.795 | + [[ ! -d /opt/stack/noVNC ]] 2014-03-11 15:00:33.796 | + [[ False = \T\r\u\e ]] 2014-03-11 15:00:33.797 | + git_timed clone https://github.com/kanaka/noVNC.git /opt/stack/noVNC 2014-03-11 15:00:33.798 | + local count=0 2014-03-11 15:00:33.799 | + local timeout=0 2014-03-11 15:00:33.801 | + [[ -n 0 ]] 2014-03-11 15:00:33.802 | + timeout=0 2014-03-11 15:00:33.803 | + timeout -s SIGINT 0 git clone https://github.com/kanaka/noVNC.git /opt/stack/noVNC 2014-03-11 15:00:33.804 | Cloning into '/opt/stack/noVNC'... 2014-03-11 15:03:13.694 | error: RPC failed; result=56, HTTP code = 200 2014-03-11 15:03:13.695 | fatal: The remote end hung up unexpectedly 2014-03-11 15:03:13.697 | fatal: early EOF 2014-03-11 15:03:13.698 | fatal: index-pack failed 2014-03-11 15:03:13.699 | + [[ 128 -ne 124 ]] 2014-03-11 15:03:13.700 | + die 596 'git call failed: [git clone' https://github.com/kanaka/noVNC.git '/opt/stack/noVNC]' 2014-03-11 15:03:13.701 | + local exitcode=0 2014-03-11 15:03:13.702 | [Call Trace] 2014-03-11 15:03:13.703
Re: [openstack-dev] any recommendations for live debugging of openstack services?
Well, for a non-interactive view of things, you can use the openstack.common.report functionality. It's currently integrated into Nova, and I believe that the other projects are working to get it integrated as well. To use it, you just send a SIGUSR1 to any Nova process, and a report of the current stack traces of threads and green threads, as well as the current configuration options, will be dumped. It doesn't look like exactly what you want, but I figured it might be useful to you anyway. Best Regards, Solly Ross - Original Message - From: Chris Friesen chris.frie...@windriver.com To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Sent: Wednesday, March 12, 2014 12:47:32 PM Subject: [openstack-dev] any recommendations for live debugging of openstack services? Are there any tools that people can recommend for live debugging of openstack services? I'm looking for a mechanism where I could take a running system that isn't behaving the way I expect and somehow poke around inside the program while it keeps running. (Sort of like tracepoints in gdb.) I've seen mention of things like twisted.manhole and eventlet.backdoor...has anyone used this sort of thing with openstack? Are there better options? Also, has anyone ever seen an implementation of watchpoints for python? By that I mean the ability to set a breakpoint if the value of a variable changes. I found https://sourceforge.net/blog/watchpoints-in-python/; but it looks pretty hacky. Thanks, Chris ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [OpenStack-Infra] Intermittent failures cloning noVNC from github.com/kanaka
@Monty: having a packaging system sounds like a good idea. Send us a pull
request on github.com/kanaka/noVNC.
Best Regards,
Solly Ross
- Original Message -
From: Monty Taylor mord...@inaugust.com
To: Sean Dague s...@dague.net, OpenStack Development Mailing List (not for
usage questions) openstack-dev@lists.openstack.org, openst...@nemebean.com
Cc: openstack-in...@lists.openstack.org
Sent: Thursday, March 13, 2014 12:09:01 PM
Subject: Re: [openstack-dev] [OpenStack-Infra] Intermittent failures cloning
noVNC from github.com/kanaka
I agree.
Solly - in addition to potentially 'adopting' noVNC - or as a parallel
train of thought ...
As we started working on storyboard in infra, we've started using the
bower tool for html/javascript packaging - and we have some ability to
cache the output of that pretty easily. Would you accept patches to
noVNC to add bower config things and/or publication of tarballs of
releases via it? Since noVNC isn't likely to be participating in the
integrated gate in either case, we could potentially split the question
of how do we get copies of it in a way that doesn't depend on OS
distros (which is why we use pip for our python depends) and does
noVNC want to have its git repo exist in OpenStack Infra systems.
Monty
On 03/13/2014 07:44 AM, Sean Dague wrote:
I think a bigger question is why are we using a git version of something
outside of OpenStack.
Where is a noNVC release we can point to and use?
In Juno I'd really be pro removing all the devstack references to git
repos not on git.openstack.org, because these kinds of failures have
real impact.
Currently we have 4 repositories that fit this bill:
SWIFT3_REPO=${SWIFT3_REPO:-http://github.com/fujita/swift3.git}
NOVNC_REPO=${NOVNC_REPO:-https://github.com/kanaka/noVNC.git}
RYU_REPO=${RYU_REPO:-https://github.com/osrg/ryu.git}
SPICE_REPO=${SPICE_REPO:-http://anongit.freedesktop.org/git/spice/spice-html5.git}
I think all of these probably need to be removed from devstack. We
should be using release versions (preferably in distros, though allowed
to be in language specific package manager).
-Sean
On 03/13/2014 10:26 AM, Solly Ross wrote:
@bnemec: I don't think that's been considered. I'm actually one of the
upstream maintainers for noVNC. The only concern that I'd have with
OpenStack adopting noVNC (there are other maintainers, as well as the
author, so I'd have to check with them as well) is that there are a few
other projects that use noVNC, so we'd need to make sure that no
OpenStack-specific code gets merged into noVNC if we adopt it. Other that
that, though, adopting noVNC doesn't sound like a horrible idea.
Best Regards,
Solly Ross
- Original Message -
From: Ben Nemec openst...@nemebean.com
To: OpenStack Development Mailing List (not for usage questions)
openstack-dev@lists.openstack.org
Cc: openstack-in...@lists.openstack.org
Sent: Wednesday, March 12, 2014 3:38:19 PM
Subject: Re: [openstack-dev] [OpenStack-Infra] Intermittent failures cloning
noVNC from github.com/kanaka
On 2014-03-11 20:34, Joshua Harlow wrote:
https://status.github.com/messages
* 'GitHub.com is operating normally, despite an ongoing DDoS attack. The
mitigations we have in place are proving effective in protecting us and
we're hopeful that we've got this one resolved.'
If you were cloning from github.org and not http://git.openstack.org then
you were likely seeing some of the DDoS attack in action.
Unfortunately I don't think novnc is in git.openstack.org because it's not
an OpenStack project. I wonder if we should investigate adopting it (if the
author(s) are amenable to that) since we're using the git version of it.
Maybe that's already been considered and I just don't know about it. :-)
-Ben
From: Sukhdev Kapur sukhdevka...@gmail.com
Reply-To: OpenStack Development Mailing List (not for usage questions)
openstack-dev@lists.openstack.org
Date: Tuesday, March 11, 2014 at 4:08 PM
To: Dane Leblanc (leblancd) lebla...@cisco.com
Cc: OpenStack Development Mailing List (not for usage questions)
openstack-dev@lists.openstack.org , openstack-in...@lists.openstack.org
openstack-in...@lists.openstack.org
Subject: Re: [openstack-dev] [OpenStack-Infra] Intermittent failures cloning
noVNC from github.com/kanaka
I have noticed that even clone of devstack has failed few times within last
couple of hours - it was running fairly smooth so far.
-Sukhdev
On Tue, Mar 11, 2014 at 5:05 PM, Sukhdev Kapur sukhdevka...@gmail.com
wrote:
[adding openstack-dev list as well ]
I have noticed that this has stated hitting my builds within last few hours.
I have noticed exact same failures on almost 10 builds.
Looks like something has happened within last few hours - perhaps the load?
-Sukhdev
On Tue, Mar 11, 2014 at 4:28 PM, Dane Leblanc (leblancd)
lebla...@cisco.com wrote:
Apologies if this is the wrong audience
Re: [openstack-dev] [Oslo] Improving oslo-incubator update.py
It would also be great if there was a way to only sync one package. When adding a new library to a project (e.g. openstack.common.report to Nova), one would want to only sync the openstack.common.report parts, and not the any changes from the rest of openstack.common. My process has been 1. Edit openstack-common.conf to only contain the packages I want 2. Run the update 3. Make sure there wasn't code that didn't get changed from 'openstack.common.xyz' to 'nova.openstack.common.xyz' (hint: this happens some times) 4. git checkout openstack-common.conf to revert the changes to openstack-common.conf IMHO, update.py needs a bit of work (well, I think the whole code copying thing needs a bit of work, but that's a different story). Best Regards, Solly Ross - Original Message - From: Jay S Bryant jsbry...@us.ibm.com To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Sent: Friday, March 14, 2014 3:36:49 PM Subject: Re: [openstack-dev] [Oslo] Improving oslo-incubator update.py From: Brant Knudson b...@acm.org To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org, Date: 03/14/2014 02:21 PM Subject: Re: [openstack-dev] [Oslo] Improving oslo-incubator update.py On Fri, Mar 14, 2014 at 2:05 PM, Jay S Bryant jsbry...@us.ibm.com wrote: It would be great if we could get the process for this automated. In the mean time, those of us doing the syncs will just have to slog through the process. Jay ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev What's the process? How do I generate the list of changes? Brant ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev Brant, My process thus far has been the following: 1. Do the sync to see what files are changed. 2. Take a look at the last commit sync'd to what is currently in master for a file. 3. Document all the commits that have come in on that file since. 4. Repeat process for all the relevant files if there is more than one. 5. If are multiples files I organize the commits with a list of the files touched by that commit. 6. Document the master level of Oslo when the sync was done for reference. Process may not be perfect, but it gets the job done. Here is an example of the format I use: https://review.openstack.org/#/c/75740/ Jay ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Duplicate code for processing REST APIs
You're far from the only one who feels this way. It should be noted, however, that it would appear that the Oslo team is trying to graduate some more of the modules into separate libraries. I talked to someone about the process, and they said that the Oslo team is trying to graduate the lowest libraries on the dependency chart first, so that any graduated libraries will have olso.xyz dependencies, and not code-sync dependencies. Best Regards, Solly Ross - Original Message - From: Jay S Bryant jsbry...@us.ibm.com To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Sent: Friday, March 14, 2014 3:12:22 PM Subject: Re: [openstack-dev] Duplicate code for processing REST APIs From: Duncan Thomas duncan.tho...@gmail.com To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org, Date: 03/14/2014 11:56 AM Subject: Re: [openstack-dev] Duplicate code for processing REST APIs On 13 March 2014 21:13, Roman Podoliaka rpodoly...@mirantis.com wrote: Hi Steven, Code from openstack/common/ dir is 'synced' from oslo-incubator. The 'sync' is effectively a copy of oslo-incubator subtree into a project source tree. As syncs are not done at the same time, the code of synced modules may indeed by different for each project depending on which commit of oslo-incubator was synced. Worth noting that there have been a few cases of projects patching OSLO bugs intheir own tree rather than fixing in OSLO then resyncing. If anybody has any tooling that can detect that, I'd love to see the results. I'm generally of the opinion that cinder is likely to be resistant to more parts of OSLO being used in cinder unless they are a proper library - syncs have caused us significant pain, code churn, review load and bugs in the last 12 months. I am but one voice among many, but I know I'm not the only member of core who feels this to be the case. Hopefully I can spend some time with OSLO core at the summit and discuss the problems I've found. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev Duncan, I will come with you for that discussion. :-) Have some thoughts and questions to share as well. Regardless, I think we need to make sure to actually get our Oslo syncs started for Cinder early in Juno. We are way behind on db and db.sqlalchemy. Planning to propose changes to that as soon as we switch over. Jay ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Oslo] Improving oslo-incubator update.py
--nodeps will only sync the modules specified on the command line: https://wiki.openstack.org/wiki/Oslo#Syncing_Code_from_Incubator Heh, whoops. Must have missed that. Is it in the README/info at the top of the update.py script? That said, it's not always safe to do that. You might sync a change in one module that depends on a change in another module and end up breaking something. It might not be caught in the sync either because the Oslo unit tests don't get synced across. Hmm... I suppose this is why we have libraries with dependencies (not meant to sound snarky). Although in the case of updating a library that you wrote, it's less likely to break things. Best Regards, Solly Ross - Original Message - From: Ben Nemec openst...@nemebean.com To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Cc: Solly Ross sr...@redhat.com Sent: Friday, March 14, 2014 4:36:03 PM Subject: Re: [openstack-dev] [Oslo] Improving oslo-incubator update.py On 2014-03-14 14:49, Solly Ross wrote: It would also be great if there was a way to only sync one package. There is. :-) --nodeps will only sync the modules specified on the command line: https://wiki.openstack.org/wiki/Oslo#Syncing_Code_from_Incubator That said, it's not always safe to do that. You might sync a change in one module that depends on a change in another module and end up breaking something. It might not be caught in the sync either because the Oslo unit tests don't get synced across. When adding a new library to a project (e.g. openstack.common.report to Nova), one would want to only sync the openstack.common.report parts, and not the any changes from the rest of openstack.common. My process has been 1. Edit openstack-common.conf to only contain the packages I want 2. Run the update 3. Make sure there wasn't code that didn't get changed from 'openstack.common.xyz' to 'nova.openstack.common.xyz' (hint: this happens some times) 4. git checkout openstack-common.conf to revert the changes to openstack-common.conf IMHO, update.py needs a bit of work (well, I think the whole code copying thing needs a bit of work, but that's a different story). Best Regards, Solly Ross - Original Message - From: Jay S Bryant jsbry...@us.ibm.com To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Sent: Friday, March 14, 2014 3:36:49 PM Subject: Re: [openstack-dev] [Oslo] Improving oslo-incubator update.py From: Brant Knudson b...@acm.org To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org, Date: 03/14/2014 02:21 PM Subject: Re: [openstack-dev] [Oslo] Improving oslo-incubator update.py On Fri, Mar 14, 2014 at 2:05 PM, Jay S Bryant jsbry...@us.ibm.com wrote: It would be great if we could get the process for this automated. In the mean time, those of us doing the syncs will just have to slog through the process. Jay ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev What's the process? How do I generate the list of changes? Brant ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev Brant, My process thus far has been the following: 1. Do the sync to see what files are changed. 2. Take a look at the last commit sync'd to what is currently in master for a file. 3. Document all the commits that have come in on that file since. 4. Repeat process for all the relevant files if there is more than one. 5. If are multiples files I organize the commits with a list of the files touched by that commit. 6. Document the master level of Oslo when the sync was done for reference. Process may not be perfect, but it gets the job done. Here is an example of the format I use: https://review.openstack.org/#/c/75740/ Jay ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] devstack: Unable to restart rabbitmq-server
I've also had devstack somehow end up starting qpid, then try to start rabbit on F20. In this case it seems sufficient to stop qpid then re-run devstack. I haven't had time to track down the issue yet. Best Regards, Solly Ross - Original Message - From: John Eckersberg jecke...@redhat.com To: Deepak C Shetty deepa...@redhat.com, openstack-dev@lists.openstack.org Sent: Monday, March 17, 2014 9:19:03 AM Subject: Re: [openstack-dev] devstack: Unable to restart rabbitmq-server Deepak C Shetty deepa...@redhat.com writes: Hi List, It been few hours and I tried everything from ensuring /etc/hosts, /etc/hostname etc (per google results) and rabbitmq-server still doesn't start. I am using latest devstack as of today on F20 There are a couple of known bugs that can prevent rabbitmq-server from starting on F20. First one (same bug, two BZs) is related to SELinux and port probing: https://bugzilla.redhat.com/show_bug.cgi?id=1032595#c8 https://bugzilla.redhat.com/show_bug.cgi?id=998682 Second one is a race condition in Erlang. If you are repeatedly unable to start rabbitmq-server, it's probably not this bug: https://bugzilla.redhat.com/show_bug.cgi?id=1059913 I have a patched rabbitmq-server package which includes the fixes for these two issues, if you'd like to try it and see if it helps your issue. And if it helps, please comment on the bug(s) to encourage the maintainer to pull them into the package :) http://jeckersb.fedorapeople.org/rabbitmq-server-3.1.5-3.fc20.noarch.rpm Hope that helps, John ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Updating libvirt in gate jobs
I agree with Dan -- I think it's important to test on newer versions as well, considering we will have people running on other versions besides Ubuntu LTS -- Fedora 20, for instance, is on 1.1.3.4. Additionally, considering bugs get fixed and features get implemented in each version of libvirt, we need to ensure that we *can* test code that uses features present in later versions of libvirt. 0.9.8 came out over two years ago making it fairly old. I think it's important to keep up-to-date on what versions we test with. Best Regards, Solly Ross - Original Message - From: Daniel P. Berrange berra...@redhat.com To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Sent: Tuesday, March 18, 2014 10:11:54 AM Subject: Re: [openstack-dev] Updating libvirt in gate jobs On Tue, Mar 18, 2014 at 07:50:15AM -0400, Davanum Srinivas wrote: Hi Team, We have 2 choices 1) Upgrade to libvirt 0.9.8+ (See [1] for details) 2) Enable UCA and upgrade to libvirt 1.2.2+ (see [2] for details) For #1, we received a patched deb from @SergeHallyn/@JamesPage and ran tests on it in review https://review.openstack.org/#/c/79816/ For #2, @SergeHallyn/@JamesPage have updated UCA (precise-proposed/icehouse) repo and we ran tests on it in review https://review.openstack.org/#/c/74889/ For IceHouse, my recommendation is to request Ubuntu folks to push the patched 0.9.8+ version we validated to public repos, then we can can install/run gate jobs with that version. This is probably the smallest risk of the 2 choices. If we've re-run the tests in that review enough times to be confident we've had a chance of exercising the race conditions, then using the patched 0.9.8 seems like a no-brainer. We know the current version in ubuntu repos is broken for us, so the sooner we address that the better. As soon as Juno begins, we can switch 1.2.2+ on UCA and request Ubuntu folks to push the verified version where we can use it. This basically re-raises the question of /what/ we should be testing in the gate, which was discussed on this list a few weeks ago, and I'm not clear that there was a definite decision in that thread http://lists.openstack.org/pipermail/openstack-dev/2014-February/027734.html Testing the lowest vs highest is targetting two different scenarios - Testing the lowest version demonstrates that OpenStack has not broken its own code by introducing use of a new feature. - Testing the highest version demonstrates that OpenStack has not been broken by 3rd party code introducing a regression. I think it is in scope for openstack to be targetting both of these scenarios. For anything in-between though, it is upto the downstream vendors to test their precise combination of versions. Currently though our testing policy for non-python bits is whatever version ubuntu ships, which may be neither the lowest or highest versions, just some arbitrary version they wish to support. So this discussion is currently more of a 'what ubuntu version should we test on' kind of decision Regards, Daniel -- |: http://berrange.com -o-http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :| ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Constructive Conversations
Since I joined the OpenStack community back in May, this is something I've though about. I think there's something to be said for thought-to-text being a lossy encoding algorithm (at least the way most people use it). I've encountered cases where it appeared the person on the other end was angry or belittling, where in fact they were just surprised (or perhaps were slightly exasperated). Sometimes it's seemed like I've had reviews where the first comment feels like OMG THIS IS THE WORST THING IMAGINABLE YOU SHOULD GO DIE IN A FIRE and then I make a change and the next comment is you're an awesome person and should feel proud of yourself. To a certain extent, things like emoji can help to rectify situations like these (that's actually why people started using them back in the BBS days), but part of it is simply knowing that certain people talk a certain way. Could some people be more conscious of how their words could be interpreted? Certainly, but it's difficult to know how one's words will be interpreted. Are there cases of people actually intending to be a dick? Probably, but I feel like these cases are dwarfed by the aforementioned cases. Just my 0.3 BTC. Best Regards, Solly Ross - Original Message - From: Chris Behrens cbehr...@codestud.com To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Sent: Tuesday, March 18, 2014 3:25:08 PM Subject: Re: [openstack-dev] Constructive Conversations On Mar 18, 2014, at 11:57 AM, Matt Riedemann mrie...@linux.vnet.ibm.com wrote: […] Not to detract from what you're saying, but this is 'meh' to me. My company has some different kind of values thing every 6 months it seems and maybe it's just me but I never really pay attention to any of it. I think I have to put something on my annual goals/results about it, but it's just fluffy wording. To me this is a self-policing community, if someone is being a dick, the others should call them on it, or the PTL for the project should stand up against it and set the tone for the community and culture his project wants to have. That's been my experience at least. Maybe some people would find codifying this helpful, but there are already lots of wikis and things that people can't remember on a daily basis so adding another isn't probably going to help the problem. Bully's don't tend to care about codes, but if people stand up against them in public they should be outcast. I agree with the goals and sentiment of Kurt’s message. But, just to add a little to Matt’s reply: Let’s face it. Everyone has a bad day now and then. It’s easier for some people to lose their cool over others. Nothing’s going to change that. - Chris ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Backwards incompatible API changes
I concur. I suspect people/organizations who are doing CD *probably* won't mind such a change as much as the people who use the versioned releases will mind backwards-incompatibility. Correct me if I'm wrong, but doing CD requires a certain willingness to roll with the punches, so to speak, whereas people using versioned releases are less likely to be as flexible. Best Regards, Solly Ross - Original Message - From: Thierry Carrez thie...@openstack.org To: openstack-dev@lists.openstack.org Sent: Thursday, March 20, 2014 6:51:26 AM Subject: Re: [openstack-dev] [nova] Backwards incompatible API changes Christopher Yeoh wrote: The patch was merged in October (just after Icehouse opened) and so has been used in clouds that do CD for quite a while. After some discussion on IRC I think we'll end up having to leave this backwards incompatible change in there - given there are most likely users who now rely on both sets of behaviour there is no good way to get out of this situation. I've added a note to the Icehouse release notes. I still think reverting before release is an option we should consider. My point is, yes we broke it back in October for people doing CD (and they might by now have gotten used to it), if we let this to release we'll then break it for everyone else. We put a high emphasis into guaranteeing backward compatibility between releases. I think there would be more damage done if we let this sail to release, compared to the damage of reverting CD followers to pre-October behavior... -- Thierry Carrez (ttx) ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Updates to Juno blueprint review process
IMHO, I feel like many of the things in the Implementation section should just be in the launchpad BP, and not in the git-tracked spec. While I think that the idea of having the BP in Gerrit is a great idea, I feel like the details of the Implementation section (assignee, etc) will lead to trivial changes that will be a burden on the people who need to approve changes and will distract from substantial changes (changes to the design parts of the blueprint, etc). Best Regards, Solly Ross - Original Message - From: Russell Bryant rbry...@redhat.com To: OpenStack Development Mailing List openstack-dev@lists.openstack.org Sent: Thursday, March 20, 2014 11:49:37 AM Subject: [openstack-dev] [Nova] Updates to Juno blueprint review process We recently discussed the idea of using gerrit to review blueprint specifications [1]. There was a lot of support for the idea so we have proceeded with putting this together before the start of the Juno development cycle. We now have a new project set up, openstack/nova-specs. You submit changes to it just like any other project in gerrit. Find the README and a template for specifications here: http://git.openstack.org/cgit/openstack/nova-specs/tree/README.rst http://git.openstack.org/cgit/openstack/nova-specs/tree/template.rst The blueprint process wiki page has also been updated to reflect that we will be using this for Nova: https://wiki.openstack.org/wiki/Blueprints#Nova Note that *all* Juno blueprints, including ones that were previously approved, must go through this new process. This will help ensure that blueprints previously approved still make sense, as well as ensure that all Juno specs follow a more complete and consistent format. Before the flood of spec reviews start, we would really like to get feedback on the content of the spec template. It includes things like deployer impact which could use more input. Feel free to provide feedback on list, or just suggest updates via proposed changes in gerrit. I suspect this process to evolve a bit throughout Juno, but I'm very excited about the positive impact it is likely to have on our overall result. Thanks! [1] http://lists.openstack.org/pipermail/openstack-dev/2014-March/029232.html -- Russell Bryant ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] instances stuck with task_state of REBOOTING
Hi Chris, Are you in the position to determine whether or not this happens with the latest master code? Either way, it definitely looks like a bug. If you could give more specific reproduction instructions, that would be most useful. Best Regards, Solly Ross - Original Message - From: Chris Friesen chris.frie...@windriver.com To: openstack-dev@lists.openstack.org Sent: Thursday, March 20, 2014 1:59:39 PM Subject: [openstack-dev] [nova] instances stuck with task_state of REBOOTING I'm running a havana install, and during some testing I've managed to get the system into a state where two instances are up and running but are reporting a task_state of REBOOTING. I can see the nova-api logs showing the soft-reboot request. I don't see a corresponding nova-compute log indicating a successful reboot or an error. I see a subsequent nova-api log for a requested soft-reboot that gets a 409 response, presumably because it thinks it's still rebooting. Has anyone seen something like this before? I'm a bit surprised that there isn't an audit that cleans this up. They've been sitting like this for hours. Does that count as a bug? Presumably a power state of RUNNING should be enough to clear the task_state. Chris ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Backwards incompatible API changes
Sorry if my meaning was unclear. I think we should revert as well. Best Regards, Solly Ross - Original Message - From: David Kranz dkr...@redhat.com To: openstack-dev@lists.openstack.org Sent: Thursday, March 20, 2014 2:20:42 PM Subject: Re: [openstack-dev] [nova] Backwards incompatible API changes On 03/20/2014 11:05 AM, Solly Ross wrote: I concur. I suspect people/organizations who are doing CD *probably* won't mind such a change as much as the people who use the versioned releases will mind backwards-incompatibility. Correct me if I'm wrong, but doing CD requires a certain willingness to roll with the punches, so to speak, whereas people using versioned releases are less likely to be as flexible. Best Regards, Solly Ross It looks like there was no existing unit test for what horizon ended up doing, where a red-flagging change to the test would have been needed. There is obviously also no tempest test. But I hope that folks doing CD would not roll with this sort of punch if they find it, but push back immediately to revert the change unless it had gone through whatever api change review process we come up with. I presume that it simply was not noticed since it is perhaps a bit of an obscure api point. Since OpenStack currently advertises 6 month cycle releases and stable apis, it would be best to revert it IMO. -David - Original Message - From: Thierry Carrez thie...@openstack.org To: openstack-dev@lists.openstack.org Sent: Thursday, March 20, 2014 6:51:26 AM Subject: Re: [openstack-dev] [nova] Backwards incompatible API changes Christopher Yeoh wrote: The patch was merged in October (just after Icehouse opened) and so has been used in clouds that do CD for quite a while. After some discussion on IRC I think we'll end up having to leave this backwards incompatible change in there - given there are most likely users who now rely on both sets of behaviour there is no good way to get out of this situation. I've added a note to the Icehouse release notes. I still think reverting before release is an option we should consider. My point is, yes we broke it back in October for people doing CD (and they might by now have gotten used to it), if we let this to release we'll then break it for everyone else. We put a high emphasis into guaranteeing backward compatibility between releases. I think there would be more damage done if we let this sail to release, compared to the damage of reverting CD followers to pre-October behavior... ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] instances stuck with task_state of REBOOTING
Well, if messages are getting dropped on the floor due to communication issues, that's not a good thing. If you have time, could you determine why the messages are getting dropped on the floor? We shouldn't be doing things that require both the controller and compute nodes until we have a connection. Best Regards, Solly Ross - Original Message - From: Chris Friesen chris.frie...@windriver.com To: openstack-dev@lists.openstack.org Sent: Thursday, March 20, 2014 2:59:55 PM Subject: Re: [openstack-dev] [nova] instances stuck with task_state of REBOOTING On 03/20/2014 12:29 PM, Chris Friesen wrote: The fact that there are no success or error logs in nova-compute.log makes me wonder if we somehow got stuck in self.driver.reboot(). Also, I'm kind of wondering what would happen if nova-compute was running reboot_instance() and we rebooted the controller at the same time. reboot_instance() could time out trying to update the instance with the the new power state and a task_state of None. Later on in _sync_power_states() we would update the power_state, but nothing would update the task_state. I don't think this is what happened to us though since I'd expect to see logs of the timeout. Actually, looking at the logs a bit more carefully it appears that what happened is something like this: We reboot the controllers. Right after they come back up something calls compute.api.API.reboot() That sets instance.task_state = task_states.REBOOTING and then calls instance.save() to update the database. Then it calls self.compute_rpcapi.reboot_instance() which does an rpc cast. That message gets dropped on the floor due to communication issues between the controller and the compute. Now we're stuck with a task_state of REBOOTING. I think that both of the RPC message loss scenarios are valid with current nova code, so we really do need an audit to clean up after this sort of thing. Chris ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [depfreeze] Exception: limit psutil to 2.0.0
Hi, I currently have a patch up for review (https://review.openstack.org/#/c/81373/) to limit psutil be 2.0.0. 2.0.0 just came out a couple weeks ago, and breaks the API in a major way. Until we can port our code to the latest version, I suggest we limit the version of psutil to 1.x (currently there's a lower bound in the 1.x range, just not an upper bound). Best Regards, Solly Ross ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [depfreeze] Exception: limit psutil to 2.0.0
Code which breaks: Glance's mutliprocessing tests will break (the reason we should limit it now). For the future, people attempting to use psutil will have no clear version target (Either they use 1.x and break with the people who install the latest version from pip, of they use 2.0.0 and break with everything that doesn't use the latest version). psutil's API is extremely unstable -- it has undergone major revisions going from 0.x to 1.x, and now 1.x to 2.0.0. Limiting psutil explicitly to a single major version (it was more or less implicitly limited before, since there was no version major version above 1) ensures that the requirements.txt file actually indicates what is necessary to use OpenStack. The alternative option would be to update the glance tests, but my concern is that 2.0.0 is not available from the package managers of most distros yet. Best Regards, Solly Ross - Original Message - From: Sean Dague s...@dague.net To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Sent: Wednesday, March 26, 2014 10:39:41 AM Subject: Re: [openstack-dev] [depfreeze] Exception: limit psutil to 2.0.0 On 03/26/2014 10:30 AM, Solly Ross wrote: Hi, I currently have a patch up for review (https://review.openstack.org/#/c/81373/) to limit psutil be 2.0.0. 2.0.0 just came out a couple weeks ago, and breaks the API in a major way. Until we can port our code to the latest version, I suggest we limit the version of psutil to 1.x (currently there's a lower bound in the 1.x range, just not an upper bound). Which code will be broken by this if it's not done? Is there an RC bug tracking it? -Sean -- Sean Dague Samsung Research America s...@dague.net / sean.da...@samsung.com http://dague.net ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [depfreeze] Exception: limit psutil to 2.0.0
What bug tracker should I file under? I tried filing one under the openstack common infrastructure tracker, but was told that it wasn't the correct place to file such a bug. Best Regards, Solly Ross - Original Message - From: Sean Dague s...@dague.net To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Sent: Wednesday, March 26, 2014 3:28:32 PM Subject: Re: [openstack-dev] [depfreeze] Exception: limit psutil to 2.0.0 Where is the RC bug tracking this? If it's that bad, we really need to be explicit about this with a critical bug at this stage of the release. -Sean On 03/26/2014 01:14 PM, Solly Ross wrote: Code which breaks: Glance's mutliprocessing tests will break (the reason we should limit it now). For the future, people attempting to use psutil will have no clear version target (Either they use 1.x and break with the people who install the latest version from pip, of they use 2.0.0 and break with everything that doesn't use the latest version). psutil's API is extremely unstable -- it has undergone major revisions going from 0.x to 1.x, and now 1.x to 2.0.0. Limiting psutil explicitly to a single major version (it was more or less implicitly limited before, since there was no version major version above 1) ensures that the requirements.txt file actually indicates what is necessary to use OpenStack. The alternative option would be to update the glance tests, but my concern is that 2.0.0 is not available from the package managers of most distros yet. Best Regards, Solly Ross - Original Message - From: Sean Dague s...@dague.net To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Sent: Wednesday, March 26, 2014 10:39:41 AM Subject: Re: [openstack-dev] [depfreeze] Exception: limit psutil to 2.0.0 On 03/26/2014 10:30 AM, Solly Ross wrote: Hi, I currently have a patch up for review (https://review.openstack.org/#/c/81373/) to limit psutil be 2.0.0. 2.0.0 just came out a couple weeks ago, and breaks the API in a major way. Until we can port our code to the latest version, I suggest we limit the version of psutil to 1.x (currently there's a lower bound in the 1.x range, just not an upper bound). Which code will be broken by this if it's not done? Is there an RC bug tracking it? -Sean -- Sean Dague Samsung Research America s...@dague.net / sean.da...@samsung.com http://dague.net ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [depfreeze] Exception: limit psutil to 2.0.0
Here's the bug for Glance: https://bugs.launchpad.net/glance/+bug/1298039 - Original Message - From: Clark Boylan clark.boy...@gmail.com To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Sent: Wednesday, March 26, 2014 4:04:12 PM Subject: Re: [openstack-dev] [depfreeze] Exception: limit psutil to 2.0.0 On Wed, Mar 26, 2014 at 12:32 PM, Solly Ross sr...@redhat.com wrote: What bug tracker should I file under? I tried filing one under the openstack common infrastructure tracker, but was told that it wasn't the correct place to file such a bug. Best Regards, Solly Ross - Original Message - From: Sean Dague s...@dague.net To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Sent: Wednesday, March 26, 2014 3:28:32 PM Subject: Re: [openstack-dev] [depfreeze] Exception: limit psutil to 2.0.0 Where is the RC bug tracking this? If it's that bad, we really need to be explicit about this with a critical bug at this stage of the release. -Sean On 03/26/2014 01:14 PM, Solly Ross wrote: Code which breaks: Glance's mutliprocessing tests will break (the reason we should limit it now). For the future, people attempting to use psutil will have no clear version target (Either they use 1.x and break with the people who install the latest version from pip, of they use 2.0.0 and break with everything that doesn't use the latest version). psutil's API is extremely unstable -- it has undergone major revisions going from 0.x to 1.x, and now 1.x to 2.0.0. Limiting psutil explicitly to a single major version (it was more or less implicitly limited before, since there was no version major version above 1) ensures that the requirements.txt file actually indicates what is necessary to use OpenStack. The alternative option would be to update the glance tests, but my concern is that 2.0.0 is not available from the package managers of most distros yet. Best Regards, Solly Ross - Original Message - From: Sean Dague s...@dague.net To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Sent: Wednesday, March 26, 2014 10:39:41 AM Subject: Re: [openstack-dev] [depfreeze] Exception: limit psutil to 2.0.0 On 03/26/2014 10:30 AM, Solly Ross wrote: Hi, I currently have a patch up for review (https://review.openstack.org/#/c/81373/) to limit psutil be 2.0.0. 2.0.0 just came out a couple weeks ago, and breaks the API in a major way. Until we can port our code to the latest version, I suggest we limit the version of psutil to 1.x (currently there's a lower bound in the 1.x range, just not an upper bound). Which code will be broken by this if it's not done? Is there an RC bug tracking it? -Sean -- Sean Dague Samsung Research America s...@dague.net / sean.da...@samsung.com http://dague.net ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev You should file the bug against the projects that don't work with latest psutil. The bug is in particular projects being incompatible with a dependency and belongs to those projects. Clark ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Regarding on iptables in openstack
Hi Shiva, This list is for development discussion and questions only. Please use the Operator List (http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators) or the general list for questions about deploying or operating OpenStack. Best Regards, Solly Ross - Original Message - From: shiva m anjane...@gmail.com To: openstack-dev@lists.openstack.org Sent: Friday, March 28, 2014 7:50:47 AM Subject: [openstack-dev] Regarding on iptables in openstack Hi, I installed devstack-havana on ubuntu-13.10. I see iptables-save with all iptable rules. Can any one please help me how do I add a new rule or edit iptables-save on Openstack? Thanks Shiva ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [OpenStack-Dev][Nova][VMWare] Enable live migration with one nova compute
Building on what John said, I'm a bit wary of introducing semantics into the Conductor's live migration code that are VMWare-specific. The conductor's live-migration code is supposed to be driver-agnostic. IMHO, it would be much better if we could handle this at a level where the code was already VMWare-specific. Best Regards, Solly Ross - Original Message - From: Jay Lau jay.lau@gmail.com To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Sent: Monday, March 31, 2014 10:36:17 AM Subject: Re: [openstack-dev] [OpenStack-Dev][Nova][VMWare] Enable live migration with one nova compute Thanks John. Yes, I also think that this should be a bp as it is going to make some changes to enable live migration with only one nova compute, will file a blueprint later. For your proposal specify the same host as the instance, this can resolve the issue of live migration with target host, but what about the case of live migration without target host? If we still allow specify the same host as the instance, the the live migration will goes to dead loop. So it seems we definitely need to find a way to specify the node for live migration, hope someone else can show some light here. Of course, I will file bp and go through the new bp review process for this feature. Thanks! 2014-03-31 21:02 GMT+08:00 John Garbutt j...@johngarbutt.com : On 31 March 2014 10:11, Jay Lau jay.lau@gmail.com wrote: Hi, Currently with VMWare VCDriver, one nova compute can manage multiple clusters/RPs, this caused cluster admin cannot do live migration between clusters/PRs if those clusters/PRs managed by one nova compute as the current live migration logic request at least two nova computes. A bug [1] was also filed to trace VMWare live migration issue. I'm now trying the following solution to see if it is acceptable for a fix, the fix wants enable live migration with one nova compute: 1) When live migration check if host are same, check both host and node for the VM instance. 2) When nova scheduler select destination for live migration, the live migration task should put (host, node) to attempted hosts. 3) Nova scheduler needs to be enhanced to support ignored_nodes. 4) nova compute need to be enhanced to check host and node when doing live migration. I also uploaded a WIP patch [2] for you to review the idea of the fix and hope can get some comments from you. [1] https://bugs.launchpad.net/nova/+bug/1192192 [2] https://review.openstack.org/#/c/84085 Long term, finding a way to unify how cells and the VMware driver manages multiple hosts, seems like the best way forward. It would be a shame for this API to be different between cells and VMware, although right now, that might not work too well :( A better short term fix, might be to allow you to specify the same host as the instance, and the scheduling of the node could be delegated to the VMware driver, which might just delegate that to vCenter. I assume we still need some way to specify the node, and I can't immediately think of a good way forward. I feel this should really be treated as a blueprint, and go through the new blueprint review process. That should help decide the right approach to take. John ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Thanks, Jay ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] [bug] unit tests sqlite regexp() function doesn't behave like mysql
IMHO,Stringifying None and then expecting the *string* to match NULL is wrong. Could we check to see if `filters[filter_name]` is None and deal with that case separately (i.e `if filters[filter_name] is None: filter = column_is_null_check else: filter = column_attr.op(db_regexp_op)( str(filters[filter_name]))`)? Best Regards, Solly Ross - Original Message - From: Chris Friesen chris.frie...@windriver.com To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Sent: Monday, March 31, 2014 10:57:04 AM Subject: Re: [openstack-dev] [nova] [bug] unit tests sqlite regexp() function doesn't behave like mysql On 03/31/2014 08:54 AM, Chris Friesen wrote: I mentioned this last week in another thread but I suspect it got lost. I forgot to mention...I've opened this as a bug (https://bugs.launchpad.net/nova/+bug/1298690) but I wanted to get some wider suggestions as to the proper way to deal with it. Chris ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Juno open for development
I should specify that I meant updating the OS version used by the CI. - Original Message - From: Solly Ross sr...@redhat.com To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Sent: Monday, March 31, 2014 5:24:12 PM Subject: Re: [openstack-dev] [Nova] Juno open for development There was some talk about updating the CI for Juno. Has this been done/when will this be done? Best Regards, Solly Ross - Original Message - From: Russell Bryant rbry...@redhat.com To: OpenStack Development Mailing List openstack-dev@lists.openstack.org Sent: Monday, March 31, 2014 2:23:32 PM Subject: [openstack-dev] [Nova] Juno open for development We just merged the change that opens the master branch for Juno development: https://review.openstack.org/#/c/83752/ We will be releasing icehouse-rc1 from the commit that precedes the above patch. If anything comes up that warrants another release candidates, please get in contact with me directly. On the Juno front, we are now ready to start accepting and reviewing Juno blueprints. The process is documented here: https://wiki.openstack.org/wiki/Blueprints#Creation If you have any questions not answered by that page, please bring them up so that we can ensure the process is adequately documented. Thanks, -- Russell Bryant ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Juno open for development
There was some talk about updating the CI for Juno. Has this been done/when will this be done? Best Regards, Solly Ross - Original Message - From: Russell Bryant rbry...@redhat.com To: OpenStack Development Mailing List openstack-dev@lists.openstack.org Sent: Monday, March 31, 2014 2:23:32 PM Subject: [openstack-dev] [Nova] Juno open for development We just merged the change that opens the master branch for Juno development: https://review.openstack.org/#/c/83752/ We will be releasing icehouse-rc1 from the commit that precedes the above patch. If anything comes up that warrants another release candidates, please get in contact with me directly. On the Juno front, we are now ready to start accepting and reviewing Juno blueprints. The process is documented here: https://wiki.openstack.org/wiki/Blueprints#Creation If you have any questions not answered by that page, please bring them up so that we can ensure the process is adequately documented. Thanks, -- Russell Bryant ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [marconi] [macaroni] Proposal: change name of Marconi to Macaroni
Hello All, As OpenStack Marconi grows, I think it's time we addressed the question on everyone's mind: why isn't the project called OpenStack Macaroni? There are several compelling reasons to change Marconi's name to Macaroni: 1. Macaroni, being tube-shaped, exemplify the concept of connecting various services such that messages (cheese sauce) can flow between them. Remember, the internet is in fact a series of tubes. 2. Everyone reads the name as Macaroni the first time they see it, anyway. 3. Everyone knows what Macaroni are, while many people do not know who Marconi is. People unable to access Wikipedia in a timely manner will greatly benefit from the name change, since they will be able to instantly recognize what Marconi/Macaroni provides to the OpenStack project (see #1). 4. Only robots dislike Macaroni and Cheese. Therefore, by holding a vote, we can identify who is, in fact, a robot, and summarily destroy them before they are able to orchestrate a SkyNet-inspired uprising. Therefore, in the name of clarity, hilarity, and the prevention of robot apocalypses, I call for a vote to change the name of the OpenStack Marconi project to OpenStack Macaroni. The vote should probably be held after OpenStack Summit, so that we can discuss the potential name change in a Summit session. Best Regards, Solly Ross P.S. Happy April Fools Day! ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Missleading and hardly parseable default neutron config files
IMHO, having Example: on a separate line from the actual option is a lot easier for humans: you just delete a single character or use your editor-of-choice's uncomment macro to activate a line. - Original Message - From: Russell Bryant rbry...@redhat.com To: openstack-dev@lists.openstack.org Sent: Wednesday, April 2, 2014 9:45:05 AM Subject: Re: [openstack-dev] Missleading and hardly parseable default neutron config files On 04/02/2014 06:38 AM, Salvatore Orlando wrote: Hi Thomas, it probably won't be a bad idea if you can share the patches you're applying to the default configuration files. I think all distros are patching them anyway, so this might allow us to provide mostly ready to use config files. Is there a chance you can push something to gerrit? How about auto generate the sample config like most other projects? -- Russell Bryant ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Help in re-running openstack
I just wanted to add that if you modify code, you can commit it into a temporary commit, and that will be preserved. Best Regards, Solly Ross - Original Message - From: Dolph Mathews dolph.math...@gmail.com To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Sent: Thursday, April 3, 2014 9:51:31 AM Subject: Re: [openstack-dev] Help in re-running openstack On Thu, Apr 3, 2014 at 8:45 AM, Anita Kuno ante...@anteaya.info wrote: On 04/03/2014 07:02 AM, Erno Kuvaja wrote: Hi Shiva, You can get into the screen after you have made the changes stop the process you have changed (ctrl-c on the correct tab) and restart it (arrow up will give you the last command ran which will be the one that has started the process by devstack). - Erno On 03/04/14 11:47, shiva m wrote: Hi, I am trying to modify code in /op/stack/* and did ./unstack.sh and ./stack.sh. But after ./stack.sh it reloading to previous values. Any one please help where to modify code and re-run. Say if I modify some python file or some configurtaion file like /etc/nova/nova.conf, how do I make these changes get effected. I have ubuntu-havana devstack setup. I am new to openstack code, correct if I am wrong. Thanks, Shiva ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev Requests of this nature are considered support requests. Although not named, this question is specifically with regard to devstack (making this the appropriate list!). Support requests belong on the general mailing list: openst...@lists.openstack.org . http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack Thank you, Anita. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [OpenStack-Dev][Nova][VMWare] Enable live migration with one nova compute
@Matt Booth: I think you make a lot of good points, but I think the main gist of the opposing argument, so to speak, is that currently, the way we differentiate between potential compute resources (whether they be an individual hypervisor or a cluster) is by having each have its own compute node. I think some of the reluctance here is to change that model -- the idea that a Nova compute node represents one resource which is, for all intents and purposes, atomic to OpenStack. While I get your point that this is an implementation detail, I think it's a rather large one, and a fundamental assumption in current OpenStack code (for the most part). If we change that assumption, we shouldn't really change it piecemeal. IMHO, this model (compute nodes as atomic resources) fits the overall design well. That being said, I personally would not be averse to something like expanding a NUMA-style API to cover the cluster, as I think this continues to fit the existing model -- a NUMA-style API breaks down an atomic resource, so for a VMWare cluster that would allow tuning to individual hypervisors, while for an individual hypervisor that would allow tuning to individual cores, etc. Best Regards, Solly Ross - Original Message - From: Matthew Booth mbo...@redhat.com To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Sent: Monday, April 7, 2014 10:47:35 AM Subject: Re: [openstack-dev] [OpenStack-Dev][Nova][VMWare] Enable live migration with one nova compute On 07/04/14 06:20, Jay Pipes wrote: On Sun, 2014-04-06 at 06:59 +, Nandavar, Divakar Padiyar wrote: Well, it seems to me that the problem is the above blueprint and the code it introduced. This is an anti-feature IMO, and probably the best solution would be to remove the above code and go back to having a single nova-compute managing a single vCenter cluster, not multiple ones. Problem is not introduced by managing multiple clusters from single nova-compute proxy node. I strongly disagree. Internally this proxy driver is still presenting the compute-node for each of the cluster its managing. In what way? What we need to think about is applicability of the live migration use case when a cluster is modelled as a compute. Since the cluster is modelled as a compute, it is assumed that a typical use case of live-move is taken care by the underlying cluster itself. With this there are other use cases which are no-op today like host maintenance mode, live move, setting instance affinity etc., In order to resolve this I was thinking of A way to expose operations on individual ESX Hosts like Putting host in maintenance mode, live move, instance affinity etc., by introducing Parent - Child compute node concept. Scheduling can be restricted to Parent compute node and Child compute node can be used for providing more drill down on compute and also enable additional compute operations.Any thoughts on this? The fundamental problem is that hacks were put in place in order to make Nova defer control to vCenter, when the design of Nova and vCenter are not compatible, and we're paying the price for that right now. All of the operations you describe above -- putting a host in maintenance mode, live-migration of an instance, ensuring a new instance is launched near or not-near another instance -- depend on a fundamental design feature in Nova: that a nova-compute worker fully controls and manages a host that provides a place to put server instances. We have internal driver interfaces for the *hypervisor*, not for the *manager of hypervisors*, because, you know, that's what Nova does. I'm going to take you to task here for use of the word 'fundamental'. What does Nova do? Apparently: 'OpenStack Nova provides a cloud computing fabric controller, supporting a wide variety of virtualization technologies, including KVM, Xen, LXC, VMware, and more. In addition to its native API, it includes compatibility with the commonly encountered Amazon EC2 and S3 APIs.' There's nothing in there about the ratio of Nova instances to hypervisors: that's an implementation detail. Now this change may or may not sit well with design decisions which have been made in the past, but the concept of managing multiple clusters from a single Nova instance is certainly not fundamentally wrong. It may not be pragmatic; it may require further changes to Nova which were not made, but there is nothing about it which is fundamentally at odds with the stated goals of the project. Why did I bother with that? I think it's in danger of being lost. Nova has been around for a while now and it has a lot of code and a lot of developers behind it. We need to remember, though, that's it's all for nothing if nobody wants to use it. VMware is different, but not wrong. Let's stay fresh. The problem with all of the vCenter stuff is that it is trying to say to Nova don't
Re: [openstack-dev] [openstack] [nova] admin user create instance for another user/tenant
Simon, please use the operators list or general list for questions such as this in the future. https://wiki.openstack.org/wiki/Mailing_Lists#General_List Best Regards, Solly Ross - Original Message - From: Xu (Simon) Chen xche...@gmail.com To: openstack-dev@lists.openstack.org Sent: Saturday, April 5, 2014 12:17:05 AM Subject: [openstack-dev] [openstack] [nova] admin user create instance for another user/tenant I wonder if there is a way to do the following. I have a user A with admin role in tenant A, and I want to create a VM in/for tenant B as user A. Obviously, I can use A's admin privilege to add itself to tenant B, but I want to avoid that. Based on the policy.json file, it seems doable: https://github.com/openstack/nova/blob/master/etc/nova/policy.json#L8 I read this as, as long as a user is an admin, it can create an instance.. Just like an admin user can remove an instance from another tenant. But in here, it looks like as long as the context project ID and target project ID don't match, an action would be rejected: https://github.com/openstack/nova/blob/master/nova/api/openstack/wsgi.py#L968 Indeed, when I try to use user A's token to create a VM (POST to v2/tenant_b/servers), I got the exception from the above link. On the other hand, according to here, VM's project_id only comes from the context: https://github.com/openstack/nova/blob/master/nova/compute/api.py#L767 I wonder if it makes sense to allow admin users to specify a project_id field (which overrides context.project_id) when creating a VM. This probably requires non-trivial code change. Or maybe there is another way of doing what I want? Thanks. -Simon ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] Improvement of Cinder API wrt https://bugs.launchpad.net/nova/+bug/1213953
So, There's currently an outstanding issue with regards to a Nova shortcut command that creates a volume from an image and then boots from it in one fell swoop. The gist of the issue is that there is currently a set timeout which can time out before the volume creation has finished (it's designed to time out in case there is an error), in cases where the image download or volume creation takes an extended period of time (e.g. under a Gluster backend for Cinder with certain network conditions). The proposed solution is a modification to the Cinder API to provide more detail on what exactly is going on, so that we could programmatically tune the timeout. My initial thought is to create a new column in the Volume table called 'status_detail' to provide more detailed information about the current status. For instance, for the 'downloading' status, we could have 'status_detail' be the completion percentage or JSON containing the total size and the current amount copied. This way, at each interval we could check to see if the amount copied had changed, and trigger the timeout if it had not, instead of blindly assuming that the operation will complete within a given amount of time. What do people think? Would there be a better way to do this? Best Regards, Solly Ross ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Improvement of Cinder API wrt https://bugs.launchpad.net/nova/+bug/1213953
Also, that's still an overly complicated process for one or two VMs. The idea behind the Nova command was to minimize the steps in the image-volume-VM process for a single VM. - Original Message - From: Chris Friesen chris.frie...@windriver.com To: openstack-dev@lists.openstack.org Sent: Tuesday, November 5, 2013 9:23:39 AM Subject: Re: [openstack-dev] Improvement of Cinder API wrt https://bugs.launchpad.net/nova/+bug/1213953 Wouldn't you still need variable timeouts? I'm assuming that copying multi-gig cinder volumes might take a while, even if it's local. (Or are you assuming copy-on-write?) Chris On 11/05/2013 01:43 AM, Caitlin Bestler wrote: Replication of snapshots is one solution to this. You create a Cinder Volume once. snapshot it. Then replicate to the hosts that need it (this is the piece currently missing). Then you clone there. I will be giving an in an hour in conference session on this and other uses of snapshots in the last time slot Wednesday. On Nov 5, 2013 5:58 AM, Solly Ross sr...@redhat.com mailto:sr...@redhat.com wrote: So, There's currently an outstanding issue with regards to a Nova shortcut command that creates a volume from an image and then boots from it in one fell swoop. The gist of the issue is that there is currently a set timeout which can time out before the volume creation has finished (it's designed to time out in case there is an error), in cases where the image download or volume creation takes an extended period of time (e.g. under a Gluster backend for Cinder with certain network conditions). The proposed solution is a modification to the Cinder API to provide more detail on what exactly is going on, so that we could programmatically tune the timeout. My initial thought is to create a new column in the Volume table called 'status_detail' to provide more detailed information about the current status. For instance, for the 'downloading' status, we could have 'status_detail' be the completion percentage or JSON containing the total size and the current amount copied. This way, at each interval we could check to see if the amount copied had changed, and trigger the timeout if it had not, instead of blindly assuming that the operation will complete within a given amount of time. What do people think? Would there be a better way to do this? Best Regards, Solly Ross ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org mailto:OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] A Tool for Making OpenStack Log Reading a Bit Easier
Hello All, The other day, I was reading through a debug-level OpenStack log, and came to the realization that reading OpenStack debug-level logs was difficult, to say the least -- they can be very busy, and it is hard to quickly filter out relevant information. Thus, I wrote a little Perl script to make reading dense debug-level logs a bit easier: https://github.com/DirectXMan12/os_log_prettifier. I figured that I'd share it with other people. Basically, the script highlights certain key details using color and bolding (via ANSI control codes), and can filter lines based on subject (in the form of 'x.y.z') or message type, using regular expressions. I hope people find it useful! Best Regards, Solly Ross ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] How to add a comment to a change review?
You just click the review button under the change list on Gerrit. It should then take you to a screen where you can see inline comments waiting to be added and enter an overall comment as well. The comment system in Gerrit really should be a bit more intuitive. Best Regards, Solly Ross - Original Message - From: Michael Bright mjbrigh...@gmail.com To: openstack-dev@lists.openstack.org Sent: Wednesday, November 6, 2013 11:34:27 AM Subject: [openstack-dev] How to add a comment to a change review? Can someone enlighten me as to how I can add a comment to one of my own changes on review.openstack.org . I suppose it's mind numbingly obvious but here for example I just cannot see how to add a comment. It says here: https://wiki.openstack.org/wiki/GerritJenkinsGit#Test_Failures Leave a comment on the review referencing the bug causing the transient failure (not the bug you're attempting to fix with your patch): To re-run the check jobs (before a change has been approved), leave a comment with the form recheck bug . To re-run the gate jobs (after a change has been approved), leave a comment with the form reverify bug . It must be so obvious ... Thanks in advance, Mike. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [horizon][novnc] Browser Usage for noVNC
It was recommended I cross-post this for visibility. Devs are welcome to provide feedback as well ;-) Best Regards, Solly Ross - Forwarded Message - From: Solly Ross sr...@redhat.com To: openstack-operat...@lists.openstack.org Sent: Tuesday, June 24, 2014 1:29:15 PM Subject: Browser Usage for noVNC Hello Operators, I'm part of the noVNC upstream development team. noVNC, for those of you who don't know, is the HTML5 VNC client integrated into Horizon (the OpenStack dashboard). We are considering removing support for some older browsers, and wanted to make sure that we wouldn't be inconveniencing anybody too much. Are there any operators who still aim to support connecting with any of the following browsers: - Firefox 11.0 - Chrome 16.0 - IE 10.0 - Safari 6.0 (insert uncommon browser here that doesn't support WebSockets natively) If so, what is your minimum browser version? Best Regards, Solly Ross ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [horizon][novnc] Browser Usage for noVNC
Hi Anne, Thanks for the numbers. I suspect the user numbers might be a bit different (deployers and OpenStack devs might have the latest versions of software on their machines, while users might have a corporate build that has older software), but it's good to get those initial statistics. Those poor people using IE 7... Best Regards, Solly Ross - Original Message - From: Anne Gentle a...@openstack.org To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Sent: Thursday, June 26, 2014 11:05:58 AM Subject: Re: [openstack-dev] [horizon][novnc] Browser Usage for noVNC On Thu, Jun 26, 2014 at 9:54 AM, Solly Ross sr...@redhat.com wrote: It was recommended I cross-post this for visibility. Devs are welcome to provide feedback as well ;-) Best Regards, Solly Ross - Forwarded Message - From: Solly Ross sr...@redhat.com To: openstack-operat...@lists.openstack.org Sent: Tuesday, June 24, 2014 1:29:15 PM Subject: Browser Usage for noVNC Hello Operators, I'm part of the noVNC upstream development team. noVNC, for those of you who don't know, is the HTML5 VNC client integrated into Horizon (the OpenStack dashboard). We are considering removing support for some older browsers, and wanted to make sure that we wouldn't be inconveniencing anybody too much. Are there any operators who still aim to support connecting with any of the following browsers: Hi Solly, I don't know if this data helps, but I took a quick look at the web analytics for people reading docs.openstack.org , and all of your listed browser versions are under 2% (most under 1%) for web site readers. Well, except IE 7 hangs in there at 4.43%. :) Let me know if that's useful. Anne - Firefox 11.0 - Chrome 16.0 - IE 10.0 - Safari 6.0 (insert uncommon browser here that doesn't support WebSockets natively) If so, what is your minimum browser version? Best Regards, Solly Ross ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [nova][libvirt] Non-readonly connection to libvirt in unit tests
Hi, I was wondering if there was a way to get a non-readonly connection to libvirt when running the unit tests on the CI. If I call `LibvirtDriver._connect(LibvirtDriver.uri())`, it works fine locally, but the upstream CI barfs with libvirtError: internal error Unable to locate libvirtd daemon in /usr/sbin (to override, set $LIBVIRTD_PATH to the name of the libvirtd binary). If I try to connect by calling libvirt.open(None), it also barfs, saying I don't have permission to connect. I could just set it to always use fakelibvirt, but it would be nice to be able to run some of the tests against a real target. The tests in question are part of https://review.openstack.org/#/c/111459/, and involve manipulating directory-based libvirt storage pools. Best Regards, Solly Ross ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [nova][libvirt][baremetal] Nova Baremetal's Usage of Components from Libvirt
Hello All, So, I'm working on https://review.openstack.org/#/c/111459/, and have encountered an issue. It seems that the Nova Baremetal driver uses the ImageCacheManager from the Libvirt driver. For various reasons (see the commit), the ImageCacheManager has been refactored to require a libvirt connection to function properly. However, the Nova Baremetal driver cannot provide such a connection. Bearing in mind that Baremetal is deprecated and slated to be replaced by Ironic, the question is such: what to do about the ImageCacheManager. One option would be to make it so that the ImageCacheManager can function without a libvirt connection. This might make sense if the Baremetal driver were around to stay; there would be somewhat less duplication than a wholesale copying of the code. However, in light of Baremetal's impending this seems to me to be a poor choice since it would involve lots of duplicate functionality, would complicate the ImageCacheManager code, and would later need to be manually removed once the Baremetal driver is removed. The second option would be to make a copy of the old ImageCacheManager in the Baremetal directory, and have the Baremetal driver use that. This seems to me to be the better option, since it means that when the Baremetal driver is removed, the old ImageCacheManager code goes with it, without someone having to manually remove it. What do you think? Best Regards, Solly Ross ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] so what do i do about libvirt-python if i'm on precise?
Just to add my two cents, while I get that people need to run on older versions of software, at a certain point you have to bump the minimum version. Even libvirt 0.9.11 is from April 3rd 2012. That's two and a third years old at this point. I think at a certain point we need to say if you want to run OpenStack on an older platform, then you'll need to run an older OpenStack or backport the required packages. Best Regards, Solly Ross - Original Message - From: Joe Gordon joe.gord...@gmail.com To: OpenStack Development Mailing List openstack-dev@lists.openstack.org Sent: Wednesday, July 30, 2014 7:07:13 PM Subject: Re: [openstack-dev] [nova] so what do i do about libvirt-python if i'm on precise? On Jul 30, 2014 3:36 PM, Clark Boylan cboy...@sapwetik.org wrote: On Wed, Jul 30, 2014, at 03:23 PM, Jeremy Stanley wrote: On 2014-07-30 13:21:10 -0700 (-0700), Joe Gordon wrote: While forcing people to move to a newer version of libvirt is doable on most environments, do we want to do that now? What is the benefit of doing so? [...] The only dog I have in this fight is that using the split-out libvirt-python on PyPI means we finally get to run Nova unit tests in virtualenvs which aren't built with system-site-packages enabled. It's been a long-running headache which I'd like to see eradicated everywhere we can. I understand though if we have to go about it more slowly, I'm just excited to see it finally within our grasp. -- Jeremy Stanley We aren't quite forcing people to move to newer versions. Only those installing nova test-requirements need newer libvirt. This does not include people using eg devstack. I think it is reasonable to expect people testing tip of nova master to have a reasonably newish test bed to test it (its not like the Infra team moves at a really fast pace :) ). Based on http://lists.openstack.org/pipermail/openstack-dev/2014-July/041457.html this patch is breaking people, which is the basis for my concerns. Perhaps we should get some further details from Salvatore. Avoiding system site packages in virtualenvs is a huge win particularly for consistency of test results. It avoids pollution of site packages that can happen differently across test machines. This particular type of inconsistency has been the cause of the previously mentioned headaches. I agree this is a huge win, but I am just concerned we don't have any deprecation cycle and just roll out a new requirement without a heads up. Clark ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] libvirtError: XML error: Missing CPU model name on 2nd level vm
Hi Kevin,
Running devstack in a VM is perfectly doable. Many developers use
devstack inside a VM (I run mine inside a VM launched using libvirt
on KVM). I can't comment on the issue that you're encountering,
but perhaps something wasn't configured correctly when you launched the
VM?
Best Regards,
Solly Ross
- Original Message -
From: Chen CH Ji jiche...@cn.ibm.com
To: openstack-dev@lists.openstack.org
Sent: Friday, August 1, 2014 5:04:16 AM
Subject: [openstack-dev] [nova] libvirtError: XML error: Missing CPU model
name on 2nd level vm
Hi
I don't have a real PC to so created a test env ,so I created a 2nd level env
(create a kvm virtual machine on top of a physical host then run devstack o
the vm)
I am not sure whether it's doable because I saw following error when start
nova-compute service , is it a bug or I need to update my configuration
instead? thanks
2014-08-01 17:04:51.532 DEBUG nova.virt.libvirt.config [-] Generated XML
('cpu\n archx86_64/arch\n topology sockets=1 cores=1
threads=1/\n/cpu\n',) from (pid=16956) to_xml
/opt/stack/nova/nova/virt/libvirt/config.py:79
Traceback (most recent call last):
File /usr/lib/python2.7/dist-packages/eventlet/hubs/hub.py, line 346, in
fire_timers
timer()
File /usr/lib/python2.7/dist-packages/eventlet/hubs/timer.py, line 56, in
__call__
cb(*args, **kw)
File /usr/lib/python2.7/dist-packages/eventlet/event.py, line 163, in
_do_send
waiter.switch(result)
File /usr/lib/python2.7/dist-packages/eventlet/greenthread.py, line 194, in
main
result = function(*args, **kwargs)
File /opt/stack/nova/nova/openstack/common/service.py, line 490, in
run_service
service.start()
File /opt/stack/nova/nova/service.py, line 164, in start
self.manager.init_host()
File /opt/stack/nova/nova/compute/manager.py, line 1055, in init_host
self.driver.init_host(host=self.host)
File /opt/stack/nova/nova/virt/libvirt/driver.py, line 633, in init_host
self._do_quality_warnings()
File /opt/stack/nova/nova/virt/libvirt/driver.py, line 616, in
_do_quality_warnings
caps = self._get_host_capabilities()
File /opt/stack/nova/nova/virt/libvirt/driver.py, line 2942, in
_get_host_capabilities
libvirt.VIR_CONNECT_BASELINE_CPU_EXPAND_FEATURES)
File /usr/lib/python2.7/dist-packages/eventlet/tpool.py, line 179, in doit
result = proxy_call(self._autowrap, f, *args, **kwargs)
File /usr/lib/python2.7/dist-packages/eventlet/tpool.py, line 139, in
proxy_call
rv = execute(f,*args,**kwargs)
File /usr/lib/python2.7/dist-packages/eventlet/tpool.py, line 77, in
tworker
rv = meth(*args,**kwargs)
File /usr/lib/python2.7/dist-packages/libvirt.py, line 3127, in baselineCPU
if ret is None: raise libvirtError ('virConnectBaselineCPU() failed',
conn=self)
libvirtError: XML error: Missing CPU model name
Best Regards!
Kevin (Chen) Ji 纪 晨
Engineer, zVM Development, CSTL
Notes: Chen CH Ji/China/IBM@IBMCN Internet: jiche...@cn.ibm.com
Phone: +86-10-82454158
Address: 3/F Ring Building, ZhongGuanCun Software Park, Haidian District,
Beijing 100193, PRC
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [keystone] event notifications issue
Hi Nader, I ran into some issues the other day with Keystone notifications. Make sure that you have (in both Keystone and the receiving project) the following configuration options set: - rpc_backend (should be rabbit, qpid, etc) - appropriate rpc backend config options (e.g. for rabbit, rabbit_hosts and rabbit_password) - notification_driver = messaging Hopefully this is helpful, as some of these options are not set by default in devstack. Best Regards, Solly Ross - Original Message - From: Nader Lahouti nader.laho...@gmail.com To: OpenStack Development Mailing List openstack-dev@lists.openstack.org Sent: Tuesday, April 8, 2014 12:50:50 PM Subject: Re: [openstack-dev] [keystone] event notifications issue Resending the question: Can someone please point me to a document/code in order to get keystone notification with the latest keystone code. Appreciate your help. On Sun, Apr 6, 2014 at 4:13 PM, Nader Lahouti nader.laho...@gmail.com wrote: Hi All, I was able to get keystone notification when creating/deleting a tenant by setting these parameters in keystone.conf: (NOTE: the brach that I was using: git branch -v * (no branch) 0d83e7e Bump stable/havana next version to 2013.2.2) ) notification_topics = Key_Notify rpc_backend = keystone.openstack.common.rpc.impl_kombu control_exchange = Key_openstack notification_driver = keystone.openstack.common.notifier.rpc_notifier Now I changed the branch to: git branch -v * master e45ff9e Merge Updated from global requirements And cannot get any notification. It seems the notifications.py changed. What I need to do, in order to get event notification with the current code and make it work? Thanks, Nader. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][cinder] create server from a volume snapshot, 180 reties is sufficient?
I think some of the objections to such patches were based on the fact that this is really just a shortcut for two separate operations: create a volume from a Glance image, and then boot off of that image. Instead of monkey-patching it, we should either figure out a way to dynamically configure the timeout based on image size, network conditions, etc, or we should just remove the shortcut entirely, and just have people do the two steps separately (the current workaround for large disk images). Best Regards, Solly Ross - Original Message - From: Ben Swartzlander ben.swartzlan...@netapp.com To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Sent: Tuesday, April 8, 2014 12:08:52 PM Subject: Re: [openstack-dev] [nova][cinder] create server from a volume snapshot, 180 reties is sufficient? Options may be bad, but hardcoded values chosen arbitrarily are worse. Unless someone can justify why the value needs to be 180 and not 179 or 181 then it should be configurable. That’s my opinion at any rate. -Ben From: Lingxian Kong [mailto:anlin.k...@gmail.com] Sent: Tuesday, April 08, 2014 11:59 AM To: OpenStack Development Mailing List Subject: [openstack-dev] [nova][cinder] create server from a volume snapshot, 180 reties is sufficient? hi there: According to the patch https://review.openstack.org/#/c/80619/ , Nova will wait for volume creation for 180s, the config option is rejected by Russell and Nikola. But the reason I raise it up is, we found the server creation failed due to timeout in our deployment, with LVM as Cinder backend. So, I wander is 180s really suitable here? Are there some guidences about when should we add an option? But at least, we should not avoid an option, just because of the existing overwhelming number of them, right? Thoughts? -- --- Lingxian Kong Huawei Technologies Co.,LTD. IT Product Line CloudOS PDU China, Xi'an Mobile: +86-18602962792 Email: konglingx...@huawei.com ; anlin.k...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][cinder] create server from a volume snapshot, 180 reties is sufficient?
Is there a blueprint for using this for Cinder-Nova interaction? Best Regards, Solly Ross - Original Message - From: Mike Perez thin...@gmail.com To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Sent: Tuesday, April 8, 2014 3:58:31 PM Subject: Re: [openstack-dev] [nova][cinder] create server from a volume snapshot, 180 reties is sufficient? On 23:58 Tue 08 Apr , Lingxian Kong wrote: hi there: According to the patch https://review.openstack.org/#/c/80619/, Nova will wait for volume creation for 180s, the config option is rejected by Russell and Nikola. But the reason I raise it up is, we found the server creation failed due to timeout in our deployment, with LVM as Cinder backend. So, I wander is 180s really suitable here? Are there some guidences about when should we add an option? But at least, we should not avoid an option, just because of the existing overwhelming number of them, right? Thoughts? It looks like this was a temporary accepted solution and the long term solution is with event callbacks [1]. [1] - https://blueprints.launchpad.net/nova/+spec/admin-event-callback-api -- Mike Perez ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] nova-specs
Just wanted to confirm what Sean said -- as someone who just joined the OpenStack community last year, going to implement a vaguely worded blueprint and then having the code review be derailed with people saying well, you probably should be using this completely different design is fairly frustrating. While you come to anticipate certain changes, IMHO it's definitely much better to decide on the design *before* you start coding, that way code reviews can focus on the code, and you don't have to completely rewrite patches as much. Best Regards, Solly Ross - Original Message - From: Sean Dague s...@dague.net To: openstack-dev@lists.openstack.org Sent: Tuesday, April 15, 2014 1:45:16 PM Subject: Re: [openstack-dev] [Nova] nova-specs On 04/15/2014 11:42 AM, Russell Bryant wrote: On 04/15/2014 11:01 AM, Brian Elliott wrote: * specs review. The new blueprint process is a work of genius, and I think its already working better than what we've had in previous releases. However, there are a lot of blueprints there in review, and we need to focus on making sure these get looked at sooner rather than later. I'd especially like to encourage operators to take a look at blueprints relevant to their interests. Phil Day from HP has been doing a really good job at this, and I'd like to see more of it. I have mixed feelings about the nova-specs repo. I dig the open collaboration of the blueprints process, but I also think there is a danger of getting too process-oriented here. Are these design documents expected to call out every detail of a feature? Ideally, I’d like to see only very high level documentation in the specs repo. Basically, the spec could include just enough detail for people to agree that they think a feature is worth inclusion. More detailed discussion could remain on the code reviews since they are the actual end work product. There is a balance to be found here. The benefit of doing more review earlier is to change direction as necessary when it's *much* easier to do so. It's a lot more time consuming to do re-work after code has been written, than re-work in a spec. Yes, it's more up front work, but I think it will speed up the process overall. It means we're much more in agreement and on the same page before code even shows up. That's huge. One of the big problems we've had in code review is the amount of churn and re-work required. That is killing our throughput in code review. If we can do more up front work that will reduce re-work later, it's going to be a *huge* help to our primary project bottleneck: the code review queue. I think the previous process is a huge demotivator to contributors, when they file a blueprint with minimal info, it gets approved, they spend months working on it, and only at the end of the process does the idea get dug into enough for people to realize that it's not what anyone wants. At that point people are so invested in the time they spent on this feature that turning that conversation productive is really hard. Catching more of these up front and being more explicit about what Nova wants in a cycle is goodness. -Sean -- Sean Dague Samsung Research America s...@dague.net / sean.da...@samsung.com http://dague.net ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] [neutron] Gerrit spec reviews
This page actually has most of the answers on it: https://wiki.openstack.org/wiki/Blueprints#Nova 1: The time limit is one release: at the end of each release, non-completed specs will be removed 2: The nova-drivers team approves specs 3: No, it's not required to have a summit session. Something like that would be problematic, as blueprints could be proposed after summit. Hope that helps. IMHO, the nova-specs gerrit repository is a great way to review blueprints. I'm really liking it so far. Best Regards, Solly Ross - Original Message - From: Kyle Mestery mest...@noironetworks.com To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Sent: Tuesday, April 15, 2014 5:14:14 PM Subject: [openstack-dev] [nova] [neutron] Gerrit spec reviews Hi Nova developers: I have a question around the new nova-specs gerrit repository. We're implementing the same thing in Neutron for Juno (I'm hoping to make this live tomorrow), but I had a few quick questions so I can build on your experience with this so far: 1. Did you implement any sort of time limit on BPs in review? Mostly around trying to triage the BPs as they come in. 2. Nova has a subset of the core-team which can actually approve BPs, is this correct? 3. For all BPs submitted using the new procedure, is it required to submit a Summit session? Any guidance appreciated here! Thanks, Kyle ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Openstack] About advice for Contribution
If you start contributing to Nova, my advice would be to start small -- find a part of Nova that you get to know well, and then branch out. Nova is a large project with complex paths through the code (RPC vs REST API vs normal method calling), so it can take a bit of time to get used to. You could also take a look at Oslo -- it's component-agnostic and the different parts tend to be a bit smaller than the other components. Hope this helps, and good luck! We're always glad to have new contributors! Best Regards, Solly Ross - Original Message - From: Mayur Patil ram.nath241...@gmail.com To: Openstack Users openst...@lists.openstack.org, Openstack Developer openstack-dev@lists.openstack.org Sent: Wednesday, April 16, 2014 3:27:24 AM Subject: [openstack-dev] [Openstack] About advice for Contribution Howdy All, I need a small advice. I am working from last two years on Eucalyptus. Recently, switched to Openstack and trying to contribute to Code-Base. My skills are: - I have good understanding of private Cloud - Total beginner in Python but somewhat good at Java Except SWIFT NEUTRON , I am clear with other components concepts. so which project/component should I study to get started for Openstack Development ? Seeking for guidance, Thanks ! -- Cheers, Mayur S. Patil, Pune. Contact : ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Looking for experienced guide to understand libvirt driver
Hi Scott, I've actually been diving through those Nova code paths recently, to work on a BP to move the Nova libvirt driver to use libvirt storage pools for image backends, so I may be able to help you. Just FYI, if my blueprint gets accepted, it may actually reduce the amount of code that you have to write, since libvirt already has a (limitted) storage driver for sheepdog (see the BP here: https://review.openstack.org/#/c/86947/) Best Regards, Solly Ross - Original Message - From: Scott Devoid dev...@anl.gov To: OpenStack Development Mailing List openstack-dev@lists.openstack.org Sent: Monday, April 21, 2014 7:38:13 PM Subject: [openstack-dev] [nova] Looking for experienced guide to understand libvirt driver Hi folks! I am working to add Sheepdog as a disk backend for the libvirt driver. I have a blueprint started and an early version of the code. However I am having trouble working my way thorough the code in the libvirt driver. The storage code doesn't feel vary modular to start with and my changes only seem to make it worse; e.g. adding more if blocks to 400 line functions. Is there an experienced contributor that could spend 30 minutes walking through parts of the code? - Blueprint: https://review.openstack.org/#/c/82584/ - Nova code: https://review.openstack.org/#/c/74148/ - Devstack code: https://review.openstack.org/#/c/89434/ Thanks, ~ Scott ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] How to know who has written the component?
Just FYI, you'll see a lot of people refer to `git blame`. They're basically the same command. Best Regards, Solly Ross - Original Message - From: Sean Dague s...@dague.net To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Sent: Thursday, April 24, 2014 10:32:41 AM Subject: Re: [openstack-dev] How to know who has written the component? git annotate is your friend, it will give you a line by line annotation of the last person that made a change to that line, and which git commit it was in. man git-annotate for more info -Sean On 04/24/2014 10:28 AM, Hao Wang wrote: Sorry for sending out the mail accidentally. # I am dealing with the bug *1308958*.**I know the root cause is the condition of SQL string, but it is unknown that why SQL is using the table externalnetworks instead of networks. # # Regarding this details, how could I know who I could contact to know more designs of the specific component? # # Thanks, Hao On Thu, Apr 24, 2014 at 10:25 AM, Hao Wang hao.1.w...@gmail.com mailto:hao.1.w...@gmail.com wrote: Hello, # I am dealing with the bug *1308958*.**I have # # # # # # # some questions with the details. # # # # # # # # ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Sean Dague http://dague.net ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova][blueprint] Accelerate the booting process of a number of vms via VMThunder
Something to be aware of when planing an image transfer library is that individual drivers might have optimized support for image transfer in certain cases (especially when dealing with transferring between different formats, like raw to qcow2, etc). This builds on what Christopher was saying -- there's actually a reason why we have code for each driver. While having a common image copying library would be nice, I think a better way to do it would be to have some sort of library composed of building blocks, such that each driver could make use of common functionality while still tailoring the operation to the quirks of the particular drivers. Best Regards, Solly Ross - Original Message - From: Christopher Lefelhocz christopher.lefel...@rackspace.com To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Sent: Thursday, April 24, 2014 11:17:41 AM Subject: Re: [openstack-dev] [Nova][blueprint] Accelerate the booting process of a number of vms via VMThunder Apologies for coming to this discussion late... On 4/22/14 6:21 PM, Jay Pipes jaypi...@gmail.com wrote: Right, a good solution would allow for some flexibility via multiple transfer drivers. +1. In particular I don't think this discussion should degenerate into zero-copy vs. pre caching. I see both as possible solutions depending on deployer/environment needs. Jay Pipes has suggested we figure out a blueprint for a separate library dedicated to the data(byte) transfer, which may be put in oslo and used by any projects in need (Hoping Jay can come in:-)). Huiba, Zhiyan, everyone else, do you think we come up with a blueprint about the data transfer in oslo can work? Yes, so I believe the most appropriate solution is to create a library -- in oslo or a standalone library like taskflow -- that would offer a simple byte streaming library that could be used by nova.image to expose a neat and clean task-based API. Right now, there is a bunch of random image transfer code spread throughout nova.image and in each of the virt drivers there seems to be different re-implementations of similar functionality. I propose we clean all that up and have nova.image expose an API so that a virt driver could do something like this: from nova.image import api as image_api ... task = image_api.copy(from_path_or_uri, to_path_or_uri) # do some other work copy_task_result = task.wait() Within nova.image.api.copy(), we would use the aforementioned transfer library to move the image bits from the source to the destination using the most appropriate method. If I understand correctly, we'll create some common library around this. It would be good to understand the details a bit better. I've thought a bit about this issue. The one area that I get stuck at is providing a common set of downloads which work across drivers effectively. Part of the reason there are a bunch or random image transfers is historical, but also because performance was already a problem. Examples include: transferring to compute first then copying to dom0 causing performance issues, needs in some drivers to download image completely to validate prior to putting in place, etc. It may be easy to say we'll push most of this to the dom0, but I know for Xen our python stack is somewhat limited so that may be an issue. By the way we've been working on proposing a simpler image pre caching system/strategy. It focuses specifically on the image caching portion of this discussion. For those interested, see the nova-spec https://review.openstack.org/#/c/85792. We'd like to leverage whatever optimized image download strategy is available. Christopher ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [OpenStack][Nova] infinite attempts left
Hi Gareth, In the future, please use the operator mailing list (see https://wiki.openstack.org/wiki/Mailing_Lists) for questions like this. To answer your question, however, it indicates that nova-manage was unable to establish an connection to the database, and will retry an infinite number of times (i.e. it won't time out). Best Regards, Solly Ross - Original Message - From: Gareth academicgar...@gmail.com To: OpenStack Development Mailing List openstack-dev@lists.openstack.org Sent: Thursday, April 24, 2014 5:42:06 AM Subject: [openstack-dev] [OpenStack][Nova] infinite attempts left When run nova-manage db sync it outputs a lot of logs and SQL connection failed. infinite attempts left What does that means? -- Gareth Cloud Computing, OpenStack, Distributed Storage, Fitness, Basketball OpenStack contributor, kun_huang@freenode My promise: if you find any spelling or grammar mistakes in my email from Mar 1 2013, notify me and I'll donate $1 or ￥1 to an open organization you specify. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [requirements][CI] Question about how to deal with bumping a library version
Hi, I've submitted a patch (https://review.openstack.org/#/c/91663/) that updates Nova to use the latest version of websockify. It would appear that the CI now pulls websockify from pypi.openstack.org, which appears not to have websockify 0.6 on it yet. What is the process for getting websockify 0.6 on pypi.openstack.org? If that process involves updating global-requirements, there's a follow up question: since websockify 0.6 breaks backwards compatibility, what happens in between the time that global-requirements gets updated and the related change gets pushed? Best Regards, Solly Ross ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova][Heat] Custom Nova Flavor creation through Heat (pt.2)
Just to expand a bit on this, flavors are supposed to be the lowest level of granularity, and the general idea is to round to the nearest flavor (so if you have a VM that requires 3GB of RAM, go with a 4GB flavor). Hence, in my mind, it doesn't make any sense to create flavors on the fly; you should have enough flavors to suit your needs, but I can't really think of a situation where you'd need so much granularity that you'd need to create new flavors on the fly (assuming, of course, that you planned ahead and created enough flavors that you don't have VMs that are extremely over-allocated). Best Regards, Solly Ross - Original Message - From: Serg Melikyan smelik...@mirantis.com To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Sent: Monday, May 5, 2014 2:18:21 AM Subject: Re: [openstack-dev] [Nova][Heat] Custom Nova Flavor creation through Heat (pt.2) Having project-scoped flavors will rid us of the identified issues, and will allow a more fine-grained way of managing physical resources. Flavors concept was introduced in clouds to solve issue with effective physical resource usage: 8Gb physical memory can be effectively splitted to two m2.my_small with 4Gb RAM or to eight m1.my_tiny with 1 GB. Let's consider example when your cloud have only 2 compute nodes with 8GB RAM: vm1 (m1.my_tiny) - node1 vm2 (m1.my_tiny) - node1 vm3 (m2.my_small) - node1 vm4 (m2.my_small) - node2 (since we could not spawn on node1) This leaves ability to spawn predictable 2 VMs with m1.my_tiny flavor on node1, and 2 VMs m1.my_tiny or 1 VM m2.my_small on node2. If user has ability to create any flavor that he likes, he can create flavor like mx.my_flavor with 3GB of RAM that could not be spawned on node1 at all, and leaves 1GB under-used on node2 when spawned there. If you will multiply number of nodes to 100 or even 1000 (like some of the OpenStack deployments) you will have very big memory under-usage. Do we really need to have ability to allocate any number of physical resources for VM? If yes, I suggest to make two ways to define number of physical resource allocation for VMs: with flavors and dynamically. Leaving to cloud admins/owners to decide which way they prefer they cloud resources to be allocated. Since small clouds may prefer flavors, and big clouds may have dynamic resource allocation (impact from under-usage will be not so big). As transition plan project-scoped flavors may do the job. On Fri, May 2, 2014 at 5:35 PM, Dimitri Mazmanov dimitri.mazma...@ericsson.com wrote: This topic has already been discussed last year and a use-case was described (see [1]). Here's a Heat blueprint for a new OS::Nova::Flavor resource: [2]. Several issues have been brought up after posting my implementation for review [3], all related to how flavors are defined/implemented in nova: * Only admin tenants can manage flavors due to the default admin rule in policy.json. * Per-stack flavor creation will pollute the global flavor list * If two stacks create a flavor with the same name, collision will occur, which will lead to the following error: ERROR (Conflict): Flavor with name dupflavor already exists. (HTTP 409) These and the ones described by Steven Hardy in [4] are related to the flavor scoping in Nova. Is there any plan/discussion to allow project scoped flavors in nova, similar to the Steven’s proposal for role-based scoping (see [4])? Currently the only purpose of the is_public flag is to hide the flavor from users without the admin role, but it’s still visible in all projects. Any plan to change this? Having project-scoped flavors will rid us of the identified issues, and will allow a more fine-grained way of managing physical resources. Dimitri [1] http://lists.openstack.org/pipermail/openstack-dev/2013-November/018744.html [2] https://wiki.openstack.org/wiki/Heat/Blueprints/dynamic-flavors [3] https://review.openstack.org/#/c/90029 [4] http://lists.openstack.org/pipermail/openstack-dev/2013-November/019099.html ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Serg Melikyan, Senior Software Engineer at Mirantis, Inc. http://mirantis.com | smelik...@mirantis.com +7 (495) 640-4904, 0261 +7 (903) 156-0836 ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova][Heat] Custom Nova Flavor creation through Heat (pt.2)
If you actually have 64 flavors, though, and it's overwhelming your users, it would be pretty trivial to implement a flavor recommender where you input your requirements and it pops out the nearest flavor. That being said, I do think you're right that you can probably mitigate flavor explosion by trimming out the outlier flavors. 20-30 flavors is still a bit much, but with some clever naming, the burden of choosing a flavor can be lessened. Additionally, if this is all for the purpose of orchestration, we have a computer dealing with choosing the correct flavor, and if your computer has a problem dealing with a choice between 64 (or even 512) different flavors, perhaps it's time to upgrade :-P. Best Regards, Solly Ross - Original Message - From: Clint Byrum cl...@fewbar.com To: openstack-dev openstack-dev@lists.openstack.org Sent: Monday, May 5, 2014 12:28:58 PM Subject: Re: [openstack-dev] [Nova][Heat] Custom Nova Flavor creation through Heat (pt.2) Excerpts from Solly Ross's message of 2014-05-05 08:23:41 -0700: Just to expand a bit on this, flavors are supposed to be the lowest level of granularity, and the general idea is to round to the nearest flavor (so if you have a VM that requires 3GB of RAM, go with a 4GB flavor). Hence, in my mind, it doesn't make any sense to create flavors on the fly; you should have enough flavors to suit your needs, but I can't really think of a situation where you'd need so much granularity that you'd need to create new flavors on the fly (assuming, of course, that you planned ahead and created enough flavors that you don't have VMs that are extremely over-allocated). I agree with the conclusion you're arguing for, but it is a bit more complex than that. Flavor defines at least three things, and possibly 4: RAM, root disk, and vcpu, with an optional ephemeral disk. Because of that, the matrix of possibilities can get extremely large. Consider if you segment RAM as 1GB, 4GB, 8GB, 16GB, vcpu as 1, 2, 4, 8, and root disk as 10G, 50G, 100G, 1T. Your matrix is now 4³, 64 flavors. If you've never heard of the paradox of choice, consumers are slowed down by having too many choices. So less flavors will not only make your resource consumption easier to optimize, it will help new users move forward with more certainty. But the reality is, nobody needs an 8 CPU, 1T, 1GB flavor. Likewise, nobody needs a 1 CPU 16GB 10G server. Both are missing the mark with the common workloads. And a lot are filled in by higher level services like Trove, LBaaS, Swift, etc. So realistically, having 20-30 flavors, with groupings around the combinations users demand, is a known pattern that seems to work well. If a user has a workload that is poorly served by any of these, it probably makes the most sense for them to over-buy and demand a better sized flavor from the provider. Dynamically allocating flavors is just going to complicate things for everybody. That said, if Nova supports it, Heat should too. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] [Heat] Custom Nova Flavor creation through Heat (pt.2)
One thing that I was discussing with @jaypipes and @dansmith over on IRC was the possibility of breaking flavors down into separate components -- i.e have a disk flavor, a CPU flavor, and a RAM flavor. This way, you still get the control of the size of your building blocks (e.g. you could restrict RAM to only 2GB, 4GB, or 16GB), but you avoid exponential flavor explosion by separating out the axes. Best Regards, Solly Ross P.S. For people who use flavor names to convey information about the workload, that probably a job better done by the VM tagging proposal (https://review.openstack.org/#/c/91444/). - Original Message - From: Dimitri Mazmanov dimitri.mazma...@ericsson.com To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Sent: Monday, May 5, 2014 1:20:09 PM Subject: Re: [openstack-dev] [Nova] [Heat] Custom Nova Flavor creation through Heat (pt.2) I guess I need to describe the use-case first. An example of Telco application is IP Multimedia Subsystem (IMS) [1], which is a fairly complex beast. Each component of IMS can have very different requirements on the computing resources. If we try to capture everything in terms of flavors the list of flavors can grow very quickly and still be specific to one single application. There¹s also many more apps to deploy. Agree, one can say, just round to the best matching flavor! Will work, but not the most efficient solution (a set of 4-5 global flavors will not provide the best fitting model for every VM we need to spawn). For such applications a flavor is not the lowest level of granularity. RAM, CPU, Disk is. Hence the question. In OpenStack, tenants are bound to think in terms flavors. And if this model is the lowest level of granularity, then dynamic creation of flavors actually supports this model and allows non-trivial applications to use flavors (I guess this is why this question had been raised last year by NSN). But, there are some issues related to this :) and these issues I have written down in my first mail. Dimitri [1] http://en.wikipedia.org/wiki/IP_Multimedia_Subsystem On 05/05/14 17:23, Solly Ross sr...@redhat.com wrote: Just to expand a bit on this, flavors are supposed to be the lowest level of granularity, and the general idea is to round to the nearest flavor (so if you have a VM that requires 3GB of RAM, go with a 4GB flavor). Hence, in my mind, it doesn't make any sense to create flavors on the fly; you should have enough flavors to suit your needs, but I can't really think of a situation where you'd need so much granularity that you'd need to create new flavors on the fly (assuming, of course, that you planned ahead and created enough flavors that you don't have VMs that are extremely over-allocated). Best Regards, Solly Ross - Original Message - From: Serg Melikyan smelik...@mirantis.com To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Sent: Monday, May 5, 2014 2:18:21 AM Subject: Re: [openstack-dev] [Nova][Heat] Custom Nova Flavor creation through Heat (pt.2) Having project-scoped flavors will rid us of the identified issues, and will allow a more fine-grained way of managing physical resources. Flavors concept was introduced in clouds to solve issue with effective physical resource usage: 8Gb physical memory can be effectively splitted to two m2.my_small with 4Gb RAM or to eight m1.my_tiny with 1 GB. Let's consider example when your cloud have only 2 compute nodes with 8GB RAM: vm1 (m1.my_tiny) - node1 vm2 (m1.my_tiny) - node1 vm3 (m2.my_small) - node1 vm4 (m2.my_small) - node2 (since we could not spawn on node1) This leaves ability to spawn predictable 2 VMs with m1.my_tiny flavor on node1, and 2 VMs m1.my_tiny or 1 VM m2.my_small on node2. If user has ability to create any flavor that he likes, he can create flavor like mx.my_flavor with 3GB of RAM that could not be spawned on node1 at all, and leaves 1GB under-used on node2 when spawned there. If you will multiply number of nodes to 100 or even 1000 (like some of the OpenStack deployments) you will have very big memory under-usage. Do we really need to have ability to allocate any number of physical resources for VM? If yes, I suggest to make two ways to define number of physical resource allocation for VMs: with flavors and dynamically. Leaving to cloud admins/owners to decide which way they prefer they cloud resources to be allocated. Since small clouds may prefer flavors, and big clouds may have dynamic resource allocation (impact from under-usage will be not so big). As transition plan project-scoped flavors may do the job. On Fri, May 2, 2014 at 5:35 PM, Dimitri Mazmanov dimitri.mazma...@ericsson.com wrote: This topic has already been discussed last year and a use-case was described (see [1]). Here's a Heat blueprint for a new OS::Nova::Flavor resource: [2]. Several issues have been brought up after posting my implementation for review [3
Re: [openstack-dev] [Nova] [Heat] Custom Nova Flavor creation through Heat (pt.2)
For your first question, I'll probably create a BP sometime today. For your second question, allowing tenants to create flavors prevents one of the main parts of the flavor idea from working -- having flavors that nicely fit together to prevent wasted host resources. For instance suppose the normal system flavors used memory in powers of 2GB (2, 4, 8, 16, 32). Now suppose someone came in, created a private flavor that used 3GB of RAM. We now have 1GB of RAM that can never be used, unless someone decides to come along and create a 1GB flavor (actually, since RAM has even more granularity than that, you could have someone specify that they wanted 1.34GB of RAM, for instance, and then you have all sorts of weird stuff going on). Best Regards, Solly Ross - Original Message - From: Dimitri Mazmanov dimitri.mazma...@ericsson.com To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Sent: Monday, May 5, 2014 3:40:08 PM Subject: Re: [openstack-dev] [Nova] [Heat] Custom Nova Flavor creation through Heat (pt.2) This is good! Is there a blueprint describing this idea? Or any plans describing it in a blueprint? Would happily share the work. Should we mix it with flavors in horizon though? I¹m thinking of having a separate ³Resources² page, wherein the user can ³define² resources. I¹m not a UX expert though. But let me come back to the project-scoped flavor creation issues. Why do you think it¹s such a bad idea to let tenants create flavors for their project specific needs? I¹ll refer again to the Steve Hardy¹s proposal: - Normal user : Can create a private flavor in a tenant where they have the Member role (invisible to any other users) - Tenant Admin user : Can create public flavors in the tenants where they have the admin role (visible to all users in the tenant) - Domain admin user : Can create public flavors in the domains where they have the admin role (visible to all users in all tenants in that domain) If you actually have 64 flavors, though, and it's overwhelming your users, ... The users won¹t see all 64 flavor, only those they have defined and public. - Dimitri On 05/05/14 20:18, Chris Friesen chris.frie...@windriver.com wrote: On 05/05/2014 11:40 AM, Solly Ross wrote: One thing that I was discussing with @jaypipes and @dansmith over on IRC was the possibility of breaking flavors down into separate components -- i.e have a disk flavor, a CPU flavor, and a RAM flavor. This way, you still get the control of the size of your building blocks (e.g. you could restrict RAM to only 2GB, 4GB, or 16GB), but you avoid exponential flavor explosion by separating out the axes. I like this idea because it allows for greater flexibility, but I think we'd need to think carefully about how to expose it via horizon--maybe separate tabs within the overall flavors page? As a simplifying view you could keep the existing flavors which group all of them, while still allowing instances to specify each one separately if desired. Chris ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] [Heat] Custom Nova Flavor creation through Heat (pt.2)
(response inline) - Original Message - From: Zane Bitter zbit...@redhat.com To: openstack-dev@lists.openstack.org Sent: Tuesday, May 6, 2014 9:09:09 PM Subject: Re: [openstack-dev] [Nova] [Heat] Custom Nova Flavor creation through Heat (pt.2) On 05/05/14 13:40, Solly Ross wrote: One thing that I was discussing with @jaypipes and @dansmith over on IRC was the possibility of breaking flavors down into separate components -- i.e have a disk flavor, a CPU flavor, and a RAM flavor. This way, you still get the control of the size of your building blocks (e.g. you could restrict RAM to only 2GB, 4GB, or 16GB), but you avoid exponential flavor explosion by separating out the axes. Dimitry and I have discussed this on IRC already (no-one changed their mind about anything as a result), but I just wanted to note here that I think even this idea is crazy. VMs are not allocated out of a vast global pool of resources. They're allocated on actual machines that have physical hardware costing real money in fixed ratios. Here's a (very contrived) example. Say your standard compute node can support 16 VCPUs and 64GB of RAM. You can sell a bunch of flavours: maybe 1 VCPU + 4GB, 2 VCPU + 8GB, 4 VCPU + 16GB... c. But if (as an extreme example) you sell a server with 1 VCPU and 64GB of RAM you have a big problem: 15 VCPUs that nobody has paid for and you can't sell. (Disks add a new dimension of wrongness to the problem.) So the simple solution is to not allow a VM that uses all of the RAM on a given host (just don't have a RAM flavor that size), and then schedule the VM on a host that has minimal RAM usage but high CPU usage. This is especially true for disk, where you may have situations where you don't care if an otherwise large VM uses no disk (disks on network stores, etc). The insight of flavours, which is fundamental to the whole concept of IaaS, is that users must pay the *opportunity cost* of their resource usage. If you allow users to opt, at their own convenience, to pay only the actual cost of the resources they use regardless of the opportunity cost to you, then your incentives are no longer aligned with your customers. You'll initially be very popular with the kind of customers who are taking advantage of you, but you'll have to hike prices across the board to make up the cost leading to a sort of dead-sea effect. A Gresham's Law of the cloud, if you will, where bad customers drive out good customers. Simply put, a cloud allowing users to define their own flavours *loses* to one with predefined flavours 10 times out of 10. Note that this proposal wouldn't prevent flavors from being used -- you could still have flavor bundles (or something of the sort) that would act the way current flavors do. In the above example, you just tell the customer: bad luck, you want 64GB of RAM, you buy 16 VCPUs whether you want them or not. It can't actually hurt to get _more_ than you wanted, even though you'd rather not pay for it (provided, of course, that everyone else *is* paying for it, and cross-subsidising you... which they won't). Again, what if you also have a user who needs lots of CPUs, but a relatively small amount of RAM or disk? Now, it's not the OpenStack project's job to prevent operators from going bankrupt. But I think at the point where we are adding significant complexity to the project just to enable people to confirm the effectiveness of a very obviously infallible strategy for losing large amounts of money, it's time to draw a line. By the way, the whole theory behind this idea seems to be that this: nova server-create --cpu-flavor=4 --ram-flavour=16G --disk-flavor=200G minimises the cognitive load on the user, whereas this: nova server-create --flavor=4-64G-200G But, flavors aren't (and shouldn't be) named 16G. Realistically, it would look more like nova create --cpu-flavor=large --ram-flavor=medium --disk-flavor=small for many clouds. Additionally, keep in mind that not everyone uses the command line. Developers often forget that many users will want to use horizon, and selecting 4-64G-200G (or even large-medium-large) from a long list can be very annoying (suppose we have 6 RAM flavors and 6 disk flavors -- that's 36 flavors that start with 4-). will cause the user's brain to explode from its combinatorial complexity. I find this theory absurd. In other words, if you really want to lose some money, it's perfectly feasible with the existing flavour implementation. The operator is only ever 3 for-loops away from setting up every combination of flavours possible from combining the CPU, RAM and disk options, and can even apply whatever constraints they desire. All that said, Heat will expose any API that Nova implements. Choose wisely. cheers, Zane. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http
Re: [openstack-dev] [Nova] [Heat] Custom Nova Flavor creation through Heat (pt.2)
The way I was thinking this would work would be to allow flavor bundles if you will, which would allow 2 or more axes in one flavor (essentially preserving the existing functionality). Thus, if you needed NUMA, you could use those. - Original Message - From: Daniel P. Berrange berra...@redhat.com To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Sent: Thursday, May 8, 2014 5:08:32 AM Subject: Re: [openstack-dev] [Nova] [Heat] Custom Nova Flavor creation through Heat (pt.2) On Mon, May 05, 2014 at 01:40:43PM -0400, Solly Ross wrote: One thing that I was discussing with @jaypipes and @dansmith over on IRC was the possibility of breaking flavors down into separate components -- i.e have a disk flavor, a CPU flavor, and a RAM flavor. This way, you still get the control of the size of your building blocks (e.g. you could restrict RAM to only 2GB, 4GB, or 16GB), but you avoid exponential flavor explosion by separating out the axes. Splitting up flavours in that way doesn't really fly, especially for CPU RAM, because the properties you want to configure for NUMA policies cross both CPU RAM so cannot be sensibly separated. Regards, Daniel -- |: http://berrange.com -o-http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :| ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] [Heat] Custom Nova Flavor creation through Heat (pt.2)
P.S. I feel like this is something that should be discussed at the design summit. Perhaps there's an existing session this could be discussed in (the unsession, perhaps?) - Original Message - From: Solly Ross sr...@redhat.com To: Daniel P. Berrange berra...@redhat.com, OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Sent: Thursday, May 8, 2014 10:51:47 AM Subject: Re: [openstack-dev] [Nova] [Heat] Custom Nova Flavor creation through Heat (pt.2) The way I was thinking this would work would be to allow flavor bundles if you will, which would allow 2 or more axes in one flavor (essentially preserving the existing functionality). Thus, if you needed NUMA, you could use those. - Original Message - From: Daniel P. Berrange berra...@redhat.com To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Sent: Thursday, May 8, 2014 5:08:32 AM Subject: Re: [openstack-dev] [Nova] [Heat] Custom Nova Flavor creation through Heat (pt.2) On Mon, May 05, 2014 at 01:40:43PM -0400, Solly Ross wrote: One thing that I was discussing with @jaypipes and @dansmith over on IRC was the possibility of breaking flavors down into separate components -- i.e have a disk flavor, a CPU flavor, and a RAM flavor. This way, you still get the control of the size of your building blocks (e.g. you could restrict RAM to only 2GB, 4GB, or 16GB), but you avoid exponential flavor explosion by separating out the axes. Splitting up flavours in that way doesn't really fly, especially for CPU RAM, because the properties you want to configure for NUMA policies cross both CPU RAM so cannot be sensibly separated. Regards, Daniel -- |: http://berrange.com -o-http://www.flickr.com/photos/dberrange/ |:| |: http://libvirt.org -o- http://virt-manager.org |:| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ |:| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc |:| ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Extending nova models
Actually, that line you linked to about IMPL is a bit misleading. In this case, under the hood IMPL is really just the sqlalchemy implementation of the Nova db api at https://github.com/openstack/nova/blob/master/nova/db/sqlalchemy/api.py#L2404. In Nova we don't insert a row in to a table per se. Instead (in general), we have sqlalchemy models which are then saved (which handles inserting, etc). So the path is api wrapper (the file you linked to) -- sqlalchemy api implementation -- sqlalchemy model. Hope this helps. Best Regards, Solly Ross - Original Message - From: Rad Gruchalski ra...@gruchalski.com To: openstack-dev@lists.openstack.org Sent: Wednesday, May 21, 2014 3:38:04 PM Subject: [openstack-dev] Extending nova models Hi everyone, This is my first question here. I hope I could get an answer for the problem I'm currently facing in the development of a nova API extension. I am trying to add a couple of API endpoints that would serve as an interface to the table storing some data. I was able to create an API endpoint by placing my extension in api/openstack/compute/contrib and modifying the policy.json file. This is now working. I then added the migration to create a table to nova/db/sqlalchemy/migrate_repo_versions/245_add_custom_table. After unstack.sh and stack.sh (I'm using devstack) I can see my table being created. Great. Next, I proceeded with creating an object definition and created a file in nova/objects. I am basing myself on keypairs.py example (https://github.com/openstack/nova/blob/2efd3faa3e07fdf16c2d91c16462e7e1e3f33e17/nova/api/openstack/compute/contrib/keypairs.py#L97) self.api.create_key_pair calls this https://github.com/openstack/nova/blob/839fe777e256d36e69e9fd7c571aed2c860b122c/nova/compute/api.py#L3512 the important part is keypair = keypair_obj.KeyPair() keypair.user_id = user_id keypair.name = key_name keypair.fingerprint = fingerprint keypair.public_key = public_key keypair.create(context) `KeyPair()` is https://github.com/openstack/nova/blob/master/nova/objects/keypair.py this has a method https://github.com/openstack/nova/blob/master/nova/objects/keypair.py#L52 and it's calling `db_keypair = db.key_pair_create(context, updates)` `db` points to `from nova import db` which I believe points to this https://github.com/openstack/nova/blob/master/nova/db/__init__.py which loads https://github.com/openstack/nova/blob/master/nova/db/api.py there's a function called https://github.com/openstack/nova/blob/master/nova/db/api.py#L922 `key_pair_create` https://github.com/openstack/nova/blob/master/nova/db/api.py#L924 `IMPL` is https://github.com/openstack/nova/blob/master/nova/db/api.py#L69-L95 but where is `IMPL.key_pair_create`? Is there an easy way to insert a record into the table? Thank you for any pointers. I’ve posted the same question on ask.openstack.org (https://ask.openstack.org/en/questions/30231). Kind regards, Radek Gruchalski ra...@gruchalski.com de.linkedin.com/in/radgruchalski/ +4917685656526 Confidentiality: This communication is intended for the above-named person and may be confidential and/or legally privileged. If it has come to you in error you must take no action based on it, nor must you copy or show it to anyone; please delete/destroy and inform the sender immediately. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Proposal: Move CPU and memory allocation ratio out of scheduler
Response inline - Original Message - From: Alex Glikson glik...@il.ibm.com To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Sent: Monday, June 9, 2014 3:13:52 PM Subject: Re: [openstack-dev] [nova] Proposal: Move CPU and memory allocation ratio out of scheduler So maybe the problem isn’t having the flavors so much, but in how the user currently has to specific an exact match from that list. If the user could say “I want a flavor with these attributes” and then the system would find a “best match” based on criteria set by the cloud admin then would that be a more user friendly solution ? Interesting idea.. Thoughts how this can be achieved? Well, that is *essentially* what a scheduler does -- you give it a set of parameters and it finds a chunk of resources (in this case, a flavor) to match those features. I'm *not* suggesting that we reuse any scheduling code, it's just one way to think about it. Another way to think about it would be to produce a distance score and choose the flavor with the smallest distance, discounting flavors that couldn't fit the target configuration. The distance score would simply be a sum of distances between the individual resources for the target and flavor. Best Regards, Solly Ross Alex From: Day, Phil philip@hp.com To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org, Date: 06/06/2014 12:38 PM Subject: Re: [openstack-dev] [nova] Proposal: Move CPU and memory allocation ratio out of scheduler From: Scott Devoid [ mailto:dev...@anl.gov ] Sent: 04 June 2014 17:36 To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [nova] Proposal: Move CPU and memory allocation ratio out of scheduler Not only live upgrades but also dynamic reconfiguration. Overcommitting affects the quality of service delivered to the cloud user. In this situation in particular, as in many situations in general, I think we want to enable the service provider to offer multiple qualities of service. That is, enable the cloud provider to offer a selectable level of overcommit. A given instance would be placed in a pool that is dedicated to the relevant level of overcommit (or, possibly, a better pool if the selected one is currently full). Ideally the pool sizes would be dynamic. That's the dynamic reconfiguration I mentioned preparing for. +1 This is exactly the situation I'm in as an operator. You can do different levels of overcommit with host-aggregates and different flavors, but this has several drawbacks: 1. The nature of this is slightly exposed to the end-user, through extra-specs and the fact that two flavors cannot have the same name. One scenario we have is that we want to be able to document our flavor names--what each name means, but we want to provide different QoS standards for different projects. Since flavor names must be unique, we have to create different flavors for different levels of service. Sometimes you do want to lie to your users! [Day, Phil] I agree that there is a problem with having every new option we add in extra_specs leading to a new set of flavors. There are a number of changes up for review to expose more hypervisor capabilities via extra_specs that also have this potential problem. What I’d really like to be able to ask for a s a user is something like “a medium instance with a side order of overcommit”, rather than have to choose from a long list of variations. I did spend some time trying to think of a more elegant solution – but as the user wants to know what combinations are available it pretty much comes down to needing that full list of combinations somewhere. So maybe the problem isn’t having the flavors so much, but in how the user currently has to specific an exact match from that list. If the user could say “I want a flavor with these attributes” and then the system would find a “best match” based on criteria set by the cloud admin (for example I might or might not want to allow a request for an overcommitted instance to use my not-overcommited flavor depending on the roles of the tenant) then would that be a more user friendly solution ? 2. If I have two pools of nova-compute HVs with different overcommit settings, I have to manage the pool sizes manually. Even if I use puppet to change the config and flip an instance into a different pool, that requires me to restart nova-compute. Not an ideal situation. [Day, Phil] If the pools are aggregates, and the overcommit is defined by aggregate meta-data then I don’t see why you need to restart nova-compute. 3. If I want to do anything complicated, like 3 overcommit tiers with good, better, best performance and allow the scheduler to pick better for a good instance if the good pool is full, this is very hard and complicated to do with the current system. [Day
Re: [openstack-dev] [Nova][devstack] Unable to boot cirros-0.3.2-x86_64-uec image
Hi Deepak, This mailing list is for development discussions only. Please use the general or operators mailing list (http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators) to ask questions about running OpenStack. Best Regards, Solly Ross - Original Message - From: Deepak Shetty dpkshe...@gmail.com To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Sent: Wednesday, June 11, 2014 3:49:01 AM Subject: [openstack-dev] [Nova][devstack] Unable to boot cirros-0.3.2-x86_64-uec image Hi, I am using the below cmd to boot cirros-0.3.2-x86_64-uec image thats present in devstack by default... nova boot --flavor m1.nano --image cirros-0.3.2-x86_64-uec --key_name mykey --security_group default myvm_nano nova list - shows the instance as ACTIVE/Running Taking the VNC console, I see that it stuck at Booting from ROM . Can someone help why the image is not booting ? thanx, deepak ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [horizon]Blueprint- showing a small message to the user for browser incompatibility
I'm not sure User Agent detection is the best way to go. Suppose you do UA sniffing and say show the message unless the UA is one of X. Then, if there's a browser which fully supports your feature set, but doesn't have a known UA (or someone set a custom UA on their browser), the message will still show, which could be confusing to users. On the other hand, if you do UA sniffing and say show the message if the UA is one of X, then a browser that didn't support the features, but didn't have a matching User Agent, wouldn't show the message. If you go with User Agent sniffing, I'd say the latter way (a blacklist) preferable, since it's probably easier to come up with currently unsupported browsers than it is to predict future browsers. Have you identified which specific browser features are needed? You could use Modernizr and then warn if the requisite feature set it not implemented. This way, you simply check for the feature set required. Best Regards, Solly Ross - Original Message - From: Nikunj Aggarwal nikunj.aggar...@hp.com To: openstack-dev@lists.openstack.org Sent: Tuesday, October 14, 2014 4:53:42 AM Subject: [openstack-dev] [horizon]Blueprint- showing a small message to the user for browser incompatibility Hi Everyone, I have submitted a blueprint which targets the issues end-users are faces when they are using Horizon in the old browsers. So, this blueprint targets to overcome this problem by showing a small message on the Horizon login page. I urge to all the Horizon community to take a look and share your views. https://blueprints.launchpad.net/horizon/+spec/detecting-browser Regards, Nikunj ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [horizon]Blueprint- showing a small message to the user for browser incompatibility
Sure, feel free to put my response in the Blueprint page. Thanks for the quick answer. Best Regards, Solly Ross - Original Message - From: Nikunj Aggarwal nikunj.aggar...@hp.com To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Cc: sr...@redhat.com Sent: Tuesday, October 14, 2014 12:30:13 PM Subject: RE: [openstack-dev] [horizon]Blueprint- showing a small message to the user for browser incompatibility Hi Solly, You are right with your questions about user setting custom UA on their browser. And during my discussion with other Horizon community members on IRC, we decided that Horizon should not care about user setting custom UA on for their browser because it is not our job to identify that and fix. If user is setting custom UA for their browser which means they want that irrespective of the outcome. Also we discussed about using Modernizr and also implementing graceful degradation but this will work for bigger features like canvas or svg but smaller css feature where it is breaking in older versions of IE like IE 9, will be a huge change and I personally think will be waste of resources and will make the code more complicated. Instead Horizon guys came to an conclusion that we will identify the browser type and version to deal with legacy browsers like older IE or firefox or any other browser and for other major features we can use feature detection with Modernizr. We are targeting all major browsers which are listed in this page - https://wiki.openstack.org/wiki/Horizon/BrowserSupport And I also think by going with this approach we will minimize the code complexity and also this small feature will greatly improve the UX experience for the end-users. Thank you for the reply and your views. Also can I put the contents of your reply as a comment into the Blueprint page? Because I think many others will also have same kind of questions. Thanks Regards, Nikunj -Original Message- From: Solly Ross [mailto:sr...@redhat.com] Sent: Tuesday, October 14, 2014 9:37 PM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [horizon]Blueprint- showing a small message to the user for browser incompatibility I'm not sure User Agent detection is the best way to go. Suppose you do UA sniffing and say show the message unless the UA is one of X. Then, if there's a browser which fully supports your feature set, but doesn't have a known UA (or someone set a custom UA on their browser), the message will still show, which could be confusing to users. On the other hand, if you do UA sniffing and say show the message if the UA is one of X, then a browser that didn't support the features, but didn't have a matching User Agent, wouldn't show the message. If you go with User Agent sniffing, I'd say the latter way (a blacklist) preferable, since it's probably easier to come up with currently unsupported browsers than it is to predict future browsers. Have you identified which specific browser features are needed? You could use Modernizr and then warn if the requisite feature set it not implemented. This way, you simply check for the feature set required. Best Regards, Solly Ross - Original Message - From: Nikunj Aggarwal nikunj.aggar...@hp.com To: openstack-dev@lists.openstack.org Sent: Tuesday, October 14, 2014 4:53:42 AM Subject: [openstack-dev] [horizon]Blueprint- showing a small message to the user for browser incompatibility Hi Everyone, I have submitted a blueprint which targets the issues end-users are faces when they are using Horizon in the old browsers. So, this blueprint targets to overcome this problem by showing a small message on the Horizon login page. I urge to all the Horizon community to take a look and share your views. https://blueprints.launchpad.net/horizon/+spec/detecting-browser Regards, Nikunj ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Any ideas on why nova-novncproxy is failing to start on devstack?
For future reference, it looks like you had an old version of websockify installed that wasn't getting updated, for some reason. Nova recently accepted a patch that removed the support for the old version of websockify Best Regards, Solly Ross - Original Message - From: Paul Michali (pcm) p...@cisco.com To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Sent: Friday, October 10, 2014 9:40:54 PM Subject: Re: [openstack-dev] Any ideas on why nova-novncproxy is failing to start on devstack? Well, I deleted /usr/local/lib/python2.7/dist-packages/websockify* and then stacked and it worked! PCM (Paul Michali) MAIL …..…. p...@cisco.com IRC ……..… pcm_ (irc.freenode.com) TW ………... @pmichali GPG Key … 4525ECC253E31A83 Fingerprint .. 307A 96BB 1A4C D2C7 931D 8D2D 4525 ECC2 53E3 1A83 On Oct 10, 2014, at 8:46 PM, Paul Michali (pcm) p...@cisco.com wrote: I had a system with devstack, which I restack with reclone, with the intention of then patching in my review diffs to update and test. Well, the stack is failing in n-nonvc, with this message: openstack@devstack-33:~/devstack$ /usr/local/bin/nova-novncproxy --config-file /etc/nova/nova.conf --web /opt/stack/noVNC echo $! /opt/stack/status/stack/n-novnc.pid; fg || echo n-novnc failed to start | tee /opt/stack/status/stack/n-novnc.failure [1] 826 /usr/local/bin/nova-novncproxy --config-file /etc/nova/nova.conf --web /opt/stack/noVNC Traceback (most recent call last): File /usr/local/bin/nova-novncproxy, line 6, in module from nova.cmd.novncproxy import main File /opt/stack/nova/nova/cmd/novncproxy.py, line 29, in module from nova.console import websocketproxy File /opt/stack/nova/nova/console/websocketproxy.py, line 110, in module websockify.ProxyRequestHandler): AttributeError: 'module' object has no attribute 'ProxyRequestHandler' n-novnc failed to start The websockify package is installed: openstack@devstack-33:~/devstack$ pip show websockify --- Name: websockify Version: 0.5.1 Location: /usr/local/lib/python2.7/dist-packages Requires: numpy However, the version required is: openstack@devstack-33:/opt/stack/nova$ grep websockify requirements.txt websockify=0.6.0,0.7 Any ideas why is does not have the right version and how to best correct? Thanks! PCM (Paul Michali) MAIL …..…. p...@cisco.com IRC ……..… pcm_ (irc.freenode.com) TW ………... @pmichali GPG Key … 4525ECC253E31A83 Fingerprint .. 307A 96BB 1A4C D2C7 931D 8D2D 4525 ECC2 53E3 1A83 ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Glance][Artifacts] Glance Artifacts Repository as a standalone service
Just my two cents, since I won't be able to make it to summit: When the artifact repository was proposed, I personally really liked the idea that images were just another artifact type eventually, even if they stayed separate for the time being. However, the pros that you bring up do seem to make a lot of sense -- being able to design an API from scratch can lead to nicer APIs, and having the ability to use different backends for images vs artifacts could be quite useful from a performance perspective. Thus, let me propose this: what if you allowed different pools of artifacts to be housed on different backends and/or endpoints? This way, you could still put images in their own little bubble without losing the image -- artifact abstraction. It would also allow you to extend some of the pros of the split to other artifact types, since a cloud admin could have other artifacts besides images that needed to be in their own little bubble for performance reasons. For instance, a cloud administrator could define three pools (for lack of a better term ATM): images, general, and critical-data, images and critical-data might be stored on specially-tuned high-performance backends, while critical-data might be on a large general-purpose backend. Best Regards, Solly Ross - Original Message - From: Alexander Tivelkov ativel...@mirantis.com To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Sent: Monday, October 27, 2014 8:08:47 AM Subject: [openstack-dev] [Glance][Artifacts] Glance Artifacts Repository as a standalone service Hi stackers, It has been some time since the announcement of Artifacts initiative within the Glance. The feature was not complete in Juno, but is being actively developed now and has good chances for landing in Kilo. However, recently on the Glance Virtual Mini-summit we had a discussions which lead to an idea to change one of the core design concepts of the Glance Artifact repository [1] Initially we planned to run Artifact repository as part of existing Glance service(s): all the APIs to handle artifacts were supposed to be hosted by glance-api service, with glance-registry as optional backend. Artifact-related endpoints were designed to be in the /v2/ branch of the API hierarchy, and were supposed to be similar in syntax and semantics to the existing v2/images endpoints. Now it is suggested to host artifacts as a standalone process listening to a different port (and probably deployed on a different host) and registered in the keystone as a separate service type. The code will be still part of the primary Glance repo - so this is not the question of starting another project or program, this is just about having a new daemon in the openstack deployment. This approach has some obvious pros and some less-obvious cons. Most important is the ability to load-balance images and artifacts independenly, being sure that any load on artifacts repo does no affect the performance of images - and vice versa. Also, this will allow us to provide different configuration options (including different backing storages) to these different components which will increase the overall flexibility and customizability of the system. We'll be able to design the artifacts API from scratch without the need to comply with the existing semantics and architecture of images-part, reusing only the components which are really needed. At the same time we'll loose the transparency between the concepts of image and artifact: initially we planned to make them very similar, so when we are finally ready to make images just one of the available artifact types, the migration may be trivial. If we now separate them into different services, we draw a strict border and potentially complicate the migration. IMHO, the pros outweigh the cons, so I personally like the idea of service separation - and all the participants of our virtual summit seemed to share the same opinion. However, it is a serious design change, so I'd like to hear the opinions of the wider audience. We have planned a design session in Paris ([2]) to discuss this change in more details (the topic is applicable not only to Artifacta, but to other initiatives under the hood of Glance as well, e.g. metadef catalog, index service etc) - please feel free to join and discuss. And please do not hesitate to share any concerns in the mailing list before the summit starts: the more opinions we have, the better solution we will make. [1] https://etherpad.openstack.org/p/kilo-glance-artifacts-current-state-of-development [2] https://etherpad.openstack.org/p/kilo-glance-summit-topics-final -- Regards, Alexander Tivelkov ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] How to connect to a serial port of an instance via websocket?
You should be able to connect like a normal WebSocket, assuming you're running
the serial console websocketproxy (it's a different command from the VNC web
socket proxy). If you want, you can ping
on IRC and I can help you debug your JS code.
Best Regards,
Solly Ross
(directxman12 on freenode IRC)
- Original Message -
From: Markus Zoeller mzoel...@de.ibm.com
To: openstack-dev@lists.openstack.org
Sent: Tuesday, October 28, 2014 10:09:44 AM
Subject: [openstack-dev] [nova] How to connect to a serial port of an
instance via websocket?
The API provides an endpoint for querying the serial console of an
instance ('os-getSerialConsole'). The nova-client interacts with this
API endpoint via the command `get-serial-console`.
nova get-serial-console myInstance
It returns a string like:
ws://127.0.0.1:6083/?token=e2b42240-375d-41fe-a166-367e4bbdce35
Q: How is one supposed to connect to such a websocket?
[1]
https://github.com/openstack/nova/blob/master/nova/api/openstack/compute/contrib/consoles.py#L111
[2]
https://ask.openstack.org/en/question/50671/how-to-connect-to-a-serial-port-of-an-instance-via-websocket/
Regards,
Markus Zoeller
IRC: markus_z
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] How to connect to a serial port of an instance via websocket?
The serial proxy will not automatically start. It's intended to be started as a service, just the like API, VNC proxy, or SPICE proxy. Best Regards, Solly Ross - Original Message - From: Markus Zoeller mzoel...@de.ibm.com To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Sent: Thursday, October 30, 2014 9:45:08 AM Subject: Re: [openstack-dev] [nova] How to connect to a serial port of an instance via websocket? The cause of this is that the serialproxy was not started. So nobody was listening to the port 6083. Validate with: $ netstat -nat | grep :608 tcp 0 0 0.0.0.0:6080 0.0.0.0:*LISTEN tcp 0 0 0.0.0.0:6081 0.0.0.0:*LISTEN tcp 0 0 192.168.122.41:60858 192.168.122.41:5672 ESTABLISHED tcp 0 0 192.168.122.41:60859 192.168.122.41:5672 ESTABLISHED tcp6 0 0 192.168.122.41:5672 192.168.122.41:60858 ESTABLISHED tcp6 0 0 192.168.122.41:5672 192.168.122.41:60859 ESTABLISHED After finding [1] all I had to do was to start this proxy manually with: $ nova-serialproxy INFO nova.console.websocketproxy [-] WebSocket server settings: INFO nova.console.websocketproxy [-] - Listen on 0.0.0.0:6083 INFO nova.console.websocketproxy [-] - Flash security policy server INFO nova.console.websocketproxy [-] - No SSL/TLS support (no cert file) INFO nova.console.websocketproxy [-] - proxying from 0.0.0.0:6083 to None:None After executing this command, the `netstat` command from above shows a listener for port 6083: $ netstat -nat | grep :608 tcp 0 0 0.0.0.0:6080 0.0.0.0:*LISTEN tcp 0 0 0.0.0.0:6081 0.0.0.0:*LISTEN tcp 0 0 0.0.0.0:6083 0.0.0.0:*LISTEN tcp 0 0 192.168.122.41:60858 192.168.122.41:5672 ESTABLISHED tcp 0 0 192.168.122.41:60859 192.168.122.41:5672 ESTABLISHED tcp6 0 0 192.168.122.41:5672 192.168.122.41:60858 ESTABLISHED tcp6 0 0 192.168.122.41:5672 192.168.122.41:60859 ESTABLISHED By using Sahids websocketclient and the URI I got from the command `nova get-serial-console instance1` the connection gets established and one will see the login screen (e.g. from cirros). I was expecting the nova-serialproxy to start automatically when the section [serial_console] has the entry `enabled=True`. Is this a bug or do I have a wrong assumption here? Thanks again Sahid for your help! Thanks to Solly too, for offering JS help! [1] OpenStack Nova Developer Docs; Websocket serial Proxy for OpenStack Nova serial ports. ; http://docs.openstack.org/developer/nova/man/nova-serialproxy.html Markus Zoeller/Germany/IBM wrote on 10/30/2014 11:29:22 AM: From: Markus Zoeller/Germany/IBM To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Date: 10/30/2014 11:29 AM Subject: [nova] How to connect to a serial port of an instance via websocket? On Wed Oct 29 09:42:52 UTC 2014, Sahid Orentino Ferdjaoui wrote: The aim of the feature is exposing an interactive web-based serial consoles through a websocket proxy. The API returns an URL with a valid token that should be used with a websocket client to read/write on the stream. Considering the service nova-serialproxy is running and well configured you can use this simple test purpose client to connect yourself on the URL returned by the API: https://gist.github.com/sahid/894c31f306bebacb2207 The general idea behind this service is for example to help debugging VMs when something was wrong with the network configuration. Hi Sahid, thanks for your great example! When I execute it I get the error socket.error: [Errno 111] Connection refused How do I debug this? Did I miss a configuration? ./lazyclient.py ws://127.0.0.1:6083/?token=39262891-2588-4872-994b-3be9b7333fd7 Traceback (most recent call last): File ./lazyclient.py, line 27, in module ws.connect() File /usr/local/lib/python2.7/dist-packages/ws4py/client/ __init__.py, line 209, in connect self.sock.connect(self.bind_addr) File /usr/lib/python2.7/socket.py, line 224, in meth return getattr(self._sock,name)(*args) socket.error: [Errno 111] Connection refused ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [oslo] Graduating oslo-incubator/reports
Hello Doug et all, I would like to get the ball rolling on graduating oslo-incubator/reports into oslo.reports. What do I have to do on my end to move forward with the graduation process? Best Regards, Solly Ross ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [OpenStack-dev][Nova] Migration stuck - resize/migrating
Indeed. Ensure you have SSH access between compute nodes (I'm working on some code to remove this requirement, but it may be a while before it gets merged). Also, if you can, could you post logs somewhere with the 'debug' config option enabled? I might be able to spot something quickly, since I've been working on the related code recently. Best Regards, Solly Ross - Original Message - From: Vishvananda Ishaya vishvana...@gmail.com To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Sent: Tuesday, November 18, 2014 4:07:31 PM Subject: Re: [openstack-dev] [OpenStack-dev][Nova] Migration stuck - resize/migrating Migrate/resize uses scp to copy files back and forth with the libvirt driver. This shouldn’t be necessary with shared storage, but it may still need ssh configured between the user that nova is running as in order to complete the migration. It is also possible that there is a bug in the code path dealing with shared storage, although I would have expected you to see a traceback somewhere. Vish On Nov 11, 2014, at 1:10 AM, Eduard Matei eduard.ma...@cloudfounders.com wrote: Hi, I'm testing our cinder volume driver in the following setup: - 2 nodes, ubuntu, devstack juno (2014.2.1) - shared storage (common backend), our custom software solution + cinder volume on shared storage - 1 instance running on node 1, /instances directory on shared storage - kvm, libvirt (with live migration flags) Live migration of instance between nodes works perfectly. Migrate simply blocks. The instance in in status Resize/Migrate, no errors in n-cpu or n-sch, and it stays like that for over 8 hours (all night). I thought it was copying the disk, but it's a 20GB sparse file with approx. 200 mb of data, and the nodes have 1Gbps link, so it should be a couple of seconds. Any difference between live migration and migration? As i said, we use a shared filesystem-like storage solution so the volume files and the instance files are visible on both nodes, so no data needs copying. I know it's tricky to debug since we use a custom cinder driver, but anyone has any ideas where to start looking? Thanks, Eduard -- Eduard Biceri Matei, Senior Software Developer www.cloudfounders.com | eduard.ma...@cloudfounders.com CloudFounders, The Private Cloud Software Company Disclaimer: This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you are not the named addressee or an employee or agent responsible for delivering this message to the named addressee, you are hereby notified that you are not authorized to read, print, retain, copy or disseminate this message or any part of it. If you have received this email in error we request you to notify us by reply e-mail and to delete all electronic files of the message. If you are not the intended recipient you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited. E-mail transmission cannot be guaranteed to be secure or error free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or contain viruses. The sender therefore does not accept liability for any errors or omissions in the content of this message, and shall have no liability for any loss or damage suffered by the user, which arise as a result of e-mail transmission. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Proposal new hacking rules
Whoops, that should say assertions not exceptions. - Original Message - From: Solly Ross sr...@redhat.com To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Sent: Monday, November 24, 2014 12:00:44 AM Subject: Re: [openstack-dev] [nova] Proposal new hacking rules Well, at least the message for exceptions in Nova says expected and observed. I suspect that it's part of our custom test case classes. Best Regards, Solly Ross - Original Message - From: Matthew Treinish mtrein...@kortar.org To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Sent: Friday, November 21, 2014 5:23:28 PM Subject: Re: [openstack-dev] [nova] Proposal new hacking rules On Fri, Nov 21, 2014 at 04:15:07PM -0500, Sean Dague wrote: On 11/21/2014 01:52 PM, Matthew Treinish wrote: On Fri, Nov 21, 2014 at 07:15:49PM +0100, jordan pittier wrote: Hey, I am not a Nova developer but I still have an opinion. Using boolean assertions I like what you propose. We should use and enforce the assert* that best matches the intention. It's about semantic and the more precise we are, the better. Using same order of arguments in equality assertions Why not. But I don't know how we can write a Hacking rule for this. So you may fix all the occurrences for this now, but it might get back in the future. Ok I'll bite, besides the enforceability issue which you pointed out, it just doesn't make any sense, you're asserting 2 things are equal: (A == B) == (B == A) and I honestly feel that it goes beyond nitpicking because of that. It's also a fallacy that there will always be an observed value and an expected value. For example: self.assertEqual(method_a(), method_b()) Which one is observed and which one is expected? I think this proposal is just reading into the parameter names a bit too much. If you are using assertEqual with 2 variable values... you are doing your test wrong. I was originally in your camp. But honestly, the error message provided to the user does say expected and observed, and teaching everyone that you have to ignore the error message is a much harder thing to do than flip the code to conform to it, creating less confusion. Uhm, no it doesn't, the default error message is A != B. [1][2][3] (both with unittest and testtools) If the error message was like that, then sure saying one way was right over the other would be fine, (assuming you didn't specify a different error message) but, that's not what it does. [1] https://github.com/testing-cabal/testtools/blob/master/testtools/testcase.py#L340 [2] https://github.com/testing-cabal/testtools/blob/master/testtools/matchers/_basic.py#L85 [3] https://hg.python.org/cpython/file/301d62ef5c0b/Lib/unittest/case.py#l508 ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Proposal new hacking rules
Well, at least the message for exceptions in Nova says expected and observed. I suspect that it's part of our custom test case classes. Best Regards, Solly Ross - Original Message - From: Matthew Treinish mtrein...@kortar.org To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Sent: Friday, November 21, 2014 5:23:28 PM Subject: Re: [openstack-dev] [nova] Proposal new hacking rules On Fri, Nov 21, 2014 at 04:15:07PM -0500, Sean Dague wrote: On 11/21/2014 01:52 PM, Matthew Treinish wrote: On Fri, Nov 21, 2014 at 07:15:49PM +0100, jordan pittier wrote: Hey, I am not a Nova developer but I still have an opinion. Using boolean assertions I like what you propose. We should use and enforce the assert* that best matches the intention. It's about semantic and the more precise we are, the better. Using same order of arguments in equality assertions Why not. But I don't know how we can write a Hacking rule for this. So you may fix all the occurrences for this now, but it might get back in the future. Ok I'll bite, besides the enforceability issue which you pointed out, it just doesn't make any sense, you're asserting 2 things are equal: (A == B) == (B == A) and I honestly feel that it goes beyond nitpicking because of that. It's also a fallacy that there will always be an observed value and an expected value. For example: self.assertEqual(method_a(), method_b()) Which one is observed and which one is expected? I think this proposal is just reading into the parameter names a bit too much. If you are using assertEqual with 2 variable values... you are doing your test wrong. I was originally in your camp. But honestly, the error message provided to the user does say expected and observed, and teaching everyone that you have to ignore the error message is a much harder thing to do than flip the code to conform to it, creating less confusion. Uhm, no it doesn't, the default error message is A != B. [1][2][3] (both with unittest and testtools) If the error message was like that, then sure saying one way was right over the other would be fine, (assuming you didn't specify a different error message) but, that's not what it does. [1] https://github.com/testing-cabal/testtools/blob/master/testtools/testcase.py#L340 [2] https://github.com/testing-cabal/testtools/blob/master/testtools/matchers/_basic.py#L85 [3] https://hg.python.org/cpython/file/301d62ef5c0b/Lib/unittest/case.py#l508 ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][glance] why do image properties control per-instance settings?
Hi Blairo, Like you mention, I think one of the major reasons for have properties set at the image level is that certain properties depend on an OS which support the features involved. In this regard, being able to say that an image supports a particular feature, and then being able to set it on a per instance basis, would be useful. On the other hand, having properties set at the image level makes sense, since you could configure the OS to support or require the feature in question, and then attach that feature to the image so that it was always there for instances booted with that image. Have properties set at the image level also aligns with the general idea of not specifying too many special things about a VM at boot time -- you specify a flavor, an image, and an SSH key pair (or use the default). Instead of having to say what's the appropriate boot setup for the XYZ app, you just say use the XYZ image and you're all set (although this could be an argument for using Heat templates as well). Best Regards, Solly Ross - Original Message - From: Blair Bethwaite blair.bethwa...@gmail.com To: openstack-dev@lists.openstack.org Sent: Tuesday, November 25, 2014 5:15:37 AM Subject: [openstack-dev] [nova][glance] why do image properties control per-instance settings? Hi all, I've just been doing some user consultation and pondering a case for use of the Qemu Guest Agent in order to get quiesced backups. In doing so I found myself wondering why on earth I need to set an image property in Glance (hw_qemu_guest_agent) to toggle such a thing for any particular instance, it doesn't make any sense that what should be an instance boot parameter (or possibly even runtime dynamic) is controlled through the cloud's image registry. There is no shortage of similar metadata properties, probably everything prefixed hw_ for a start. It looks like this has even come up on reviews before, e.g. https://review.openstack.org/#/c/43513/ The last comment from DanielB is: For setting per-instance, rather than doing something that only works for passing kernel command line, it would be desirable to have a way to pass in arbitrary key,value attribute pairs to the 'boot' API call, because I can see this being useful for things beyond just the kernel command line. In some cases I imagine image properties could be useful to indicate that the image has a certain *capability*, which could be used as a step to verify it can support some requested feature (e.g., qemu-ga) for any particular instance launch. Is there similar work underway? Would it make sense to build such functionality via the existing instance metadata API? -- Cheers, ~Blairo ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] How should libvirt pools work with distributed storage drivers?
Hi! Some days ago, a bunch of Nova specs were approved for Kilo. Among them was https://blueprints.launchpad.net/nova/+spec/use-libvirt-storage-pools Now, while I do recognize the wisdom of using storage pools, I do see a couple of possible problems with this, especially in the light of my upcoming spec proposal for using StorPool distributed storage for the VM images. My main concern is with the explicit specification that the libvirt pools should be of the directory type, meaning that all the images should be visible as files in a single directory. Would it be possible to extend the specification to allow other libvirt pool types, or to allow other ways of pointing Nova at the filesystem path of the VM image? The specification was never intended to restrict storage pools to being file-based. In fact, it was my intention that all different types of pools be supported. The specification dedicates several paragraphs to discussing file-based pools, since transitioning from legacy file-based backends to the storage pool backend requires a bit of work, while other backends, like LVM, can simply be turned into a pool without any movement or renaming of the underlying volumes. In fact, LVM works excellently (it's one of the pool types I use frequently in testing to make sure migration works regardless of source and destination pool type). Where this is coming from is that StorPool volumes (which we intend to write a DiskImage subclass for) appear in the host filesystem as /dev/storpool/volumename special files (block devices). Thus, it would be... interesting... to find ways to make them show up under a specific directory (yes, we could do lots and lots of symlink magic, but we've been down that road before and it doesn't necessarily lead to Good Things(tm)). I see that the spec has several mentions of yeah, we should special-case Ceph/RBD here, since they do things in a different way- well, StorPool does things in a slightly different way, too :) The reason that I wrote something about Ceph/RBD is that the Ceph storage driver in libvirt is incomplete -- it doesn't yet have support for virStorageVolCreateXMLFrom, so we need to work around that. And yes, we do have work in progress to expose the StorPool cluster's volumes as a libvirt pool, but this might take a bit of time to complete and then it will most probably take much more time to get into the libvirt upstream *and* into the downstream distributions, so IMHO okay, let's use different libvirt pool types might not be entirely enough for us, although it would be a possible workaround. The intention was that new storage pool types should try to get themselves in as new libvirt storage pool drivers, and then they should just work in Nova (there is one line that needs to be modified, but other than that, you should just be able to start using them). Of course, it's entirely possible that I have not completely understood the proposed mechanism; I do see some RBD patches in the previous incarnations of this blueprint, and if I read them right, it *might* be trivial to subclass the new libvirt storage pool support thing and provide the /dev/storpool/volumename paths to the upper layers. If this is so, feel free to let me know I've wasted your time in reading this e-mail, in strong terms if necessary :) I dislike using strong terms ;-), but I do think you may have misread the spec. If you are unclear, you can catch me next week on freenode as directxman12 and we can discuss further (I'm out on PTO this week). Best Regards, Solly Ross ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Issues with IPTables
Here you go. Keep in mind that I structured them more like their own install
guide. Basic tweaks were integrated into the steps, but larger issues are
noted at the bottom under the notes section.
Best Regards,
Solly Ross
- Original Message -
From: Qing He qing...@radisys.com
To: Solly Ross sr...@redhat.com
Cc: OpenStack Development Mailing List openstack-dev@lists.openstack.org
Sent: Monday, September 16, 2013 1:37:02 PM
Subject: RE: Issues with IPTables
Solly,
It would be great if you can share the notes. The reason I asked the question
is that I'm trying to decide If I need to allocate development time in
installation following the installation guide. The usual wisdom is that
installation with detailed instruction would take no time. However, your
experience and mine showed the contrary. I have not finished mine following the
Ubuntu installation guide. Thus, I was interested in knowing your effort spent
on it so that I would know that it was not just me who had issues with the
supposedly plug and play installation with the packages.
Thanks,
Qing
-Original Message-
From: Solly Ross [mailto:sr...@redhat.com]
Sent: Monday, September 16, 2013 10:24 AM
To: Qing He
Cc: OpenStack Development Mailing List
Subject: Re: Issues with IPTables
Quite a while. RDO's documentation for configuring multinode Packstack with
Neutron was a bit lacking, so after attempting to get that working for a while,
I switched to following the Basic Install Guide
(http://docs.openstack.org/trunk/basic-install/content/basic-install_intro.html).
I also found the basic install guide catered for Fedora
(http://docs.openstack.org/trunk/basic-install/yum/content/basic-install_intro.html),
but that is sorely lacking in the actual instruction department, and is
missing several steps.
If you would like, I can attach the raw draft of my notes. Eventually, some of
the changes or clairifications should make their way into the actual OpenStack
Docs.
Best Regards,
Solly Ross
- Original Message -
From: Qing He qing...@radisys.com
To: sr...@redhat.com
Sent: Monday, September 16, 2013 1:14:42 PM
Subject: RE: Issues with IPTables
Solly,
A side question, how long did this process take you?
Thanks,
Qing
-Original Message-
From: Solly Ross [mailto:sr...@redhat.com]
Sent: Monday, September 16, 2013 10:11 AM
To: OpenStack Development Mailing List
Subject: [openstack-dev] Issues with IPTables
In a enfort to improve/verify the Openstack Documentation with regards to RHEL
and Fedora, I've been attempting to follow the basic install guides. I've
managed to create a working installation and set of instructions. However, to
do so I needed to disable the Neutron IPTables firewall, as it was blocking
non-VM traffic. Namely, it was blocking the GRE packets being used by Neutron.
Did I miss something, or is this a bug?
Best Regards,
Solly Ross
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Intro
=
We will be following the basic install guide at
http://docs.openstack.org/trunk/basic-install/.
Layout
==
We have three networks:
* vmnet10 (NAT, 192.168.0.x) -- management network
* vmnet11 (host-only, 10.10.10.x) -- data network
* vmnet12 (NAT, 192.168.230.x) -- external/API network
All networks have x.x.x.1 assigned as the host's IP, and NAT networks have
x.x.x.2 set as the default gateway/NAT box. For this reason, we will start
all IPs at x.x.x.3 instead of x.x.x.1 (just add 2 to every IP in the guide)
controller.rdo-test
---
* eth0: 192.168.0.3 (mgmt)
* eth1: 129.168.230.7 (ext)
compute.rdo-test
* eth0: 192.168.0.5 (mgmt)
* eth1: 10.10.10.4 (data)
network.rdo-test
* eth0: 192.168.0.4 (mgmt)
* eth1: 10.10.10.3 (data)
* eth2: 192.168.230.8 (ext)
Setup
=
NOTE: make sure that the outside network is reachable
(for example, in our VMWare setup, add `DNS1=192.168.0.2`
and `GATEWAY=192.168.0.2` to /etc/sysconfig/network-scripts/ifcfg-eth0)
Controller Node (controller.rdo-test)
-
1. Add the repositories:
1. `yum-config-manager --add-repo
http://repos.fedorapeople.org/repos/openstack/openstack-trunk/el6-openstack-trunk.repo`
(RDO)
2. `yum install -y
http://dl.fedoraproject.org/pub/epel/6Server/x86_64/epel-release-6-8.noarch.rpm`
(EPEL)
2. Update to grab the new kernel, and reboot to use it
`yum -y update shutdown -r now`
3. Edit the network scripts to contain the correct lines:
`$EDITOR /etc/sysconfig/network-scripts/ifcfg-eth{0,1}`
1. `ONBOOT=yes`
2. `NETMASK=255.255.255.0`
3. `GATEWAY=x.x.x.2` (replace the `x.x.x` with the appropriate prefix)
4. `BOOTPROTO=none`
5. `IPADDR=[SEE LAYOUT SECTION]`
4. Edit sysctl.conf to disable route verification
`$EDITOR /etc/sysctl.conf`
1. `net.ipv4.conf.all.rp_filter = 0`
2
Re: [openstack-dev] Issues with IPTables
Quite a while. RDO's documentation for configuring multinode Packstack with Neutron was a bit lacking, so after attempting to get that working for a while, I switched to following the Basic Install Guide (http://docs.openstack.org/trunk/basic-install/content/basic-install_intro.html). I also found the basic install guide catered for Fedora (http://docs.openstack.org/trunk/basic-install/yum/content/basic-install_intro.html), but that is sorely lacking in the actual instruction department, and is missing several steps. If you would like, I can attach the raw draft of my notes. Eventually, some of the changes or clairifications should make their way into the actual OpenStack Docs. Best Regards, Solly Ross - Original Message - From: Qing He qing...@radisys.com To: sr...@redhat.com Sent: Monday, September 16, 2013 1:14:42 PM Subject: RE: Issues with IPTables Solly, A side question, how long did this process take you? Thanks, Qing -Original Message- From: Solly Ross [mailto:sr...@redhat.com] Sent: Monday, September 16, 2013 10:11 AM To: OpenStack Development Mailing List Subject: [openstack-dev] Issues with IPTables In a enfort to improve/verify the Openstack Documentation with regards to RHEL and Fedora, I've been attempting to follow the basic install guides. I've managed to create a working installation and set of instructions. However, to do so I needed to disable the Neutron IPTables firewall, as it was blocking non-VM traffic. Namely, it was blocking the GRE packets being used by Neutron. Did I miss something, or is this a bug? Best Regards, Solly Ross ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] Nova Migrate/Resize Libvirt Methods
Hello Fellow OpenStackers, I was working on a bug (https://bugzilla.redhat.com/show_bug.cgi?id=975014) involving the Nova resize functionality in the libvirt driver. So, it turns out that the bug is caused by nova code trying to ssh directly into another compute node. Since it was not authorized to do so, the command failed. So, my question is this: should nova really be SSHing directly into a peer machine to create a directory? Doesn't that kind of defeat the purpose of the centralized RPC mechanism? However, if this is the intended functionality, should Packstack be smarter about setting up permissions (this was a packstack install) when there are two or more compute nodes listed? Best Regards, Solly Ross P.S. Yes, I know that was more than one question :-P ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [horizon] [ux] Changing how the modals are closed in Horizon
Just throwing in my two cents: Multiple times, I've lost typed in data in Horizon because I've accidentally pressed escape without thinking (I use Vim, so pressing escape when I'm done typing is second nature). While clicking the 'x' button is often deliberate, pressing escape can be accidental (either through habit or through accidentally pressing the key). Best Regards, Solly Ross - Original Message - From: Aaron Sahlin asah...@linux.vnet.ibm.com To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Sent: Thursday, December 4, 2014 12:00:13 PM Subject: Re: [openstack-dev] [horizon] [ux] Changing how the modals are closed in Horizon The more I think on it. I agree with Rob Cresswell comment While clicking off the modal is relatively easy to do my accident, hitting Esc or ‘X’ are fairly distinct actions. While there is nothing wrong with warning the user that they will lose data after they clicked the 'x' / 'esc'... That was a deliberate action by them. So might be over engineering this. My vote is to just keep it simple and go with changing the default behavior to 'static'. On 12/4/2014 8:08 AM, Timur Sufiev wrote: Hi Aaron, The only way to combine 2 aforementioned solutions I've been thinking of is to implement David's solution as the 4th option (in addition to true|false|static) on a per-form basis, leaving the possibility to change the default value in configs. I guess this sort of combining would be as simple as just putting both patches together (perhaps, changing a bit David's js-code for catching 'click' event - to work only for the modal forms with [data-modal-backdrop='confirm']). On Thu, Dec 4, 2014 at 1:30 AM, Aaron Sahlin asah...@linux.vnet.ibm.com wrote: I would be happy with either the two proposed solutions (both improvements over the what we have now). Any thoughts on combining them? Only close if esc or 'x' is clicked, but also warn them if data was entered. On 12/3/2014 7:21 AM, Rob Cresswell (rcresswe) wrote: +1 to changing the behaviour to ‘static'. Modal inside a modal is potentially slightly more useful, but looks messy and inconsistent, which I think outweighs the functionality. Rob On 2 Dec 2014, at 12:21, Timur Sufiev tsuf...@mirantis.com wrote: Hello, Horizoneers and UX-ers! The default behavior of modals in Horizon (defined in turn by Bootstrap defaults) regarding their closing is to simply close the modal once user clicks somewhere outside of it (on the backdrop element below and around the modal). This is not very convenient for the modal forms containing a lot of input - when it is closed without a warning all the data the user has already provided is lost. Keeping this in mind, I've made a patch [1] changing default Bootstrap 'modal_backdrop' parameter to 'static', which means that forms are not closed once the user clicks on a backdrop, while it's still possible to close them by pressing 'Esc' or clicking on the 'X' link at the top right border of the form. Also the patch [1] allows to customize this behavior (between 'true'-current one/'false' - no backdrop element/'static') on a per-form basis. What I didn't know at the moment I was uploading my patch is that David Lyle had been working on a similar solution [2] some time ago. It's a bit more elaborate than mine: if the user has already filled some some inputs in the form, then a confirmation dialog is shown, otherwise the form is silently dismissed as it happens now. The whole point of writing about this in the ML is to gather opinions which approach is better: * stick to the current behavior; * change the default behavior to 'static'; * use the David's solution with confirmation dialog (once it'll be rebased to the current codebase). What do you think? [1] https://review.openstack.org/#/c/113206/ [2] https://review.openstack.org/#/c/23037/ P.S. I remember that I promised to write this email a week ago, but better late than never :). -- Timur Sufiev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Timur Sufiev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http
[openstack-dev] [nova] Feature Freeze Exception Request (Use libvirt storage pools)
Hi, I would like to request a non-priority feature freeze exception for the Use libvirt storage pools blueprint [1]. The blueprint introduces a new image backed type that uses libvirt storage pools, and is designed to supercede several of the existing image backends for Nova. Using libvirt storage pools simplifies both the maintenance of existing code and the introduction of future storage pool types (since we can support any libvirt storage pool format that supports the createXMLFrom API call). It also paves the way for potentially using the storage pool API to assist with SSH-less migration of disks (not part of this blueprint). The blueprint also provides a way to migrate disks using legacy backends to the new backend on cold migrations/resizes, reboots (soft and hard), and live block migrations. The code [2] is up and working, and is split into (hopefully) manageable chunks. Best Regards, Solly Ross [1] http://specs.openstack.org/openstack/nova-specs/specs/kilo/approved/use-libvirt-storage-pools.html [2] https://review.openstack.org/#/c/152348/ and onward P.S. I would really like to get this in, since this would be the second time that this has been deferred, and took a good bit of manual rebasing to create the Kilo version from the Juno version. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [nova] Feature Freeze Exception Request
Hi, I would like to request a feature freeze exception for the Websockify security proxy framework blueprint [1]. The blueprint introduces a framework for defining security drivers for the connections between the websocket proxy and the hypervisor, and provides a TLS driver for VNC connections using the VeNCrypt RFB auth method. The two patches [2] have sat in place with one +2 (Dan Berrange) and multiple +1s for a while now (the first does not currently show any votes because of a merge conflict that I had to deal with recently). Best Regards, Solly Ross [1] http://specs.openstack.org/openstack/nova-specs/specs/kilo/approved/websocket-proxy-to-host-security.html [2] https://review.openstack.org/#/c/115483/ and https://review.openstack.org/#/c/115484/ __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
