Re: [openstack-dev] [nova] fair standards for all hypervisor drivers
On Fri, 2014-08-08 at 09:06 -0400, Russell Bryant wrote: On 08/07/2014 08:06 PM, Michael Still wrote: It seems to me that the tension here is that there are groups who would really like to use features in newer libvirts that we don't CI on in the gate. Is it naive to think that a possible solution here is to do the following: - revert the libvirt version_cap flag I don't feel strongly either way on this. It seemed useful at the time for being able to decouple upgrading libvirt and enabling features that come with that. Right, I suggested the flag as a more deliberate way of avoiding the issue that was previously seen in the gate with live snapshots. I still think it's a pretty elegant and useful little feature, and don't think we need to use it as proxy battle over testing requirements for new libvirt features. Mark. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] Python3 search path order when building Debian package
Hi, During the build of oslo packages for OpenStack, I'm having the issue that python3 seems to first look into /usr/lib/python3.4/dist-packages/olso, finds stuff in it, and then do not search in the local directory. Because of that, unit tests are failing during the build. For example, when trying to build python-oslo.utils (new package, not yet available in Debian), I get: == FAIL: unittest.loader.ModuleImportFailure.tests.test_excutils unittest.loader.ModuleImportFailure.tests.test_excutils -- _StringException: Traceback (most recent call last): ImportError: Failed to import test module: tests.test_excutils Traceback (most recent call last): File /usr/lib/python3.4/unittest/loader.py, line 312, in _find_tests module = self._get_module_from_name(name) File /usr/lib/python3.4/unittest/loader.py, line 290, in _get_module_from_name __import__(name) File /home/zigo/sources/openstack/juno/python-oslo.utils/build-area/python-oslo.utils-0.1.1/tests/test_excutils.py, line 22, in module from oslo.utils import excutils ImportError: No module named 'oslo.utils' note that I am running testr the following way: testr-python3 init PYTHON=python3.4 testr-python3 run --subunit | subunit2pyunit Of course, if I first install python3-oslo.utils, then the unit tests are passing without any issue. Though at build time, the package isn't installed in the system yet. What's very surprising to me, is that I have this issue only with Python 3, not with python 2.7, which passes the tests without any issue, and does the path hunt for the python module correctly. For other oslo libraries, I have declared some Build-Conflicts, to make sure that they aren't installed in the system when building. Though that can't be done with oslo.utils, because it depends on oslo.config and oslo.i18n. I think this highlight once more that using namespace should be avoided if possible. This creates more issues than it solves problems. Any idea, comment or whatever that may help would be appreciated. Cheers, Thomas Goirand (zigo) P.S: By the way, there's issues with python-oslo.utils with Python 3.4, after solving the above problem... ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Neutron][LBaaS] Improvements to current reviews
I agree with Doug as well. We should update the current patch. Susanne On Sun, Aug 10, 2014 at 8:18 AM, Gary Kotton gkot...@vmware.com wrote: Hi, I took a look at https://review.openstack.org/#/c/105331/ and had one minor issue which I think can be addressed. Prior to approving we need to understand what the state of the V2 API will be. Thanks Gary From: Vijay Venkatachalam vijay.venkatacha...@citrix.com Reply-To: OpenStack List openstack-dev@lists.openstack.org Date: Sunday, August 10, 2014 at 2:57 PM To: OpenStack List openstack-dev@lists.openstack.org Subject: Re: [openstack-dev] [Neutron][LBaaS] Improvements to current reviews Thanks Brandon for constant improvisation. I agree with Doug. Please update current one. We already hv more number of reviews :-). It will be difficult to manage if we add more. Thanks, Vijay Sent using CloudMagic On Sun, Aug 10, 2014 at 3:23 AM, Doug Wiegley do...@a10networks.com wrote: I think you should update the current reviews (new patch set, not additional review.) Doug On Aug 9, 2014, at 3:34 PM, Brandon Logan brandon.lo...@rackspace.com wrote: So I've done some work on improving the code on the current pending reviews. And would like to get peoples' opinions on whether I should add antoher patch set to those reviews, or add the changes as as another review dependent on the pending ones. To be clear, no matter what the first review in the chain will not change: https://review.openstack.org/#/c/105331/ However, if adding another patch set is preferrable then plugin and db implementation review would get another patch set and then obviously anything depending on that. https://review.openstack.org/#/c/105609/ My opinion is that I'd like to get both of these in as a new patch set. Reason being that the reviews don't have any +2's and there is uncertainty because of the GBP discussion. So, I don't think it'd be a major issue if a new patch set was created. Speak up if you think otherwise. I'd like to get as many people's thoughts as possible. The changes are: 1) Added data models, which are just plain python object mimicking the sql alchemy models, but don't have the overhead or dynamic nature of being sql alchemy models. These data models are now returned by the database methods, instead of the sql alchemy objects. Also, I moved the definition of the sql alchemy models into its own module. I've been wanting to do this but since I thought I was running out of time I left it for later. These shouldn't cause many merge/rebase conflicts, but it probably cause a few because the sql alchemy models were moved to a different module. 2) The LoadBalancerPluginv2 no longer inherits from the LoadBalancerPluginDbv2. The database is now a composite attribute of the plugin (i.e. plugin.db.get_loadbalancer()). This cleans up the code a bit and removes the necessity for _delete_db_entity methods that the drivers needed because now they can actually call the database methods. Also, this makes testing more clear, though I have not added any tests for this because the previous tests are sufficient for now. Adding the appropriate tests would add 1k or 2k lines most likely. This will likely cause more conflicts because the _db_delete_entity methods have been removed. However, the new driver interface/mixins defined a db_method for all drivers to use, so if that is being used there shouldn't be any problems. Thanks, Brandon ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Fwd: FW: [Neutron] Group Based Policy and the way forward
Hi Sumit, First I want to say I'm not opposed to GBP itself, but has many confusion about it's core resource model and how it will integrate with neutron core. Do you mean for whatever GBP backend configured in any future Neutron deployment, so long as they are in tree, then ML2 core plugin shall always be there to expose the Neutron core resource: Network/Subnet/Port? Then since Network/Subnet/Port will never be treated just as LEGACY COMPATIBLE role, there is no need to extend Nova-Neutron interface to follow the GBP resource. Anyway, one of optional service plugins inside Neutron shouldn't has any impact on Nova side. If we agree on this point, core model of GBP should be reviewed, but not just focus on naming convention whether it should be called EP or policy target, then leaving some words here to emphasize GBP is only complementary. In fact, EP/EPG/BD/RD has been designed to be ABLE TO REPLACE Neutron core resource, with mapping as the first step to keep compatible. In fact, if Neutron core source shall never to be swapped out, GBP core object could be greatly simplified, because mapping already means redundancy :-) Only policy-group is meaningful, behind which are very important policy concept: consumer/producer contracts. After PG is defined, it should be directly applied to existing Neutron core resource, but not create similar concept of EP/L2P/L3P, then mapping to them. Mapping is redundant, and I can understand it's necessity only if someday those neutron core resource are planned to be swapped out. Simple conclusion: if GBP is just an optional complementary, then after defining policy template, directly APPLYING, but not create similar redundant resource and then MAPPING it to existing neutron core resource. On Sat, Aug 9, 2014 at 3:35 AM, Jay Pipes jaypi...@gmail.com wrote: On 08/08/2014 12:29 PM, Sumit Naiksatam wrote: Hi Jay, To extend Ivar's response here, the core resources and core plugin configuration does not change with the addition of these extensions. The mechanism to implement the GBP extensions is via a service plugin. So even in a deployment where a GBP service plugin is deployed with a driver which interfaces with a backend that perhaps directly understands some of the GBP constructs, that system would still need to have a core plugin configured that honors Neutron's core resources. Hence my earlier comment that GBP extensions are complementary to the existing core resources (in much the same way as the existing extensions in Neutron). OK, thanks Sumit. That clearly explains things for me. Best, -jay ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Octavia] Minutes from 8/6/2014 meeting
Great notes! thanks it helped me catch up after vacation. :) On Thu, Aug 7, 2014 at 4:33 AM, Stephen Balukoff sbaluk...@bluebox.net wrote: On where to capture notes like this long-term: I would say the wiki is more searchable for now. When we make the transition to IRC meetings, then the meeting bots will capture minutes and transcripts in the usual way and we can link to these from the wiki. On Thu, Aug 7, 2014 at 1:29 AM, Stephen Balukoff sbaluk...@bluebox.net wrote: Wow, Trevor! Thanks for capturing all that! On Wed, Aug 6, 2014 at 9:47 PM, Trevor Vardeman trevor.varde...@rackspace.com wrote: Agenda items are numbered, and topics, as discussed, are described beneath in list format. 1) Octavia Constitution and Project Direction Documents (Road map) a) Constitution and Road map will potentially be adopted after another couple days; providing those who were busy more time to review the information 2) Octavia Design Proposals a) Difference between version 0.5 and 1.0 isn't huge b) Version 2 has many network topology changes and Layer 4 routing + This includes N node Active-Active + Would like to avoid Layer 2 connectivity with Load Balancers (included in version 1 however) + Layer router driver + Layer router controller + Long term solution c) After refining Version 1 document (with some scrutiny) all changes will be propagated to the Version 2 document d) Version 0.5 is unpublished e) All control layer, anything connected to the intermediate message bus in version 1, will be collapsed down to 1 daemon. + No scale-able control, but scale-able service delivery + Version 1 will be the first large operator compatible version, that will have both scale-able control and scale-able service delivery + 0.5 will be a good start - laying out ground work - rough topology for the end users - must be approved by the networking teams for each contributing company f) The portions under control of neutron lbaas is the User API and the driver (for neutron lbaas) g) If neutron LBaaS is a sufficient front-end (user API doesn't suck), then Octavia will be kept as a vendor driver h) Potentially including a REST API on top of Octavia + Octavia is initially just a vendor driver, no real desire for another API in front of Octavia + If someone wants it, the work is trivial and can be done in another project at another time i) Octavia should have a loose coupling with Neutron; use a shim for network connectivity (one specifically for Neutron communication in the start) + This is going to hold any dirty hacks that would be required to get something done, keeping Octavia clean - Example: changing the mac address on a port 3) Operator Network Topology Requirements a) One requirement is floating IPs. b) IPv6 is in demand, but is currently not supported reliably on Neutron + IPv6 would be represented as a different load balancer entity, and possibly include co-location with another Load Balancer c) Network interface plug-ability (potentially) d) Sections concerning front-end connectivity should be forwarded to each company's network specialists for review + Share findings in the mailing list, and dissect the proposals with the information and comment what requirements are needing added etc. 4) HA/Failover Options/Solutions a) Rackspace may have a solution to this, but the conversation will be pushed off to the next meeting (at least) + Will gather more information from another member in Rackspace to provide to the ML for initial discussions b) One option for HA: Spare pool option (similar to Libra) + Poor recovery time is a big problem c) Another option for HA: Active/Passive + Bluebox uses one active and one passive configuration, and has sub-second fail over. However is not resource-sufficient Questions: Q: What is the expectation for a release time-frame A: Wishful thinking; Octavia version 0.5 beta for Juno (probably not, but would be awesome to push for that) Notes: + We need to pressure the Neutron core reviewers to review the Neutron LBaaS changes to get merges. + Version 2 front-end topology is different than the Version 1. Please review them individually, and thoroughly PS. I re-wrote most of the information from the recording (thanks again Doug). I have one question for everyone: should I just email this out after each meeting to the Octavia mailing list, or should I also add it to a page in an Octavia wiki for Meeting Notes/Minutes or something for review by anyone? What are your thoughts? ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [Heat] SoftwareDeployment resource is always in progress
Hello, I 'm trying to create a simple stack with heat (Icehouse release). The template contains SoftwareConfig, SoftwareDeployment and a single server resources. The problem is that the SoftwareDeployment resource is always in progress ! After waiting for more than an hour the stack deployment failed and I got this error: TRACE heat.engine.resource HTTPUnauthorized: ERROR: Authentication failed. Please try again with option --include-password or export HEAT_INCLUDE_PASSWORD=1 TRACE heat.engine.resource Authentication required When I checked the log file (/var/log/heat/heat-engine.log), it shows the following message(every second): 2014-08-10 19:41:09.622 2391 INFO urllib3.connectionpool [-] Starting new HTTP connection (1): 192.168.122.10 2014-08-10 19:41:10.648 2391 INFO urllib3.connectionpool [-] Starting new HTTP connection (1): 192.168.122.10 2014-08-10 19:41:11.671 2391 INFO urllib3.connectionpool [-] Starting new HTTP connection (1): 192.168.122.10 2014-08-10 19:41:12.690 2391 INFO urllib3.connectionpool [-] Starting new HTTP connection (1): 192.168.122.10 Here the template I am using : https://github.com/openstack/heat-templates/blob/master/hot/software-config/example-templates/wordpress/WordPress_software-config_1-instance.yaml Please help ! Kind Regards, David ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Neutron] Simple proposal for stabilizing new features in-tree
gustavo panizzo (gfa) wrote: only one think i didn't like it why all url,api, etc has to include the word 'preview'? i imagine that i would be consuming the new feature using heat, puppet, local scripts, custom horizon, whatever. Why do you make me to change all them when the feature moves out of preview? it could be a lot of rework (for consumers) without gain. I would totally support other big fat warnings everywhere (logs, documentation, startup log of neutron-server) but don't change the API if is not necessary I see two issues with this proposal: the first one is what Gustavo just said: the use of the preview package/configoption/API creates friction when the feature needs to go mainstream (package renaming, change in configuration for deployers, change in API calls for users...). My understanding is that the goal is to make it easy for people to try the preview feature, and keeping the experimental feature in-tree is seen as simpler to experiment with. But the pain from this friction imho outweighs the pain of deploying an out-of-tree plugin for deployers. The second issue is that once the feature is in preview in tree, it moves the responsibility/burden of making it official (or removed) to the core developers (as Salvatore mentioned). I kind of like the approach where experimental features are developed in faster iterations out-of-tree and when they are celebrated by experimenters and are ready to be stable-supported, they are moved in tree. Regards, -- Thierry Carrez (ttx) ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [keystone] how to generate conf sample in havana?
Hello, Right now, i customerized keystone to meet our requeriments. I want to add some configuration options in keystone.conf.sample. But, i do not find generate_sample.sh in directory tools in Havana keystone. So i want to know how to generate conf sample in havana keystone. I know, i can write directly to keystone.conf.sample, but should there be something like generate_sample.sh in keystone?___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] The future of the integrated release
Jeremy Stanley wrote: On 2014-08-08 09:43:54 -0700 (-0700), Devananda van der Veen wrote: [...] this sounds like it will also solve the current problem when a core reviewer has gone on vacation after blocking something for procedural reasons. I don't really think so, unless I'm misunderstanding which vacation problem you're talking about. We can either make a vote carry over on subsequent patchsets (in which case you need someone to reverse or delete the -2 before you can merge it) or not (in which case you will lose that -2 on the next uploaded patchset and have to keep reapplying it manually). Unless perhaps you're suggesting that we should allow approved changes to merge even with a workflow -2 from another reviewer (keeping in mind that we don't currently allow changes with a workflow -1 to merge). Hmm.. Ideally we'd want a blocking vote that carries over patchsets, but which can be set or unset by any core/driver/ptl (to be determined). So if the PTL sets the procedural Workflow-2 at feature freeze, I can clear it when RC1 is published. Not sure that's supported in Gerrit though :) -- Thierry Carrez (ttx) ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [Neutron][LBaaS] Use cases with regards to VIP and routers
Gang, I was asked the following questions around our Neutron LBaaS use cases: 1. Will there be a scenario where the “VIP” port will be in a different Node, from all the Member “VMs” in a pool. 2. Also how likely is it for the LBaaS configured subnet to not have a “router” and just use the “extra_routes” option. 3. Is there a valid use case where customers will be using the “extra_routes” with subnets instead of the “routers”. ( It would be great if you have some use case picture for this). Feel free to chime in here and I'll summaries the answers. Regards Susanne ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Fuel] Using host networking for docker containers
Hi, we're running only 3 containers in privileged mode: cobbler, rsyslog and mcollective. Running all the containers in privileged mode is not a good idea for security reasons. Docker manages DNAT forwarding itself, so it does not create any overhead for us. Is there any real benefits of using separate namespaces in security terms? Of course, for example only ports specified in EXPOSE line in Dockerfile are exposed to the host network. So if you start any additional tcp/udp listeners inside the containers, their ports won't be accessible from the host network. On Sat, Aug 9, 2014 at 10:39 AM, Dmitriy Shulyak dshul...@mirantis.com wrote: Hi team, I want to discuss benefits of using host networking [1] for docker containers, on master node. This feature was added in docker 0.11 and basicly means - reuse host networking stack, without creating separate namespace for each container. In my opinion it will result in much more stable install/upgrade of master node. 1. There will be no need for dhcrelay/dhcrelay_monitor on host 2. No dnat port forwarding 3. Performance improvement for pxe boot ??? Is there any real benefits of using separate namespaces in security terms? To implement this we will need: 1. Update docker to recent version 0.12/1.x, we will do it anyway, yes? 2. Run docker containers with --net=host Ofcourse it will require running containers in privileged mode, but afaik we are already doing this for other reasons. So, what do you think? [1] https://github.com/docker/docker/issues/2012 [2] https://docs.docker.com/articles/networking/ ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] The future of the integrated release
Eoghan Glynn wrote: The TC has begun to scrutinize new projects more carefully on technical grounds, particularly since some recently-integrated projects have run into scaling limitations that have hampered their adoption. I believe this sort of technical guidance (or curation, if you will) is an essential function of the TC. We've learned that integration bestows legitimacy, as well as assumptions of stability, performance, and scalability, upon projects which are integrated. While that wasn't the intent, I think we need to accept that that is where we stand. We will be faced with some hard decisions regarding projects, both incubated and already integrated, which are clearly not meeting those expectations today. How does this relate to the recent gap analysis undertaken by the TC for already integrated projects, in order to measure their status against the steadily rising bar for integration? It is almost orthogonal. During the Icehouse cycle the TC created clear requirements for projects wishing to incubate or graduate for incubation. But it was unclear how well the *currently-incubated* projects fulfilled those requirements. Since it was a bit unfair to hold new entrants to different standards, during the Juno cycle we completed a gap analysis of currently-integrated projects, created gap coverage plans to address any identified gap, and reviewed progress on those after each milestone. So the idea that being (and remaining) in the integrated release should also be judged on technical merit is a slightly different effort. It's always been a factor in our choices, but like Devananda says, it's more difficult than just checking a number of QA/integration checkboxes. In some cases, blessing one project in a problem space stifles competition, innovation and alternate approaches. In some other cases, we reinvent domain-specific solutions rather than standing on the shoulders of domain-specific giants in neighboring open source projects. This all has created a world where you need to be *in* OpenStack to matter, or to justify the investment. This has created a world where everything and everyone wants to be in the OpenStack integrated release. This has created more pressure to add new projects, and less pressure to fix and make the existing projects perfect. 4 years in, we might want to inflect that trajectory and take steps to fix this world. -- Thierry Carrez (ttx) ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][libvirt] Non-readonly connection to libvirt in unit tests
On Mon, Aug 04, 2014 at 06:46:13PM -0400, Solly Ross wrote: Hi, I was wondering if there was a way to get a non-readonly connection to libvirt when running the unit tests on the CI. If I call `LibvirtDriver._connect(LibvirtDriver.uri())`, it works fine locally, but the upstream CI barfs with libvirtError: internal error Unable to locate libvirtd daemon in /usr/sbin (to override, set $LIBVIRTD_PATH to the name of the libvirtd binary). If I try to connect by calling libvirt.open(None), it also barfs, saying I don't have permission to connect. I could just set it to always use fakelibvirt, but it would be nice to be able to run some of the tests against a real target. The tests in question are part of https://review.openstack.org/#/c/111459/, and involve manipulating directory-based libvirt storage pools. Nothing in the unit tests should rely on being able to connect to the libvirt daemon, as the unit tests should still be able to pass when the daemon is not running at all. We should be either using fakelibvirt or mocking any libvirt APIs that need to be tested Regards, Daniel -- |: http://berrange.com -o-http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :| ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] fair standards for all hypervisor drivers
On Mon, Aug 11, 2014 at 08:57:31AM +1000, Michael Still wrote: On Sun, Aug 10, 2014 at 2:33 AM, Jeremy Stanley fu...@yuggoth.org wrote: On 2014-08-08 09:06:29 -0400 (-0400), Russell Bryant wrote: [...] We've seen several times that building and maintaining 3rd party CI is a *lot* of work. Building and maintaining *any* CI is a *lot* of work, not the least of which is the official OpenStack project CI (I believe Monty mentioned in #openstack-infra last night that our CI is about twice the size of Travis-CI now, not sure what metric he's comparing there though). Like you said in [1], doing this in infra's CI would be ideal. I think 3rd party should be reserved for when running it in the project's infrastructure is not an option for some reason (requires proprietary hw or sw, for example). Add to the not an option for some reason list, software which is not easily obtainable through typical installation channels (PyPI, Linux distro-managed package repositories for their LTS/server releases, et cetera) or which requires gyrations which destabilize or significantly complicate maintenance of the overall system as well as reproducibility for developers. It may be possible to work around some of these concerns via access from multiple locations coupled with heavy caching, but adding that in for a one-off source is hard to justify the additional complexity too. My understanding is that Fedora has a PPA equivalent which ships a latest and greated libvirt. So, it would be packages if we went the Fedora route, which should be less work. Yes, there is the 'virt preview' repository http://fedoraproject.org/wiki/Virtualization_Preview_Repository Regards, Daniel -- |: http://berrange.com -o-http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :| ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] fair standards for all hypervisor drivers
On Fri, Aug 08, 2014 at 09:06:29AM -0400, Russell Bryant wrote: On 08/07/2014 08:06 PM, Michael Still wrote: It seems to me that the tension here is that there are groups who would really like to use features in newer libvirts that we don't CI on in the gate. Is it naive to think that a possible solution here is to do the following: - revert the libvirt version_cap flag I don't feel strongly either way on this. It seemed useful at the time for being able to decouple upgrading libvirt and enabling features that come with that. I'd like to let Dan get back from vacation and weigh in on it, though. Yes, I think that version cap feature is valuable no matter what we do about CI testing, which is why I +2'd it originally. I wonder if the job could be as simple as one with an added step in the config to install latest libvirt from source. Dan, do you think someone could add a libvirt-current.tar.gz to http://libvirt.org/sources/ ? Using the latest release seems better than master from git. I'd strongly recommend against using GIT master. It will cause openstack CI more pain than benefits. Using the latest stable release is a better bet I'll mess around and see if I can spin up an experimental job. There is work to add support for this in devestack already which I prefer since it makes it easy for developers to get an environment which matches the build system: https://review.openstack.org/#/c/108714/ Regards, Daniel -- |: http://berrange.com -o-http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :| ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Fuel] Using host networking for docker containers
Moving to host networking would reduce our ability to do zero downtime upgrades in the future. It means you must kill the old container in order to start the new one, rather than allowing for the possibility to remap the network configuration in iptables. It's something we don't have now, but we may be able to do in the future. With regards to security issues, we have some more restrictive firewall rules in place now. I don't think this is a major issue. I don't think it makes a huge difference in performance to switch to host networking, but it's worth testing. On Mon, Aug 11, 2014 at 1:16 PM, Aleksandr Didenko adide...@mirantis.com wrote: Hi, we're running only 3 containers in privileged mode: cobbler, rsyslog and mcollective. Running all the containers in privileged mode is not a good idea for security reasons. Docker manages DNAT forwarding itself, so it does not create any overhead for us. Is there any real benefits of using separate namespaces in security terms? Of course, for example only ports specified in EXPOSE line in Dockerfile are exposed to the host network. So if you start any additional tcp/udp listeners inside the containers, their ports won't be accessible from the host network. On Sat, Aug 9, 2014 at 10:39 AM, Dmitriy Shulyak dshul...@mirantis.com wrote: Hi team, I want to discuss benefits of using host networking [1] for docker containers, on master node. This feature was added in docker 0.11 and basicly means - reuse host networking stack, without creating separate namespace for each container. In my opinion it will result in much more stable install/upgrade of master node. 1. There will be no need for dhcrelay/dhcrelay_monitor on host 2. No dnat port forwarding 3. Performance improvement for pxe boot ??? Is there any real benefits of using separate namespaces in security terms? To implement this we will need: 1. Update docker to recent version 0.12/1.x, we will do it anyway, yes? 2. Run docker containers with --net=host Ofcourse it will require running containers in privileged mode, but afaik we are already doing this for other reasons. So, what do you think? [1] https://github.com/docker/docker/issues/2012 [2] https://docs.docker.com/articles/networking/ ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] fair standards for all hypervisor drivers
In the past there was smokestack that used a fedora version of libvirt. Any idea what happened to that? On 8/11/14, 12:53 PM, Daniel P. Berrange berra...@redhat.com wrote: On Fri, Aug 08, 2014 at 09:06:29AM -0400, Russell Bryant wrote: On 08/07/2014 08:06 PM, Michael Still wrote: It seems to me that the tension here is that there are groups who would really like to use features in newer libvirts that we don't CI on in the gate. Is it naive to think that a possible solution here is to do the following: - revert the libvirt version_cap flag I don't feel strongly either way on this. It seemed useful at the time for being able to decouple upgrading libvirt and enabling features that come with that. I'd like to let Dan get back from vacation and weigh in on it, though. Yes, I think that version cap feature is valuable no matter what we do about CI testing, which is why I +2'd it originally. I wonder if the job could be as simple as one with an added step in the config to install latest libvirt from source. Dan, do you think someone could add a libvirt-current.tar.gz to https://urldefense.proofpoint.com/v1/url?u=http://libvirt.org/sources/k= oIvRg1%2BdGAgOoM1BIlLLqw%3D%3D%0Ar=eH0pxTUZo8NPZyF6hgoMQu%2BfDtysg45MkPh CZFxPEq8%3D%0Am=rPfYVc630ymUlJf3W60dbNbCgcW5TimMFx1ooqIdr38%3D%0As=0298 f3506d0ac41e81fce6b0b3096455c3e78ba578970e42290997a766b0811e ? Using the latest release seems better than master from git. I'd strongly recommend against using GIT master. It will cause openstack CI more pain than benefits. Using the latest stable release is a better bet I'll mess around and see if I can spin up an experimental job. There is work to add support for this in devestack already which I prefer since it makes it easy for developers to get an environment which matches the build system: https://review.openstack.org/#/c/108714/ Regards, Daniel -- |: https://urldefense.proofpoint.com/v1/url?u=http://berrange.com/k=oIvRg1%2 BdGAgOoM1BIlLLqw%3D%3D%0Ar=eH0pxTUZo8NPZyF6hgoMQu%2BfDtysg45MkPhCZFxPEq8% 3D%0Am=rPfYVc630ymUlJf3W60dbNbCgcW5TimMFx1ooqIdr38%3D%0As=d7e4cac70435cd 1066f4f7ec8795f16c345c0ca4529bb4b3033762552817e775 -o- https://urldefense.proofpoint.com/v1/url?u=http://www.flickr.com/photos/db errange/k=oIvRg1%2BdGAgOoM1BIlLLqw%3D%3D%0Ar=eH0pxTUZo8NPZyF6hgoMQu%2BfD tysg45MkPhCZFxPEq8%3D%0Am=rPfYVc630ymUlJf3W60dbNbCgcW5TimMFx1ooqIdr38%3D% 0As=0f30863dcc9a34ffe520b0cee787f76bb67334963ebcd57d9b53b3890f0265ea :| |: https://urldefense.proofpoint.com/v1/url?u=http://libvirt.org/k=oIvRg1%2B dGAgOoM1BIlLLqw%3D%3D%0Ar=eH0pxTUZo8NPZyF6hgoMQu%2BfDtysg45MkPhCZFxPEq8%3 D%0Am=rPfYVc630ymUlJf3W60dbNbCgcW5TimMFx1ooqIdr38%3D%0As=07044ea24f9483f b87f1c67511c4c62a3e2b8971ad8418b7a0a61068f623f529 -o- https://urldefense.proofpoint.com/v1/url?u=http://virt-manager.org/k=oIvR g1%2BdGAgOoM1BIlLLqw%3D%3D%0Ar=eH0pxTUZo8NPZyF6hgoMQu%2BfDtysg45MkPhCZFxP Eq8%3D%0Am=rPfYVc630ymUlJf3W60dbNbCgcW5TimMFx1ooqIdr38%3D%0As=185d42ae1a 5b0422c1fa3306e55397461140024c38530f3a6986a72c51edec08 :| |: https://urldefense.proofpoint.com/v1/url?u=http://autobuild.org/k=oIvRg1% 2BdGAgOoM1BIlLLqw%3D%3D%0Ar=eH0pxTUZo8NPZyF6hgoMQu%2BfDtysg45MkPhCZFxPEq8 %3D%0Am=rPfYVc630ymUlJf3W60dbNbCgcW5TimMFx1ooqIdr38%3D%0As=2e295d841588d 8cb243a6c37eb0c17007760ba0506cd621fff34de6a6f7c7b03 -o- https://urldefense.proofpoint.com/v1/url?u=http://search.cpan.org/~danberr /k=oIvRg1%2BdGAgOoM1BIlLLqw%3D%3D%0Ar=eH0pxTUZo8NPZyF6hgoMQu%2BfDtysg45M kPhCZFxPEq8%3D%0Am=rPfYVc630ymUlJf3W60dbNbCgcW5TimMFx1ooqIdr38%3D%0As=39 b473e1829f6e6c4f7bd4e27304334df8c0d1e9ac2cf7432490ac72fbe111e6 :| |: https://urldefense.proofpoint.com/v1/url?u=http://entangle-photo.org/k=oI vRg1%2BdGAgOoM1BIlLLqw%3D%3D%0Ar=eH0pxTUZo8NPZyF6hgoMQu%2BfDtysg45MkPhCZF xPEq8%3D%0Am=rPfYVc630ymUlJf3W60dbNbCgcW5TimMFx1ooqIdr38%3D%0As=c100dc7d e638a7bd0477a6d9723599041f442e80207213597a220e1a2cd34c74 -o- https://urldefense.proofpoint.com/v1/url?u=http://live.gnome.org/gtk-vnck =oIvRg1%2BdGAgOoM1BIlLLqw%3D%3D%0Ar=eH0pxTUZo8NPZyF6hgoMQu%2BfDtysg45MkPh CZFxPEq8%3D%0Am=rPfYVc630ymUlJf3W60dbNbCgcW5TimMFx1ooqIdr38%3D%0As=5c43d 0b2c8ad31d53813420ce9ab1885943b4ef88458006edcb2295f47c46846 :| ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [Neutron][LBaaS] Continuing on Calling driver interface on every API request
Hi: Continuing from last week's LBaaS meeting... Currently an entity cannot be sent to driver unless it is linked to loadbalancer because loadbalancer is the root object and driver information is only available with loadbalancer. The request to the driver is delayed because of which error propagation becomes tricky. Let's say a monitor was configured with timeout delay there would be no error then. When a listener is configured there will be a monitor creation/deployment error like timeout configured greater than delay. Unless the error is very clearly crafted the user won't be able to understand the error. I am half-heartedly OK with current approach. But, I would prefer Brandon's Solution - make provider an attribute in each of the entities to get rid of this problem. What do others think? Thanks, Vijay V. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] fair standards for all hypervisor drivers
On 08/11/2014 05:53 AM, Daniel P. Berrange wrote: There is work to add support for this in devestack already which I prefer since it makes it easy for developers to get an environment which matches the build system: https://review.openstack.org/#/c/108714/ Ah, cool. Devstack is indeed a better place to put the build scripting. So, I think we should: 1) Get the above patch working, and then merged. 2) Get an experimental job going to use the above while we work on #3 3) Before the job can move into the check queue and potentially become voting, it needs to not rely on downloading the source on every run. IIRC, we can have nodepool build an image to use for these jobs that includes the bits already installed. I'll switch my efforts over to helping get the above completed. -- Russell Bryant ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] fair standards for all hypervisor drivers
On Mon, Aug 11, 2014 at 07:58:41AM -0400, Russell Bryant wrote: On 08/11/2014 05:53 AM, Daniel P. Berrange wrote: There is work to add support for this in devestack already which I prefer since it makes it easy for developers to get an environment which matches the build system: https://review.openstack.org/#/c/108714/ Ah, cool. Devstack is indeed a better place to put the build scripting. So, I think we should: 1) Get the above patch working, and then merged. 2) Get an experimental job going to use the above while we work on #3 3) Before the job can move into the check queue and potentially become voting, it needs to not rely on downloading the source on every run. Don't we have the ability to mirror downloads locally to the build system for python ? The proposed patch allows an alternate download URL to be set via an env variable so it could point to a local mirror instead of libvirt.org / qemu.org IIRC, we can have nodepool build an image to use for these jobs that includes the bits already installed. Regards, Daniel -- |: http://berrange.com -o-http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :| ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] fair standards for all hypervisor drivers
On 08/09/2014 12:33 PM, Jeremy Stanley wrote: On 2014-08-08 09:06:29 -0400 (-0400), Russell Bryant wrote: [...] We've seen several times that building and maintaining 3rd party CI is a *lot* of work. Building and maintaining *any* CI is a *lot* of work, not the least of which is the official OpenStack project CI (I believe Monty mentioned in #openstack-infra last night that our CI is about twice the size of Travis-CI now, not sure what metric he's comparing there though). Dang, I'd love to see those numbers. :-) Like you said in [1], doing this in infra's CI would be ideal. I think 3rd party should be reserved for when running it in the project's infrastructure is not an option for some reason (requires proprietary hw or sw, for example). Add to the not an option for some reason list, software which is not easily obtainable through typical installation channels (PyPI, Linux distro-managed package repositories for their LTS/server releases, et cetera) or which requires gyrations which destabilize or significantly complicate maintenance of the overall system as well as reproducibility for developers. It may be possible to work around some of these concerns via access from multiple locations coupled with heavy caching, but adding that in for a one-off source is hard to justify the additional complexity too. Understood. Some questions ... is building an image that has libvirt and qemu pre-installed from source good enough? It avoids the dependency on job runs, but moves it to image build time though, so it still exists. If the above still doesn't seem like a workable setup, then I think we should just go straight to an image with fedora + virt-preview repo, which kind of sounds easier, anyway. -- Russell Bryant ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] fair standards for all hypervisor drivers
On 08/11/2014 07:58 AM, Russell Bryant wrote: On 08/11/2014 05:53 AM, Daniel P. Berrange wrote: There is work to add support for this in devestack already which I prefer since it makes it easy for developers to get an environment which matches the build system: https://review.openstack.org/#/c/108714/ Ah, cool. Devstack is indeed a better place to put the build scripting. So, I think we should: 1) Get the above patch working, and then merged. 2) Get an experimental job going to use the above while we work on #3 3) Before the job can move into the check queue and potentially become voting, it needs to not rely on downloading the source on every run. IIRC, we can have nodepool build an image to use for these jobs that includes the bits already installed. I'll switch my efforts over to helping get the above completed. I still think the devstack patch is good, but after some more thought, I think a better long term CI job setup would just be a fedora image with the virt-preview repo. I think I'll try that ... -- Russell Bryant ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] fair standards for all hypervisor drivers
On 08/11/2014 08:01 AM, Daniel P. Berrange wrote: On Mon, Aug 11, 2014 at 07:58:41AM -0400, Russell Bryant wrote: On 08/11/2014 05:53 AM, Daniel P. Berrange wrote: There is work to add support for this in devestack already which I prefer since it makes it easy for developers to get an environment which matches the build system: https://review.openstack.org/#/c/108714/ Ah, cool. Devstack is indeed a better place to put the build scripting. So, I think we should: 1) Get the above patch working, and then merged. 2) Get an experimental job going to use the above while we work on #3 3) Before the job can move into the check queue and potentially become voting, it needs to not rely on downloading the source on every run. Don't we have the ability to mirror downloads locally to the build system for python ? The proposed patch allows an alternate download URL to be set via an env variable so it could point to a local mirror instead of libvirt.org / qemu.org There's a pypi mirror at least. I'm not sure about mirroring other things. -- Russell Bryant ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] The future of the integrated release
On 2014-08-11 11:02:52 +0200 (+0200), Thierry Carrez wrote: Ideally we'd want a blocking vote that carries over patchsets, but which can be set or unset by any core/driver/ptl (to be determined). So if the PTL sets the procedural Workflow-2 at feature freeze, I can clear it when RC1 is published. Not sure that's supported in Gerrit though :) As far as I know it's not... otherwise we'd have made WIP do the same as you describe. -- Jeremy Stanley ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [Nova] PCI support
Hi, At the moment all of the drivers are required CI support. Are there any plans regarding the PIC support. I understand that this is something that requires specific hardware. Are there any plans to add this? Thanks Gary ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Fwd: FW: [Neutron] Group Based Policy and the way forward
loy wolfe [mailto:loywo...@gmail.com] wrote: Then since Network/Subnet/Port will never be treated just as LEGACY COMPATIBLE role, there is no need to extend Nova-Neutron interface to follow the GBP resource. Anyway, one of optional service plugins inside Neutron shouldn't has any impact on Nova side. This gets to the root of why I was getting confused about Jay and others having Nova related concerns. I was/am assuming that GBP is simply an *additional* mechanism for manipulating Neutron, not a deprecation of any part of the existing Neutron API. I think Jay's concern and the reason why he keeps mentioning Nova as the biggest and most important consumer of Neutron's API stems from an assumption that Nova would need to change to use the GBP API. If I've understood the follow on discussions correctly, there's no need for Nova to use the GBP API at all until/unless the Nova developers see benefit in it because they can continue to accomplish everything with the existing API. The GBP API simply provides a more application centric rather than network centric representation of the same thing. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] fair standards for all hypervisor drivers
On 2014-08-11 08:04:34 -0400 (-0400), Russell Bryant wrote: Dang, I'd love to see those numbers. :-) Me too. Now that I'm not travelling I'll see if I can find out what he meant by that. Understood. Some questions ... is building an image that has libvirt and qemu pre-installed from source good enough? It avoids the dependency on job runs, but moves it to image build time though, so it still exists. Moving complex stability risks to image creation time still causes us to potentially fail to update our worker images as often, which means tests randomly run on increasingly stale systems in some providers/regions until the issue is noticed, identified and addressed. That said, we do already compile some things during job runs today (in particular, library bindings which get install-time linked by some Python modules). In reality, depending on more things gathered from different places on the Internet (be it Git repository sites like GitHub/Bitbucket, or private package collections) decreases our overall stability far more than compiling things does. If the above still doesn't seem like a workable setup, then I think we should just go straight to an image with fedora + virt-preview repo, which kind of sounds easier, anyway. If it's published from EPEL or whatever Fedora's equivalent is, then that's probably fine. If it's served from a separate site, then that increases the chances that we run into network issues either at image build time or job run time. Also, we would want to make sure whatever solution we settle on is well integrated within DevStack itself, so that individual developers can recreate these conditions themselves without a lot of additional work. One other thing to keep in mind... Fedora's lifecycle is too short for us to support outside of jobs for our master branches, so this would not be a solution beyond release time (we couldn't continue to run these jobs for Juno once released if the solution hinges on Fedora). Getting the versions we want developers and deployers to use into Ubuntu 14.04 Cloud Archive and CentOS (RHEL) 7 EPEL on the other hand would be a much more viable long-term solution. -- Jeremy Stanley ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [keystone] how to generate keystone.conf.sample in havana release?
Unfortunately, there is no way to generate a sample conf in the Havana release. The etc/keystone.conf.sample included in Havana was maintained manually. In icehouse (and master), you can generate a sample configuration file using tox: $ tox -e sample_config Or run generate_sample.sh directly if you already have all the dependencies available: $ sh tools/config/generate_sample.sh -- Forwarded message -- From: jazeltq jaze...@163.com Date: Mon, Aug 11, 2014 at 5:06 AM Subject: how to generate keystone.conf.sample in havana release? To: Dolph Mathews dolph.math...@gmail.com Hello Is there a way to auto-generate keystone.conf.sample in havana release keystone? -- This message was sent from Launchpad by jazeltq (https://launchpad.net/~jazeltq-k) using the Contact this team's admins link on the Keystone Drivers team page (https://launchpad.net/~keystone-drivers). For more information see https://help.launchpad.net/YourAccount/ContactingPeople ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [sahara] Backports for 2014.1.2
There were no proposals for additional backports for sahara 2014.1.2, so, I'm going to release it later today. On Thu, Aug 7, 2014 at 10:48 PM, Sergey Lukjanov slukja...@mirantis.com wrote: Hey sahara folks, I'm going to push 2014.1.2 tag to stable/icehouse branch next week, so, please, propose backports before the weekend and ping us to backport some sensitive fixes. Thanks you! -- Sincerely yours, Sergey Lukjanov Sahara Technical Lead (OpenStack Data Processing) Principal Software Engineer Mirantis Inc. -- Sincerely yours, Sergey Lukjanov Sahara Technical Lead (OpenStack Data Processing) Principal Software Engineer Mirantis Inc. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] fair standards for all hypervisor drivers
On 08/11/2014 09:17 AM, Jeremy Stanley wrote: On 2014-08-11 08:04:34 -0400 (-0400), Russell Bryant wrote: Dang, I'd love to see those numbers. :-) Me too. Now that I'm not travelling I'll see if I can find out what he meant by that. Understood. Some questions ... is building an image that has libvirt and qemu pre-installed from source good enough? It avoids the dependency on job runs, but moves it to image build time though, so it still exists. Moving complex stability risks to image creation time still causes us to potentially fail to update our worker images as often, which means tests randomly run on increasingly stale systems in some providers/regions until the issue is noticed, identified and addressed. That said, we do already compile some things during job runs today (in particular, library bindings which get install-time linked by some Python modules). In reality, depending on more things gathered from different places on the Internet (be it Git repository sites like GitHub/Bitbucket, or private package collections) decreases our overall stability far more than compiling things does. If the above still doesn't seem like a workable setup, then I think we should just go straight to an image with fedora + virt-preview repo, which kind of sounds easier, anyway. If it's published from EPEL or whatever Fedora's equivalent is, then that's probably fine. If it's served from a separate site, then that increases the chances that we run into network issues either at image build time or job run time. Also, we would want to make sure whatever solution we settle on is well integrated within DevStack itself, so that individual developers can recreate these conditions themselves without a lot of additional work. EPEL is a repo produced by the Fedora project for RHEL and its derivatives. The virt-preview repo is hosted on fedorapeople.org, which is where custom repos live. I'd say it's more analogous to Ubuntu's PPAs. https://fedorapeople.org/groups/virt/virt-preview/ One other thing to keep in mind... Fedora's lifecycle is too short for us to support outside of jobs for our master branches, so this would not be a solution beyond release time (we couldn't continue to run these jobs for Juno once released if the solution hinges on Fedora). Getting the versions we want developers and deployers to use into Ubuntu 14.04 Cloud Archive and CentOS (RHEL) 7 EPEL on the other hand would be a much more viable long-term solution. Yep, makes sense. For testing bleeding edge, I've also got my eye on how we could do this with CentOS. There is a virt SIG in CentOS that I'm hoping will produce something similar to Fedora's virt-preview repo, but it's not there yet. I'm going to go off and discuss this with the SIG there. http://wiki.centos.org/SpecialInterestGroup/Virtualization -- Russell Bryant ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [sahara] new client release (0.7.1)
Hey sahara folks, the latest sahara client release was at Mar 29 (right after the fully completed renaming process) and so we have a lot of unreleased fixes and improvements. You can check diff [0] and launchpad's release page [1]. I'm going to release 0.7.1 this week, so, please propose changes to be included into the new client release in the next few days. Thanks. [0] https://github.com/openstack/python-saharaclient/compare/0.7.0...HEAD [1] https://launchpad.net/python-saharaclient/+milestone/0.7.1 -- Sincerely yours, Sergey Lukjanov Sahara Technical Lead (OpenStack Data Processing) Principal Software Engineer Mirantis Inc. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] fair standards for all hypervisor drivers
On 2014-08-11 08:07:02 -0400 (-0400), Russell Bryant wrote: There's a pypi mirror at least. I'm not sure about mirroring other things. Right, that's a specific solution for mirroring the pypi.python.org cheeseshop. We've got our (Infra) sights set on mirroring Ubuntu and CentOS package repositories to similarly reduce the incidence of job run-time failures we see getting updated packages and indexes from distro sites and cloud provider mirrors, but those too are nontrivial efforts and will be a while yet before we have them in place. Mirroring things is generally complex, since different kinds of files/data need widely differing retrieval, indexing and caching solutions--there's no one-size-fits-all option really. Perhaps another good example is the Fedora qcow2 image we download and cache on DevStack worker images so that Heat can perform some of its more complex integration tests... failures encountered when obtaining that image from dl.fedoraproject.org are (last time I checked anyway) our most frequent cause of nodepool update problems. We could set up our own mirror of that file of course, but to some extent that's still just moving the problem--each additional mirroring solution is something new we have to monitor, maintain and troubleshoot so we must ask ourselves whether the increased management burden from that new complexity is balanced by potential decreases in management burden found by improving stability in other parts of the system. -- Jeremy Stanley ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [designate] [neutron] designate and neutron integration
Hi, I want to ask about neutron and designate integration. I think dnsmasq fowards DNS request from instance to designate is better. ++ |DNS server(designate) | ++ | -+--+-- Network1 | ++ |dnsmasq | ++ | -+--+-- Network2 | +-+ |instance | +-+ Because it's simpler than virtual router connects Network1 and Network2. If router connects network, instance should know where DNS server is. it is complicated. dnsmasq returns its ip address as dns server in DHCP replay by ordinary, so, I think good dnsmasq becomes like a gateway to designate. But, I can't connect dnsmasq to Network1. because of today's neutron design. Question: Does designate design team have a plan such as above integration? or other integration design? *1: Network1 and Network2 are deployed by neutron. *2: neutron deploys dnsmasq as a dhcp server. dnsmasq can forward DNS request. Thanks, kazuhiro MIYASHITA ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [OSSN 0022] Nova Networking does not enforce security group rules following a soft reboot of an instance
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Nova Networking does not enforce security group rules following a soft reboot of an instance - --- ### Summary ### In deployments using Nova Networking, security group rules associated with an instance may not be enforced after a soft reboot. Nova is designed to apply the configured security group rules to an instance when certain operations are performed, such as a normal boot operation. If an operation has been performed that results in the clearing of security group rules, such as restarting the nova compute service, then performing a soft reboot of that instance will cause it to be started without security group rules being applied. Deployments using Neutron are not impacted. ### Affected Services / Software ### Nova, Havana, Grizzly ### Discussion ### In Nova deployments using Nova Networking, security groups are implemented using iptables, which is used to configure and control network traffic into Nova instances. When an instance is first booted using the normal boot method (nova boot instance_id), the security group rules are applied to that instance. When an instance is rebooted using the soft reboot method (nova reboot instance_id), the security group rules are not reapplied since they should have been already applied when the instance was initially booted. If the security group rules have not been applied following an event that resulted in their clearing, such as restarting the compute service, the instance will be brought up without security group enforcement. This situation is most likely to arise in cases where the Nova compute service has been terminated or restarted, which removes all iptables rules. If a stopped instance is then started by using a soft reboot, it will not have any security group rules applied. A hard reboot (nova reboot --hard instance_id) reapplies the security group rules, so it is not susceptible to this issue. Depending on the deployment architecture, this could breach security assumptions and leave an instance vulnerable to network based attacks. This issue only affects the Havana and Grizzly releases. The Icehouse release does not allow a stopped instance to be started using a soft reboot, therefore this issue does not affect the Icehouse release. ### Recommended Actions ### Do not to use the soft reboot method to start instances from the stopped state. If instances are in the stopped state, boot using nova boot instance_id or reboot using nova reboot --hard instance_id to force the security group rules to be applied. ### Contacts / References ### This OSSN : https://wiki.openstack.org/wiki/OSSN/OSSN-0022 Original LaunchPad Bug : https://bugs.launchpad.net/nova/+bug/1316822 OpenStack Security ML : openstack-secur...@lists.openstack.org OpenStack Security Group : https://launchpad.net/~openstack-ossg -BEGIN PGP SIGNATURE- Version: GnuPG v1 Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQEcBAEBAgAGBQJT6MtUAAoJEJa+6E7Ri+EV2t4H/RwM5auF4ik9tdZuRNVLI/Rv Si1SBx+EZA5QOksJ1r476mm2nA9+OnhCZRVnz8bHGiKtzC7eRV82c1OO1q0R/w1/ zHuTd6ZsZaAyBshfv6YoEVoRn6I2duYiR6gRHz/+hfrItt5E+SYBXqFskJazJ6dF PZBA16qrIPmeir1eDpDvFnpbMEkAMUDKGZNba/DXwEZgdKVdFaWPI3cNIsHVP6bL fsqMX+7bldB8kBDKfyFleFYqBUI5Anonzrq1tu6TS0vXl4gsfD2LE5LphPTYaiC+ EMypsACSafd19RhpeHS7/I1iVhf/Xa26Bvne0T8HWMLuDd1M5TT9E7nnas45k24= =1x8I -END PGP SIGNATURE- ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] fair standards for all hypervisor drivers
On Mon, Aug 11, 2014 at 01:45:56PM +, Jeremy Stanley wrote: On 2014-08-11 08:07:02 -0400 (-0400), Russell Bryant wrote: There's a pypi mirror at least. I'm not sure about mirroring other things. Right, that's a specific solution for mirroring the pypi.python.org cheeseshop. We've got our (Infra) sights set on mirroring Ubuntu and CentOS package repositories to similarly reduce the incidence of job run-time failures we see getting updated packages and indexes from distro sites and cloud provider mirrors, but those too are nontrivial efforts and will be a while yet before we have them in place. Mirroring things is generally complex, since different kinds of files/data need widely differing retrieval, indexing and caching solutions--there's no one-size-fits-all option really. If there are specific things we could do to libvirt.org / qemu.org download sites, to make mirroring easier or more reliable for OpenStack, we could certainly explore options there, since we know the right people involved in both projects. Regards, Daniel -- |: http://berrange.com -o-http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :| ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Neutron][LBaaS] Continuing on Calling driver interface on every API request
Hi, Validations such as timeout delay should be performed on the API level before it reaches the driver. For a configuration tree (lb, listeners, pools, etc.), there should be one provider. Having provider defined in multiple places does not make sense. -San. From: Vijay Venkatachalam [mailto:vijay.venkatacha...@citrix.com] Sent: Monday, August 11, 2014 2:43 PM To: OpenStack Development Mailing List (openstack-dev@lists.openstack.org) Subject: [openstack-dev] [Neutron][LBaaS] Continuing on Calling driver interface on every API request Hi: Continuing from last week's LBaaS meeting... Currently an entity cannot be sent to driver unless it is linked to loadbalancer because loadbalancer is the root object and driver information is only available with loadbalancer. The request to the driver is delayed because of which error propagation becomes tricky. Let's say a monitor was configured with timeout delay there would be no error then. When a listener is configured there will be a monitor creation/deployment error like timeout configured greater than delay. Unless the error is very clearly crafted the user won't be able to understand the error. I am half-heartedly OK with current approach. But, I would prefer Brandon's Solution - make provider an attribute in each of the entities to get rid of this problem. What do others think? Thanks, Vijay V. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] fair standards for all hypervisor drivers
On 2014-08-11 14:56:27 +0100 (+0100), Daniel P. Berrange wrote: If there are specific things we could do to libvirt.org / qemu.org download sites, to make mirroring easier or more reliable for OpenStack, we could certainly explore options there, since we know the right people involved in both projects. Mirroring is generally a reaction to stability issues more than anything. Providing those files from multiple locations behind a global cache/load-balancing solution (maybe a CDN) to ensure stability would likely help prevent us from needing to have yet another one-off mirror for a handful of files. It's worth adding to https://wiki.openstack.org/wiki/Meetings/InfraTeamMeeting#Agenda_for_next_meeting and discussing tomorrow, if you're around, so we can be sure to get input from more of the Infra team. -- Jeremy Stanley ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] PCI support
Gary, Cisco is adding it in our CI testbed. I guess that mlnx is doing the same for their MD as well. —Robert On 8/11/14, 9:05 AM, Gary Kotton gkot...@vmware.commailto:gkot...@vmware.com wrote: Hi, At the moment all of the drivers are required CI support. Are there any plans regarding the PIC support. I understand that this is something that requires specific hardware. Are there any plans to add this? Thanks Gary ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Neutron] Is network ordering of vNICs guaranteed?
Armando M. [mailto:arma...@gmail.com] wrote: On 9 August 2014 10:16, Jay Pipes jaypi...@gmail.com wrote: Paul, does this friend of a friend have a reproduceable test script for this? We would also need to know the OpenStack release where this issue manifest itself. A number of bugs have been raised in the past around this type of issue, and the last fix I recall is this one: https://bugs.launchpad.net/nova/+bug/1300325 It's possible that this might have regressed, though. The reason I called it friend of a friend is because I think the info has filtered through a series of people and is not firsthand observation. I'll ask them to track back to who actually observed the behavior, how long ago, and with what version. It could be a regression, or it could just be old info that people have continued to assume is true without realizing it was considered a bug all along and has been fixed. Thanks! The moment I first heard it my first reaction was that it was almost certainly a bug and had probably already been fixed. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [keystone] how to generate keystone.conf.sample in havana release?
Ok， thank you all the same. At 2014-08-11 09:31:19, Dolph Mathews dolph.math...@gmail.com wrote: Unfortunately, there is no way to generate a sample conf in the Havana release. The etc/keystone.conf.sample included in Havana was maintained manually. In icehouse (and master), you can generate a sample configuration file using tox: $ tox -e sample_config Or run generate_sample.sh directly if you already have all the dependencies available: $ sh tools/config/generate_sample.sh -- Forwarded message -- From: jazeltqjaze...@163.com Date: Mon, Aug 11, 2014 at 5:06 AM Subject: how to generate keystone.conf.sample in havana release? To: Dolph Mathews dolph.math...@gmail.com Hello Is there a way to auto-generate keystone.conf.sample in havana release keystone? -- This message was sent from Launchpad by jazeltq (https://launchpad.net/~jazeltq-k) using the Contact this team's admins link on the Keystone Drivers team page (https://launchpad.net/~keystone-drivers). For more information see https://help.launchpad.net/YourAccount/ContactingPeople ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] PCI support
Thanks for the update. From: Robert Li (baoli) ba...@cisco.commailto:ba...@cisco.com Reply-To: OpenStack List openstack-dev@lists.openstack.orgmailto:openstack-dev@lists.openstack.org Date: Monday, August 11, 2014 at 5:08 PM To: OpenStack List openstack-dev@lists.openstack.orgmailto:openstack-dev@lists.openstack.org Subject: Re: [openstack-dev] [Nova] PCI support Gary, Cisco is adding it in our CI testbed. I guess that mlnx is doing the same for their MD as well. -Robert On 8/11/14, 9:05 AM, Gary Kotton gkot...@vmware.commailto:gkot...@vmware.com wrote: Hi, At the moment all of the drivers are required CI support. Are there any plans regarding the PIC support. I understand that this is something that requires specific hardware. Are there any plans to add this? Thanks Gary ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [OpenStack][InstanceGroup] metadetails and metadata in instance_group.py
On 08/10/2014 10:36 PM, Jay Lau wrote: I was asking this because I got a -2 for https://review.openstack.org/109505 , just want to know why this new term metadetails was invented when we already have details, metadata, system_metadata, instance_metadata, and properties (on images and volumes). As the person who -2'd the review, I'm thankful you raised this issue on the ML, Jay. Much appreciated. Eagerly awaiting answers, -jay ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [TripleO][heat] a small experiment with Ansible in TripleO
Excerpts from Steve Baker's message of 2014-08-10 15:33:26 -0700: On 02/08/14 04:07, Allison Randal wrote: A few of us have been independently experimenting with Ansible as a backend for TripleO, and have just decided to try experimenting together. I've chatted with Robert, and he says that TripleO was always intended to have pluggable backends (CM layer), and just never had anyone interested in working on them. (I see it now, even in the early docs and talks, I guess I just couldn't see the forest for the trees.) So, the work is in line with the overall goals of the TripleO project. We're starting with a tiny scope, focused only on updating a running TripleO deployment, so our first work is in: - Create an Ansible Dynamic Inventory plugin to extract metadata from Heat - Improve/extend the Ansible nova_compute Cloud Module (or create a new one), for Nova rebuild - Develop a minimal handoff from Heat to Ansible, particularly focused on the interactions between os-collect-config and Ansible We're merging our work in this repo, until we figure out where it should live: https://github.com/allisonrandal/tripleo-ansible We've set ourselves one week as the first sanity-check to see whether this idea is going anywhere, and we may scrap it all at that point. But, it seems best to be totally transparent about the idea from the start, so no-one is surprised later. Having pluggable backends for configuration seems like a good idea, and Ansible is a great choice for the first alternative backend. TripleO is intended to be loosely coupled for many components, not just in-instance configuration. However what this repo seems to be doing at the moment is bypassing heat to do a stack update, and I can only assume there is an eventual goal to not use heat at all for stack orchestration too. Granted, until blueprint update-failure-recovery lands[1] then doing a stack-update is about as much fun as russian roulette. But this effort is tactical rather than strategic, especially given TripleO's mission statement. We intend to stay modular. Ansible won't replace Heat from end to end. Right now we're stuck with an update that just doesn't work. It isn't just about update-failure-recovery, which is coming along nicely, but it is also about the lack of signals to control rebuild, poor support for addressing machines as groups, and unacceptable performance in large stacks. We remain committed to driving these things into Heat, which will allow us to address these things the way a large scale operation will need to. But until we can land those things in Heat, we need something more flexible like Ansible to go around Heat and do things in the exact order we need them done. Ansible doesn't have a REST API, which is a non-starter for modern automation, but the need to control workflow is greater than the need to have a REST API at this point. If I were to use Ansible for TripleO configuration I would start with something like the following: * Install an ansible software-config hook onto the image to be triggered by os-refresh-config[2][3] * Incrementally replace StructuredConfig resources in tripleo-heat-templates with SoftwareConfig resources that include the ansible playbooks via get_file * The above can start in a fork of tripleo-heat-templates, but can eventually be structured using resource providers so that the deployer chooses what configuration backend to use by selecting the environment file that contains the appropriate config resources Now you have a cloud orchestrated by heat and configured by Ansible. If it is still deemed necessary to do an out-of-band update to the stack then you're in a much better position to do an ansible push, since you can use the same playbook files that heat used to bring up the stack. That would be a good plan if we wanted to fix issues with os-*-config, but that is the opposite of reality. We are working around Heat orchestration issues with Ansible. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Neutron] Is network ordering of vNICs guaranteed?
Thanks, Paul! On 08/11/2014 10:10 AM, CARVER, PAUL wrote: Armando M. [mailto:arma...@gmail.com] wrote: On 9 August 2014 10:16, Jay Pipes jaypi...@gmail.com wrote: Paul, does this friend of a friend have a reproduceable test script for this? We would also need to know the OpenStack release where this issue manifest itself. A number of bugs have been raised in the past around this type of issue, and the last fix I recall is this one: https://bugs.launchpad.net/nova/+bug/1300325 It's possible that this might have regressed, though. The reason I called it friend of a friend is because I think the info has filtered through a series of people and is not firsthand observation. I'll ask them to track back to who actually observed the behavior, how long ago, and with what version. It could be a regression, or it could just be old info that people have continued to assume is true without realizing it was considered a bug all along and has been fixed. Thanks! The moment I first heard it my first reaction was that it was almost certainly a bug and had probably already been fixed. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [OpenStack][InstanceGroup] metadetails and metadata in instance_group.py
As the person who -2'd the review, I'm thankful you raised this issue on the ML, Jay. Much appreciated. The metadetails term isn't being invented in this patch, of course. I originally complained about the difference when this was being added: https://review.openstack.org/#/c/109505/1/nova/api/openstack/compute/contrib/server_groups.py,cm As best I can tell, the response in that patch set about why it's being translated is wrong (backwards). I expect that the API extension at the time called it metadetails and they decided to make the object the same and do the translation there. From what I can tell, the actual server_group API extension that made it into the tree never got the ability to set/change/etc the metadata/metadetails anyway, so there's no reason (AFAICT) to add it in wrongly. If we care to have this functionality, then I propose we change the attribute on the object (we can handle this with versioning) and reflect it as metadata in the API. However, I have to ask: do we really need another distinct metadata store attached to server_groups? If not, how about we just remove it from the database and the object, clean up the bit of residue that is still in the API extension and be done with it? --Dan signature.asc Description: OpenPGP digital signature ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [OpenStack][InstanceGroup] metadetails and metadata in instance_group.py
On 08/11/2014 11:06 AM, Dan Smith wrote: As the person who -2'd the review, I'm thankful you raised this issue on the ML, Jay. Much appreciated. The metadetails term isn't being invented in this patch, of course. I originally complained about the difference when this was being added: https://review.openstack.org/#/c/109505/1/nova/api/openstack/compute/contrib/server_groups.py,cm As best I can tell, the response in that patch set about why it's being translated is wrong (backwards). I expect that the API extension at the time called it metadetails and they decided to make the object the same and do the translation there. From what I can tell, the actual server_group API extension that made it into the tree never got the ability to set/change/etc the metadata/metadetails anyway, so there's no reason (AFAICT) to add it in wrongly. If we care to have this functionality, then I propose we change the attribute on the object (we can handle this with versioning) and reflect it as metadata in the API. However, I have to ask: do we really need another distinct metadata store attached to server_groups? No. If not, how about we just remove it from the database and the object, clean up the bit of residue that is still in the API extension and be done with it? +1 -jay ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [TripleO][heat] a small experiment with Ansible in TripleO
On 11/08/14 10:46, Clint Byrum wrote: Right now we're stuck with an update that just doesn't work. It isn't just about update-failure-recovery, which is coming along nicely, but it is also about the lack of signals to control rebuild, poor support for addressing machines as groups, and unacceptable performance in large stacks. Are there blueprints/bugs filed for all of these issues? -ZB ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Neutron] Simple proposal for stabilizing new features in-tree
On 8/11/14, 4:52 AM, Thierry Carrez wrote: gustavo panizzo (gfa) wrote: only one think i didn't like it why all url,api, etc has to include the word 'preview'? i imagine that i would be consuming the new feature using heat, puppet, local scripts, custom horizon, whatever. Why do you make me to change all them when the feature moves out of preview? it could be a lot of rework (for consumers) without gain. I would totally support other big fat warnings everywhere (logs, documentation, startup log of neutron-server) but don't change the API if is not necessary I see two issues with this proposal: the first one is what Gustavo just said: the use of the preview package/configoption/API creates friction when the feature needs to go mainstream (package renaming, change in configuration for deployers, change in API calls for users...). Hi Thierry, I completely agree with you and with Gustavo that mangling the REST URIs to include preview may have more cost (i.e. friction when the API becomes stable) than benefit. I'm happy to drop that particular part of the proposal. The email was intended to kick off discussion of these sorts of details. My understanding is that the goal is to make it easy for people to try the preview feature, and keeping the experimental feature in-tree is seen as simpler to experiment with. But the pain from this friction imho outweighs the pain of deploying an out-of-tree plugin for deployers. I agree out-of-tree is a better option for truly experimental features. This in-tree stabilization is intended for a beta release, as opposed to a prototype. The second issue is that once the feature is in preview in tree, it moves the responsibility/burden of making it official (or removed) to the core developers (as Salvatore mentioned). I kind of like the approach where experimental features are developed in faster iterations out-of-tree and when they are celebrated by experimenters and are ready to be stable-supported, they are moved in tree. I don't think we are really disagreeing here. There are clearly situations where rapid iteration out-of-tree, without the burden of the core review process, is most appropriate. But this proposal is intended for features that are on the cusp of being declared stable, rather than for experimentation. The intent is absolutely to have all changes to the code go through the regular core review process during this stabilization phase. This enables the feature to be fully reviewed and integrated (also including CLIs, Horizon and Heat support, documentation, etc.) at the point when the decision is made that no further incompatible API changes will be needed. Once the feature is declared stable, from the end-user perspective, its just a matter of removing the preview label. Moving the feature's code from the preview subtree to its normal locations in the tree will not effect most users or operators. Note that the GBP team had implemented a proof-of-concept prior to the start of the Juno cycle out-of-tree. Our initial plan was to get this PoC code reviewed and merged at the beginning of Juno, and then iteratively improve it throughout the cycle. But we got a lot of resistance to the idea of merging a large body of code that had been developed outside the Neutron development and review process. We've instead had to break it into multiple pieces, and make sure each of those is production ready, to have any chance of getting through the review process during Juno. Its not really clear that something significant developed externally can ever be moved in tree, at least without major upheaval, including incompatible API changes, as it goes through the review/merge process. Finally, consider that many interesting potential features for Neutron involve integrations with external back-ends, such as ODL or vendor-specific devices or controllers, along with a reference implementation that doesn't depend on external systems. To really validate that the API, the model, and the driver framework code for a new feature are all stable, it is necessary to implement and deploy several of these back-end integrations along with the reference implementation. But vendors may not be willing to invest substantially in this integration effort without the assurances about the quality and relative stability of the interfaces involved that comes from the core review process, and without the clear path to market that comes with in-tree development of an approved blueprint targeted at a specific Neutron release. -Bob Regards, ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Manila] Incubation request
I just saw the agenda for tomorrow's TC meeting and we're on it. I plan to be there. https://wiki.openstack.org/wiki/Meetings#Technical_Committee_meeting -Ben From: Swartzlander, Ben [mailto:ben.swartzlan...@netapp.com] Sent: Monday, July 28, 2014 9:53 PM To: openstack-dev@lists.openstack.org Subject: [openstack-dev] [Manila] Incubation request Manila has come a long way since we proposed it for incubation last autumn. Below are the formal requests. https://wiki.openstack.org/wiki/Manila/Incubation_Application https://wiki.openstack.org/wiki/Manila/Program_Application Anyone have anything to add before I forward these to the TC? -Ben Swartzlander ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [Fuel] 6.0 blueprints cleanup
We've moved all blueprints from 6.0 to 'next' milestone. It has been done in order to better view of stuff that we really want to implement in 6.0. Feature freeze for 6.0 release is planned to 18th of September. If you are going to merge your blueprint before that date, you can move it to 6.0 milestone and 6.0.x series. But blueprint must have fixed scope and must be assigned to person who will lead this activity. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Fuel] 6.0 blueprints cleanup
All, Please refer to this email for a list of information that has to be populated in a blueprint before it can be assigned and scheduled: http://lists.openstack.org/pipermail/openstack-dev/2014-August/042042.html Please don't schedule a blueprint to 6.0 until it has these details and its assignees are confirmed. On Mon, Aug 11, 2014 at 8:30 AM, Dmitry Pyzhov dpyz...@mirantis.com wrote: We've moved all blueprints from 6.0 to 'next' milestone. It has been done in order to better view of stuff that we really want to implement in 6.0. Feature freeze for 6.0 release is planned to 18th of September. If you are going to merge your blueprint before that date, you can move it to 6.0 milestone and 6.0.x series. But blueprint must have fixed scope and must be assigned to person who will lead this activity. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Dmitry Borodaenko ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Neutron][LBaaS] Continuing on Calling driver interface on every API request
Yeah what I meant was the only solution I could come up with so that the driver gets passed every call is to have the every entity have a provider. I do believe this is a bit cumbersome for a user, and extra validation would be needed to verify that two entities linked together cannot have different providers, but that would be pretty easy. In my opinion, it'd be a bit weird to have them all have a provider. However, there are some pros to it such as: 1) Driver always gets the create, update, and delete calls. 2) If drivers support a varying range of values for certain entity attributes, that validation can be caught immediately if thats something people wanted. 3) Will remove the necessity of a DEFERRED status for some drivers (This also brings up a CON, in that some drivers may use DEFERRED and some may not, which leads to an inconsistent UX). 4) Status management in some drivers will become a bit easier. Still I don't think it is something that should be done because having the user give a provider for every entity is a bit cumbersome. Though if enough people want this then a larger discussion about it should probably happen. Thanks, Brandon On Mon, 2014-08-11 at 13:57 +, Samuel Bercovici wrote: Hi, Validations such as “timeout delay” should be performed on the API level before it reaches the driver. For a configuration tree (lb, listeners, pools, etc.), there should be one provider. Having provider defined in multiple places does not make sense. -San. From: Vijay Venkatachalam [mailto:vijay.venkatacha...@citrix.com] Sent: Monday, August 11, 2014 2:43 PM To: OpenStack Development Mailing List (openstack-dev@lists.openstack.org) Subject: [openstack-dev] [Neutron][LBaaS] Continuing on Calling driver interface on every API request Hi: Continuing from last week’s LBaaS meeting… Currently an entity cannot be sent to driver unless it is linked to loadbalancer because loadbalancer is the root object and driver information is only available with loadbalancer. The request to the driver is delayed because of which error propagation becomes tricky. Let’s say a monitor was configured with timeout delay there would be no error then. When a listener is configured there will be a monitor creation/deployment error like “timeout configured greater than delay”. Unless the error is very clearly crafted the user won’t be able to understand the error. I am half-heartedly OK with current approach. But, I would prefer Brandon’s Solution – make provider an attribute in each of the entities to get rid of this problem. What do others think? Thanks, Vijay V. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] devstack local.conf file
Hi, I have gone through devstack links. They are not clear like openstack.org documents. For Example: when i am using below local.conf file in devstack,hp_msa_driver is not coming in enabled_backends in cinder.conf after running stack.sh. [[local|localrc]] ADMIN_PASSWORD=vedams123 DATABASE_PASSWORD=$ADMIN_PASSWORD RABBIT_PASSWORD=$ADMIN_PASSWORD SERVICE_PASSWORD=$ADMIN_PASSWORD SERVICE_TOKEN=ADMIN FLAT_INTERFACE=eth0 FIXED_RANGE=192.168.2.170/29 HOST_IP=192.168.2.151 LOGFILE=$DEST/logs/stack.sh.log SCREEN_LOGDIR=$DEST/logs/screen SYSLOG=True SYSLOG_HOST=$HOST_IP SYSLOG_PORT=516 TEMPEST_VOLUME_DRIVER=hp_msa_fc TEMPEST_VOLUME_VENDOR=Hewlett-Packard TEMPEST_STORAGE_PROTOCOL=FC [[post-config|$CINDER_CONF]] [hp_msa_driver] volume_driver = cinder.volume.drivers.san.hp.hp_msa_fc.HPMSAFCDriver san_ip = 192.168.2.192 san_login = manage san_password =!manage volume_backend_name=HPMSA_FC [lvmdriver-1] volume_group=stack-volumes-1 volume_driver=cinder.volume.drivers.lvm.LVMISCSIDriver volume_backend_name=LVM_iSCSI *I am getting below cinder.conf file after running stack.sh script* [keystone_authtoken] auth_uri = http://192.168.2.151:5000/v2.0 signing_dir = /var/cache/cinder admin_password = vedams123 admin_user = cinder admin_tenant_name = service cafile = identity_uri = http://192.168.2.151:35357 [DEFAULT] rabbit_password = vedams123 rabbit_hosts = 192.168.2.151 rpc_backend = cinder.openstack.common.rpc.impl_kombu use_syslog = True *default_volume_type = lvmdriver-1* *enabled_backends = lvmdriver-1* enable_v1_api = true periodic_interval = 60 lock_path = /opt/stack/data/cinder state_path = /opt/stack/data/cinder osapi_volume_extension = cinder.api.contrib.standard_extensions rootwrap_config = /etc/cinder/rootwrap.conf api_paste_config = /etc/cinder/api-paste.ini sql_connection = mysql://root:vedams123@127.0.0.1/cinder?charset=utf8 iscsi_helper = tgtadm my_ip = 192.168.2.151 verbose = True debug = True auth_strategy = keystone [lvmdriver-1] volume_group = stack-volumes-1 volume_driver = cinder.volume.drivers.lvm.LVMISCSIDriver volume_backend_name = LVM_iSCSI [hp_msa_driver] volume_backend_name = HPMSA_FC san_password = !manage san_login = manage san_ip = 192.168.2.192 volume_driver = cinder.volume.drivers.san.hp.hp_msa_fc.HPMSAFCDriver *Then i analyzed source code of stack.sh,and added in local.conf this line:* *CINDER_ENABLED_BACKENDS=hp_msa:hp_msa_driver,lvm:lvmdriver-1* Now i am getting hp_msa_fc in cinder.conf in enabled_backends Regards Nikesh ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Neutron][LBaaS] Continuing on Calling driver interface on every API request
Hi all, Validations such as ³timeout delay² should be performed on the API level before it reaches the driver. For a configuration tree (lb, listeners, pools, etc.), there should be one provider. You¹re right, but I think the point of Vijay¹s example was to highlight the combo error problem with populating all of the driver objects at once (in short, the driver interface isn¹t well suited to that model.) That his one example can be covered by API validators is irrelevant. Consider a backend that does not support APP_COOKIE¹s, or HTTPS_TERMINATED (but has multiple listeners) instead. Should the entire load balancer create fail, or should it offer degraded service? Do all drivers have to implement a transaction rollback; wait, the interface makes that very hard. That¹s his point. The driver is no longer just glue code between interfaces; it¹s now a mini-object error handler. Having provider defined in multiple places does not make sense. Channeling Brandon, who can yell if I get this wrong, the point is not to have a potentially different provider on each object. It¹s to allow a provider to be assigned when the first object in the tree is created, so that future related objects will always get routed to the same provider. Not knowing which provider should get all the objects is why we have to wait until we see a LoadBalancer object. All of this sort of edge case nonsense is because we (the royal we, the community), wanted all load balancer objects to be ³root² objects, even though only one of them is an actual root today, to support many-to-many relationships among all of them, at some future date, without an interface change. If my bias is showing that I¹m not a fan of adding this complexity for that, I¹m not surprised. Thanks, doug On 8/11/14, 7:57 AM, Samuel Bercovici samu...@radware.com wrote: Hi, Validations such as ³timeout delay² should be performed on the API level before it reaches the driver. For a configuration tree (lb, listeners, pools, etc.), there should be one provider. Having provider defined in multiple places does not make sense. -San. From: Vijay Venkatachalam [mailto:vijay.venkatacha...@citrix.com] Sent: Monday, August 11, 2014 2:43 PM To: OpenStack Development Mailing List (openstack-dev@lists.openstack.org) Subject: [openstack-dev] [Neutron][LBaaS] Continuing on Calling driver interface on every API request Hi: Continuing from last week¹s LBaaS meetingŠ Currently an entity cannot be sent to driver unless it is linked to loadbalancer because loadbalancer is the root object and driver information is only available with loadbalancer. The request to the driver is delayed because of which error propagation becomes tricky. Let¹s say a monitor was configured with timeout delay there would be no error then. When a listener is configured there will be a monitor creation/deployment error like ³timeout configured greater than delay². Unless the error is very clearly crafted the user won¹t be able to understand the error. I am half-heartedly OK with current approach. But, I would prefer Brandon¹s Solution make provider an attribute in each of the entities to get rid of this problem. What do others think? Thanks, Vijay V. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Which program for Rally
On 08/06/2014 05:48 PM, John Griffith wrote: I have to agree with Duncan here. I also don't know if I fully understand the limit in options. Stress test seems like it could/should be different (again overlap isn't a horrible thing) and I don't see it as siphoning off resources so not sure of the issue. We've become quite wrapped up in projects, programs and the like lately and it seems to hinder forward progress more than anything else. I'm also not convinced that Tempest is where all things belong, in fact I've been thinking more and more that a good bit of what Tempest does today should fall more on the responsibility of the projects themselves. For example functional testing of features etc, ideally I'd love to have more of that fall on the projects and their respective teams. That might even be something as simple to start as saying if you contribute a new feature, you have to also provide a link to a contribution to the Tempest test-suite that checks it. Sort of like we do for unit tests, cross-project tracking is difficult of course, but it's a start. The other idea is maybe functional test harnesses live in their respective projects. Honestly I think who better to write tests for a project than the folks building and contributing to the project. At some point IMO the QA team isn't going to scale. I wonder if maybe we should be thinking about proposals for delineating responsibility and goals in terms of functional testing? All good points. Your last paragraph was discussed by the QA team leading up to and at the Atlanta summit. The conclusion was that the api/functional tests focused on a single project should be part of that project. As Sean said, we can envision there being half (or some other much smaller number) as many such tests in tempest going forward. Details are under discussion, but the way this is likely to play out is that individual projects will start by creating their own functional tests outside of tempest. Swift already does this and neutron seems to be moving in that direction. There is a spec to break out parts of tempest (https://github.com/openstack/qa-specs/blob/master/specs/tempest-library.rst) into a library that might be used by projects implementing functional tests. Once a project has sufficient functional testing, we can consider removing its api tests from tempest. This is a bit tricky because tempest needs to cover *all* cross-project interactions. In this respect, there is no clear line in tempest between scenario tests which have this goal explicitly, and api tests which may also involve interactions that might not be covered in a scenario. So we will need a principled way to make sure there is complete cross-project coverage in tempest with a smaller number of api tests. -David ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Fuel] 6.0 blueprints cleanup
+2, yes please On Mon, Aug 11, 2014 at 7:42 PM, Dmitry Borodaenko dborodae...@mirantis.com wrote: All, Please refer to this email for a list of information that has to be populated in a blueprint before it can be assigned and scheduled: http://lists.openstack.org/pipermail/openstack-dev/2014-August/042042.html Please don't schedule a blueprint to 6.0 until it has these details and its assignees are confirmed. On Mon, Aug 11, 2014 at 8:30 AM, Dmitry Pyzhov dpyz...@mirantis.com wrote: We've moved all blueprints from 6.0 to 'next' milestone. It has been done in order to better view of stuff that we really want to implement in 6.0. Feature freeze for 6.0 release is planned to 18th of September. If you are going to merge your blueprint before that date, you can move it to 6.0 milestone and 6.0.x series. But blueprint must have fixed scope and must be assigned to person who will lead this activity. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Dmitry Borodaenko ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Mike Scherbakov #mihgen ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [OpenStack][InstanceGroup] metadetails and metadata in instance_group.py
On 8/11/14, 6:06 PM, Dan Smith d...@danplanet.com wrote: As the person who -2'd the review, I'm thankful you raised this issue on the ML, Jay. Much appreciated. The metadetails term isn't being invented in this patch, of course. I originally complained about the difference when this was being added: https://review.openstack.org/#/c/109505/1/nova/api/openstack/compute/contr ib/server_groups.py,cm As best I can tell, the response in that patch set about why it's being translated is wrong (backwards). I expect that the API extension at the time called it metadetails and they decided to make the object the same and do the translation there. From what I can tell, the actual server_group API extension that made it into the tree never got the ability to set/change/etc the metadata/metadetails anyway, so there's no reason (AFAICT) to add it in wrongly. If we care to have this functionality, then I propose we change the attribute on the object (we can handle this with versioning) and reflect it as metadata in the API. However, I have to ask: do we really need another distinct metadata store attached to server_groups? If not, how about we just remove it from the database and the object, clean up the bit of residue that is still in the API extension and be done with it? The initial version of the feature did not make use of this. The reason was that we chose for a very Limited subset to be used, that is, affinity and anti affinity. Moving forwards we would like to implement A number of different policies with this. We can drop it at the moment due to the fact that it is not used. I think that Yathi may be using this for the constrain scheduler. But I am not 100% sure. --Dan ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Neutron][LBaaS] Use cases with regards to VIP and routers
Hi Susanne, While there are a few operators involved with LBaaS that would have good input, you might want to also ask this on the non-dev mailing list, for a larger sample size. Thanks, doug On 8/11/14, 3:05 AM, Susanne Balle sleipnir...@gmail.com wrote: Gang, I was asked the following questions around our Neutron LBaaS use cases: 1. Will there be a scenario where the ³VIP² port will be in a different Node, from all the Member ³VMs² in a pool. 2. Also how likely is it for the LBaaS configured subnet to not have a ³router² and just use the ³extra_routes² option. 3. Is there a valid use case where customers will be using the ³extra_routes² with subnets instead of the ³routers². ( It would be great if you have some use case picture for this). Feel free to chime in here and I'll summaries the answers. Regards Susanne ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [neutron] [third-party] Freescale CI log site is being blocked
Hi-
Today I contacted the service provider regarding the malware in the website.
Got a response that website is truly functional and malware free.
I too have verified all the directories, subdirectories and log files manually
for any malware injected into the website. I have detected none.
There is no anonymous login enabled for FTP or cPanel login to the server. The
FTP is protected with Strong Passcode.
This is an update from my end.
Like all other CI's now normal browsing of logs is available for view.
The CI is made down to take the above explained changes to take place. Now the
CI is active running the jobs.
All the old logs are present as in place.
You may browse the old logs using the url
For ML2 Mechanism driver :
http://fslopenstackci.com/{change_number}/{change_patchset}/Freescale-ML2-Mechanism-Driver
For FWaaS Plugin :
http://fslopenstackci.com/{change_number}/{change_patchset}/Freescale-FWaaS-Plugin
Now I have updated the CI to create a BUILD directory as well to showcase logs
for rechecks.
With this new change the log URL will be
For ML2 Mechanism driver :
http://fslopenstackci.com/{build_number}/{change_number}/{change_patchset}/Freescale-ML2-Mechanism-Driver
For FWaaS Plugin :
http://fslopenstackci.com/{build_number}/{change_number}/{change_patchset}/Freescale-FWaaS-Plugin
Hi Mestery-
Kindly please verify the access to the site.
Also, if it's still in blocking mode, kindly mail me the logs with Cisco WSA to
verify the reason behind this blocking.
Kindly help me with your review on my code at
https://review.openstack.org/#/c/109659/
Thanking you all.
--
Trinath Somanchi - B39208
trinath.soman...@freescale.com | Mob: +91 9866 235 130
-Original Message-
From: Anita Kuno [mailto:ante...@anteaya.info]
Sent: Saturday, August 09, 2014 11:09 AM
To: openstack-dev@lists.openstack.org
Subject: Re: [openstack-dev] [neutron] [third-party] Freescale CI log site is
being blocked
On 08/08/2014 11:27 PM, trinath.soman...@freescale.com wrote:
Thanks anita for the reply.
Previously the existing server is accessible by kyle. But now its not being
accessible.
For the paid hosting I have its administered by godaddy
If you are paying godaddy to administer the server, have you asked them why one
of your users has acknowledged your site is blacklisted by Cisco WSA appliances?
If you are paying them to administer your server, answering your question falls
within their job.
You need to find out the reason behind Cisco security blocking, that is what I
am asking you to do. It if fine if you don't know, but it is your
responsibility to find out.
Thanks Trinath,
Anita.
and the FTP is only accessed by Jenkins.
I can try relocating FTP web based file browser script and provide a normal
view of files.
Don't know the reason behind Cisco Security blocking the access where it has
given access to view the website before.
Thanks a lot again for the brief email.
--
Trinath Somanchi - B39208
trinath.soman...@freescale.com | extn: 4048
-Original Message-
From: Anita Kuno [mailto:ante...@anteaya.info]
Sent: Saturday, August 09, 2014 10:21 AM
To: openstack-dev@lists.openstack.org
Subject: Re: [openstack-dev] [neutron] [third-party] Freescale CI log
site is being blocked
On 08/08/2014 10:06 PM, trinath.soman...@freescale.com wrote:
Hi Sumit-
When I try to paste a large log text into paste.openstack, It is giving me
image verification and says its spam.
Let's not confuse paste.openstack.org's spam blocker from spam blockers on
servers. They are two separate functionalities and the conversation does not
move forward if we try to pretend they are the same thing or even remotely
related, which they are not.
If you recall, Trinath, the first server you had got hacked since you had not
hardened it appropriately. Having hosting via go daddy or any other paid
hosting service does not absolve you of the responsibility of having a well
maintained server. If you need help maintaining your server, I suggest you
contract a server administrator to advise you or do the work. We have to
assume a certain level of competence here, due to the responsibility involved
I don't think you are going to get many responses to questions if you don't
know how to maintain your server.
This isn't really the the place to ask. Running your third party ci system
and copying the logs, sure this is the place, basic server maintenance is
your responsibility.
If you recall, a basic evaluation of your server logs told you you had been
hacked the last time. This might be a place to start now.
In any case, please maintain your server and please address Kyle's concerns.
Thank you Trianth,
Anita.
I don't know why its taken as spam/malware. It's a paid hosting I had from
GODADDY.
--
Trinath Somanchi - B39208
trinath.soman...@freescale.com | extn: 4048
-Original Message-
From: Sumit Naiksatam
Re: [openstack-dev] [Neutron][LBaaS] Continuing on Calling driver interface on every API request
Hi Doug, In some implementations Driver !== Device. I think this is also true for HA Proxy. This might mean that there is a difference between creating a logical object and when there is enough information to actually schedule/place this into a device. The ability to express such errors (detecting an error on a logical object after it was created but when it actually get scheduled) should be discussed and addressed anyway. -Sam. -Original Message- From: Doug Wiegley [mailto:do...@a10networks.com] Sent: Monday, August 11, 2014 6:55 PM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [Neutron][LBaaS] Continuing on Calling driver interface on every API request Hi all, Validations such as ³timeout delay² should be performed on the API level before it reaches the driver. For a configuration tree (lb, listeners, pools, etc.), there should be one provider. You¹re right, but I think the point of Vijay¹s example was to highlight the combo error problem with populating all of the driver objects at once (in short, the driver interface isn¹t well suited to that model.) That his one example can be covered by API validators is irrelevant. Consider a backend that does not support APP_COOKIE¹s, or HTTPS_TERMINATED (but has multiple listeners) instead. Should the entire load balancer create fail, or should it offer degraded service? Do all drivers have to implement a transaction rollback; wait, the interface makes that very hard. That¹s his point. The driver is no longer just glue code between interfaces; it¹s now a mini-object error handler. Having provider defined in multiple places does not make sense. Channeling Brandon, who can yell if I get this wrong, the point is not to have a potentially different provider on each object. It¹s to allow a provider to be assigned when the first object in the tree is created, so that future related objects will always get routed to the same provider. Not knowing which provider should get all the objects is why we have to wait until we see a LoadBalancer object. All of this sort of edge case nonsense is because we (the royal we, the community), wanted all load balancer objects to be ³root² objects, even though only one of them is an actual root today, to support many-to-many relationships among all of them, at some future date, without an interface change. If my bias is showing that I¹m not a fan of adding this complexity for that, I¹m not surprised. Thanks, doug On 8/11/14, 7:57 AM, Samuel Bercovici samu...@radware.com wrote: Hi, Validations such as ³timeout delay² should be performed on the API level before it reaches the driver. For a configuration tree (lb, listeners, pools, etc.), there should be one provider. Having provider defined in multiple places does not make sense. -San. From: Vijay Venkatachalam [mailto:vijay.venkatacha...@citrix.com] Sent: Monday, August 11, 2014 2:43 PM To: OpenStack Development Mailing List (openstack-dev@lists.openstack.org) Subject: [openstack-dev] [Neutron][LBaaS] Continuing on Calling driver interface on every API request Hi: Continuing from last week¹s LBaaS meetingŠ Currently an entity cannot be sent to driver unless it is linked to loadbalancer because loadbalancer is the root object and driver information is only available with loadbalancer. The request to the driver is delayed because of which error propagation becomes tricky. Let¹s say a monitor was configured with timeout delay there would be no error then. When a listener is configured there will be a monitor creation/deployment error like ³timeout configured greater than delay². Unless the error is very clearly crafted the user won¹t be able to understand the error. I am half-heartedly OK with current approach. But, I would prefer Brandon¹s Solution make provider an attribute in each of the entities to get rid of this problem. What do others think? Thanks, Vijay V. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] devstack local.conf file
Hi Nikesh, You need to set the enabled_backends in the local.conf file. e.g. [[post-config|$CINDER_CONF]] [DEFAULT] enabled_backends=hp_msa_driver [hp_msa_driver] volume_driver = cinder.volume.drivers.san.hp.hp_msa_fc.HPMSAFCDriver … Ramy From: Nikesh Kumar Mahalka [mailto:nikeshmaha...@vedams.com] Sent: Monday, August 11, 2014 8:55 AM To: openst...@lists.openstack.org; openstack-dev@lists.openstack.org Subject: [openstack-dev] devstack local.conf file Hi, I have gone through devstack links. They are not clear like openstack.orghttp://openstack.org documents. For Example: when i am using below local.conf file in devstack,hp_msa_driver is not coming in enabled_backends in cinder.conf after running stack.sh. [[local|localrc]] ADMIN_PASSWORD=vedams123 DATABASE_PASSWORD=$ADMIN_PASSWORD RABBIT_PASSWORD=$ADMIN_PASSWORD SERVICE_PASSWORD=$ADMIN_PASSWORD SERVICE_TOKEN=ADMIN FLAT_INTERFACE=eth0 FIXED_RANGE=192.168.2.170/29http://192.168.2.170/29 HOST_IP=192.168.2.151 LOGFILE=$DEST/logs/stack.sh.log SCREEN_LOGDIR=$DEST/logs/screen SYSLOG=True SYSLOG_HOST=$HOST_IP SYSLOG_PORT=516 TEMPEST_VOLUME_DRIVER=hp_msa_fc TEMPEST_VOLUME_VENDOR=Hewlett-Packard TEMPEST_STORAGE_PROTOCOL=FC [[post-config|$CINDER_CONF]] [hp_msa_driver] volume_driver = cinder.volume.drivers.san.hp.hp_msa_fc.HPMSAFCDriver san_ip = 192.168.2.192 san_login = manage san_password =!manage volume_backend_name=HPMSA_FC [lvmdriver-1] volume_group=stack-volumes-1 volume_driver=cinder.volume.drivers.lvm.LVMISCSIDriver volume_backend_name=LVM_iSCSI I am getting below cinder.conf file after running stack.sh script [keystone_authtoken] auth_uri = http://192.168.2.151:5000/v2.0 signing_dir = /var/cache/cinder admin_password = vedams123 admin_user = cinder admin_tenant_name = service cafile = identity_uri = http://192.168.2.151:35357 [DEFAULT] rabbit_password = vedams123 rabbit_hosts = 192.168.2.151 rpc_backend = cinder.openstack.common.rpc.impl_kombu use_syslog = True default_volume_type = lvmdriver-1 enabled_backends = lvmdriver-1 enable_v1_api = true periodic_interval = 60 lock_path = /opt/stack/data/cinder state_path = /opt/stack/data/cinder osapi_volume_extension = cinder.api.contrib.standard_extensions rootwrap_config = /etc/cinder/rootwrap.conf api_paste_config = /etc/cinder/api-paste.ini sql_connection = mysql://root:vedams123@127.0.0.1/cinder?charset=utf8http://root:vedams123@127.0.0.1/cinder?charset=utf8 iscsi_helper = tgtadm my_ip = 192.168.2.151 verbose = True debug = True auth_strategy = keystone [lvmdriver-1] volume_group = stack-volumes-1 volume_driver = cinder.volume.drivers.lvm.LVMISCSIDriver volume_backend_name = LVM_iSCSI [hp_msa_driver] volume_backend_name = HPMSA_FC san_password = !manage san_login = manage san_ip = 192.168.2.192 volume_driver = cinder.volume.drivers.san.hp.hp_msa_fc.HPMSAFCDriver Then i analyzed source code of stack.sh,and added in local.conf this line: CINDER_ENABLED_BACKENDS=hp_msa:hp_msa_driver,lvm:lvmdriver-1 Now i am getting hp_msa_fc in cinder.conf in enabled_backends Regards Nikesh ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Neutron][LBaaS] Continuing on Calling driver interface on every API request
Hi Sam, Very true. I think that Vijay’s objection is that we are currently imposing a logical structure on the driver, when it should be a driver decision. Certainly, it goes both ways. And I also agree that the mechanism for returning multiple errors, and the ability to specify whether those errors are fatal or not, individually, is currently weak. Doug On 8/11/14, 10:21 AM, Samuel Bercovici samu...@radware.com wrote: Hi Doug, In some implementations Driver !== Device. I think this is also true for HA Proxy. This might mean that there is a difference between creating a logical object and when there is enough information to actually schedule/place this into a device. The ability to express such errors (detecting an error on a logical object after it was created but when it actually get scheduled) should be discussed and addressed anyway. -Sam. -Original Message- From: Doug Wiegley [mailto:do...@a10networks.com] Sent: Monday, August 11, 2014 6:55 PM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [Neutron][LBaaS] Continuing on Calling driver interface on every API request Hi all, Validations such as ³timeout delay² should be performed on the API level before it reaches the driver. For a configuration tree (lb, listeners, pools, etc.), there should be one provider. You¹re right, but I think the point of Vijay¹s example was to highlight the combo error problem with populating all of the driver objects at once (in short, the driver interface isn¹t well suited to that model.) That his one example can be covered by API validators is irrelevant. Consider a backend that does not support APP_COOKIE¹s, or HTTPS_TERMINATED (but has multiple listeners) instead. Should the entire load balancer create fail, or should it offer degraded service? Do all drivers have to implement a transaction rollback; wait, the interface makes that very hard. That¹s his point. The driver is no longer just glue code between interfaces; it¹s now a mini-object error handler. Having provider defined in multiple places does not make sense. Channeling Brandon, who can yell if I get this wrong, the point is not to have a potentially different provider on each object. It¹s to allow a provider to be assigned when the first object in the tree is created, so that future related objects will always get routed to the same provider. Not knowing which provider should get all the objects is why we have to wait until we see a LoadBalancer object. All of this sort of edge case nonsense is because we (the royal we, the community), wanted all load balancer objects to be ³root² objects, even though only one of them is an actual root today, to support many-to-many relationships among all of them, at some future date, without an interface change. If my bias is showing that I¹m not a fan of adding this complexity for that, I¹m not surprised. Thanks, doug On 8/11/14, 7:57 AM, Samuel Bercovici samu...@radware.com wrote: Hi, Validations such as ³timeout delay² should be performed on the API level before it reaches the driver. For a configuration tree (lb, listeners, pools, etc.), there should be one provider. Having provider defined in multiple places does not make sense. -San. From: Vijay Venkatachalam [mailto:vijay.venkatacha...@citrix.com] Sent: Monday, August 11, 2014 2:43 PM To: OpenStack Development Mailing List (openstack-dev@lists.openstack.org) Subject: [openstack-dev] [Neutron][LBaaS] Continuing on Calling driver interface on every API request Hi: Continuing from last week¹s LBaaS meetingŠ Currently an entity cannot be sent to driver unless it is linked to loadbalancer because loadbalancer is the root object and driver information is only available with loadbalancer. The request to the driver is delayed because of which error propagation becomes tricky. Let¹s say a monitor was configured with timeout delay there would be no error then. When a listener is configured there will be a monitor creation/deployment error like ³timeout configured greater than delay². Unless the error is very clearly crafted the user won¹t be able to understand the error. I am half-heartedly OK with current approach. But, I would prefer Brandon¹s Solution make provider an attribute in each of the entities to get rid of this problem. What do others think? Thanks, Vijay V. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [OpenStack][InstanceGroup] metadetails and metadata in instance_group.py
Le 11/08/2014 18:03, Gary Kotton a écrit : On 8/11/14, 6:06 PM, Dan Smith d...@danplanet.com wrote: As the person who -2'd the review, I'm thankful you raised this issue on the ML, Jay. Much appreciated. The metadetails term isn't being invented in this patch, of course. I originally complained about the difference when this was being added: https://review.openstack.org/#/c/109505/1/nova/api/openstack/compute/contr ib/server_groups.py,cm As best I can tell, the response in that patch set about why it's being translated is wrong (backwards). I expect that the API extension at the time called it metadetails and they decided to make the object the same and do the translation there. From what I can tell, the actual server_group API extension that made it into the tree never got the ability to set/change/etc the metadata/metadetails anyway, so there's no reason (AFAICT) to add it in wrongly. If we care to have this functionality, then I propose we change the attribute on the object (we can handle this with versioning) and reflect it as metadata in the API. However, I have to ask: do we really need another distinct metadata store attached to server_groups? If not, how about we just remove it from the database and the object, clean up the bit of residue that is still in the API extension and be done with it? The initial version of the feature did not make use of this. The reason was that we chose for a very Limited subset to be used, that is, affinity and anti affinity. Moving forwards we would like to implement A number of different policies with this. We can drop it at the moment due to the fact that it is not used. I think that Yathi may be using this for the constrain scheduler. But I am not 100% sure. Unless I'm wrong, I can't see where this metadata is being used in the scheduler, either for filtering or for other reasons. So, please give us context why this is currently useful ? If this is something for the next future, I would love discussing it with regards to the current split. Thanks, -Sylvain --Dan ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Neutron] Simple proposal for stabilizing new features in-tree
On 8/8/14, 6:28 PM, Salvatore Orlando wrote: If we want to keep everything the way it is, we have to change everything [1] This is pretty much how I felt after reading this proposal, and I felt that this quote, which Ivar will probably appreciate, was apt to the situation. Recent events have spurred a discussion about the need for a change in process. It is not uncommon indeed to believe that by fixing the process, things will inevitably change for better. While no-one argues that flaws in processes need to be fixed, no process change will ever change anything, in my opinion, unless it is aimed at spurring change in people as well. From what I understand, this proposal starts with the assumption that any new feature which is committed to Neutron (ie: has a blueprint approved), and is not a required neutron component should be considered as a preview. This is not different from the process, which, albeit more informally, has been adopted so far. Load Balancing, Firewall, VPN, have all been explicitly documented as experimental in their first release; I would argue that even if not experimental anymore, they may not be considered stable until their stability was proven by upstream QA with API and scenario tests - but this is not sanctioned anywhere currently, I think. Correct, this proposal is not so much a new process or change in process as a formalization of what we've already been doing, and a suggested adaptation to clarify the current expectations around stability of new APIs. According to this proposal, for preview features: - all the code would be moved to a preview package Yes. - Options will be marked as preview Yes. - URIs should be prefixed with preview That's what I suggested, but, as several people have pointed out, this does seem like its worth the cost of breaking the API compatibility just at the point when it is being declared stable. I'd like to withdraw this item. - CLIs will note the features are preview in their help strings Yes. - Documentation will explicitly state this feature is preview (I think we already mark them as experimental, frankly I don't think there are a lot of differences in terminology here) Yes. Again to me, failure is one likely outcome of an experiment. The term preview is intended to imply more of a commitment to quickly reach stability. - Database migrations will be in the main alembic path as usual Right. - CLI, Devstack and Heat support will be available Right, as appropriate for the feature. - Can be used by non-preview neutron code No, I suggested No non-preview Neutron code should import code from anywhere under the neutron.preview module, - Will undergo the usual review process Right. This is key for the code to not have to jump through a new major upheaval at right as it becomes stable. - QA will be desirable, but will done either with WIP tempest patches or merging the relevant scenario tests in the preview feature iself More than desirable. We need a way to maintain and run the tempest-like API and scenario tests during the stabilization process, but to let then evolve with the feature. - The feature might be promoted or removed, but the process for this is not yet defined. Any suggestions? I did try to address preventing long-term stagnation of preview features. As a starting point, reviewing and merging a patch that moves the code from the preview sub-tree to its intended location could be a lightweight promotion process. I don't think this change in process will actually encourage better behaviour both by contributors and core reviewers. Encouraging better behavior might be necessary, but wasn't the main intent of this proposal. This proposal was intended to clarify and formalize the stability expectations around the initial releases of new features. It was specifically intended to address the conundrum currently faced by reviewers regarding patches that meet all applicable quality standards, but may not yet have (somehow, miraculously) achieved the maturity associated with stable APIs and features fully supported for widespread deployment. I reckon that better behaviour might be encouraged by forcing developer and reviewers to merge in the neutron source code tree only code which meets the highest quality standards. A change in process should enforce this - and when I think about the criteria, I think at the same kind of criteria we're being imposed to declare parity with nova. Proven reliability, and scalability should be a must. Proven usability should be a requirement for all new APIs. I agree regarding the quality standards for merging of code, and am not suggesting relaxing those one bit. But proving all of the desirable system-level attributes of a complex new feature before merging anything precludes any kind of iterative development process. I think we should consider enforcing things like proven reliability, scalability, and usability at the point where the
Re: [openstack-dev] [designate] [neutron] designate and neutron integration
kazuhiro MIYASHITA, I have done a lot of thinking about this. I have a blueprint on hold until Kilo for Neutron/Designate integration [1]. However, my blueprint doesn't quite address what you are going after here. An assumption that I have made is that Designate is an external or internet facing service so a Neutron router needs to be in the datapath to carry requests from dnsmasq to an external network. The advantage of this is that it is how Neutron works today so there is no new development needed. Could you elaborate on the advantages of connecting dnsmasq directly to the external network where Designate will be available? Carl [1] https://review.openstack.org/#/c/88624/ On Mon, Aug 11, 2014 at 7:51 AM, Miyashita, Kazuhiro miy...@jp.fujitsu.com wrote: Hi, I want to ask about neutron and designate integration. I think dnsmasq fowards DNS request from instance to designate is better. ++ |DNS server(designate) | ++ | -+--+-- Network1 | ++ |dnsmasq | ++ | -+--+-- Network2 | +-+ |instance | +-+ Because it's simpler than virtual router connects Network1 and Network2. If router connects network, instance should know where DNS server is. it is complicated. dnsmasq returns its ip address as dns server in DHCP replay by ordinary, so, I think good dnsmasq becomes like a gateway to designate. But, I can't connect dnsmasq to Network1. because of today's neutron design. Question: Does designate design team have a plan such as above integration? or other integration design? *1: Network1 and Network2 are deployed by neutron. *2: neutron deploys dnsmasq as a dhcp server. dnsmasq can forward DNS request. Thanks, kazuhiro MIYASHITA ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Fwd: FW: [Neutron] Group Based Policy and the way forward
On Fri, Aug 8, 2014 at 7:13 PM, Armando M. arma...@gmail.com wrote: On Fri, Aug 8, 2014 at 5:38 PM, Armando M. arma...@gmail.com wrote: One advantage of the service plugin is that one can leverage the neutron common framework such as Keystone authentication where common scoping is done. It would be important in the policy type of framework to have such scoping The framework you're referring to is common and already reusable, it's not a prerogative of Neutron. Are you suggesting that Service Plugins, L3, IPAM etc become individual endpoints, resulting in redundant authentication round-trips for each of the components. Wouldn't this result in degraded performance and potential consistency issues? The endpoint - in the OpenStack lingo - that exposes the API abstractions (concepts and operations) can be, logically and physically, different from the worker that implements these abstractions; authentication is orthogonal to this and I am not suggesting what you mention. From what I understand, you are saying that the implementation could be done via a mechanism different than a service plugin. Would this be done by implementing the service plugin as a different process? This would imply making changes to the the neutron server - plugin interface. If this is the case, wouldn't it be better to use the existing mechanism to avoid introducing any instability at this stage of the Juno cycle. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [designate] [neutron] designate and neutron integration
kazuhiro MIYASHITA, As designate progresses with server pools, we aim to have support for a 'private' dns server, that could run within a neutron network - is that the level of integration you were referring to? That is, for the time being, a long term goal, and not covered by Carl's Kilo blueprint. We talked with both people from both Neutron and Nova in Atlanta, and worked out the first steps for designate / neutron integration (auto provisioning of records) For that level of integration, we are assuming that a neutron router will be involved in DNS queries within a network. Long term I would prefer to see a 'private pool' connecting directly to the Network2 (like any other service VM (LBaaS etc)) and have dnsmasq pass on only records hosted by that 'private pool' to designate. This is all yet to be fleshed out, so I am open to suggestions. It requires that we complete server pools, and that work is only just starting (it was the main focus of our mid-cycle 2 weeks ago). Graham On Mon, 2014-08-11 at 11:02 -0600, Carl Baldwin wrote: kazuhiro MIYASHITA, I have done a lot of thinking about this. I have a blueprint on hold until Kilo for Neutron/Designate integration [1]. However, my blueprint doesn't quite address what you are going after here. An assumption that I have made is that Designate is an external or internet facing service so a Neutron router needs to be in the datapath to carry requests from dnsmasq to an external network. The advantage of this is that it is how Neutron works today so there is no new development needed. Could you elaborate on the advantages of connecting dnsmasq directly to the external network where Designate will be available? Carl [1] https://review.openstack.org/#/c/88624/ On Mon, Aug 11, 2014 at 7:51 AM, Miyashita, Kazuhiro miy...@jp.fujitsu.com wrote: Hi, I want to ask about neutron and designate integration. I think dnsmasq fowards DNS request from instance to designate is better. ++ |DNS server(designate) | ++ | -+--+-- Network1 | ++ |dnsmasq | ++ | -+--+-- Network2 | +-+ |instance | +-+ Because it's simpler than virtual router connects Network1 and Network2. If router connects network, instance should know where DNS server is. it is complicated. dnsmasq returns its ip address as dns server in DHCP replay by ordinary, so, I think good dnsmasq becomes like a gateway to designate. But, I can't connect dnsmasq to Network1. because of today's neutron design. Question: Does designate design team have a plan such as above integration? or other integration design? *1: Network1 and Network2 are deployed by neutron. *2: neutron deploys dnsmasq as a dhcp server. dnsmasq can forward DNS request. Thanks, kazuhiro MIYASHITA ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [oslo] oslo.concurrency repo review
On Mon, Aug 11, 2014 at 5:44 AM, Joshua Harlow harlo...@outlook.com wrote: One question from me: Will there be later fixes to remove oslo.config dependency/usage from oslo.concurrency? I still don't understand how oslo.concurrency can be used as a library with the configuration being set in a static manner via oslo.config (let's use the example of `lock_path` @ https://github.com/YorikSar/ oslo.concurrency/blob/master/oslo/concurrency/lockutils.py#L41). For example: Library X inside application Z uses lockutils (via the nice oslo.concurrency library) and sets the configuration `lock_path` to its desired settings, then library Y (also a user of oslo.concurrency) inside same application Z sets the configuration for `lock_path` to its desired settings. Now both have some unknown set of configuration they have set and when library X (or Y) continues to use lockutils they will be using some mix of configuration (likely some mish mash of settings set by X and Y); perhaps to a `lock_path` that neither actually wants to be able to write to... This doesn't seem like it will end well; and will just cause headaches during debug sessions, testing, integration and more... The same question can be asked about the `set_defaults()` function, how is library Y or X expected to use this (are they?)?? I hope one of the later changes is to remove/fix this?? Thoughts? -Josh I'd be happy to remove lock_path config variable altogether. It's basically never used. There are two basic branches in code wrt lock_path: - when you provide lock_path argument to lock (and derivative functions), file-based lock is used and CONF.lock_path is ignored; - when you don't provide lock_path in arguments, semaphore-based lock is used and CONF.lock_path is just a prefix for its name (before hashing). I wonder if users even set lock_path in their configs as it has almost no effect. So I'm all for removing it, but... From what I understand, every major change in lockutils drags along a lot of headache for everybody (and risk of bugs that would be discovered very late). So is such change really worth it? And if so, it will require very thorough research of lockutils usage patterns. -- Kind regards, Yuriy. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [tc][ceilometer] Some background on the gnocchi project
Hi Eoghan, Thanks for the note below. However, one thing the overview below does not cover is why InfluxDB (http://influxdb.com/) is not being leveraged. Many folks feel that this technology is a viable solution for the problem space discussed below. Thanks, Brad Brad Topol, Ph.D. IBM Distinguished Engineer OpenStack (919) 543-0646 Internet: bto...@us.ibm.com Assistant: Kendra Witherspoon (919) 254-0680 From: Eoghan Glynn egl...@redhat.com To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org, Date: 08/06/2014 11:17 AM Subject:[openstack-dev] [tc][ceilometer] Some background on the gnocchi project Folks, It's come to our attention that some key individuals are not fully up-to-date on gnocchi activities, so it being a good and healthy thing to ensure we're as communicative as possible about our roadmap, I've provided a high-level overview here of our thinking. This is intended as a precursor to further discussion with the TC. Cheers, Eoghan What gnocchi is: === Gnocchi is a separate, but related, project spun up on stackforge by Julien Danjou, with the objective of providing efficient storage and retrieval of timeseries-oriented data and resource representations. The goal is to experiment with a potential approach to addressing an architectural misstep made in the very earliest days of ceilometer, specifically the decision to store snapshots of some resource metadata alongside each metric datapoint. The core idea is to move to storing datapoints shorn of metadata, and instead allow the resource-state timeline to be reconstructed more cheaply from much less frequently occurring events (e.g. instance resizes or migrations). What gnocchi isn't: == Gnocchi is not a large-scale under-the-radar rewrite of a core OpenStack component along the lines of keystone-lite. The change is concentrated on the final data-storage phase of the ceilometer pipeline, so will have little initial impact on the data-acquiring agents, or on transformation phase. We've been totally open at the Atlanta summit and other forums about this approach being a multi-cycle effort. Why we decided to do it this way: The intent behind spinning up a separate project on stackforge was to allow the work progress at arms-length from ceilometer, allowing normalcy to be maintained on the core project and a rapid rate of innovation on gnocchi. Note that that the developers primarily contributing to gnocchi represent a cross-section of the core team, and there's a regular feedback loop in the form of a recurring agenda item at the weekly team meeting to avoid the effort becoming silo'd. But isn't re-architecting frowned upon? == Well, the architecture of other OpenStack projects have also under-gone change as the community understanding of the implications of prior design decisions has evolved. Take for example the move towards nova no-db-compute the unified-object-model in order to address issues in the nova architecture that made progress towards rolling upgrades unneccessarily difficult. The point, in my understanding, is not to avoid doing the course-correction where it's deemed necessary. Rather, the principle is more that these corrections happen in an open and planned way. The path forward: A subset of the ceilometer community will continue to work on gnocchi in parallel with the ceilometer core over the remainder of the Juno cycle and into the Kilo timeframe. The goal is to have an initial implementation of gnocchi ready for tech preview by the end of Juno, and to have the integration/migration/ co-existence questions addressed in Kilo. Moving the ceilometer core to using gnocchi will be contingent on it demonstrating the required performance characteristics and providing the semantics needed to support a v3 ceilometer API that's fit-for-purpose. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [TripleO][heat] a small experiment with Ansible in TripleO
Excerpts from Zane Bitter's message of 2014-08-11 08:16:56 -0700: On 11/08/14 10:46, Clint Byrum wrote: Right now we're stuck with an update that just doesn't work. It isn't just about update-failure-recovery, which is coming along nicely, but it is also about the lack of signals to control rebuild, poor support for addressing machines as groups, and unacceptable performance in large stacks. Are there blueprints/bugs filed for all of these issues? Convergnce addresses the poor performance for large stacks in general. We also have this: https://bugs.launchpad.net/heat/+bug/1306743 Which shows how slow metadata access can get. I have worked on patches but haven't been able to complete them. We made big strides but we are at a point where 40 nodes polling Heat every 30s is too much for one CPU to handle. When we scaled Heat out onto more CPUs on one box by forking we ran into eventlet issues. We also ran into issues because even with many processes we can only use one to resolve templates for a single stack during update, which was also excessively slow. We haven't been able to come back around to those yet, but you can see where this has turned into a bit of a rat hole of optimization. action-aware-sw-config is sort of what we want for rebuild. We collaborated with the trove devs on how to also address it for resize a while back but I have lost track of that work as it has taken a back seat to more pressing issues. Addressing groups is a general problem that I've had a hard time articulating in the past. Tomas Sedovic has done a good job with this TripleO spec, but I don't know that we've asked for an explicit change in a bug or spec in Heat just yet: https://review.openstack.org/#/c/97939/ There are a number of other issues noted in that spec which are already addressed in Heat, but require refactoring in TripleO's templates and tools, and that work continues. The point remains: we need something that works now, and doing an alternate implementation for updates is actually faster than addressing all of these issues. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [oslo.db]A proposal for DB read/write separation
Hi Li, comments inline. On 08/08/2014 12:03 AM, Li Ma wrote: Getting a massive amount of information from data storage to be displayed is where most of the activity happens in OpenStack. The two activities of reading data and writing (creating, updating and deleting) data are fundamentally different. The optimization for these two opposite database activities can be done by physically separating the databases that service these two different activities. All the writes go to database servers, which then replicates the written data to the database server(s) dedicated to servicing the reads. Currently, AFAIK, many OpenStack deployment in production try to take advantage of MySQL (includes Percona or MariaDB) multi-master Galera cluster. It is possible to design and implement a read/write separation schema for such a DB cluster. The above does not really make sense for MySQL Galera/PXC clusters *if only Galera nodes are used in the cluster*. Since Galera is synchronously replicated, there's no real point in segregating writers from readers, IMO. Better to just spread the write AND read load equally among all Galera cluster nodes. However, if you have a Galera cluster that then slaves off to one or more standard MySQL slaves, then certainly doing writer/reader segregation could be useful, especially for directing readers of aggregate or report-type data to the read-only slaves. Actually, OpenStack has a method for read scalability via defining master_connection and slave_connection in configuration, but this method lacks of flexibility due to deciding master or slave in the logical context(code). It's not transparent for application developer. As a result, it is not widely used in all the OpenStack projects. So, I'd like to propose a transparent read/write separation method for oslo.db that every project may happily takes advantage of it without any code modification. I've never seen a writer/reader segregation proxy or middleware piece that was properly able to send the right reads to the slaves. Unfortunately, determining what are the right reads to send to the slaves is highly application-dependent, since the application knows when it can tolerate slave lags. Moreover, I'd like to put it in the mailing list in advance to make sure it is acceptable for oslo.db. I think oslo.db is not the right place for this. I believe the efforts that Mike Wilson has been doing in the slavification blueprints are the more appropriate place to add this slave-aware code. Best, -jay I'd appreciate any comments. br. Li Ma ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [TripleO][heat] a small experiment with Ansible in TripleO
On Mon, Aug 11, 2014 at 11:20:50AM -0700, Clint Byrum wrote: Excerpts from Zane Bitter's message of 2014-08-11 08:16:56 -0700: On 11/08/14 10:46, Clint Byrum wrote: Right now we're stuck with an update that just doesn't work. It isn't just about update-failure-recovery, which is coming along nicely, but it is also about the lack of signals to control rebuild, poor support for addressing machines as groups, and unacceptable performance in large stacks. Are there blueprints/bugs filed for all of these issues? Convergnce addresses the poor performance for large stacks in general. We also have this: https://bugs.launchpad.net/heat/+bug/1306743 Which shows how slow metadata access can get. I have worked on patches but haven't been able to complete them. We made big strides but we are at a point where 40 nodes polling Heat every 30s is too much for one CPU to handle. When we scaled Heat out onto more CPUs on one box by forking we ran into eventlet issues. We also ran into issues because even with many processes we can only use one to resolve templates for a single stack during update, which was also excessively slow. Related to this, and a discussion we had recently at the TripleO meetup is this spec I raised today: https://review.openstack.org/#/c/113296/ It's following up on the idea that we could potentially address (or at least mitigate, pending the fully convergence-ified heat) some of these scalability concerns, if TripleO moves from the one-giant-template model to a more modular nested-stack/provider model (e.g what Tomas has been working on) I've not got into enough detail on that yet to be sure if it's acheivable for Juno, but it seems initially to be complex-but-doable. I'd welcome feedback on that idea and how it may fit in with the more granular convergence-engine model. Can you link to the eventlet/forking issues bug please? I thought since bug #1321303 was fixed that multiple engines and multiple workers should work OK, and obviously that being true is a precondition to expending significant effort on the nested stack decoupling plan above. Steve ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [oslo] oslo.concurrency repo review
On 08/11/2014 01:02 PM, Yuriy Taraday wrote: On Mon, Aug 11, 2014 at 5:44 AM, Joshua Harlow harlo...@outlook.com wrote: One question from me: Will there be later fixes to remove oslo.config dependency/usage from oslo.concurrency? I still don't understand how oslo.concurrency can be used as a library with the configuration being set in a static manner via oslo.config (let's use the example of `lock_path` @ https://github.com/YorikSar/ oslo.concurrency/blob/master/oslo/concurrency/lockutils.py#L41). For example: Library X inside application Z uses lockutils (via the nice oslo.concurrency library) and sets the configuration `lock_path` to its desired settings, then library Y (also a user of oslo.concurrency) inside same application Z sets the configuration for `lock_path` to its desired settings. Now both have some unknown set of configuration they have set and when library X (or Y) continues to use lockutils they will be using some mix of configuration (likely some mish mash of settings set by X and Y); perhaps to a `lock_path` that neither actually wants to be able to write to... This doesn't seem like it will end well; and will just cause headaches during debug sessions, testing, integration and more... The same question can be asked about the `set_defaults()` function, how is library Y or X expected to use this (are they?)?? I hope one of the later changes is to remove/fix this?? Thoughts? -Josh I'd be happy to remove lock_path config variable altogether. It's basically never used. There are two basic branches in code wrt lock_path: - when you provide lock_path argument to lock (and derivative functions), file-based lock is used and CONF.lock_path is ignored; - when you don't provide lock_path in arguments, semaphore-based lock is used and CONF.lock_path is just a prefix for its name (before hashing). I wonder if users even set lock_path in their configs as it has almost no effect. So I'm all for removing it, but... From what I understand, every major change in lockutils drags along a lot of headache for everybody (and risk of bugs that would be discovered very late). So is such change really worth it? And if so, it will require very thorough research of lockutils usage patterns. Two things lock_path has to stay for: Windows and consumers who require file-based locking semantics. Neither of those use cases are trivial to remove, so IMHO it would not be appropriate to do it as part of the graduation. If we were going to alter the API that much it needed to happen in incubator. As far as lock_path mismatches, that shouldn't be a problem unless a consumer is doing something very unwise. Oslo libs get their configuration from the application using them, so unless the application passes two separate conf objects to library X and Y they're both going to get consistent settings. If someone _is_ doing that, then I think it's their responsibility to make sure the options in both config files are compatible with each other. -Ben ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [Infra] Meeting Tuesday August 12th at 19:00 UTC
Hi everyone, The OpenStack Infrastructure (Infra) team is hosting our weekly meeting on Tuesday August 12th, at 19:00 UTC in #openstack-meeting Meeting agenda available here: https://wiki.openstack.org/wiki/Meetings/InfraTeamMeeting (anyone is welcome to to add agenda items) Everyone interested in infrastructure and process surrounding automated testing and deployment is encouraged to attend. -- Elizabeth Krumbach Joseph || Lyz || pleia2 http://www.princessleia.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [TripleO][heat] a small experiment with Ansible in TripleO
Excerpts from Steven Hardy's message of 2014-08-11 11:40:07 -0700: On Mon, Aug 11, 2014 at 11:20:50AM -0700, Clint Byrum wrote: Excerpts from Zane Bitter's message of 2014-08-11 08:16:56 -0700: On 11/08/14 10:46, Clint Byrum wrote: Right now we're stuck with an update that just doesn't work. It isn't just about update-failure-recovery, which is coming along nicely, but it is also about the lack of signals to control rebuild, poor support for addressing machines as groups, and unacceptable performance in large stacks. Are there blueprints/bugs filed for all of these issues? Convergnce addresses the poor performance for large stacks in general. We also have this: https://bugs.launchpad.net/heat/+bug/1306743 Which shows how slow metadata access can get. I have worked on patches but haven't been able to complete them. We made big strides but we are at a point where 40 nodes polling Heat every 30s is too much for one CPU to handle. When we scaled Heat out onto more CPUs on one box by forking we ran into eventlet issues. We also ran into issues because even with many processes we can only use one to resolve templates for a single stack during update, which was also excessively slow. Related to this, and a discussion we had recently at the TripleO meetup is this spec I raised today: https://review.openstack.org/#/c/113296/ It's following up on the idea that we could potentially address (or at least mitigate, pending the fully convergence-ified heat) some of these scalability concerns, if TripleO moves from the one-giant-template model to a more modular nested-stack/provider model (e.g what Tomas has been working on) I've not got into enough detail on that yet to be sure if it's acheivable for Juno, but it seems initially to be complex-but-doable. I'd welcome feedback on that idea and how it may fit in with the more granular convergence-engine model. Can you link to the eventlet/forking issues bug please? I thought since bug #1321303 was fixed that multiple engines and multiple workers should work OK, and obviously that being true is a precondition to expending significant effort on the nested stack decoupling plan above. That was the issue. So we fixed that bug, but we never un-reverted the patch that forks enough engines to use up all the CPU's on a box by default. That would likely help a lot with metadata access speed (we could manually do it in TripleO but we tend to push defaults. :) ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [TripleO] reverting the HOT migration? // dealing with lockstep changes
Hi, so shortly after the HOT migration landed, we hit https://bugs.launchpad.net/tripleo/+bug/1354305 which is that on even quite recently deployed clouds, the migrated templates were just too new. A partial revert (of just the list_join bit) fixes that, but a deeper problem emerged which is that stack-update to get from a non-HOT to HOT template appears broken (https://bugs.launchpad.net/heat/+bug/1354962). I think we need to revert the HOT migration today, as forcing a scorched earth recreation of a cloud is not a great answer for folk that have deployed versions - its a backwards compat issue. Its true that our release as of icehouse isn't really useable, so we could try to wiggle our way past this one, but I think as the first real test of our new backwards compat policy, that that would be a mistake. What we need to be able to land it again, is some way whereby an existing cloud can upgrade their undercloud (via stack-update against the old heat deployed in the seed [today, old heat in the undercloud itself in future]) and then once that is deployed subsequent templates can use the new features. We're likely to run into such lockstep changes in future, so we also need to be able to recognise them in review / design, and call them out so we can fix them early rather than deep down the pike. -Rob -- Robert Collins rbtcoll...@hp.com Distinguished Technologist HP Converged Cloud ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [oslo] usage patterns for oslo.config
On Aug 8, 2014, at 7:22 PM, Devananda van der Veen devananda@gmail.com wrote: On Fri, Aug 8, 2014 at 12:41 PM, Doug Hellmann d...@doughellmann.com wrote: That’s right. The preferred approach is to put the register_opt() in *runtime* code somewhere before the option will be used. That might be in the constructor for a class that uses an option, for example, as described in http://docs.openstack.org/developer/oslo.config/cfg.html#registering-options Doug Interesting. I've been following the prevailing example in Nova, which is to register opts at the top of a module, immediately after defining them. Is there a situation in which one approach is better than the other? The approach used in Nova is the “old” way of doing it. It works, but assumes that all of the application code is modifying a global configuration object. The runtime approach allows you to pass a configuration object to a library, which makes it easier to mock the configuration for testing and avoids having the configuration options bleed into the public API of the library. We’ve started using the runtime approach in new Oslo libraries that have configuration options, but changing the implementation in existing application code isn’t strictly necessary. Doug Thanks, Devananda ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [tc][ceilometer] Some background on the gnocchi project
Hi Eoghan, Thanks for the note below. However, one thing the overview below does not cover is why InfluxDB ( http://influxdb.com/ ) is not being leveraged. Many folks feel that this technology is a viable solution for the problem space discussed below. Great question Brad! As it happens we've been working closely with Paul Dix (lead developer of InfluxDB) to ensure that this metrics store would be usable as a backend driver. That conversation actually kicked off at the Juno summit in Atlanta, but it really got off the ground at our mid-cycle meet-up in Paris on in early July. I wrote a rough strawman version of an InfluxDB driver in advance of the mid-cycle to frame the discussion, and Paul Dix traveled to the meet-up so we could have the discussion face-to-face. The conclusion was that InfluxDB would indeed potentially be a great fit, modulo some requirements that we identified during the detailed discussions: * shard-space-based retention backgrounded deletion * capability to merge individual timeseries for cross-aggregation * backfill-aware downsampling The InfluxDB folks have committed to implementing those features in over July and August, and have made concrete progress on that score. I hope that provides enough detail to answer to your question? Cheers, Eoghan Thanks, Brad Brad Topol, Ph.D. IBM Distinguished Engineer OpenStack (919) 543-0646 Internet: bto...@us.ibm.com Assistant: Kendra Witherspoon (919) 254-0680 From: Eoghan Glynn egl...@redhat.com To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org, Date: 08/06/2014 11:17 AM Subject: [openstack-dev] [tc][ceilometer] Some background on the gnocchi project Folks, It's come to our attention that some key individuals are not fully up-to-date on gnocchi activities, so it being a good and healthy thing to ensure we're as communicative as possible about our roadmap, I've provided a high-level overview here of our thinking. This is intended as a precursor to further discussion with the TC. Cheers, Eoghan What gnocchi is: === Gnocchi is a separate, but related, project spun up on stackforge by Julien Danjou, with the objective of providing efficient storage and retrieval of timeseries-oriented data and resource representations. The goal is to experiment with a potential approach to addressing an architectural misstep made in the very earliest days of ceilometer, specifically the decision to store snapshots of some resource metadata alongside each metric datapoint. The core idea is to move to storing datapoints shorn of metadata, and instead allow the resource-state timeline to be reconstructed more cheaply from much less frequently occurring events (e.g. instance resizes or migrations). What gnocchi isn't: == Gnocchi is not a large-scale under-the-radar rewrite of a core OpenStack component along the lines of keystone-lite. The change is concentrated on the final data-storage phase of the ceilometer pipeline, so will have little initial impact on the data-acquiring agents, or on transformation phase. We've been totally open at the Atlanta summit and other forums about this approach being a multi-cycle effort. Why we decided to do it this way: The intent behind spinning up a separate project on stackforge was to allow the work progress at arms-length from ceilometer, allowing normalcy to be maintained on the core project and a rapid rate of innovation on gnocchi. Note that that the developers primarily contributing to gnocchi represent a cross-section of the core team, and there's a regular feedback loop in the form of a recurring agenda item at the weekly team meeting to avoid the effort becoming silo'd. But isn't re-architecting frowned upon? == Well, the architecture of other OpenStack projects have also under-gone change as the community understanding of the implications of prior design decisions has evolved. Take for example the move towards nova no-db-compute the unified-object-model in order to address issues in the nova architecture that made progress towards rolling upgrades unneccessarily difficult. The point, in my understanding, is not to avoid doing the course-correction where it's deemed necessary. Rather, the principle is more that these corrections happen in an open and planned way. The path forward: A subset of the ceilometer community will continue to work on gnocchi in parallel with the ceilometer core over the remainder of the Juno cycle and into the Kilo timeframe. The goal is to have an initial implementation of gnocchi ready for tech preview by the end of Juno, and to have the integration/migration/ co-existence questions addressed in Kilo. Moving the ceilometer core to using gnocchi will be contingent on it
Re: [openstack-dev] [Neutron][LBaaS] Continuing on Calling driver interface on every API request
Yes, the point was to say the plugin need not restrict and let driver decide what to do with the API. Even if the call was made to driver instantaneously, I understand, the driver might decide to ignore first and schedule later. But, if the call is present, there is scope for validation. Also, the driver might be scheduling an async-api to backend, in which case deployment error cannot be shown to the user instantaneously. W.r.t. identifying a provider/driver, how would it be to make tenant the default root object? tenantid is already associated with each of these entities, so no additional pain. For the tenant who wants to override let him specify provider in each of the entities. If you think of this in terms of the UI, let's say if the loadbalancer configuration is exposed as a single wizard (which has loadbalancer, listener, pool, monitor properties) then provider is chosen only once. Curious question, is flavour framework expected to address this problem? Thanks, Vijay V. -Original Message- From: Doug Wiegley [mailto:do...@a10networks.com] Sent: 11 August 2014 22:02 To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [Neutron][LBaaS] Continuing on Calling driver interface on every API request Hi Sam, Very true. I think that Vijay’s objection is that we are currently imposing a logical structure on the driver, when it should be a driver decision. Certainly, it goes both ways. And I also agree that the mechanism for returning multiple errors, and the ability to specify whether those errors are fatal or not, individually, is currently weak. Doug On 8/11/14, 10:21 AM, Samuel Bercovici samu...@radware.com wrote: Hi Doug, In some implementations Driver !== Device. I think this is also true for HA Proxy. This might mean that there is a difference between creating a logical object and when there is enough information to actually schedule/place this into a device. The ability to express such errors (detecting an error on a logical object after it was created but when it actually get scheduled) should be discussed and addressed anyway. -Sam. -Original Message- From: Doug Wiegley [mailto:do...@a10networks.com] Sent: Monday, August 11, 2014 6:55 PM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [Neutron][LBaaS] Continuing on Calling driver interface on every API request Hi all, Validations such as ³timeout delay² should be performed on the API level before it reaches the driver. For a configuration tree (lb, listeners, pools, etc.), there should be one provider. You¹re right, but I think the point of Vijay¹s example was to highlight the combo error problem with populating all of the driver objects at once (in short, the driver interface isn¹t well suited to that model.) That his one example can be covered by API validators is irrelevant. Consider a backend that does not support APP_COOKIE¹s, or HTTPS_TERMINATED (but has multiple listeners) instead. Should the entire load balancer create fail, or should it offer degraded service? Do all drivers have to implement a transaction rollback; wait, the interface makes that very hard. That¹s his point. The driver is no longer just glue code between interfaces; it¹s now a mini-object error handler. Having provider defined in multiple places does not make sense. Channeling Brandon, who can yell if I get this wrong, the point is not to have a potentially different provider on each object. It¹s to allow a provider to be assigned when the first object in the tree is created, so that future related objects will always get routed to the same provider. Not knowing which provider should get all the objects is why we have to wait until we see a LoadBalancer object. All of this sort of edge case nonsense is because we (the royal we, the community), wanted all load balancer objects to be ³root² objects, even though only one of them is an actual root today, to support many-to-many relationships among all of them, at some future date, without an interface change. If my bias is showing that I¹m not a fan of adding this complexity for that, I¹m not surprised. Thanks, doug On 8/11/14, 7:57 AM, Samuel Bercovici samu...@radware.com wrote: Hi, Validations such as ³timeout delay² should be performed on the API level before it reaches the driver. For a configuration tree (lb, listeners, pools, etc.), there should be one provider. Having provider defined in multiple places does not make sense. -San. From: Vijay Venkatachalam [mailto:vijay.venkatacha...@citrix.com] Sent: Monday, August 11, 2014 2:43 PM To: OpenStack Development Mailing List (openstack-dev@lists.openstack.org) Subject: [openstack-dev] [Neutron][LBaaS] Continuing on Calling driver interface on every API request Hi: Continuing from last week¹s LBaaS meetingŠ Currently an entity cannot be sent to driver unless it
Re: [openstack-dev] [oslo] oslo.concurrency repo review
On Mon, Aug 11, 2014 at 11:39 AM, Ben Nemec openst...@nemebean.com wrote: On 08/11/2014 01:02 PM, Yuriy Taraday wrote: On Mon, Aug 11, 2014 at 5:44 AM, Joshua Harlow harlo...@outlook.com wrote: One question from me: Will there be later fixes to remove oslo.config dependency/usage from oslo.concurrency? I still don't understand how oslo.concurrency can be used as a library with the configuration being set in a static manner via oslo.config (let's use the example of `lock_path` @ https://github.com/YorikSar/ oslo.concurrency/blob/master/oslo/concurrency/lockutils.py#L41). For example: Library X inside application Z uses lockutils (via the nice oslo.concurrency library) and sets the configuration `lock_path` to its desired settings, then library Y (also a user of oslo.concurrency) inside same application Z sets the configuration for `lock_path` to its desired settings. Now both have some unknown set of configuration they have set and when library X (or Y) continues to use lockutils they will be using some mix of configuration (likely some mish mash of settings set by X and Y); perhaps to a `lock_path` that neither actually wants to be able to write to... This doesn't seem like it will end well; and will just cause headaches during debug sessions, testing, integration and more... The same question can be asked about the `set_defaults()` function, how is library Y or X expected to use this (are they?)?? I hope one of the later changes is to remove/fix this?? Thoughts? -Josh I'd be happy to remove lock_path config variable altogether. It's basically never used. There are two basic branches in code wrt lock_path: - when you provide lock_path argument to lock (and derivative functions), file-based lock is used and CONF.lock_path is ignored; - when you don't provide lock_path in arguments, semaphore-based lock is used and CONF.lock_path is just a prefix for its name (before hashing). I wonder if users even set lock_path in their configs as it has almost no effect. So I'm all for removing it, but... From what I understand, every major change in lockutils drags along a lot of headache for everybody (and risk of bugs that would be discovered very late). So is such change really worth it? And if so, it will require very thorough research of lockutils usage patterns. Two things lock_path has to stay for: Windows and consumers who require file-based locking semantics. Neither of those use cases are trivial to remove, so IMHO it would not be appropriate to do it as part of the graduation. If we were going to alter the API that much it needed to happen in incubator. As far as lock_path mismatches, that shouldn't be a problem unless a consumer is doing something very unwise. Oslo libs get their configuration from the application using them, so unless the application passes two separate conf objects to library X and Y they're both going to get consistent settings. If someone _is_ doing that, then I think it's their responsibility to make sure the options in both config files are compatible with each other. Why would it be assumed they would pass the same settings (how is that even possible to know ahead of time? especially if library X pulls in a new library ZZ that requires a new configuration setting). For example, one directory for `lock_path` may be reasonable for tooz and another may be reasonable for taskflow (completly depends on there intended usage), it would not likely desireable to have them go to the same location. Forcing application Z to know the inner workings of library X and library Y (or future unknown library ZZ) is just pushing the problem onto the library user, which seems inappropriate and breaks the whole point of having abstractions APIs in the first place... This IMHO is part of the problem with having statically set *action at a distance* type of configuration, the libraries themselves are not in control of their own configuration, which breaks abstractions APIs left and right. If some application Y can go under a library and pull the rug out from under it, how is that a reasonable thing to expect the library to be able to predict handle? This kind of requirement has always made me wonder how other libraries (like tooz, or taskflow) actually interact with any of the oslo.* libraries in any predicatable way (since those library could be interacting with oslo.* libraries that have configuration that can be switched out from underneath them, making those libraries have *secret* APIs that appear and disappear depending on what used oslo.* library was newly added as a dependency and what newly added configuration that library sucked in/exposed...). -Josh -Ben ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [TripleO] reverting the HOT migration? // dealing with lockstep changes
On Tue, 2014-08-12 at 06:58 +1200, Robert Collins wrote: Hi, so shortly after the HOT migration landed, we hit https://bugs.launchpad.net/tripleo/+bug/1354305 which is that on even quite recently deployed clouds, the migrated templates were just too new. A partial revert (of just the list_join bit) fixes that, but a deeper problem emerged which is that stack-update to get from a non-HOT to HOT template appears broken (https://bugs.launchpad.net/heat/+bug/1354962). I think we need to revert the HOT migration today, as forcing a scorched earth recreation of a cloud is not a great answer for folk that have deployed versions - its a backwards compat issue. Its true that our release as of icehouse isn't really useable, so we could try to wiggle our way past this one, but I think as the first real test of our new backwards compat policy, that that would be a mistake. Hmmm. We blocked a good bit of changes to get these HOT templates in so I hate to see us revert them. Also, It isn't clear to me how much work it would be to fully support the non-HOT to HOT templates upgrade path. How much work is this? And is that something we really want to spend time on instead of all the other things? Why not just create a branch of the old templates that works on the existing underclouds? Users would then need to use these templates as a one-time upgrade step to be able to upgrade to a heat HOT capable undercloud first. With regards to the seed... Would a tool that allowed us to migrate the stateful data from an old seed to a new seed be a better use of time here? A re-seeder might be a useful tool to future proof upgrade paths that rely on the software versions in the seed (Heat etc.) that aren't easily up-gradable. Packages would help here too, but not everyone uses them... Dan What we need to be able to land it again, is some way whereby an existing cloud can upgrade their undercloud (via stack-update against the old heat deployed in the seed [today, old heat in the undercloud itself in future]) and then once that is deployed subsequent templates can use the new features. We're likely to run into such lockstep changes in future, so we also need to be able to recognise them in review / design, and call them out so we can fix them early rather than deep down the pike. -Rob ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [oslo] oslo.concurrency repo review
On Mon, Aug 11, 2014 at 11:02 AM, Yuriy Taraday yorik@gmail.com wrote: On Mon, Aug 11, 2014 at 5:44 AM, Joshua Harlow harlo...@outlook.com wrote: One question from me: Will there be later fixes to remove oslo.config dependency/usage from oslo.concurrency? I still don't understand how oslo.concurrency can be used as a library with the configuration being set in a static manner via oslo.config (let's use the example of `lock_path` @ https://github.com/YorikSar/oslo.concurrency/blob/master/oslo/concurrency/lockutils.py#L41). For example: Library X inside application Z uses lockutils (via the nice oslo.concurrency library) and sets the configuration `lock_path` to its desired settings, then library Y (also a user of oslo.concurrency) inside same application Z sets the configuration for `lock_path` to its desired settings. Now both have some unknown set of configuration they have set and when library X (or Y) continues to use lockutils they will be using some mix of configuration (likely some mish mash of settings set by X and Y); perhaps to a `lock_path` that neither actually wants to be able to write to... This doesn't seem like it will end well; and will just cause headaches during debug sessions, testing, integration and more... The same question can be asked about the `set_defaults()` function, how is library Y or X expected to use this (are they?)?? I hope one of the later changes is to remove/fix this?? Thoughts? -Josh I'd be happy to remove lock_path config variable altogether. It's basically never used. There are two basic branches in code wrt lock_path: - when you provide lock_path argument to lock (and derivative functions), file-based lock is used and CONF.lock_path is ignored; - when you don't provide lock_path in arguments, semaphore-based lock is used and CONF.lock_path is just a prefix for its name (before hashing). Agreed, it just seems confusing (and bad) to have parts of the API come in from `CONF.lock_path` (or other `CONF.*` options) and other parts of the API come in via function parameters. This just makes understanding the API and knowing how to interact with it that much harder (after all what is the right way of using XYZ feature when it can be changed via a out-of-band *hidden* API call via configuration adjustments under the covers?)... This makes it really hard to use oslo.concurrency in taskflow (and likely other libraries that would like to consume oslo.concurrency, seeing that it will be on pypi, I would expect this number to grow...) since taskflow would really appreciate and desire to have stable APIs that don't change by some configuration that can be set by some party via some out-of-band method (for example some other library or program calling `set_defaults()`). This kind of way of using an API (half of the settings from config, half of the settings from the functions API...) may be ok for applications but it's not IMHO ok for libraries (or clients) that want to use oslo.concurrency. Hopefully it can be fixed some that it works via both ways? Oslo.db I believe made this work better by allowing for configuration to come in via a configuration object that can be provided by the user of oslo.db, this makes the API that oslo.db exposes strongly tied to the attributes documentation of that object. I still don't think thats perfect either since its likely that the documentation for what that objects attributes should be is not as update to date or easy to change as updating function/method documentation... I wonder if users even set lock_path in their configs as it has almost no effect. So I'm all for removing it, but... From what I understand, every major change in lockutils drags along a lot of headache for everybody (and risk of bugs that would be discovered very late). So is such change really worth it? And if so, it will require very thorough research of lockutils usage patterns. Sounds like tech debt to me, it always requires work to make something better. Are we the type of community that will avoid changing things (for the better) because we fear introducing new bugs that may be found along the way? I for one hope that we are not that type of community (that type of community will die due to its own *fake* fears...). -- Kind regards, Yuriy. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [oslo] oslo.concurrency repo review
On Aug 11, 2014, at 3:26 PM, Joshua Harlow harlo...@outlook.com wrote: On Mon, Aug 11, 2014 at 11:02 AM, Yuriy Taraday yorik@gmail.com wrote: On Mon, Aug 11, 2014 at 5:44 AM, Joshua Harlow harlo...@outlook.com wrote: One question from me: Will there be later fixes to remove oslo.config dependency/usage from oslo.concurrency? I still don't understand how oslo.concurrency can be used as a library with the configuration being set in a static manner via oslo.config (let's use the example of `lock_path` @ https://github.com/YorikSar/oslo.concurrency/blob/master/oslo/concurrency/lockutils.py#L41). For example: Library X inside application Z uses lockutils (via the nice oslo.concurrency library) and sets the configuration `lock_path` to its desired settings, then library Y (also a user of oslo.concurrency) inside same application Z sets the configuration for `lock_path` to its desired settings. Now both have some unknown set of configuration they have set and when library X (or Y) continues to use lockutils they will be using some mix of configuration (likely some mish mash of settings set by X and Y); perhaps to a `lock_path` that neither actually wants to be able to write to... This doesn't seem like it will end well; and will just cause headaches during debug sessions, testing, integration and more... The same question can be asked about the `set_defaults()` function, how is library Y or X expected to use this (are they?)?? I hope one of the later changes is to remove/fix this?? Thoughts? -Josh I'd be happy to remove lock_path config variable altogether. It's basically never used. There are two basic branches in code wrt lock_path: - when you provide lock_path argument to lock (and derivative functions), file-based lock is used and CONF.lock_path is ignored; - when you don't provide lock_path in arguments, semaphore-based lock is used and CONF.lock_path is just a prefix for its name (before hashing). Agreed, it just seems confusing (and bad) to have parts of the API come in from `CONF.lock_path` (or other `CONF.*` options) and other parts of the API come in via function parameters. This just makes understanding the API and knowing how to interact with it that much harder (after all what is the right way of using XYZ feature when it can be changed via a out-of-band *hidden* API call via configuration adjustments under the covers?)... This makes it really hard to use oslo.concurrency in taskflow (and likely other libraries that would like to consume oslo.concurrency, seeing that it will be on pypi, I would The libraries placed in the oslo namespace are very much NOT meant to be used by anything other than OpenStack. They are intended to be the glue layer between OpenStack and some other implementation libraries. oslo.concurrency wraps pylockfile and the plan is to move the actual lock code into pylockfile without the oslo.config dependency. That will make pylockfile reusable by taskflow and tooz, and the locking stuff in oslo.concurrency a smaller API with consistent configuration for use by applications. expect this number to grow...) since taskflow would really appreciate and desire to have stable APIs that don't change by some configuration that can be set by some party via some out-of-band method (for example some other library or program calling `set_defaults()`). This kind of way of using an API (half of the settings from config, half of the settings from the functions API...) may be ok for applications but it's not IMHO ok for libraries (or clients) that want to use oslo.concurrency. Hopefully it can be fixed some that it works via both ways? Oslo.db I believe made this work better by allowing for configuration to come in via a configuration object that can be provided by the user of oslo.db, this makes the API that oslo.db exposes strongly tied to the attributes documentation of that object. I still don't think thats perfect either since its likely that the documentation for what that objects attributes should be is not as update to date or easy to change as updating function/method documentation… That technique of having the configuration object passed to the oslo library will be repeated in the other new libraries we are creating if they already depend on configuration settings of some sort. The configuration options are not part of the public API of the library, so they and their definitions will be hidden from the caller, but the library has to be given a configuration object in order to load the settings for itself. I wonder if users even set lock_path in their configs as it has almost no effect. So I'm all for removing it, but... From what I understand, every major change in lockutils drags along a lot of headache for everybody (and risk of bugs that would be discovered very late). So is such change really worth it? And if so, it will require very thorough
Re: [openstack-dev] [tc][ceilometer] Some background on the gnocchi project
On 8/11/2014 4:22 PM, Eoghan Glynn wrote: Hi Eoghan, Thanks for the note below. However, one thing the overview below does not cover is why InfluxDB ( http://influxdb.com/ ) is not being leveraged. Many folks feel that this technology is a viable solution for the problem space discussed below. Great question Brad! As it happens we've been working closely with Paul Dix (lead developer of InfluxDB) to ensure that this metrics store would be usable as a backend driver. That conversation actually kicked off at the Juno summit in Atlanta, but it really got off the ground at our mid-cycle meet-up in Paris on in early July. ... The InfluxDB folks have committed to implementing those features in over July and August, and have made concrete progress on that score. I hope that provides enough detail to answer to your question? I guess it begs the question, if influxdb will do what you want and it's open source (MIT) as well as commercially supported, how does gnocchi differentiate? Cheers, Eoghan Thanks, Brad Brad Topol, Ph.D. IBM Distinguished Engineer OpenStack (919) 543-0646 Internet: bto...@us.ibm.com Assistant: Kendra Witherspoon (919) 254-0680 From: Eoghan Glynn egl...@redhat.com To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org, Date: 08/06/2014 11:17 AM Subject: [openstack-dev] [tc][ceilometer] Some background on the gnocchi project Folks, It's come to our attention that some key individuals are not fully up-to-date on gnocchi activities, so it being a good and healthy thing to ensure we're as communicative as possible about our roadmap, I've provided a high-level overview here of our thinking. This is intended as a precursor to further discussion with the TC. Cheers, Eoghan What gnocchi is: === Gnocchi is a separate, but related, project spun up on stackforge by Julien Danjou, with the objective of providing efficient storage and retrieval of timeseries-oriented data and resource representations. The goal is to experiment with a potential approach to addressing an architectural misstep made in the very earliest days of ceilometer, specifically the decision to store snapshots of some resource metadata alongside each metric datapoint. The core idea is to move to storing datapoints shorn of metadata, and instead allow the resource-state timeline to be reconstructed more cheaply from much less frequently occurring events (e.g. instance resizes or migrations). What gnocchi isn't: == Gnocchi is not a large-scale under-the-radar rewrite of a core OpenStack component along the lines of keystone-lite. The change is concentrated on the final data-storage phase of the ceilometer pipeline, so will have little initial impact on the data-acquiring agents, or on transformation phase. We've been totally open at the Atlanta summit and other forums about this approach being a multi-cycle effort. Why we decided to do it this way: The intent behind spinning up a separate project on stackforge was to allow the work progress at arms-length from ceilometer, allowing normalcy to be maintained on the core project and a rapid rate of innovation on gnocchi. Note that that the developers primarily contributing to gnocchi represent a cross-section of the core team, and there's a regular feedback loop in the form of a recurring agenda item at the weekly team meeting to avoid the effort becoming silo'd. But isn't re-architecting frowned upon? == Well, the architecture of other OpenStack projects have also under-gone change as the community understanding of the implications of prior design decisions has evolved. Take for example the move towards nova no-db-compute the unified-object-model in order to address issues in the nova architecture that made progress towards rolling upgrades unneccessarily difficult. The point, in my understanding, is not to avoid doing the course-correction where it's deemed necessary. Rather, the principle is more that these corrections happen in an open and planned way. The path forward: A subset of the ceilometer community will continue to work on gnocchi in parallel with the ceilometer core over the remainder of the Juno cycle and into the Kilo timeframe. The goal is to have an initial implementation of gnocchi ready for tech preview by the end of Juno, and to have the integration/migration/ co-existence questions addressed in Kilo. Moving the ceilometer core to using gnocchi will be contingent on it demonstrating the required performance characteristics and providing the semantics needed to support a v3 ceilometer API that's fit-for-purpose. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org
Re: [openstack-dev] [oslo] oslo.concurrency repo review
On 08/11/2014 02:20 PM, Joshua Harlow wrote: On Mon, Aug 11, 2014 at 11:39 AM, Ben Nemec openst...@nemebean.com wrote: On 08/11/2014 01:02 PM, Yuriy Taraday wrote: On Mon, Aug 11, 2014 at 5:44 AM, Joshua Harlow harlo...@outlook.com wrote: One question from me: Will there be later fixes to remove oslo.config dependency/usage from oslo.concurrency? I still don't understand how oslo.concurrency can be used as a library with the configuration being set in a static manner via oslo.config (let's use the example of `lock_path` @ https://github.com/YorikSar/ oslo.concurrency/blob/master/oslo/concurrency/lockutils.py#L41). For example: Library X inside application Z uses lockutils (via the nice oslo.concurrency library) and sets the configuration `lock_path` to its desired settings, then library Y (also a user of oslo.concurrency) inside same application Z sets the configuration for `lock_path` to its desired settings. Now both have some unknown set of configuration they have set and when library X (or Y) continues to use lockutils they will be using some mix of configuration (likely some mish mash of settings set by X and Y); perhaps to a `lock_path` that neither actually wants to be able to write to... This doesn't seem like it will end well; and will just cause headaches during debug sessions, testing, integration and more... The same question can be asked about the `set_defaults()` function, how is library Y or X expected to use this (are they?)?? I hope one of the later changes is to remove/fix this?? Thoughts? -Josh I'd be happy to remove lock_path config variable altogether. It's basically never used. There are two basic branches in code wrt lock_path: - when you provide lock_path argument to lock (and derivative functions), file-based lock is used and CONF.lock_path is ignored; - when you don't provide lock_path in arguments, semaphore-based lock is used and CONF.lock_path is just a prefix for its name (before hashing). I wonder if users even set lock_path in their configs as it has almost no effect. So I'm all for removing it, but... From what I understand, every major change in lockutils drags along a lot of headache for everybody (and risk of bugs that would be discovered very late). So is such change really worth it? And if so, it will require very thorough research of lockutils usage patterns. Two things lock_path has to stay for: Windows and consumers who require file-based locking semantics. Neither of those use cases are trivial to remove, so IMHO it would not be appropriate to do it as part of the graduation. If we were going to alter the API that much it needed to happen in incubator. As far as lock_path mismatches, that shouldn't be a problem unless a consumer is doing something very unwise. Oslo libs get their configuration from the application using them, so unless the application passes two separate conf objects to library X and Y they're both going to get consistent settings. If someone _is_ doing that, then I think it's their responsibility to make sure the options in both config files are compatible with each other. Why would it be assumed they would pass the same settings (how is that even possible to know ahead of time? especially if library X pulls in a new library ZZ that requires a new configuration setting). For example, one directory for `lock_path` may be reasonable for tooz and another may be reasonable for taskflow (completly depends on there intended usage), it would not likely desireable to have them go to the same location. The only reason I can see that you would want to do this is to avoid lock name collisions, but I think I'd rather namespace lib locks than require users to manage multiple lock paths. Even the one lock path has been a hassle. Forcing application Z to know the inner workings of library X and library Y (or future unknown library ZZ) is just pushing the problem onto the library user, which seems inappropriate and breaks the whole point of having abstractions APIs in the first place... This IMHO is part of the problem with having statically set *action at a distance* type of configuration, the libraries themselves are not in control of their own configuration, which breaks abstractions APIs left and right. If some application Y can go under a library and pull the rug out from under it, how is that a reasonable thing to expect the library to be able to predict handle? The application doesn't have to, and shouldn't, know about lib options. The libraries have a list_opts method that returns the list of options the lib needs, which is sufficient for the config generator to do the right thing (a bit of an oversimplification maybe, but the point is there's an API for handling the options so the app doesn't need to). Besides that the application doesn't
Re: [openstack-dev] [oslo] oslo.concurrency repo review
On Mon, Aug 11, 2014 at 12:47 PM, Doug Hellmann d...@doughellmann.com wrote: On Aug 11, 2014, at 3:26 PM, Joshua Harlow harlo...@outlook.com wrote: On Mon, Aug 11, 2014 at 11:02 AM, Yuriy Taraday yorik@gmail.com wrote: On Mon, Aug 11, 2014 at 5:44 AM, Joshua Harlow harlo...@outlook.com wrote: One question from me: Will there be later fixes to remove oslo.config dependency/usage from oslo.concurrency? I still don't understand how oslo.concurrency can be used as a library with the configuration being set in a static manner via oslo.config (let's use the example of `lock_path` @ https://github.com/YorikSar/oslo.concurrency/blob/master/oslo/concurrency/lockutils.py#L41). For example: Library X inside application Z uses lockutils (via the nice oslo.concurrency library) and sets the configuration `lock_path` to its desired settings, then library Y (also a user of oslo.concurrency) inside same application Z sets the configuration for `lock_path` to its desired settings. Now both have some unknown set of configuration they have set and when library X (or Y) continues to use lockutils they will be using some mix of configuration (likely some mish mash of settings set by X and Y); perhaps to a `lock_path` that neither actually wants to be able to write to... This doesn't seem like it will end well; and will just cause headaches during debug sessions, testing, integration and more... The same question can be asked about the `set_defaults()` function, how is library Y or X expected to use this (are they?)?? I hope one of the later changes is to remove/fix this?? Thoughts? -Josh I'd be happy to remove lock_path config variable altogether. It's basically never used. There are two basic branches in code wrt lock_path: - when you provide lock_path argument to lock (and derivative functions), file-based lock is used and CONF.lock_path is ignored; - when you don't provide lock_path in arguments, semaphore-based lock is used and CONF.lock_path is just a prefix for its name (before hashing). Agreed, it just seems confusing (and bad) to have parts of the API come in from `CONF.lock_path` (or other `CONF.*` options) and other parts of the API come in via function parameters. This just makes understanding the API and knowing how to interact with it that much harder (after all what is the right way of using XYZ feature when it can be changed via a out-of-band *hidden* API call via configuration adjustments under the covers?)... This makes it really hard to use oslo.concurrency in taskflow (and likely other libraries that would like to consume oslo.concurrency, seeing that it will be on pypi, I would The libraries placed in the oslo namespace are very much NOT meant to be used by anything other than OpenStack. They are intended to be the glue layer between OpenStack and some other implementation libraries. oslo.concurrency wraps pylockfile and the plan is to move the actual lock code into pylockfile without the oslo.config dependency. That will make pylockfile reusable by taskflow and tooz, and the locking stuff in oslo.concurrency a smaller API with consistent configuration for use by applications. Sounds great, I've been wondering why https://github.com/stackforge/tooz/commit/f3e11e40f9871f8328 happened/merged (maybe it should be changed?). I see that https://review.openstack.org/#/c/102202/ merged so that's good news and hopefully makes the underlying lockutils functionality more useful to outside of openstack users in the near-term future (which includes taskflow, being that it is being used in outside openstack by various entities). expect this number to grow...) since taskflow would really appreciate and desire to have stable APIs that don't change by some configuration that can be set by some party via some out-of-band method (for example some other library or program calling `set_defaults()`). This kind of way of using an API (half of the settings from config, half of the settings from the functions API...) may be ok for applications but it's not IMHO ok for libraries (or clients) that want to use oslo.concurrency. Hopefully it can be fixed some that it works via both ways? Oslo.db I believe made this work better by allowing for configuration to come in via a configuration object that can be provided by the user of oslo.db, this makes the API that oslo.db exposes strongly tied to the attributes documentation of that object. I still don't think thats perfect either since its likely that the documentation for what that objects attributes should be is not as update to date or easy to change as updating function/method documentation… That technique of having the configuration object passed to the oslo library will be repeated in the other new libraries we are creating if they already depend on configuration settings of some sort. The configuration options are not part of the public API of the library, so they
Re: [openstack-dev] [Neutron][LBaaS] Continuing on Calling driver interface on every API request
Hi folks, That actually going in opposite direction to what flavor framework is trying to do (and for dispatching it's doing the same as providers). REST call dispatching should really go via the root object. I don't quite get the issue with health monitors. If HM is incorrectly configured prior to association with a pool - API layer should handle that. I don't think driver implementations should be different at constraints to HM parameters. So I'm -1 on adding provider (or flavor) to each entity. After all, it looks just like data denormalization which actually will affect lots of API aspects in negative way. Thanks, Eugene. On Mon, Aug 11, 2014 at 11:20 PM, Vijay Venkatachalam vijay.venkatacha...@citrix.com wrote: Yes, the point was to say the plugin need not restrict and let driver decide what to do with the API. Even if the call was made to driver instantaneously, I understand, the driver might decide to ignore first and schedule later. But, if the call is present, there is scope for validation. Also, the driver might be scheduling an async-api to backend, in which case deployment error cannot be shown to the user instantaneously. W.r.t. identifying a provider/driver, how would it be to make tenant the default root object? tenantid is already associated with each of these entities, so no additional pain. For the tenant who wants to override let him specify provider in each of the entities. If you think of this in terms of the UI, let's say if the loadbalancer configuration is exposed as a single wizard (which has loadbalancer, listener, pool, monitor properties) then provider is chosen only once. Curious question, is flavour framework expected to address this problem? Thanks, Vijay V. -Original Message- From: Doug Wiegley [mailto:do...@a10networks.com] Sent: 11 August 2014 22:02 To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [Neutron][LBaaS] Continuing on Calling driver interface on every API request Hi Sam, Very true. I think that Vijay’s objection is that we are currently imposing a logical structure on the driver, when it should be a driver decision. Certainly, it goes both ways. And I also agree that the mechanism for returning multiple errors, and the ability to specify whether those errors are fatal or not, individually, is currently weak. Doug On 8/11/14, 10:21 AM, Samuel Bercovici samu...@radware.com wrote: Hi Doug, In some implementations Driver !== Device. I think this is also true for HA Proxy. This might mean that there is a difference between creating a logical object and when there is enough information to actually schedule/place this into a device. The ability to express such errors (detecting an error on a logical object after it was created but when it actually get scheduled) should be discussed and addressed anyway. -Sam. -Original Message- From: Doug Wiegley [mailto:do...@a10networks.com] Sent: Monday, August 11, 2014 6:55 PM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [Neutron][LBaaS] Continuing on Calling driver interface on every API request Hi all, Validations such as ³timeout delay² should be performed on the API level before it reaches the driver. For a configuration tree (lb, listeners, pools, etc.), there should be one provider. You¹re right, but I think the point of Vijay¹s example was to highlight the combo error problem with populating all of the driver objects at once (in short, the driver interface isn¹t well suited to that model.) That his one example can be covered by API validators is irrelevant. Consider a backend that does not support APP_COOKIE¹s, or HTTPS_TERMINATED (but has multiple listeners) instead. Should the entire load balancer create fail, or should it offer degraded service? Do all drivers have to implement a transaction rollback; wait, the interface makes that very hard. That¹s his point. The driver is no longer just glue code between interfaces; it¹s now a mini-object error handler. Having provider defined in multiple places does not make sense. Channeling Brandon, who can yell if I get this wrong, the point is not to have a potentially different provider on each object. It¹s to allow a provider to be assigned when the first object in the tree is created, so that future related objects will always get routed to the same provider. Not knowing which provider should get all the objects is why we have to wait until we see a LoadBalancer object. All of this sort of edge case nonsense is because we (the royal we, the community), wanted all load balancer objects to be ³root² objects, even though only one of them is an actual root today, to support many-to-many relationships among all of them, at some future date, without an interface change. If my bias is showing that I¹m not a fan of adding this complexity
Re: [openstack-dev] Which program for Rally
I apologize for the delay in my response to this thread, between travelling and having a stuck 'a' key on my laptop this is the earliest I could respond. I opted for a separate branch on this thread to summarize my views and I'll respond inline later on some of the previous discussion. On Wed, Aug 06, 2014 at 12:30:35PM +0200, Thierry Carrez wrote: Hi everyone, At the TC meeting yesterday we discussed Rally program request and incubation request. We quickly dismissed the incubation request, as Rally appears to be able to live happily on top of OpenStack and would benefit from having a release cycle decoupled from the OpenStack integrated release. That leaves the question of the program. OpenStack programs are created by the Technical Committee, to bless existing efforts and teams that are considered *essential* to the production of the OpenStack integrated release and the completion of the OpenStack project mission. There are 3 ways to look at Rally and official programs at this point: 1. Rally as an essential QA tool Performance testing (and especially performance regression testing) is an essential QA function, and a feature that Rally provides. If the QA team is happy to use Rally to fill that function, then Rally can obviously be adopted by the (already-existing) QA program. That said, that would put Rally under the authority of the QA PTL, and that raises a few questions due to the current architecture of Rally, which is more product-oriented. There needs to be further discussion between the QA core team and the Rally team to see how that could work and if that option would be acceptable for both sides. So ideally this is where Rally would belong, the scope of what Rally is attempting to do is definitely inside the scope of the QA program. I don't see any reason why that isn't the case. The issue is with what Rally is in it's current form. It's scope is too large and monolithic, and it duplicates much of the functionality we either already have or need in current QA or Infra projects. But, nothing in Rally is designed to be used outside of it. I actually feel pretty strongly that in it's current form Rally should *not* be a part of any OpenStack program. All of the points Sean was making in the other branch on this thread (which I'll probably respond to later) are a huge concerns I share with Rally. He basically summarized most of my views on the topic, so I'll try not to rewrite everything. But, the fact that all of this duplicate functionality was implemented in a completely separate manner which is Rally specific and can't really be used unless all of Rally is used is of a large concern. What I think the path forward here is to have both QA and Rally work together on getting common functionality that is re-usable and shareable. Additionally, I have some concerns over the methodology that Rally uses for it's performance measurement. But, I'll table that discussion because I think it would partially derail this discussion. So one open question is long-term where would this leave Rally if we want to bring it in under the QA program. (after splitting up the functionality to more conducive with all our existing tools and projects) The one thing Rally does here which we don't have an analogous solution for is, for lack of better term, the post processing layer. The part that generates the performs the analysis on the collected data and generates the graphs. That is something that we'll have an eventually need for and that is something that we can work on turning rally into as we migrate everything to actually work together. There are probably also other parts of Rally which don't fit into an existing QA program project, (or the QA program in general) and in those cases we probably should split them off as smaller projects to implement that bit. For example, the SLA stuff Rally has that probably should be a separate entity as well, but I'm unsure if that fits under QA program. My primary concern is the timing for doing all of this work. We're approaching J-3 and honestly this feels like it would take the better part of an entire cycle to analyze, plan, and then implement. Starting an analysis of how to do all of the work at this point I feel would just distract everyone from completing our dev goals for the cycle. Probably the Rally team, if they want to move forward here, should start the analysis of this structural split and we can all pick this up together post-juno. 2. Rally as an essential operator tool Regular benchmarking of OpenStack deployments is a best practice for cloud operators, and a feature that Rally provides. With a bit of a stretch, we could consider that benchmarking is essential to the completion of the OpenStack project mission. That program could one day evolve to include more such operations best practices tools. In addition to the slight stretch already mentioned, one concern here is that we still want to have performance testing in QA
Re: [openstack-dev] [tc][ceilometer] Some background on the gnocchi project
On 8/11/2014 4:22 PM, Eoghan Glynn wrote: Hi Eoghan, Thanks for the note below. However, one thing the overview below does not cover is why InfluxDB ( http://influxdb.com/ ) is not being leveraged. Many folks feel that this technology is a viable solution for the problem space discussed below. Great question Brad! As it happens we've been working closely with Paul Dix (lead developer of InfluxDB) to ensure that this metrics store would be usable as a backend driver. That conversation actually kicked off at the Juno summit in Atlanta, but it really got off the ground at our mid-cycle meet-up in Paris on in early July. ... The InfluxDB folks have committed to implementing those features in over July and August, and have made concrete progress on that score. I hope that provides enough detail to answer to your question? I guess it begs the question, if influxdb will do what you want and it's open source (MIT) as well as commercially supported, how does gnocchi differentiate? Hi Sandy, One of the ideas behind gnocchi is to combine resource representation and timeseries-oriented storage of metric data, providing an efficient and convenient way to query for metric data associated with individual resources. Also, having an API layered above the storage driver avoids locking in directly with a particular metrics-oriented DB, allowing for the potential to support multiple storage driver options (e.g. to choose between a canonical implementation based on Swift, an InfluxDB driver, and an OpenTSDB driver, say). A less compelling reason would be to provide a well-defined hook point to innovate with aggregation/analytic logic not supported natively in the underlying drivers (e.g. period-spanning statistics such as exponentially-weighted moving average or even Holt-Winters). Cheers, Eoghan Cheers, Eoghan Thanks, Brad Brad Topol, Ph.D. IBM Distinguished Engineer OpenStack (919) 543-0646 Internet: bto...@us.ibm.com Assistant: Kendra Witherspoon (919) 254-0680 From: Eoghan Glynn egl...@redhat.com To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org, Date: 08/06/2014 11:17 AM Subject: [openstack-dev] [tc][ceilometer] Some background on the gnocchi project Folks, It's come to our attention that some key individuals are not fully up-to-date on gnocchi activities, so it being a good and healthy thing to ensure we're as communicative as possible about our roadmap, I've provided a high-level overview here of our thinking. This is intended as a precursor to further discussion with the TC. Cheers, Eoghan What gnocchi is: === Gnocchi is a separate, but related, project spun up on stackforge by Julien Danjou, with the objective of providing efficient storage and retrieval of timeseries-oriented data and resource representations. The goal is to experiment with a potential approach to addressing an architectural misstep made in the very earliest days of ceilometer, specifically the decision to store snapshots of some resource metadata alongside each metric datapoint. The core idea is to move to storing datapoints shorn of metadata, and instead allow the resource-state timeline to be reconstructed more cheaply from much less frequently occurring events (e.g. instance resizes or migrations). What gnocchi isn't: == Gnocchi is not a large-scale under-the-radar rewrite of a core OpenStack component along the lines of keystone-lite. The change is concentrated on the final data-storage phase of the ceilometer pipeline, so will have little initial impact on the data-acquiring agents, or on transformation phase. We've been totally open at the Atlanta summit and other forums about this approach being a multi-cycle effort. Why we decided to do it this way: The intent behind spinning up a separate project on stackforge was to allow the work progress at arms-length from ceilometer, allowing normalcy to be maintained on the core project and a rapid rate of innovation on gnocchi. Note that that the developers primarily contributing to gnocchi represent a cross-section of the core team, and there's a regular feedback loop in the form of a recurring agenda item at the weekly team meeting to avoid the effort becoming silo'd. But isn't re-architecting frowned upon? == Well, the architecture of other OpenStack projects have also under-gone change as the community understanding of the implications of prior design decisions has evolved. Take for example the move towards nova no-db-compute the unified-object-model in order to address issues in the nova architecture that made progress towards rolling upgrades unneccessarily difficult. The point, in my
Re: [openstack-dev] [Congress] congress-server fails to start
Hi Rajdeep, What version of pip are you running? Please try installing the latest version (https://pip.pypa.io/en/latest/installing.html) and run 'sudo pip install -r requirements.txt'. - Peter From: Rajdeep Dua rajdeep@gmail.commailto:rajdeep@gmail.com Reply-To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.orgmailto:openstack-dev@lists.openstack.org Date: Monday, August 11, 2014 at 11:27 AM To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.orgmailto:openstack-dev@lists.openstack.org Subject: [openstack-dev] [Congress] congress-server fails to start Hi All, command to start the congress-server fails $ ./bin/congress-server --config-file etc/congress.conf.sample Error : ImportError: No module named keystonemiddleware.auth_token Installing keystonemiddleware manually also fails $ sudo pip install keystonemiddleware Could not find a version that satisfies the requirement oslo.config=1.4.0.0a3 (from keystonemiddleware) (from versions: ) No distributions matching the version for oslo.config=1.4.0.0a3 (from keystonemiddleware) Thanks Rajdeep ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [TripleO][heat] a small experiment with Ansible in TripleO
On 11/08/14 14:49, Clint Byrum wrote: Excerpts from Steven Hardy's message of 2014-08-11 11:40:07 -0700: On Mon, Aug 11, 2014 at 11:20:50AM -0700, Clint Byrum wrote: Excerpts from Zane Bitter's message of 2014-08-11 08:16:56 -0700: On 11/08/14 10:46, Clint Byrum wrote: Right now we're stuck with an update that just doesn't work. It isn't just about update-failure-recovery, which is coming along nicely, but it is also about the lack of signals to control rebuild, poor support for addressing machines as groups, and unacceptable performance in large stacks. Are there blueprints/bugs filed for all of these issues? Convergnce addresses the poor performance for large stacks in general. We also have this: https://bugs.launchpad.net/heat/+bug/1306743 Which shows how slow metadata access can get. I have worked on patches but haven't been able to complete them. We made big strides but we are at a point where 40 nodes polling Heat every 30s is too much for one CPU This sounds like the same figure I heard at the design summit; did the DB call optimisation work that Steve Baker did immediately after that not have any effect? to handle. When we scaled Heat out onto more CPUs on one box by forking we ran into eventlet issues. We also ran into issues because even with many processes we can only use one to resolve templates for a single stack during update, which was also excessively slow. Related to this, and a discussion we had recently at the TripleO meetup is this spec I raised today: https://review.openstack.org/#/c/113296/ It's following up on the idea that we could potentially address (or at least mitigate, pending the fully convergence-ified heat) some of these scalability concerns, if TripleO moves from the one-giant-template model to a more modular nested-stack/provider model (e.g what Tomas has been working on) I've not got into enough detail on that yet to be sure if it's acheivable for Juno, but it seems initially to be complex-but-doable. I'd welcome feedback on that idea and how it may fit in with the more granular convergence-engine model. Can you link to the eventlet/forking issues bug please? I thought since bug #1321303 was fixed that multiple engines and multiple workers should work OK, and obviously that being true is a precondition to expending significant effort on the nested stack decoupling plan above. That was the issue. So we fixed that bug, but we never un-reverted the patch that forks enough engines to use up all the CPU's on a box by default. That would likely help a lot with metadata access speed (we could manually do it in TripleO but we tend to push defaults. :) Right, and we decided we wouldn't because it's wrong to do that to people by default. In some cases the optimal running configuration for TripleO will differ from the friendliest out-of-the-box configuration for Heat users in general, and in those cases - of which this is one - TripleO will need to specify the configuration. cheers, Zane. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Which program for Rally
On 08/11/2014 04:21 PM, Matthew Treinish wrote: I apologize for the delay in my response to this thread, between travelling and having a stuck 'a' key on my laptop this is the earliest I could respond. I opted for a separate branch on this thread to summarize my views and I'll respond inline later on some of the previous discussion. On Wed, Aug 06, 2014 at 12:30:35PM +0200, Thierry Carrez wrote: Hi everyone, At the TC meeting yesterday we discussed Rally program request and incubation request. We quickly dismissed the incubation request, as Rally appears to be able to live happily on top of OpenStack and would benefit from having a release cycle decoupled from the OpenStack integrated release. That leaves the question of the program. OpenStack programs are created by the Technical Committee, to bless existing efforts and teams that are considered *essential* to the production of the OpenStack integrated release and the completion of the OpenStack project mission. There are 3 ways to look at Rally and official programs at this point: 1. Rally as an essential QA tool Performance testing (and especially performance regression testing) is an essential QA function, and a feature that Rally provides. If the QA team is happy to use Rally to fill that function, then Rally can obviously be adopted by the (already-existing) QA program. That said, that would put Rally under the authority of the QA PTL, and that raises a few questions due to the current architecture of Rally, which is more product-oriented. There needs to be further discussion between the QA core team and the Rally team to see how that could work and if that option would be acceptable for both sides. So ideally this is where Rally would belong, the scope of what Rally is attempting to do is definitely inside the scope of the QA program. I don't see any reason why that isn't the case. The issue is with what Rally is in it's current form. It's scope is too large and monolithic, and it duplicates much of the functionality we either already have or need in current QA or Infra projects. But, nothing in Rally is designed to be used outside of it. I actually feel pretty strongly that in it's current form Rally should *not* be a part of any OpenStack program. All of the points Sean was making in the other branch on this thread (which I'll probably respond to later) are a huge concerns I share with Rally. He basically summarized most of my views on the topic, so I'll try not to rewrite everything. But, the fact that all of this duplicate functionality was implemented in a completely separate manner which is Rally specific and can't really be used unless all of Rally is used is of a large concern. What I think the path forward here is to have both QA and Rally work together on getting common functionality that is re-usable and shareable. Additionally, I have some concerns over the methodology that Rally uses for it's performance measurement. But, I'll table that discussion because I think it would partially derail this discussion. So one open question is long-term where would this leave Rally if we want to bring it in under the QA program. (after splitting up the functionality to more conducive with all our existing tools and projects) The one thing Rally does here which we don't have an analogous solution for is, for lack of better term, the post processing layer. The part that generates the performs the analysis on the collected data and generates the graphs. That is something that we'll have an eventually need for and that is something that we can work on turning rally into as we migrate everything to actually work together. There are probably also other parts of Rally which don't fit into an existing QA program project, (or the QA program in general) and in those cases we probably should split them off as smaller projects to implement that bit. For example, the SLA stuff Rally has that probably should be a separate entity as well, but I'm unsure if that fits under QA program. My primary concern is the timing for doing all of this work. We're approaching J-3 and honestly this feels like it would take the better part of an entire cycle to analyze, plan, and then implement. Starting an analysis of how to do all of the work at this point I feel would just distract everyone from completing our dev goals for the cycle. Probably the Rally team, if they want to move forward here, should start the analysis of this structural split and we can all pick this up together post-juno. 2. Rally as an essential operator tool Regular benchmarking of OpenStack deployments is a best practice for cloud operators, and a feature that Rally provides. With a bit of a stretch, we could consider that benchmarking is essential to the completion of the OpenStack project mission. That program could one day evolve to include more such operations best practices tools. In addition to the slight stretch
Re: [openstack-dev] [Neutron][LBaaS] Continuing on Calling driver interface on every API request
Hi Eugene, An example of the HM issue (and really this can happen with any entity) is if the driver the API sends the configuration to does not actually support the value of an attribute. For example: Provider A support PING health monitor type, Provider B does not. API allows the PING health monitor type to go through. Once a load balancer has been linked with that health monitor and the LoadBalancer chose to use Provider B, that entire configuration is then sent to the driver. The driver errors out not on the LoadBalancer create, but on the health monitor create. I think that's the issue. Thanks, Brandon On Tue, 2014-08-12 at 00:17 +0400, Eugene Nikanorov wrote: Hi folks, That actually going in opposite direction to what flavor framework is trying to do (and for dispatching it's doing the same as providers). REST call dispatching should really go via the root object. I don't quite get the issue with health monitors. If HM is incorrectly configured prior to association with a pool - API layer should handle that. I don't think driver implementations should be different at constraints to HM parameters. So I'm -1 on adding provider (or flavor) to each entity. After all, it looks just like data denormalization which actually will affect lots of API aspects in negative way. Thanks, Eugene. On Mon, Aug 11, 2014 at 11:20 PM, Vijay Venkatachalam vijay.venkatacha...@citrix.com wrote: Yes, the point was to say the plugin need not restrict and let driver decide what to do with the API. Even if the call was made to driver instantaneously, I understand, the driver might decide to ignore first and schedule later. But, if the call is present, there is scope for validation. Also, the driver might be scheduling an async-api to backend, in which case deployment error cannot be shown to the user instantaneously. W.r.t. identifying a provider/driver, how would it be to make tenant the default root object? tenantid is already associated with each of these entities, so no additional pain. For the tenant who wants to override let him specify provider in each of the entities. If you think of this in terms of the UI, let's say if the loadbalancer configuration is exposed as a single wizard (which has loadbalancer, listener, pool, monitor properties) then provider is chosen only once. Curious question, is flavour framework expected to address this problem? Thanks, Vijay V. -Original Message- From: Doug Wiegley [mailto:do...@a10networks.com] Sent: 11 August 2014 22:02 To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [Neutron][LBaaS] Continuing on Calling driver interface on every API request Hi Sam, Very true. I think that Vijay’s objection is that we are currently imposing a logical structure on the driver, when it should be a driver decision. Certainly, it goes both ways. And I also agree that the mechanism for returning multiple errors, and the ability to specify whether those errors are fatal or not, individually, is currently weak. Doug On 8/11/14, 10:21 AM, Samuel Bercovici samu...@radware.com wrote: Hi Doug, In some implementations Driver !== Device. I think this is also true for HA Proxy. This might mean that there is a difference between creating a logical object and when there is enough information to actually schedule/place this into a device. The ability to express such errors (detecting an error on a logical object after it was created but when it actually get scheduled) should be discussed and addressed anyway. -Sam. -Original Message- From: Doug Wiegley [mailto:do...@a10networks.com] Sent: Monday, August 11, 2014 6:55 PM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [Neutron][LBaaS] Continuing on Calling driver interface on every API request Hi all, Validations such as ³timeout delay² should be performed on the API level before it reaches the driver. For a configuration tree (lb, listeners, pools, etc.), there should be one provider. You¹re right, but I think the point of Vijay¹s example was to highlight
Re: [openstack-dev] [TripleO][Nova][Neutron] multiple hypervisors on one compute host - neutron agent and compute hostnames
On Tue, Aug 5, 2014 at 4:17 PM, Robert Collins robe...@robertcollins.net
wrote:
Hi!
James has run into an issue implementing the multi-hypervisor spec
(
http://git.openstack.org/cgit/openstack/tripleo-specs/tree/specs/juno/tripleo-juno-deploy-cloud-hypervisor-type.rst
)
which we're hoping to use to reduce infrastructure overheads by
deploying OpenStack control plane services in VMs, without requiring
dedicated VM hypervisor machines in the deploy cloud.
The issue we've hit is that the Neutron messages for VIF plugging are
sent to the Neutron agent with an exactly matching hostname to the
Nova-compute process. However, we have unique hostnames for the
nova-compute processes on one machine (one for -kvm, one for -docker,
one for -ironic etc) for a variety of reasons: so we can see if all
the processes are up, so that we don't get messages for the wrong
process from nova-api etc.
So you are running multiple nova-computes on a single node? This goes
against the model that nova is operating under. Instead of hacking a
workaround, if we think this is a use case nova/openstack should support,
why not have that discussion before deciding that the best solution is a
shallow patch.
I think a reasonable step might be to allow the agent host option to
be a list - e.g.
[DEFAULT]
hosts={{nova.compute_hostname}}-libvirt,{{nova.compute_hostname}}-docker
we'd just make it listen to all the nova-compute hostnames we may have
on the machine.
That seems like a fairly shallow patch to me: add a new hosts option
with no default, change the code to listen to N queues when hosts is
set, and to report state N times as well (for consistency).
Alternatively, with a DB migration, we could record N hosts against
one agent status.
Alternatively we could run N ovs-agents on one machine (with a
separate integration bridge each), but I worry that we'd encounter
unexpected cross-chatter between them on things like external bridge
flows.
Thoughts?
For now, we're going to have to run with a limitation of only one
vif-plugging hypervisor type per machine - we'll make the agent
hostname match that of the nova compute that needs VIFs plugged ;)
-Rob
--
Robert Collins rbtcoll...@hp.com
Distinguished Technologist
HP Converged Cloud
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [tc][ceilometer] Some background on the gnocchi project
On 8/11/2014 5:29 PM, Eoghan Glynn wrote: On 8/11/2014 4:22 PM, Eoghan Glynn wrote: Hi Eoghan, Thanks for the note below. However, one thing the overview below does not cover is why InfluxDB ( http://influxdb.com/ ) is not being leveraged. Many folks feel that this technology is a viable solution for the problem space discussed below. Great question Brad! As it happens we've been working closely with Paul Dix (lead developer of InfluxDB) to ensure that this metrics store would be usable as a backend driver. That conversation actually kicked off at the Juno summit in Atlanta, but it really got off the ground at our mid-cycle meet-up in Paris on in early July. ... The InfluxDB folks have committed to implementing those features in over July and August, and have made concrete progress on that score. I hope that provides enough detail to answer to your question? I guess it begs the question, if influxdb will do what you want and it's open source (MIT) as well as commercially supported, how does gnocchi differentiate? Hi Sandy, One of the ideas behind gnocchi is to combine resource representation and timeseries-oriented storage of metric data, providing an efficient and convenient way to query for metric data associated with individual resources. Doesn't InfluxDB do the same? Also, having an API layered above the storage driver avoids locking in directly with a particular metrics-oriented DB, allowing for the potential to support multiple storage driver options (e.g. to choose between a canonical implementation based on Swift, an InfluxDB driver, and an OpenTSDB driver, say). Right, I'm not suggesting to remove the storage abstraction layer. I'm just curious what gnocchi does better/different than InfluxDB? Or, am I missing the objective here and gnocchi is the abstraction layer and not an influxdb alternative? If so, my apologies for the confusion. A less compelling reason would be to provide a well-defined hook point to innovate with aggregation/analytic logic not supported natively in the underlying drivers (e.g. period-spanning statistics such as exponentially-weighted moving average or even Holt-Winters). Cheers, Eoghan Cheers, Eoghan Thanks, Brad Brad Topol, Ph.D. IBM Distinguished Engineer OpenStack (919) 543-0646 Internet: bto...@us.ibm.com Assistant: Kendra Witherspoon (919) 254-0680 From: Eoghan Glynn egl...@redhat.com To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org, Date: 08/06/2014 11:17 AM Subject: [openstack-dev] [tc][ceilometer] Some background on the gnocchi project Folks, It's come to our attention that some key individuals are not fully up-to-date on gnocchi activities, so it being a good and healthy thing to ensure we're as communicative as possible about our roadmap, I've provided a high-level overview here of our thinking. This is intended as a precursor to further discussion with the TC. Cheers, Eoghan What gnocchi is: === Gnocchi is a separate, but related, project spun up on stackforge by Julien Danjou, with the objective of providing efficient storage and retrieval of timeseries-oriented data and resource representations. The goal is to experiment with a potential approach to addressing an architectural misstep made in the very earliest days of ceilometer, specifically the decision to store snapshots of some resource metadata alongside each metric datapoint. The core idea is to move to storing datapoints shorn of metadata, and instead allow the resource-state timeline to be reconstructed more cheaply from much less frequently occurring events (e.g. instance resizes or migrations). What gnocchi isn't: == Gnocchi is not a large-scale under-the-radar rewrite of a core OpenStack component along the lines of keystone-lite. The change is concentrated on the final data-storage phase of the ceilometer pipeline, so will have little initial impact on the data-acquiring agents, or on transformation phase. We've been totally open at the Atlanta summit and other forums about this approach being a multi-cycle effort. Why we decided to do it this way: The intent behind spinning up a separate project on stackforge was to allow the work progress at arms-length from ceilometer, allowing normalcy to be maintained on the core project and a rapid rate of innovation on gnocchi. Note that that the developers primarily contributing to gnocchi represent a cross-section of the core team, and there's a regular feedback loop in the form of a recurring agenda item at the weekly team meeting to avoid the effort becoming silo'd. But isn't re-architecting frowned upon? == Well, the architecture of other OpenStack projects have also under-gone change as the community understanding of the implications of prior design decisions has
Re: [openstack-dev] [qa][ceilometer] swapping the roles of mongodb and sqlalchemy for ceilometer in Tempest
On Sat, Aug 9, 2014 at 6:29 AM, Eoghan Glynn egl...@redhat.com wrote: Hi Folks, Dina Belova has recently landed some infra patches[1,2] to create an experimental mongodb-based Tempest job. This effectively just overrides the ceilometer storage backend config so that mongodb is used instead of sql-alchemy. The new job has been running happily for a few days so I'd like now to consider the path forwards with this. One of our Juno goals under the TC gap analysis was to more fully gate against mongodb, given that this is the storage backend recommended/supported by many distros. The sql-alchemy backend, on the other hand, is more suited for proofs of concept or small deployments. However up to now we've been hampered from reflecting that reality in the gate, due to the gate being stuck on Precise for a long time, as befits LTS, and the version of mongodb needed by ceilometer (i.e. 2.4) effectively unavailable on that Ubuntu release (in fact it was limited to 2.0.4). So the orientation towards gating on sql-alchemy was mostly driven by legacy issues in the gate's usage of Precise, as opposed to this being considered the most logical basket in which to put all our testing eggs. However, we're now finally in the brave new world of Trusty :) So I would like to make the long-delayed change over soon. This would involve transposing the roles of sql-alchemy and mongodb in the gate - the mongodb variant becomes the blessed job run by default, whereas the sql-alchemy based job to relegated to the second tier. So my questions are: (a) would the QA side of the house be agreeable to this switch? and: (b) how long would the mongodb job need to be stable in this experimental mode before we pull the trigger on swicthing? If the answer to (a) is yes, we can get infra patches proposed early next week to make the swap. Cheers, Eoghan [1] https://review.openstack.org/#/q/project:openstack-infra/config+branch:master+topic:ceilometer-mongodb-job,n,z [2] https://review.openstack.org/#/q/project:openstack-infra/devstack-gate+branch:master+topic:ceilometer-backend,n,z My interpretation of the gap analysis [1] is merely that you have coverage, not that you switch to it and relegate the SQLAlchemy tests to second chair. I believe that's a dangerous departure from current standards. A dependency on mongodb, due to it's AGPL license, and lack of sufficient support for a non-AGPL storage back end, has consistently been raised as a blocking issue for Marconi. [2] Sure, the main goal is to have full mongodb-based coverage in the gate. So, if the QA/infra folks are prepared to host *both* jobs, then I'd be happy to change my request to simply: let's promote the mongodb-based Tempest variant to the first tier, to run alongside the current sqlalchemy-based job Ignoring the question of is it ok to say: 'to run ceilometer in any sort of non-trivial deployment you must manager yet another underlying service, mongodb' I would prefer not adding an addition gate variant to all projects. With the effort to reduce the number of gate variants we have [0] I would prefer to see just ceilometer gate on both mongodb and sqlalchemy and the main integrated gate [1] pick just one. [0] http://lists.openstack.org/pipermail/openstack-dev/2014-July/041057.html [1] http://git.openstack.org/cgit/openstack-infra/config/tree/modules/openstack_project/files/zuul/layout.yaml#n238 Does that work for you Devananda? Cheers, Eoghan -Deva [1] https://wiki.openstack.org/wiki/Governance/TechnicalCommittee/Ceilometer_Gap_Coverage [2] http://lists.openstack.org/pipermail/openstack-dev/2014-March/030510.html is a very articulate example of this objection ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [tc][ceilometer] Some background on the gnocchi project
On 2014-08-11 5:13 PM, Sandy Walsh wrote: Right, I'm not suggesting to remove the storage abstraction layer. I'm just curious what gnocchi does better/different than InfluxDB? I was at the OpenStack Design Summit when Gnocchi was presented. Soon after the basic goals and technical details of Gnocchi were presented, people wondered why InfluxDB wasn't used. AFAIK, people presenting Gnocchi didn't know about InfluxDB so they weren't able to answer the question. I don't really blame them. At that time, I didn't know anything about Gnocchi, even less about InfluxDB but rapidly learned that both are DataSeries databases/services. What I would have answered to that question is (IMO): Gnocchi is a new project tackling the need for a DataSeries database/storage as a service. Pandas/Swift is used as an implementation reference. Some people love Swift and will use it everywhere they can, nothing wrong with it. (or lets not go down that path) Or, am I missing the objective here and gnocchi is the abstraction layer and not an influxdb alternative? If so, my apologies for the confusion. InfluxDB can't be used as-is by OpenStack services. There needs to be an abstraction layer somewhere. As Gnocchi is (or will be) well written, people will be free to drop the Swift implementation and replace it by whatever they want: InfluxDB, Blueflood, RRD, Whisper, plain text files, in-memory, /dev/null, etc. But we first need to start somewhere with one implementation and Pandas/Swift was chosen. I'm confident people will soon start proposing alternative storage backends/implementations better fitting their needs and tastes. -- Mathieu ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Neutron][LBaaS] Use cases with regards to VIP and routers
Susanne, Are you asking in the context of Load Balancer services in general, or in terms of the Neutron LBaaS project or the Octavia project? Stephen On Mon, Aug 11, 2014 at 9:04 AM, Doug Wiegley do...@a10networks.com wrote: Hi Susanne, While there are a few operators involved with LBaaS that would have good input, you might want to also ask this on the non-dev mailing list, for a larger sample size. Thanks, doug On 8/11/14, 3:05 AM, Susanne Balle sleipnir...@gmail.com wrote: Gang, I was asked the following questions around our Neutron LBaaS use cases: 1. Will there be a scenario where the ³VIP² port will be in a different Node, from all the Member ³VMs² in a pool. 2. Also how likely is it for the LBaaS configured subnet to not have a ³router² and just use the ³extra_routes² option. 3. Is there a valid use case where customers will be using the ³extra_routes² with subnets instead of the ³routers². ( It would be great if you have some use case picture for this). Feel free to chime in here and I'll summaries the answers. Regards Susanne ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Stephen Balukoff Blue Box Group, LLC (800)613-4305 x807 ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Which program for Rally
Hi stackers, I would like to put some more details on current situation. The issue is with what Rally is in it's current form. It's scope is too large and monolithic, and it duplicates much of the functionality we either already have or need in current QA or Infra projects. But, nothing in Rally is designed to be used outside of it. I actually feel pretty strongly that in it's current form Rally should *not* be a part of any OpenStack program Rally is not just a bunch of scripts like tempest, it's more like Nova, Cinder, and other projects that works out of box and resolve Operators Dev use cases in one click. This architectural design is the main key of Rally success, and why we got such large adoption and community. So I'm opposed to this option. It feels to me like this is only on the table because the Rally team has not done a great job of communicating or working with anyone else except for when it comes to either push using Rally, or this conversation about adopting Rally. Actually Rally team done already a bunch of useful work including cross projects and infra stuff. Keystone, Glance, Cinder, Neutron and Heat are running rally performance jobs, that can be used for performance testing, benchmarking, regression testing (already now). These jobs supports in-tree plugins for all components (scenarios, load generators, benchmark context) and they can use Rally fully without interaction with Rally team at all. More about these jobs: https://docs.google.com/a/mirantis.com/document/d/1s93IBuyx24dM3SmPcboBp7N47RQedT8u4AJPgOHp9-A/ So I really don't see anything like this in tempest (even in observed future) I would like to mention work on OSprofiler (cross service/project profiler) https://github.com/stackforge/osprofiler (that was done by Rally team) https://review.openstack.org/#/c/105096/ (btw Glance already accepted it https://review.openstack.org/#/c/105635/ ) My primary concern is the timing for doing all of this work. We're approaching J-3 and honestly this feels like it would take the better part of an entire cycle to analyze, plan, and then implement. Starting an analysis of how to do all of the work at this point I feel would just distract everyone from completing our dev goals for the cycle. Probably the Rally team, if they want to move forward here, should start the analysis of this structural split and we can all pick this up together post-juno Matt, Sean - seriously community is about convincing people, not about forcing people to do something against their wiliness. You are making huge architectural decisions without deep knowledge about what is Rally, what are use cases, road map, goals and auditory. IMHO community in my opinion is thing about convincing people. So QA program should convince Rally team (at least me) to do such changes. Key secret to convince me, is to say how this will help OpenStack to perform better. Currently Rally team see a lot of issues related to this decision: 1) It breaks already existing performance jobs (Heat, Glance, Cinder, Neutron, Keystone) 2) It breaks functional testing of Rally (that is already done in gates) 2) It makes Rally team depending on Tempest throughput, and what I heard multiple times from QA team is that performance work is very low priority and that major goals are to keep gates working. So it will slow down work of performance team. 3) Brings a ton of questions what should be in Rally and what should be in Tempest. That are at the moment quite resolved. https://docs.google.com/a/pavlovic.ru/document/d/137zbrz0KJd6uZwoZEu4BkdKiR_Diobantu0GduS7HnA/edit#heading=h.9ephr9df0new 4) It breaks existing OpenStack team, that is working 100% on performance, regressions and sla topics. Sorry but there is no such team in tempest. This directory is not active developed: https://github.com/openstack/tempest/commits/master/tempest/stress Matt, Sean, David - what are the real goals of merging Rally to Tempest? I see a huge harm for OpenStack and companies that are using Rally, and don't see actually any benefits. What I heard for now is something like this decision will make tempest better.. But do you care more about Tempest than OpenStack? Best regards, Boris Pavlovic On Tue, Aug 12, 2014 at 12:37 AM, David Kranz dkr...@redhat.com wrote: On 08/11/2014 04:21 PM, Matthew Treinish wrote: I apologize for the delay in my response to this thread, between travelling and having a stuck 'a' key on my laptop this is the earliest I could respond. I opted for a separate branch on this thread to summarize my views and I'll respond inline later on some of the previous discussion. On Wed, Aug 06, 2014 at 12:30:35PM +0200, Thierry Carrez wrote: Hi everyone, At the TC meeting yesterday we discussed Rally program request and incubation request. We quickly dismissed the incubation request, as Rally appears to be able to live happily on top of OpenStack and would benefit from having a release
Re: [openstack-dev] [tc][ceilometer] Some background on the gnocchi project
On 8/11/2014 4:22 PM, Eoghan Glynn wrote: Hi Eoghan, Thanks for the note below. However, one thing the overview below does not cover is why InfluxDB ( http://influxdb.com/ ) is not being leveraged. Many folks feel that this technology is a viable solution for the problem space discussed below. Great question Brad! As it happens we've been working closely with Paul Dix (lead developer of InfluxDB) to ensure that this metrics store would be usable as a backend driver. That conversation actually kicked off at the Juno summit in Atlanta, but it really got off the ground at our mid-cycle meet-up in Paris on in early July. ... The InfluxDB folks have committed to implementing those features in over July and August, and have made concrete progress on that score. I hope that provides enough detail to answer to your question? I guess it begs the question, if influxdb will do what you want and it's open source (MIT) as well as commercially supported, how does gnocchi differentiate? Hi Sandy, One of the ideas behind gnocchi is to combine resource representation and timeseries-oriented storage of metric data, providing an efficient and convenient way to query for metric data associated with individual resources. Doesn't InfluxDB do the same? InfluxDB stores timeseries data primarily. Gnocchi in intended to store strongly-typed OpenStack resource representations (instance, images, etc.) in addition to providing a means to access timeseries data associated with those resources. So to answer your question: no, IIUC, it doesn't do the same thing. Though of course these things are not a million miles from each other, one is just a step up in the abstraction stack, having a wider and more OpenStack-specific scope. Also, having an API layered above the storage driver avoids locking in directly with a particular metrics-oriented DB, allowing for the potential to support multiple storage driver options (e.g. to choose between a canonical implementation based on Swift, an InfluxDB driver, and an OpenTSDB driver, say). Right, I'm not suggesting to remove the storage abstraction layer. I'm just curious what gnocchi does better/different than InfluxDB? Or, am I missing the objective here and gnocchi is the abstraction layer and not an influxdb alternative? If so, my apologies for the confusion. No worries :) The intention is for gnocchi to provide an abstraction over timeseries, aggregation, downsampling and archiving/retention policies, with a number of drivers mapping onto real timeseries storage options. One of those drivers is based on Swift, another is in the works based on InfluxDB, and a third based on OpenTSDB has also been proposed. Cheers, Eoghan ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
