Re: [openstack-dev] [neutron] Neutron scaling datapoints?
And for another recent one that came out yesterday: Interesting to read for those who are using mongodb + openstack... https://aphyr.com/posts/322-call-me-maybe-mongodb-stale-reads -Josh Joshua Harlow wrote: Joshua Harlow wrote: Kevin Benton wrote: Timestamps are just one way (and likely the most primitive), using redis (or memcache) key/value and expiry are another (and letting memcache or redis expire using its own internal algorithms), using zookeeper ephemeral nodes[1] are another... The point being that its backend specific and tooz supports varying backends. Very cool. Is the backend completely transparent so a deployer could choose a service they are comfortable maintaining, or will that change the properties WRT to resiliency of state on node restarts, partitions, etc? Of course... we tried to make it 'completely' transparent, but in reality certain backends (zookeeper which uses a paxos-like algorithm and redis with sentinel support...) are better (more resilient, more consistent, handle partitions/restarts better...) than others (memcached is after all just a distributed cache). This is just the nature of the game... And for some more reading fun: https://aphyr.com/posts/315-call-me-maybe-rabbitmq https://aphyr.com/posts/291-call-me-maybe-zookeeper https://aphyr.com/posts/283-call-me-maybe-redis https://aphyr.com/posts/316-call-me-maybe-etcd-and-consul ... (aphyr.com has alot of these neat posts)... The Nova implementation of Tooz seemed pretty straight-forward, although it looked like it had pluggable drivers for service management already. Before I dig into it much further I'll file a spec on the Neutron side to see if I can get some other cores onboard to do the review work if I push a change to tooz. Sounds good to me. On Sun, Apr 12, 2015 at 9:38 AM, Joshua Harlow harlo...@outlook.com mailto:harlo...@outlook.com wrote: Kevin Benton wrote: So IIUC tooz would be handling the liveness detection for the agents. That would be nice to get ride of that logic in Neutron and just register callbacks for rescheduling the dead. Where does it store that state, does it persist timestamps to the DB like Neutron does? If so, how would that scale better? If not, who does a given node ask to know if an agent is online or offline when making a scheduling decision? Timestamps are just one way (and likely the most primitive), using redis (or memcache) key/value and expiry are another (and letting memcache or redis expire using its own internal algorithms), using zookeeper ephemeral nodes[1] are another... The point being that its backend specific and tooz supports varying backends. However, before (what I assume is) the large code change to implement tooz, I would like to quantify that the heartbeats are actually a bottleneck. When I was doing some profiling of them on the master branch a few months ago, processing a heartbeat took an order of magnitude less time (50ms) than the 'sync routers' task of the l3 agent (~300ms). A few query optimizations might buy us a lot more headroom before we have to fall back to large refactors. Sure, always good to avoid prematurely optimizing things... Although this is relevant for u I think anyway: https://review.openstack.org/#__/c/138607/ https://review.openstack.org/#/c/138607/ (same thing/nearly same in nova)... https://review.openstack.org/#__/c/172502/ https://review.openstack.org/#/c/172502/ (a WIP implementation of the latter). [1] https://zookeeper.apache.org/__doc/trunk/__zookeeperProgrammers.html#__Ephemeral+Nodes https://zookeeper.apache.org/doc/trunk/zookeeperProgrammers.html#Ephemeral+Nodes Kevin Benton wrote: One of the most common is the heartbeat from each agent. However, I don't think we can't eliminate them because they are used to determine if the agents are still alive for scheduling purposes. Did you have something else in mind to determine if an agent is alive? Put each agent in a tooz[1] group; have each agent periodically heartbeat[2], have whoever needs to schedule read the active members of that group (or use [3] to get notified via a callback), profit... Pick from your favorite (supporting) driver at: http://docs.openstack.org/developer/tooz/compatibility.html http://docs.openstack.org/__developer/tooz/compatibility.__html http://docs.openstack.org/__developer/tooz/compatibility.__html http://docs.openstack.org/developer/tooz/compatibility.html [1] http://docs.openstack.org/developer/tooz/compatibility.html#grouping http://docs.openstack.org/__developer/tooz/compatibility.__html#grouping http://docs.openstack.org/__developer/tooz/compatibility.__html#grouping http://docs.openstack.org/developer/tooz/compatibility.html#grouping [2] https://github.com/openstack/tooz/blob/0.13.1/tooz/coordination.py#L315 https://github.com/openstack/__tooz/blob/0.13.1/tooz/__coordination.py#L315 https://github.com/openstack/__tooz/blob/0.13.1/tooz/__coordination.py#L315
[openstack-dev] TC candidacy
Hello OpenStack world! My name is Dean Troyer and I am running for the OpenStack Technical Committee. I have been part of the OpenStack community for a long time and heavily involved in three projects: I was an early contributor to DevStack and served as its PTL during its short tenure as a stand-alone program, I started Grenade to use DevStack as the basis for upgrade testing. I also am the PTL of the OpenStackClient project that recently became the first project brought into the expanded official project definition. The common thread of my work in OpenStack has been with things that cross the traditional vertical service projects. This has highlighted the scope and consequences of differences between projects for me. Differences that affect project developers is one thing, and sometimes a good thing even, but differences that adversely affect deployers and consumers of OpenStack clouds is another thing altogether. I believe this is where the focus of encouraging projects to converge should be placed. Efforts like the API Working Group and the Log Working Group need to be encouraged where they improve the experiences of our customers (where 'our' == 'OpenStack' and 'customers' == 'everyone downstream from OpenStack'). I believe the TC has a good bit of work ahead to follow through implementing the vision of inclusiveness. Communicating the state of projects is a huge part of this. We should set up the goals and see who can score. Those projects that do score will be the ones rewarded not by the TC but by the rest of the community. One specific example that I think the TC should facilitate is describing the technical relationships between projects. The TC should bless, if not create, a common vocabulary to describe the groups of projects and their relationships. While this may be expressed in 'tags', it is more than just tag definitions. The Technical Committee is unique in that in some way it touches every aspect of OpenStack: projects and project developers, application developers, distributors and VARs, cloud deployers and operators, end users, and the OpenStack Board of Directors (DefCore!). The TC is the duct tape of OpenStack! No, it is better than that, it is by design the group that oversees enabling everything else to succeed. I believe that TC members must have a broad vision and insight into many parts of OpenStack and depth into at least a couple of areas. And I believe that I have both of those attributes. I would appreciate your consideration and vote. As I mentioned above, I have been part of the OpenStack community since before there was one, working on the original Nova deployment at NASA. That was followed by a stint at Rackspace where DevStack and OSC were born, and on to Nebula where we tossed Grenade into the world. After the recent events at Nebula I will be joining Intel soon and again be focused on upstream OpenStack projects. Thanks again for your time and consideration dt -- Dean Troyer dtro...@gmail.com __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [oslo][policy][neutron] oslo.policy API is not powerful enough to switch Neutron to it
Excerpts from Ihar Hrachyshka's message of 2015-04-22 12:33:52 +0200: On 04/22/2015 05:01 AM, Doug Hellmann wrote: Excerpts from Ihar Hrachyshka's message of 2015-04-17 14:45:58 +0200: Hi, tl;dr neutron has special semantics for policy targets that relies on private symbols from oslo.policy, and it's impossible to introduce this semantics into oslo.policy itself due to backwards compatibility concerns, meaning we need to expose some more symbols as part of public API for the library to facilitate neutron switch to it. === oslo.policy was graduated during Kilo [1]. Neutron considered the switch to it [2], but failed to achieve it because some library symbols that were originally public (or at least looked like public) in policy.py from oslo-incubator, became private in oslo.policy. Specifically, Neutron policy code [3] relies on the following symbols that are now hidden inside oslo_policy._checks (note the underscore in the name of the module that suggests we cannot use the module directly): - - RoleCheck - - RuleCheck - - AndCheck I'm not sure I followed all of the rest of what you wrote below, but it seems like this is the real problem. We are pretty aggressive about making things private when we release a new library, because it's easier to make them public later if we need to than it is to make public things private. In this case, it looks like we made some symbols private even though they were already being used, and that seems like a mistake on our part. So, if we make those public, would that address the issues with neutron adopting the library? Yes, that would help. I will also check Adam's suggestion, maybe it will allow us to avoid exposing RoleCheck. Keeping stuff private is less of a priority if we can demonstrate that having it public makes it more useful. Those symbols are used for the following matters: (all the relevant neutron code is in neutron/policy.py) 1. debug logging in case policy does not authorize an action (RuleCheck, AndCheck) [log_rule_list] 2. filling in admin context with admin roles (RoleCheck, RuleCheck, AndCheck/OrCheck internals) [get_admin_roles] 3. aggregating core, attribute and subattribute policies (RuleCheck, AndCheck) [_prepare_check] == 1. debug logging in case policy does not authorize an action == Neutron logs rules that failed to match if policy module does not authorize an action. Not sure whether Neutron developers really want to have those debug logs, and whether we cannot just kill them to avoid this specific usage of private symbols; though it also seems that we could easily use __str__ that is present for all types of Checks instead. So it does not look like a blocker for the switch. == 2. filling in admin context with admin roles == Admin context object is filled with .roles attribute that is a list of roles considered granting admin permissions [4]. The attribute would then be used by plugins that would like to do explicit policy checks. As per Salvatore, this attribute can probably be dropped now that all plugins and services don't rely on it (Salvatore mentioned lbaas mixins as the ones that previously relied on it, but are now not doing it since service split from neutron tree (?)). The problem with dropping the .roles attribute from context object in Liberty is that we, as a responsible upstream with lots of plugins maintained out-of-tree (see the ongoing vendor decomposition effort) would need to support the attribute while it's marked as deprecated for at least one cycle, meaning that if we don't get those oslo.policy internals we rely on in Liberty, we would need to postpone the switch till Mizzle, or rely on private symbols during the switch (while a new release of oslo.policy can easily break us). (BTW the code to extract admin roles is not really robust and has bugs, f.e. it does not handle AndChecks that could be used in context_is_admin. In theory, 'and' syntax would mean that both roles are needed to claim someone is an admin, while the code to extract admin roles handles 'and' the same way as 'or'. For the deprecation time being, we may need to document this limitation.) == 3. aggregating core, attribute and subattribute policies == That's the most interesting issue. For oslo.policy, policies are described as target: rule, where rule is interpreted as per registered checks, while target is opaque to the library. Neutron extended the syntax for target as: target[:attribute[:subattribute]]. This extension of the syntax is a bit more problematic. We should talk about a way to fold that into the library so it can be used consistently across projects. FTR, making it less easy to extend common behaviors in application-specific ways is one reason we like to make symbols private whenever possible.
Re: [openstack-dev] [all] Liberty Design Summit - Proposed room / time layout
Thierry Carrez wrote: Ben Swartzlander wrote: My main request was that the Manila sessions didn't overlap with the Cinder sessions, since there are some key people who are involved in both projects. It looks like some attempt was made to avoid overlaps (the fishbowl sessions at least), but the working sessions could be adjusted to overlap less (space permitting). I would prefer that some of the Wednesday Manila working sessions were moved to the end of Thursday (after the Cinder ones). I'll shuffle a few things around. No miracle to expect though: there are only 18 time slots available and Cinder requested 13, and Manila 7. So there will at least be two conflicts. OK, I pushed a number of changes to accommodate the various requests. They are highlighted in green in the proposed layout. https://docs.google.com/spreadsheets/d/1VsFdRYGbX5eCde81XDV7TrPBfEC7cgtOFikruYmqbPY/edit#gid=569963128 I expect to make it official tomorrow, so let me know if any of the changes break the world. -- Thierry Carrez (ttx) __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] TC candidacy
Please consider my candidacy for membership on the technical committee. My name is Anita Kuno. I consider my home project to be Infra but I tend to move around wherever I feel the need is greatest. I have worked on OpenStack since mid-grizzly starting as an intern with the GNOME Outreach Program for Women (which is now known as Outreachy) with my mentor Iccha Sethi. I moved around until I found Infra and have considered that my home base ever since, mostly because Infra, and my job, allows me so much flexibility. During the Hong Kong summit I launched myself into Neutron to see if there was anything I could do to support improvement, not because I knew anything about Neutron but because I live by the axiom don't ask anyone to do anything you wouldn't be prepared to do yourself. I realized that Neutron developers couldn't even find each other to talk to each other due to the crowd and organized a Neutron Tempest code sprint in Montreal in January 2014. Coming out of that, I have been involved with the third party ci space ever since, a difficult and demanding space and those involved with have many opinions on whether I have done and am doing a good job or a poor job. I split the project-config repo out of what was the Infra config repo (and is now system-config) at the end of Juno and have been a core reviewer on the project ever since. I was able to help Neutron split out their 'as a service' repos at the Sprint in Lehi in December last year due to having repo-split experience myself. I had known that the Nova-net to Neutron migration work was/is important and had attended the Paris summit session on the issue, which had some people indicate they would drive the work so stepped back believing it was taken care of. Until December when I saw that not enough work had taken place for anything to happen in Kilo. I got involved to support Oleg Bondarev's work and help find more people to provide design direction and feedback. We had a design and got some code written (way to go group) however the feedback from the ops summit was such that it became evident that the current solution even if finished would be insufficient to address the issue. I curtailed our work (with agreement of those at the meeting) in favour of opening a larger discussion on the mailing list. I consider the work those involved put in to be valuable, as it is possible we might not have gotten the level of detail in the feedback we currently have without the code, thank you to all who participated. At present I have agreed to chair the discussion in Vancouver for the session addressing Nova-net and Neutron. I ask that those involved and affected by this work find it in their hearts to bring a positive outlook to this issue. I'm grateful for your support. Last cycle I attended the Neutron, Keystone, Nova and Cinder mid-cycles, to help with third party work, the nova-net to neutron migration as well as helping project devs better understand how infra works and how to maximize efficient use of infra tools such as gerrit as well as how to offer an elastic-recheck fingerprint. I tend to gravitate towards work that needs to get done but which noone else wants to do. I have been operating from the belief that this is for the benefit of OpenStack. I will admit the big tent movement has thrown me off in regards to what is beneficial for OpenStack. Thierry's blog post helps in that regard. I would like to look and work on issues that affect the health of OpenStack long term including our vision of our targeted user. I am an astrologer at heart and as such look at large patterns and cycles of activity as well at their results. OpenStack is in a unique position to redefine software creation as a process that has outcomes that can be negotiated as beneficial for all concerned. The way it does so is by incorporating unlimited freedom with careful evaluation of structure and limits of resources by balancing culture and social responsibility to seeing and respecting both ends of the spectrum in actions. When we do this (and we have been able to) then everyone wins and feels nourished as a result. When one side of OpenStack (the freedom side, for instance) needs to accomplish its goal at the expense of the other side (careful evaluation of structure and limits of resources) then we all lose. It is a powerful energy structure and requires balance. I also served the technical committee as election official for 4 election seasons. I want to thank you co-election officials for your guidance and support in that process, Monty Taylor, Thierry Carrez and Tristan Cacqueray (who is currently serving as an election official). I would also like to acknowledge Elizabeth K. Joseph who has replaced me, thank you to you as well, Elizabeth. Please feel free to ask me any questions if my post has failed to present my perspective and position. I will continue to serve OpenStack to the best of my ability regardless of what position the community chooses to bestow
[openstack-dev] Re: [oslo] eventlet 0.17.3 is now fully Python 3 compatible
On 4/22/15 7:19 AM, Victor Sergeyev wrote: Hi, All, My 2c are: - yes, oslo.db supports python 3 (unittests passes, at least :) ) - MySQL-python still default MySQL DB driver in OpenStack, but at the moment the only DB driver for MySQL in python3 environment is PyMySQL, so I think, it's ok to use it with python 3. the same maintainer of PyMySQL has also ported MySQL-Python to Python 3, and this driver works pretty well also: https://github.com/PyMySQL/mysqlclient-python __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Required data migrations in Kilo, need Turbo Hipster tests updated
On 4/22/15 1:20 AM, Dan Smith wrote: The migrate_flavor_data command didn't actually work on the CLI (unless I'm missing something or did something odd). See https://review.openstack.org/#/c/175890/ where I fix the requirement of max_number. This likely means that operators have not bothered to do or test the migrate_flavor_data yet which could be a concern. Yep, I saw and +2d, thanks. I think it's pretty early in kilo testing and since you don't *have* to do this for kilo, it's not really been an issue yet. Sure, but for people doing continuous deployment, they clearly haven't ran the migrate_flavor_data (or if they have, they haven't filed any bugs about it not working[0]). I also found what I believe to be a bug with the flavor migration code. I started working on a fix by my limited knowledge of nova's objects has hindered me. Any thoughts on the legitimacy of the bug would be helpful though: https://bugs.launchpad.net/nova/+bug/1447132 . Basically for some of the datasets that turbo-hipster uses there are no entries in the new instance_extra table stopping any flavor migration from actually running. Then in your change (174480) you check the metadata table instead of the extras table causing the migration to fail even though migrate_flavor_data thinks there is nothing to do. I think it's worth noting that your change (174480) will require operators (particularly continuous deployers) to run migrate_flavor_data and given the difficulties I've found I'm not sure it's ready to be ran. If we encounter bugs against real datasets with migrate_flavor_data then deployers will be unable to upgrade until migrate_flavor_data is fixed. This may make things awkward if a deployer updates their codebase and can't run the migrations. Clearly they'll have to roll-back the changes. This is the scenario turbo-hipster is meant to catch - migrations that don't work on real datasets. If migrate_flavor_data is broken a backport into Kilo will be needed so that if Liberty requires all the flavor migrations to be finished they can indeed be ran before upgrading to Liberty. This may give reason not to block on having the flavors migrated until the M-release but I realise that has other undersiable consequences (ie high code maintenance). Cheers, Josh [0] I found another one even: https://review.openstack.org/#/c/176172/ That said, I'm surprised migrate_flavor_data isn't just done as a migration. I'm guessing there is a reason it's a separate tool and that it has been discussed before, but if we are going to have a migration enforcing the data to be migrated, then wouldn't it be sensible enough to just do it at that point? The point of this is that it's *not* done as a migration. Doing this transition as a DB migration would require hours of downtime for large deployments while rolling over this migration. Instead, the code in kilo can handle the data being in either place, and converts it on update. The nova-manage command allows background conversion (hence the max-number limit for throttling) to take place while the system is running. Thanks for fixing up nova-manage and for making T-H aware! --Dan __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [neutron][QoS] service-plugin or not discussion
Hi everybody, In the latest QoS meeting, one of the topics was a discussion about how to implement QoS [1] either as in core, or as a service plugin, in, or out-tree. It’s my feeling, and Mathieu’s that it looks more like a core feature, as we’re talking of port properties that we define at high level, and most plugins (QoS capable) may want to implement at dataplane/controlplane level, and also that it’s something requiring a good amount of review. In the other hand Irena and Sean were more concerned about having a good separation of concerns (I agree actually with that part), and being able to do quicker iterations on a separate stackforge repo. Since we didn’t seem to find an agreement, and I’m probably missing some details, I’d like to loop in our core developers and PTL to provide an opinion on this. [1] http://eavesdrop.openstack.org/meetings/neutron_qos/2015/neutron_qos.2015-04-21-14.03.log.html#l-192 Thanks for your patience, Miguel Angel Ajo __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] About Sahara EDP New Ideas for Liberty
Hi Ken, responses inline On Wed, 2015-04-22 at 12:36 +, Chen, Ken wrote: Hi Trevor, I saw below items in Proposed Sprint Topics of sahara liberty. https://etherpad.openstack.org/p/sahara-liberty-proposed-sessions. I guess these are the EDP ideas we want to discuss on Vancouver design summit. We have some comments as below: Yes, feel free to add anything else to the pad. We'll talk about as much as we have time for. I'm thinking that most of it will be covered on Friday, or in between sessions during the week if folks are around. o job scheduler (proposed by weiting) we already have a spec on this, please help review it and give your comments and ideas. https://review.openstack.org/#/c/175719/ Great! thanks o more complex workflows (job dependencies, DAGs, etc. Do we rely on Oozie, or something else? Huichun is now figuring this. I am not whether you guys already have some detail ideas about this? If needed we can contribute some effort. If no details are ready, we can help draw a draft version first. No work on this so far, although we have talked about it off and on for a few cycles. Oozie has a lot of capabilities for coordination, but we are not Oozie-only, so what do we do? This is the central question. o job interface mapping https://blueprints.launchpad.net/sahara/+spec/unified-job-interface-map proposed in Kilo but moved to Liberty ++ high priority in my opinion. Should be done early, awesome feature seems interesting. We agree EDP UI should be improved. In fact we have some unclear thinking about EDP inside our team. Some guys do not like current EDP design, and think it is more like a re-design of oozie or spark UI, instead of a universal interface to users. However, we have not a clear strategy on this part. Yes, Oozie had a heavy influence on EDP. This is partly historical -- EDP was written rapidly between Havanna and Icehouse and based on Oozie since it offered handling of jobs and multiple types out of the box. It was a quick path to EDP functionality. However, EDP should be more of a universal interface. We only support a few conceptual operations -- run job, cancel job, and job status. With those three operations, we should be able to run anything. For example, recently Telles has been working on Storm support. The job interface mapping will help generalize how arguments are passed to jobs and allow us to remove some assumptions about jobs. I am all for other generalizations that will move EDP further in the direction of a general interface. o early error detection to help transient clusters -- how many things can we detect early that can go wrong with an EDP job so that we return an error before spinning up the cluster (only to find that the job fails once the cluster is launched?) Ex, bad swift paths seems easier, but may include some trivial work. Some of this will be folded into the job interface mapping. Ethan has just updated the spec to include input_datasource and output_datasource as argument types. If we know what will be done with a datasource, we can potentially validate it before the job runs. • Spark plugins -- we have an independent Spark plugin, but we also have Spark supported by mapr, and in the future it will be supported by Ambari. Should we continue to carry a simple Spark standalone plugin? Or should we work toward shifting our Spark support to one or more vendor plugins? Not sure what this will impact. Thought about this more. Overlap in the plugins is fine, as long as there is someone in the community willing and able to support it. -Ken __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] StackTach.v3 now in production ...
(sorry for cross-post, but this is appropriate to both audiences) Hey y'all! For those of you that don't know, StackTach is a notification-based debugging, monitoring and usage tool for OpenStack. We're happy to announce that we've recently rolled StackTach.v3 into production at one of the Rax datacenters with a plan to roll out to the rest asap. Once we do, we'll be bumping all the library versions to 1.0, but we encourage you to start playing with the system now. The docs and screencasts are at www.stacktach.com We live on #stacktach on freenode (for v2 and v3 questions) All the StackTach code is on stackforge https://github.com/stackforge?query=stacktach This a very exciting time for us. With StackTach.v3 we've: * solved many of the scaling, redundancy and idempotency problems of v2 * modularized the entire system (use only the parts you want) * made the system less rigid with respect to Nova and Glance. Now, nearly any JSON notification can be handled (even outside of OpenStack) * created a very flexible REST API with pluggable implementation drivers. So, if you don't like our solution but want to keep a compatible API, all the pieces are there for you, including cmdline tools and client libraries. * included a devstack-like sandbox for you to play in that doesn't require an OpenStack installation to generate notifications * developed a way to run STv3 side-by-side with your existing notification consumers for safe trials. We can split notification queues without requiring any changes to your openstack deployment (try *that* with oslo-messaging ;) If you haven't looked at your OpenStack deployment from the perspective of notifications you're really missing out. It's the most powerful way to debug your installations. And, for usage metering, there is really no better option. We feel StackTach.v3 is the best solution out there for all your event-processing needs. Let us know how we can help! We're in a good place to squash bugs quickly. Cheers -Sandy, Dragon and the rest of the StackTach.v3 team/contributors __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] About Sahara EDP New Ideas for Liberty
Hi Trevor, I saw below items in Proposed Sprint Topics of sahara liberty. https://etherpad.openstack.org/p/sahara-liberty-proposed-sessions. I guess these are the EDP ideas we want to discuss on Vancouver design summit. We have some comments as below: • EDP Priorities in Liberty - At the last 2 summits, we've looked at possible work for EDP in the cycle and prioritized it. This is helpful, since there is probably more that could be done than can be done in a single cycle :) o job scheduler (proposed by weiting) we already have a spec on this, please help review it and give your comments and ideas. https://review.openstack.org/#/c/175719/ o more complex workflows (job dependencies, DAGs, etc. Do we rely on Oozie, or something else? Huichun is now figuring this. I am not whether you guys already have some detail ideas about this? If needed we can contribute some effort. If no details are ready, we can help draw a draft version first. o job interface mapping https://blueprints.launchpad.net/sahara/+spec/unified-job-interface-map proposed in Kilo but moved to Liberty ++ high priority in my opinion. Should be done early, awesome feature seems interesting. We agree EDP UI should be improved. In fact we have some unclear thinking about EDP inside our team. Some guys do not like current EDP design, and think it is more like a re-design of oozie or spark UI, instead of a universal interface to users. However, we have not a clear strategy on this part. o early error detection to help transient clusters -- how many things can we detect early that can go wrong with an EDP job so that we return an error before spinning up the cluster (only to find that the job fails once the cluster is launched?) Ex, bad swift paths seems easier, but may include some trivial work. • Spark plugins -- we have an independent Spark plugin, but we also have Spark supported by mapr, and in the future it will be supported by Ambari. Should we continue to carry a simple Spark standalone plugin? Or should we work toward shifting our Spark support to one or more vendor plugins? Not sure what this will impact. -Ken -Original Message- From: Trevor McKay [mailto:tmc...@redhat.com] Sent: Tuesday, March 24, 2015 10:49 PM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] About Sahara EDP New Ideas for Liberty Weiting, Andrew, Agreed, great ideas! As Andrew noted, we have discussed some of these things before and it would be great to discuss them in Vancouver. I think that a Sahara-side workflow manager is the right approach. Oozie has a lot of capability for job coordination, but it won't work for all of our cluster and job types. Notes on Spark in particular -- when we implemented Spark EDP, we looked at various implementations for a Spark job server. One was to extend Oozie, one was to use the Ooyala Spark job server, and one was to use ssh around spark-submit. We chose the last, notes are here: https://etherpad.openstack.org/p/sahara_spark_edp We could potentially revisit the Ooyala job server. My impression at the time was that for the functions we wanted, it was pretty heavy. But if we are going to add job coordination as a general feature, it may be appropriate. I believe in the Spark community it is the dominant solution for job management, open source is here: https://github.com/spark-jobserver/spark-jobserver As part of the Spark investigation, I posted on this JIRA, too. This is a JIRA for developing a REST api to the spark job server, which may be enough for us to build our own coordination system: https://issues.apache.org/jira/browse/SPARK-3644 Best, Trevor On Tue, 2015-03-24 at 01:55 +, Chen, Weiting wrote: Hi Andrew. Thanks for response. My reply in line. From: Andrew Lazarev [mailto:alaza...@mirantis.com] Sent: Saturday, March 21, 2015 12:10 AM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] About Sahara EDP New Ideas for Liberty Hi Weiting, 1. Add a schedule feature to run the jobs on time: This request comes from the customer, they usually run the job in a specific time every day. So it should be great if there is a scheduler to help arrange the regular job to run. Looks like a great feature. And should be quite easy to implement. Feel free to create spec for that. [Weiting] We are working on the spec and the bp has already been registered in https://blueprints.launchpad.net/sahara/+spec/enable-scheduled-edp-jobs. 2. A more complex workflow design in Sahara EDP: Current EDP only provide one job that is running on one cluster. Yes. And ability to run several jobs in one oozie workflow is discussed on every summit (e.g. 'coordinated jobs' at https://etherpad.openstack.org/p/kilo-summit-sahara-edp). But for now it was not
Re: [openstack-dev] [heat][novaclient] heat gate is broken because of new novaclient
On 04/22/2015 07:07 AM, Sean Dague wrote: On 04/22/2015 04:09 AM, Angus Salkeld wrote: Hi Can we please have a new release of novaclient (after the below fix)? Heat's unit tests pass fine with: python-novaclient (2.23.0) but python-novaclient 2.24.0 introduces this bug: https://bugs.launchpad.net/python-novaclient/+bug/1437244 We still need this patch in: https://review.openstack.org/176228 We should also update requirements to exclude 2.24.0 I've got an alternate fix here - https://review.openstack.org/#/c/176252/ I was -1 for a long time on the complex repr stuff in the introducing patch, and I think that's just going to get us into other problems down the road. The repr fall back for Resource is totally fine, and we should just use that. python-novaclient 2.24.1 has been released with https://review.openstack.org/#/c/176252/ in place. Hopefully that fixes things for Heat. -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [mistral] Break_on in Retry policy
So, in this case I guess 'break-on' will work correctly now: https://github.com/stackforge/mistral/blob/master/mistral/engine/policies.py#L295-L296 On Wed, Apr 22, 2015 at 2:58 PM, Renat Akhmerov rakhme...@mirantis.com wrote: Lingxian, yes, that’s basically what I suggest too. Renat Akhmerov @ Mirantis Inc. On 22 Apr 2015, at 16:03, Lingxian Kong anlin.k...@gmail.com wrote: Hi all, In my understanding, the meaning of the 'break-on' in 'retry' policy is just give an oppotunity to end the task execution earlier, i.e. we don't need to wait for all the iterations. I prefer that we keep the ERROR state, and keep it simple. On Wed, Apr 22, 2015 at 5:06 PM, Renat Akhmerov rakhme...@mirantis.com wrote: Ok, after all thinking my suggestion is to leave break-on” but clarify its semantics and behavior a little bit as follow: As Dmitri wrote before “success-on” and “error-on” may be easily confused with “on-success” and “on-error”. “retry policy loop may only stop if: Task action/workflow completed successfully (no need to retry anymore). This case has nothing to do with “break-on”. Task action/workflow failed and some condition (“break-on”) evaluates to True. So in this case I don’t think we need to give a user opportunity to change semantics of task behavior and be able to say “although task action/workflow failed I want this task to finish with success”. IMO, it may be 1) confusing 2) error-prone 3) complicating our understanding of how workflow works. In other works, I’m now against of this behavior and I think that success/error of action/workflow should exactly match to success/error of task. Based on the previous thoughts evaluation of “break-on” condition should work against task inbound context that doesn’t contain a task result itself (because the action failed) but may include results of other tasks completed in parallel branches. The general use case for this is to “stop waiting for something if we see that fundamental requirements for that are not met”. Thoughts? Renat Akhmerov @ Mirantis Inc. On 21 Apr 2015, at 14:11, Nikolay Makhotkin nmakhot...@mirantis.com wrote: Any more suggestions/thoughts here? Dmitri? More words: succeed-on / fail-on, success-expr / error-expr. -- Best Regards, Nikolay __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Regards! --- Lingxian Kong __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Best Regards, Nikolay __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [puppet] [infra] Moving repositories to OpenStack namespace
Hi, Puppet OpenStack is now part of the big tent [1] and we would like to move our repositories from Stackforge to OpenStack namespace. How we are going to proceed * Patch project-config: https://review.openstack.org/#/c/176326/ (WIP for now) * Github: Repositories will be transfered to the other namespace, and redirection will be managed by Github * git.openstack.org will move the repositories but no redirection is possible here, you'll have to change remote urls. * Make a request to openstack-infra about the redirection [2]. Impact == * We need to schedule it because it will involve some Gerrit outage, to make changes in its database and filesystem, and rebuild search indexes. * This will be postponed after the kilo release to not slow-down other projects. * We will have to patch all our modules for this change (.fixtures, metadata.json, beaker, etc...). In theory, that should not break our CI, but that could happen. We will make sure to have some Puppet OpenStack folks around during this time. * If you are not using Github, you'll have to change your remote-url otherwise. @infra: please let me know when we can proceed. @puppet: any feedback/question on this change is welcome. Thanks, [1] https://review.openstack.org/#/c/172112/ [2] https://wiki.openstack.org/wiki/Meetings/InfraTeamMeeting#Upcoming_Project_Renames -- Emilien Macchi signature.asc Description: OpenPGP digital signature __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [puppet] Moving forward with puppet-keystone CI (beaker tests)
Hi, Some important work is being done on Keystone v3 API support in puppet-keystone. We've clearly seen there is a lack of review and I think we all worry about breaking something. Spencer I are working on beaker tests lately and the jobs are non-voting for now. I propose: * to review (and eventually merge) the beaker-tests patches [1] [2] for Keystone openstacklib. * to patch project-config [3] to make vote Beaker jobs in Puppet OpenStack gate for puppet-keystone puppet-openstacklib. Why voting? Because otherwise I'm not sure people will notice the failure and some patches will be merged while beaker is red. So we can have a good set of tests that will help us to detect some issues in the future. I don't think we will catch all mistakes we can do, but this is a good start. To vote this proposal, you can use the gerrit patches and let any feedback. Thanks, [1] puppet-keystone: https://review.openstack.org/#/c/155873/ [2] puppet-openstacklib: https://review.openstack.org/#/c/176098/ [3] project-config: https://review.openstack.org/176343 -- Emilien Macchi signature.asc Description: OpenPGP digital signature __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Required data migrations in Kilo, need Turbo Hipster tests updated
Sure, but for people doing continuous deployment, they clearly haven't ran the migrate_flavor_data (or if they have, they haven't filed any bugs about it not working[0]). Hence the usefulness of T-H here, right? The point of the migration check is to make sure that people _do_ run it before the leave kilo. Right now, they have nothing other than the big note in the release notes about doing it. Evidence seems to show that they're not seeing it, which is exactly why we need the check :) I also found what I believe to be a bug with the flavor migration code. I started working on a fix by my limited knowledge of nova's objects has hindered me. Any thoughts on the legitimacy of the bug would be helpful though: https://bugs.launchpad.net/nova/+bug/1447132 . Basically for some of the datasets that turbo-hipster uses there are no entries in the new instance_extra table stopping any flavor migration from actually running. Then in your change (174480) you check the metadata table instead of the extras table causing the migration to fail even though migrate_flavor_data thinks there is nothing to do. I don't think this has anything to do with the objects, it's probably just a result of my lack of sqlalchemy-fu. Sounds like you weren't close to a fix, so I'll try to cook something up. So, a question here: These data sets were captured at some point in time, right? Does that mean that they were from say Icehouse era and have had nothing done to them since? Meaning, are there data sets that actually had juno or kilo running on them? This instance_extra thing would be the case for any instance that hasn't been touched in a long time, so it's legit. However, as we move to more online migration of data, I do wonder if taking an ancient dataset, doing schema migrations forward to current and then expecting it to work is really reflective of reality. Can you shed some light on what is really going on? I think it's worth noting that your change (174480) will require operators (particularly continuous deployers) to run migrate_flavor_data and given the difficulties I've found I'm not sure it's ready to be ran. Right, that's the point. If we encounter bugs against real datasets with migrate_flavor_data then deployers will be unable to upgrade until migrate_flavor_data is fixed. This may make things awkward if a deployer updates their codebase and can't run the migrations. Clearly they'll have to roll-back the changes. This is the scenario turbo-hipster is meant to catch - migrations that don't work on real datasets. Right, that's why we're holding until we make sure that it works. If migrate_flavor_data is broken a backport into Kilo will be needed so that if Liberty requires all the flavor migrations to be finished they can indeed be ran before upgrading to Liberty. This may give reason not to block on having the flavors migrated until the M-release but I realise that has other undersiable consequences (ie high code maintenance). Backports to fix this are fine IMHO, and just like any other bug found in actual running of things. It's too bad that none of our continuous deployment people seem to have found this yet, but that's a not uncommon occurrence. Obviously if we hit something that makes it too painful to get right in kilo, then we can punt for another cycle. Thanks! --Dan __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [oslo] eventlet 0.17.3 is now fully Python 3 compatible
Hi, All, My 2c are: - yes, oslo.db supports python 3 (unittests passes, at least :) ) - MySQL-python still default MySQL DB driver in OpenStack, but at the moment the only DB driver for MySQL in python3 environment is PyMySQL, so I think, it's ok to use it with python 3. Hi. I'm maintainer of PyMySQL and mysqlclient. mysqlclient is fork of MySQL-python. It uses libmysqlclient.so. It fixes some bugs, build issues and it support Python 3. For example: * Support MariaDB's libmysqlclient.so * Support microsecond in TIME column I recommend to use mysqlclient instead of MySQL-python even on Python 2. https://pypi.python.org/pypi/mysqlclient https://github.com/PyMySQL/mysqlclient-python -- INADA Naoki songofaca...@gmail.com __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [mistral] Break_on in Retry policy
may be not quite - please advice how it works in these cases 1) if break-on expression contains the reference to task result, like break-on: % $.my_task.foo.bar = true % but action returns ERROR and task payload is None (desired behavior: don’t puke, evaluate to false and don’t break) 2) if break-on contains the value from (e.g. published variable, updated by other branch of workflow) - desired behavior - evaluate my_global_flag on every iteration: break-on % $.my_global_flag = true % 3) a combination of the two break-on % $.my_global_counter $.my_task.counter % We need functional tests for all 3 cases (may be unit tests but these cases become difficult to simulate/mock). DZ. On Apr 22, 2015, at 6:55 AM, Nikolay Makhotkin nmakhot...@mirantis.com wrote: So, in this case I guess 'break-on' will work correctly now: https://github.com/stackforge/mistral/blob/master/mistral/engine/policies.py#L295-L296 On Wed, Apr 22, 2015 at 2:58 PM, Renat Akhmerov rakhme...@mirantis.com wrote: Lingxian, yes, that’s basically what I suggest too. Renat Akhmerov @ Mirantis Inc. On 22 Apr 2015, at 16:03, Lingxian Kong anlin.k...@gmail.com wrote: Hi all, In my understanding, the meaning of the 'break-on' in 'retry' policy is just give an oppotunity to end the task execution earlier, i.e. we don't need to wait for all the iterations. I prefer that we keep the ERROR state, and keep it simple. On Wed, Apr 22, 2015 at 5:06 PM, Renat Akhmerov rakhme...@mirantis.com wrote: Ok, after all thinking my suggestion is to leave break-on” but clarify its semantics and behavior a little bit as follow: As Dmitri wrote before “success-on” and “error-on” may be easily confused with “on-success” and “on-error”. “retry policy loop may only stop if: Task action/workflow completed successfully (no need to retry anymore). This case has nothing to do with “break-on”. Task action/workflow failed and some condition (“break-on”) evaluates to True. So in this case I don’t think we need to give a user opportunity to change semantics of task behavior and be able to say “although task action/workflow failed I want this task to finish with success”. IMO, it may be 1) confusing 2) error-prone 3) complicating our understanding of how workflow works. In other works, I’m now against of this behavior and I think that success/error of action/workflow should exactly match to success/error of task. Based on the previous thoughts evaluation of “break-on” condition should work against task inbound context that doesn’t contain a task result itself (because the action failed) but may include results of other tasks completed in parallel branches. The general use case for this is to “stop waiting for something if we see that fundamental requirements for that are not met”. Thoughts? Renat Akhmerov @ Mirantis Inc. On 21 Apr 2015, at 14:11, Nikolay Makhotkin nmakhot...@mirantis.com wrote: Any more suggestions/thoughts here? Dmitri? More words: succeed-on / fail-on, success-expr / error-expr. -- Best Regards, Nikolay __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Regards! --- Lingxian Kong __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Best Regards, Nikolay __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [neutron][lbaas] adding lbaas core
Congratulations Phil! Thank you for your work so far. -Sam. -Original Message- From: Doug Wiegley [mailto:doug...@parksidesoftware.com] Sent: Tuesday, April 21, 2015 7:55 PM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [neutron][lbaas] adding lbaas core It’s been a week, welcome Phil. Thanks, doug On Apr 13, 2015, at 2:39 PM, Doug Wiegley doug...@parksidesoftware.com wrote: Hi all, I'd like to nominate Philip Toohill as a neutron-lbaas core. Good guy, did a bunch of work on the ref impl for lbaasv2, and and I'll let the numbers[1] speak for themselves. Existing lbaas cores, please vote. All three of us. :-) [1] http://stackalytics.com/report/contribution/neutron-lbaas/30 Thanks, doug __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [oslo] eventlet 0.17.3 is now fully Python 3 compatible
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 On 04/22/2015 05:02 PM, INADA Naoki wrote: Hi, All, My 2c are: - yes, oslo.db supports python 3 (unittests passes, at least :) ) - MySQL-python still default MySQL DB driver in OpenStack, but at the moment the only DB driver for MySQL in python3 environment is PyMySQL, so I think, it's ok to use it with python 3. ?Hi.? I'm maintainer of PyMySQL and mysqlclient. mysqlclient is fork of MySQL-python. It uses libmysqlclient.so. It fixes some bugs, build issues and it support Python 3. For example: * Support MariaDB's libmysqlclient.so * Support microsecond in TIME column I recommend to use mysqlclient instead of MySQL-python even on Python 2. https://pypi.python.org/pypi/mysqlclient https://github.com/PyMySQL/mysqlclient-python Is it packaged in popular distributions? RHEL? Fedora? SuSe? Ubuntu? Debian? Gentoo? /Ihar -BEGIN PGP SIGNATURE- Version: GnuPG v2 iQEcBAEBCAAGBQJVN7qEAAoJEC5aWaUY1u5715kH/3YWVEAKqM/KIPGVBnycchZx qGiQlkLuk989WLDvELJ4iwnYaeWfLv3O0RozHOJNdetKbmxJnSJS5BZvX7RUGWrU NomHc8LfGeKyE4M3DAuyJUBeih2/YFOuvq404iPtl7YlvQPyMsoSm6lnmm/YuQlV hlB9erx0P/UPCeeRpWGKIV31L1KMLPgl+Mr7TItsfnmlKDkwtOBijIfw3ECxknqI CzUwMLTwvIL3IRfXWke+FBqzoUIZr/tXXJAaqsfWjjX31AZp0Py8LsW8AXnkbCrN GPH+raU8gkAdpgYMM34dBoxI18Y5xCrLyWXzHRIoTp42dIvzPBE24/UHldNKG7I= =z6qY -END PGP SIGNATURE- __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [puppet] Moving forward with puppet-keystone CI (beaker tests)
Emillen, Do you see shelling out to openstackclient in the keystone test to verify that the keystone resources have been created? Do you see trying to hit the api from something like aviator? Ultimately I'd like to see us spin up an entire openstack in one test then hit it with tempest. It may be possible to use a very narrow version of tempest to validate just keystone. Thanks, Spencer On Wed, Apr 22, 2015 at 7:51 AM, Emilien Macchi emil...@redhat.com wrote: Hi, Some important work is being done on Keystone v3 API support in puppet-keystone. We've clearly seen there is a lack of review and I think we all worry about breaking something. Spencer I are working on beaker tests lately and the jobs are non-voting for now. I propose: * to review (and eventually merge) the beaker-tests patches [1] [2] for Keystone openstacklib. * to patch project-config [3] to make vote Beaker jobs in Puppet OpenStack gate for puppet-keystone puppet-openstacklib. Why voting? Because otherwise I'm not sure people will notice the failure and some patches will be merged while beaker is red. So we can have a good set of tests that will help us to detect some issues in the future. I don't think we will catch all mistakes we can do, but this is a good start. To vote this proposal, you can use the gerrit patches and let any feedback. Thanks, [1] puppet-keystone: https://review.openstack.org/#/c/155873/ [2] puppet-openstacklib: https://review.openstack.org/#/c/176098/ [3] project-config: https://review.openstack.org/176343 -- Emilien Macchi -- To unsubscribe from this group and stop receiving emails from it, send an email to puppet-openstack+unsubscr...@puppetlabs.com. -- Spencer Krum (619)-980-7820 __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Glance] Open glance-drivers to all glance-cores
Hello all, With all due respect to all the Glance core-reviewers (who are doing an excellent job, by the way), please NO. First reaction that came to my mind after reading the title: What might be the thinking behind this, what is the direction this is driving the project towards, what’s next, open Glance core reviewers to all committers? I think not. The prime reason:- === Glance “drivers” is a role and very much like any other role, it revolves around responsibilities. The authority aspect of this role is a side-effect and a privilege given to help perform these responsibilities effectively. Similarly, Glance core-reviewers more commonly called as Glance cores is another responsibility. It revolves around managing the code that is proposed to be merged to the project code bases. So, how can something that’s approved by the drivers and not approved by the core reviewers get into the project? Although, the role played and the authority imposed may be different by both these groups, however that effect is observed by the community on the code proposed. The specs are open to the community and have set expectations for providing the details around subtle aspects like Deployer Impact, Security Impact, Developer Impact, etc. So, all of these groups can point out to the author if the respective expectations aren’t met. And the timely provided feedback will have to be weighed in by the drivers. As the name of the role suggests, these people are expected to get things resolved and help drive the priorities of the program. How were the priorities set? = Well, this is very well known; during the Summits, mini summits, various meetings, mailing list discussions, etc. What are the factors a driver must look into while providing feedback? = We are contributing to a Foundation that supports Open Source software. We promote Open Community discussions. Besides these important considerations, a thorough guideline for providing feedback is documented at [1]. How do they help drive the program? *They provide feedback that help support the important paradigms of (open but in general) software evolution: Supportability, Maintainability, Elasticity, Scalability, Stability, etc. * They are proactive in providing feedback on different fronts: design patterns, OpenStack coherence, cross-project interactions, developer perspectives, operator perspectives, security perspective, testing, dependency, use-cases, adoptability that can include subset of user research, market research, competition research, interoperability etc * They help prioritize the code that is planned to be reviewed in a cycle and sometimes take ownership of a spec to see it though by discussing with different groups, reviewers, cross project liaisons, meetings within and outside of the project. * More importantly they provide timely feedback of the specs that have been prioritized in the beginning of cycle and attempt to provide prudent feedback on other specs. While I see that some of the core reviewers help the project in many of the above aspects and are good candidates for drivers, being a driver is an added responsibility. We should make every effort to set the right expectations on the same and encourage great developers become core-reviewers without being bogged down by this added burden. Hence, we have a clear separation of concern and do not have a strong dependency on either of the responsibility. About the ability to scale and the ACLs on the specs: === I have to agree that our feedback time and thoroughness has lacked severely for the past few cycles. However, we must note that the community is growing and sometimes we need to bear through the transition phase. We have had a mission statement change and a few related features are still spunkily trying to get merged. I am glad that you brought up the feedback time on the specs, as this also applies to the feedback time on feature-code. For example, Artifacts reviews did not get much attention from the existing set of core-reviewers. How do we solve that first? If we are going to scale drivers, we first need to commit to be able to review features that are earlier promised to land. Adding more features that come later on the priority list of the program with the help of a bigger driver team and not providing early feedback to top priority reviews doesn’t make much sense. Clarity and transparency: = Historically, the feedback has primarily been given at the summits and at mini-summits. Any strong objections have been sincerely discussed and I’ve been part of most of them over the last few years. So, personally I did not have issues around clarity and transparency of the feedback. I have seen any features that needed feedback from variety of groups have been discussed at
Re: [openstack-dev] [Openstack-operators] [Neutron] The specs process, effective operators feedback and product management
The rally process (email based) doesn’t seem scalable enough for the neutron case IMHO, but I agree that a voting system doesn’t differ too much from launchpad and that it can be gamed. On 22/4/2015, at 1:21, Assaf Muller amul...@redhat.com wrote: Just to offer some closure, it seems like the voting idea was shot down with the energy of a trillion stars, yet the general idea of offering an easy way for users to request features makes sense. Expect to see ideas of how to implement this soon... - Original Message - On Apr 10, 2015, at 11:04 AM, Boris Pavlovic bo...@pavlovic.me wrote: Hi, I believe that specs are too detailed and too dev oriented for managers, operators and devops. They actually don't want/have time to write/read all the stuff in specs and that's why the communication between dev operators community is a broken. I would recommend to think about simpler approaches like making mechanism for proposing features/changes in projects. Like we have in Rally: https://rally.readthedocs.org/en/latest/feature_requests.html This is similar to specs but more concentrate on WHAT rather than HOW. +1 I think the line between HOW and WHAT are too often blurred in Neutron. Unless we’re able to improve our ability to communicate at an appropriate level of abstraction with non-dev stakeholders, meeting their needs will continue to be a struggle. Maru __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev Miguel Angel Ajo __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [neutron][lbaas] adding lbaas core
Congrats Phil. Santosh On Wed, Apr 22, 2015 at 9:38 AM, Vijay Venkatachalam vijay.venkatacha...@citrix.com wrote: Congratulations Phil! -Original Message- From: Tom Creighton [mailto:tom.creigh...@rackspace.com] Sent: Wednesday, 22 April 2015 12:14 AM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [neutron][lbaas] adding lbaas core Congratulations Phil! On Apr 21, 2015, at 11:54 AM, Doug Wiegley doug...@parksidesoftware.com wrote: It’s been a week, welcome Phil. Thanks, doug On Apr 13, 2015, at 2:39 PM, Doug Wiegley doug...@parksidesoftware.com wrote: Hi all, I'd like to nominate Philip Toohill as a neutron-lbaas core. Good guy, did a bunch of work on the ref impl for lbaasv2, and and I'll let the numbers[1] speak for themselves. Existing lbaas cores, please vote. All three of us. :-) [1] http://stackalytics.com/report/contribution/neutron-lbaas/30 Thanks, doug __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Santosh __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [neutron] [docs] Networking Guide Doc Day - April 23rd
Dear Ruijig, As Sean already mentioned, those are not private branches. Let me explain you, in order to facilitate the collaboration from new authors on this new guide, we decided to use these git repos with .rst format. So, part of the work needed for the networking guide during this doc day is to move the content from this repos to the openstack-manuals repo under the networking section. Again, here an example: https://review.openstack.org/#/c/174693/ Thanks, Edgar From: Guo, Ruijing ruijing@intel.commailto:ruijing@intel.com Reply-To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.orgmailto:openstack-dev@lists.openstack.org Date: Tuesday, April 21, 2015 at 5:38 PM To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.orgmailto:openstack-dev@lists.openstack.org Subject: Re: [openstack-dev] [neutron] [docs] Networking Guide Doc Day - April 23rd Hi, Edgar, The following documents are still in private branch: Scenario 1a - Legacy Content - Ready for conversion to RSThttps://github.com/ionosphere80/openstack-networking-guide/blob/master/scenario-legacy-ovs/scenario-legacy-ovs.md Scenario 1b - Legacy Content - Ready for conversion to RSThttps://github.com/ionosphere80/openstack-networking-guide/blob/master/scenario-legacy-lb/scenario-legacy-lb.md Scenario 2 - High availability (DVR and Open vSwitch) Content - Ready for conversion to RSThttps://github.com/ionosphere80/openstack-networking-guide/blob/master/scenario-dvr/scenario-dvr.md Scenario 3b - High availability (L3 HA and Linux Bridge) Content - Work in Progresshttps://github.com/phil-hopkins-a/openstack-networking-guide Thanks, -Ruijing From: Edgar Magana [mailto:edgar.mag...@workday.com] Sent: Tuesday, April 21, 2015 10:13 PM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [neutron] [docs] Networking Guide Doc Day - April 23rd Hello, Which Docs are you referring? Please, point me to the specific Docs in private branches to find out the process to move them to public. Thanks, Edgar From: Guo, Ruijing ruijing@intel.commailto:ruijing@intel.com Reply-To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.orgmailto:openstack-dev@lists.openstack.org Date: Monday, April 20, 2015 at 9:52 PM To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.orgmailto:openstack-dev@lists.openstack.org Subject: Re: [openstack-dev] [neutron] [docs] Networking Guide Doc Day - April 23rd Hi, Edgar Some docs are still in private branch. Can we move to public branch for reviewing submitting patches? Thanks, -Ruijing From: Edgar Magana [mailto:edgar.mag...@workday.com] Sent: Saturday, April 18, 2015 2:25 AM To: OpenStack Development Mailing List (not for usage questions) Subject: [openstack-dev] [neutron] [docs] Networking Guide Doc Day - April 23rd Hello Folks, I would like to invite all available contributors to help us to complete the OpenStack Networking Guide. We are having a Networking Doc Day on April 23rd in order to review the current guide and make a big push on its content. Let's use both the Neutron and Docs IRC channels: #openstack-neutron #openstack-doc All the expected content is being described in the TOC: https://wiki.openstack.org/wiki/NetworkingGuide/TOC Information for Doc contributors in here: https://wiki.openstack.org/wiki/Documentation/HowTo#Edit_OpenStack_RST_and.2For_DocBook_documentation We have prepared an etherpad to coordinate who is doing what: https://etherpad.openstack.org/p/networking-guide There are so many ways you can contribute: * Assign to yourself one of the available chapters * Review the current content and open bugs if needed * Review the existing gerrit commit if you are familiar with the information * Be available on IRC to answer some questions about configuration details or functionality * Cheering at the contributors! Do not hesitate in contacting me for any questions. Cheers! Edgar Magana IRC: emagana emag...@gmail.commailto:emag...@gmail.com edgar.mag...@workday.commailto:edgar.mag...@workday.com __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Add config option for real deletes insteadof soft-deletes
Hi, Artom I checked my cluster (20 compute nodes fully operated for 5 month, with 258 VMs and 112 users), the datebase size of nova only 1.5MB. So, is it necessary to do the cleanup? -- Luo gangyiluogan...@chinamobile.com -- Original -- From: Artom Lifshitz;alifs...@redhat.com; Date: Wed, Apr 22, 2015 05:42 AM To: openstack-devopenstack-dev@lists.openstack.org; Subject: [openstack-dev] [Nova] Add config option for real deletes insteadof soft-deletes Hello, I'd like to gauge acceptance of introducing a feature that would give operators a config option to perform real database deletes instead of soft deletes. There's definitely a need for *something* that cleans up the database. There have been a few attempts at a DB purge engine [1][2][3][4][5], and archiving to shadow tables has been merged [6] (though that currently has some issues [7]). DB archiving notwithstanding, the general response to operators when they mention the database becoming too big seems to be DIY cleanup. I would like to propose a different approach: add a config option that turns soft-deletes into real deletes, and start telling operators if you turn this on, it's DIY backups. Would something like that be acceptable and feasible? I'm ready to put in the work to implement this, however searching the mailing list indicates that it would be somewhere between non trivial and impossible [8]. Before I start, I would like some confidence that it's closer to the former than the latter :) Cheers! [1] https://blueprints.launchpad.net/nova/+spec/db-purge-engine [2] https://blueprints.launchpad.net/nova/+spec/db-purge2 [3] https://blueprints.launchpad.net/nova/+spec/remove-db-archiving [4] https://blueprints.launchpad.net/nova/+spec/database-purge [5] https://blueprints.launchpad.net/nova/+spec/db-archiving [6] https://review.openstack.org/#/c/18493/ [7] https://review.openstack.org/#/c/109201/ [8] http://lists.openstack.org/pipermail/openstack-operators/2014-November/005591.html __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev__ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [neutron-lbaas] [third-party] trying to set up 3rd party CI, neutron-lbaas tox fails to import some neutron
At first glance it seems like you're trying to run these tests with a
neutron repo which is not up to date.
Recently Neutron unit tests were reorganized [1]. Have you tried pulling
again from git the neutron repo?
Salvatore
[1] https://review.openstack.org/#/c/158811/
On 22 April 2015 at 19:38, Lenny Verkhovsky len...@mellanox.com wrote:
Hi,
We had some issues with tox lately,
The fix was removing ~/.pip and some other packages from this folder that
were used as cache for pip
And reinstalling devstack.
*Lenny Verkhovsky*
*From:* Shane McGough [mailto:smcgo...@kemptechnologies.com]
*Sent:* Wednesday, April 22, 2015 1:30 PM
*To:* OpenStack Development Mailing List (not for usage questions)
*Subject:* [openstack-dev] [neutron-lbaas] [third-party] trying to set up
3rd party CI, neutron-lbaas tox fails to import some neutron
Hi all
I am having trouble running tox tests on neutron-lbaas on a default clone.
I can see from the tox logs that it downloads the neutron egg just fine,
however, when running some of the tests it gets import errors when trying
to import from the neutron side of things.
I checked the neutron repo and it does indeed seem like the files its
trying to import do not exist within the neutron repo tox downloads. Some
neutron files do successfully import apparently but majority are
referencing files that do not exist in the location its referencing.
Am I missing something fundamental here?
I included some of the errors below just to give an idea of what fails.
Any help would be appreciated
I am using Ubuntu Server 14.04.2 LTS
Thanks
Shane
py27 runtests: PYTHONHASHSEED='0'
py27 runtests: commands[0] | sh tools/pretty_tox.sh
running testr
Non-zero exit code (2) from test listing.
error: testr failed (3)
running=OS_STDOUT_CAPTURE=${OS_STDOUT_CAPTURE:-1}
OS_STDERR_CAPTURE=${OS_STDERR_CAPTURE:-1}
OS_LOG_CAPTURE=${OS_LOG_CAPTURE:-1} ${PYTHON:-python} -m subunit.run
discover -t ./ ${OS_TEST_PATH:-./neutron_lbaas/tests/unit} --list
--- import errors ---
Failed to import test module: neutron_lbaas.tests.unit.agent.test_agent
Traceback (most recent call last):
File
/home/shane/work/neutron-lbaas/.tox/py27/lib/python2.7/site-packages/unittest2/loader.py,
line 445, in _find_test_path
module = self._get_module_from_name(name)
File
/home/shane/work/neutron-lbaas/.tox/py27/lib/python2.7/site-packages/unittest2/loader.py,
line 384, in _get_module_from_name
__import__(name)
File neutron_lbaas/tests/unit/agent/test_agent.py, line 21, in module
from neutron_lbaas.tests import base
File neutron_lbaas/tests/base.py, line 18, in module
from neutron.tests.unit.db import test_db_base_plugin_v2
ImportError: cannot import name test_db_base_plugin_v2
Failed to import test module: neutron_lbaas.tests.unit.agent.test_agent_api
Traceback (most recent call last):
File
/home/shane/work/neutron-lbaas/.tox/py27/lib/python2.7/site-packages/unittest2/loader.py,
line 445, in _find_test_path
module = self._get_module_from_name(name)
File
/home/shane/work/neutron-lbaas/.tox/py27/lib/python2.7/site-packages/unittest2/loader.py,
line 384, in _get_module_from_name
__import__(name)
File neutron_lbaas/tests/unit/agent/test_agent_api.py, line 21, in
module
from neutron_lbaas.tests import base
File neutron_lbaas/tests/base.py, line 18, in module
from neutron.tests.unit.db import test_db_base_plugin_v2
ImportError: cannot import name test_db_base_plugin_v2
Failed to import test module:
neutron_lbaas.tests.unit.agent.test_agent_manager
Traceback (most recent call last):
File
/home/shane/work/neutron-lbaas/.tox/py27/lib/python2.7/site-packages/unittest2/loader.py,
line 445, in _find_test_path
module = self._get_module_from_name(name)
File
/home/shane/work/neutron-lbaas/.tox/py27/lib/python2.7/site-packages/unittest2/loader.py,
line 384, in _get_module_from_name
__import__(name)
File neutron_lbaas/tests/unit/agent/test_agent_manager.py, line 24, in
module
from neutron_lbaas.tests import base
File neutron_lbaas/tests/base.py, line 18, in module
from neutron.tests.unit.db import test_db_base_plugin_v2
ImportError: cannot import name test_db_base_plugin_v2
Failed to import test module:
neutron_lbaas.tests.unit.common.cert_manager.test_barbican
Traceback (most recent call last):
File
/home/shane/work/neutron-lbaas/.tox/py27/lib/python2.7/site-packages/unittest2/loader.py,
line 445, in _find_test_path
module = self._get_module_from_name(name)
File
/home/shane/work/neutron-lbaas/.tox/py27/lib/python2.7/site-packages/unittest2/loader.py,
line 384, in _get_module_from_name
__import__(name)
File neutron_lbaas/tests/unit/common/cert_manager/test_barbican.py,
line 26, in module
from neutron_lbaas.tests import base
File neutron_lbaas/tests/base.py, line 18, in module
from neutron.tests.unit.db import
Re: [openstack-dev] [neutron-lbaas] [third-party] trying to set up 3rd party CI, neutron-lbaas tox fails to import some neutron
Hi,
We had some issues with tox lately,
The fix was removing ~/.pip and some other packages from this folder that were
used as cache for pip
And reinstalling devstack.
Lenny Verkhovsky
From: Shane McGough [mailto:smcgo...@kemptechnologies.com]
Sent: Wednesday, April 22, 2015 1:30 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: [openstack-dev] [neutron-lbaas] [third-party] trying to set up 3rd
party CI, neutron-lbaas tox fails to import some neutron
Hi all
I am having trouble running tox tests on neutron-lbaas on a default clone. I
can see from the tox logs that it downloads the neutron egg just fine, however,
when running some of the tests it gets import errors when trying to import from
the neutron side of things.
I checked the neutron repo and it does indeed seem like the files its trying to
import do not exist within the neutron repo tox downloads. Some neutron files
do successfully import apparently but majority are referencing files that do
not exist in the location its referencing.
Am I missing something fundamental here?
I included some of the errors below just to give an idea of what fails.
Any help would be appreciated
I am using Ubuntu Server 14.04.2 LTS
Thanks
Shane
py27 runtests: PYTHONHASHSEED='0'
py27 runtests: commands[0] | sh tools/pretty_tox.sh
running testr
Non-zero exit code (2) from test listing.
error: testr failed (3)
running=OS_STDOUT_CAPTURE=${OS_STDOUT_CAPTURE:-1}
OS_STDERR_CAPTURE=${OS_STDERR_CAPTURE:-1} OS_LOG_CAPTURE=${OS_LOG_CAPTURE:-1}
${PYTHON:-python} -m subunit.run discover -t ./
${OS_TEST_PATH:-./neutron_lbaas/tests/unit} --list
--- import errors ---
Failed to import test module: neutron_lbaas.tests.unit.agent.test_agent
Traceback (most recent call last):
File
/home/shane/work/neutron-lbaas/.tox/py27/lib/python2.7/site-packages/unittest2/loader.py,
line 445, in _find_test_path
module = self._get_module_from_name(name)
File
/home/shane/work/neutron-lbaas/.tox/py27/lib/python2.7/site-packages/unittest2/loader.py,
line 384, in _get_module_from_name
__import__(name)
File neutron_lbaas/tests/unit/agent/test_agent.py, line 21, in module
from neutron_lbaas.tests import base
File neutron_lbaas/tests/base.py, line 18, in module
from neutron.tests.unit.db import test_db_base_plugin_v2
ImportError: cannot import name test_db_base_plugin_v2
Failed to import test module: neutron_lbaas.tests.unit.agent.test_agent_api
Traceback (most recent call last):
File
/home/shane/work/neutron-lbaas/.tox/py27/lib/python2.7/site-packages/unittest2/loader.py,
line 445, in _find_test_path
module = self._get_module_from_name(name)
File
/home/shane/work/neutron-lbaas/.tox/py27/lib/python2.7/site-packages/unittest2/loader.py,
line 384, in _get_module_from_name
__import__(name)
File neutron_lbaas/tests/unit/agent/test_agent_api.py, line 21, in module
from neutron_lbaas.tests import base
File neutron_lbaas/tests/base.py, line 18, in module
from neutron.tests.unit.db import test_db_base_plugin_v2
ImportError: cannot import name test_db_base_plugin_v2
Failed to import test module: neutron_lbaas.tests.unit.agent.test_agent_manager
Traceback (most recent call last):
File
/home/shane/work/neutron-lbaas/.tox/py27/lib/python2.7/site-packages/unittest2/loader.py,
line 445, in _find_test_path
module = self._get_module_from_name(name)
File
/home/shane/work/neutron-lbaas/.tox/py27/lib/python2.7/site-packages/unittest2/loader.py,
line 384, in _get_module_from_name
__import__(name)
File neutron_lbaas/tests/unit/agent/test_agent_manager.py, line 24, in
module
from neutron_lbaas.tests import base
File neutron_lbaas/tests/base.py, line 18, in module
from neutron.tests.unit.db import test_db_base_plugin_v2
ImportError: cannot import name test_db_base_plugin_v2
Failed to import test module:
neutron_lbaas.tests.unit.common.cert_manager.test_barbican
Traceback (most recent call last):
File
/home/shane/work/neutron-lbaas/.tox/py27/lib/python2.7/site-packages/unittest2/loader.py,
line 445, in _find_test_path
module = self._get_module_from_name(name)
File
/home/shane/work/neutron-lbaas/.tox/py27/lib/python2.7/site-packages/unittest2/loader.py,
line 384, in _get_module_from_name
__import__(name)
File neutron_lbaas/tests/unit/common/cert_manager/test_barbican.py, line
26, in module
from neutron_lbaas.tests import base
File neutron_lbaas/tests/base.py, line 18, in module
from neutron.tests.unit.db import test_db_base_plugin_v2
ImportError: cannot import name test_db_base_plugin_v2
Failed to import test module:
neutron_lbaas.tests.unit.common.cert_manager.test_local
Traceback (most recent call last):
File
/home/shane/work/neutron-lbaas/.tox/py27/lib/python2.7/site-packages/unittest2/loader.py,
line 445, in _find_test_path
module = self._get_module_from_name(name)
File
Re: [openstack-dev] [puppet] Moving forward with puppet-keystone CI (beaker tests)
On 04/22/2015 11:53 AM, Spencer Krum wrote: Emillen, Do you see shelling out to openstackclient in the keystone test to verify that the keystone resources have been created? Do you see trying to hit the api from something like aviator? Ultimately I'd like to see us spin up an entire openstack in one test then hit it with tempest. * shell testing: yes because it's the way we wrote our providers. * aviator: no because we gave up some months ago in favor of using openstackclient. I'm not in favor of using aviator which would add yet another dependency and complexity. Maybe I'm wrong though. * tempest: well... tempest aims to test API features while we only want to check Keystone is actually running. I think serverspec + some shelling could help. Having tempest is (to me) overkill and could slow down our CI if something's wrong in Tempest. It may be possible to use a very narrow version of tempest to validate just keystone. Like, only a small set of tests that we would run with testr? Thanks, Spencer On Wed, Apr 22, 2015 at 7:51 AM, Emilien Macchi emil...@redhat.com mailto:emil...@redhat.com wrote: Hi, Some important work is being done on Keystone v3 API support in puppet-keystone. We've clearly seen there is a lack of review and I think we all worry about breaking something. Spencer I are working on beaker tests lately and the jobs are non-voting for now. I propose: * to review (and eventually merge) the beaker-tests patches [1] [2] for Keystone openstacklib. * to patch project-config [3] to make vote Beaker jobs in Puppet OpenStack gate for puppet-keystone puppet-openstacklib. Why voting? Because otherwise I'm not sure people will notice the failure and some patches will be merged while beaker is red. So we can have a good set of tests that will help us to detect some issues in the future. I don't think we will catch all mistakes we can do, but this is a good start. To vote this proposal, you can use the gerrit patches and let any feedback. Thanks, [1] puppet-keystone: https://review.openstack.org/#/c/155873/ [2] puppet-openstacklib: https://review.openstack.org/#/c/176098/ [3] project-config: https://review.openstack.org/176343 -- Emilien Macchi -- To unsubscribe from this group and stop receiving emails from it, send an email to puppet-openstack+unsubscr...@puppetlabs.com mailto:puppet-openstack%2bunsubscr...@puppetlabs.com. -- Spencer Krum (619)-980-7820 -- To unsubscribe from this group and stop receiving emails from it, send an email to puppet-openstack+unsubscr...@puppetlabs.com mailto:puppet-openstack+unsubscr...@puppetlabs.com. -- Emilien Macchi signature.asc Description: OpenPGP digital signature __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] TC candidacy
confirmed On Wed, Apr 22, 2015 at 9:46 AM, Anita Kuno ante...@anteaya.info wrote: Please consider my candidacy for membership on the technical committee. My name is Anita Kuno. I consider my home project to be Infra but I tend to move around wherever I feel the need is greatest. I have worked on OpenStack since mid-grizzly starting as an intern with the GNOME Outreach Program for Women (which is now known as Outreachy) with my mentor Iccha Sethi. I moved around until I found Infra and have considered that my home base ever since, mostly because Infra, and my job, allows me so much flexibility. During the Hong Kong summit I launched myself into Neutron to see if there was anything I could do to support improvement, not because I knew anything about Neutron but because I live by the axiom don't ask anyone to do anything you wouldn't be prepared to do yourself. I realized that Neutron developers couldn't even find each other to talk to each other due to the crowd and organized a Neutron Tempest code sprint in Montreal in January 2014. Coming out of that, I have been involved with the third party ci space ever since, a difficult and demanding space and those involved with have many opinions on whether I have done and am doing a good job or a poor job. I split the project-config repo out of what was the Infra config repo (and is now system-config) at the end of Juno and have been a core reviewer on the project ever since. I was able to help Neutron split out their 'as a service' repos at the Sprint in Lehi in December last year due to having repo-split experience myself. I had known that the Nova-net to Neutron migration work was/is important and had attended the Paris summit session on the issue, which had some people indicate they would drive the work so stepped back believing it was taken care of. Until December when I saw that not enough work had taken place for anything to happen in Kilo. I got involved to support Oleg Bondarev's work and help find more people to provide design direction and feedback. We had a design and got some code written (way to go group) however the feedback from the ops summit was such that it became evident that the current solution even if finished would be insufficient to address the issue. I curtailed our work (with agreement of those at the meeting) in favour of opening a larger discussion on the mailing list. I consider the work those involved put in to be valuable, as it is possible we might not have gotten the level of detail in the feedback we currently have without the code, thank you to all who participated. At present I have agreed to chair the discussion in Vancouver for the session addressing Nova-net and Neutron. I ask that those involved and affected by this work find it in their hearts to bring a positive outlook to this issue. I'm grateful for your support. Last cycle I attended the Neutron, Keystone, Nova and Cinder mid-cycles, to help with third party work, the nova-net to neutron migration as well as helping project devs better understand how infra works and how to maximize efficient use of infra tools such as gerrit as well as how to offer an elastic-recheck fingerprint. I tend to gravitate towards work that needs to get done but which noone else wants to do. I have been operating from the belief that this is for the benefit of OpenStack. I will admit the big tent movement has thrown me off in regards to what is beneficial for OpenStack. Thierry's blog post helps in that regard. I would like to look and work on issues that affect the health of OpenStack long term including our vision of our targeted user. I am an astrologer at heart and as such look at large patterns and cycles of activity as well at their results. OpenStack is in a unique position to redefine software creation as a process that has outcomes that can be negotiated as beneficial for all concerned. The way it does so is by incorporating unlimited freedom with careful evaluation of structure and limits of resources by balancing culture and social responsibility to seeing and respecting both ends of the spectrum in actions. When we do this (and we have been able to) then everyone wins and feels nourished as a result. When one side of OpenStack (the freedom side, for instance) needs to accomplish its goal at the expense of the other side (careful evaluation of structure and limits of resources) then we all lose. It is a powerful energy structure and requires balance. I also served the technical committee as election official for 4 election seasons. I want to thank you co-election officials for your guidance and support in that process, Monty Taylor, Thierry Carrez and Tristan Cacqueray (who is currently serving as an election official). I would also like to acknowledge Elizabeth K. Joseph who has replaced me, thank you to you as well, Elizabeth. Please feel free to ask me any questions if my post has failed to
Re: [openstack-dev] TC candidacy
confirmed On Wed, Apr 22, 2015 at 9:43 AM, Dean Troyer dtro...@gmail.com wrote: Hello OpenStack world! My name is Dean Troyer and I am running for the OpenStack Technical Committee. I have been part of the OpenStack community for a long time and heavily involved in three projects: I was an early contributor to DevStack and served as its PTL during its short tenure as a stand-alone program, I started Grenade to use DevStack as the basis for upgrade testing. I also am the PTL of the OpenStackClient project that recently became the first project brought into the expanded official project definition. The common thread of my work in OpenStack has been with things that cross the traditional vertical service projects. This has highlighted the scope and consequences of differences between projects for me. Differences that affect project developers is one thing, and sometimes a good thing even, but differences that adversely affect deployers and consumers of OpenStack clouds is another thing altogether. I believe this is where the focus of encouraging projects to converge should be placed. Efforts like the API Working Group and the Log Working Group need to be encouraged where they improve the experiences of our customers (where 'our' == 'OpenStack' and 'customers' == 'everyone downstream from OpenStack'). I believe the TC has a good bit of work ahead to follow through implementing the vision of inclusiveness. Communicating the state of projects is a huge part of this. We should set up the goals and see who can score. Those projects that do score will be the ones rewarded not by the TC but by the rest of the community. One specific example that I think the TC should facilitate is describing the technical relationships between projects. The TC should bless, if not create, a common vocabulary to describe the groups of projects and their relationships. While this may be expressed in 'tags', it is more than just tag definitions. The Technical Committee is unique in that in some way it touches every aspect of OpenStack: projects and project developers, application developers, distributors and VARs, cloud deployers and operators, end users, and the OpenStack Board of Directors (DefCore!). The TC is the duct tape of OpenStack! No, it is better than that, it is by design the group that oversees enabling everything else to succeed. I believe that TC members must have a broad vision and insight into many parts of OpenStack and depth into at least a couple of areas. And I believe that I have both of those attributes. I would appreciate your consideration and vote. As I mentioned above, I have been part of the OpenStack community since before there was one, working on the original Nova deployment at NASA. That was followed by a stint at Rackspace where DevStack and OSC were born, and on to Nebula where we tossed Grenade into the world. After the recent events at Nebula I will be joining Intel soon and again be focused on upstream OpenStack projects. Thanks again for your time and consideration dt -- Dean Troyer dtro...@gmail.com __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Elizabeth Krumbach Joseph || Lyz || pleia2 __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] TC Candidacy
confirmed On Wed, Apr 22, 2015 at 5:24 AM, Ed Leafe e...@leafe.com wrote: Greetings Stackers! I'm announcing my candidacy for the Technical Comitee Elections. Those of you who have been around OpenStack for a while know me, as I was one of the original developers involved since the inception of OpenStack back in 2010, when I worked for Rackspace's Cloud Server team. I was a contributor and core reviewer for Nova until a job change at Rackspace removed me from direct OpenStack development around the Essex release. I was still peripherally involved, though, as my work as a Developer Advocate involved creating the first Python SDK for OpenStack [0], which gave me a great education on how frustrating inconsistent APIs can be! In order to help improve the experience of developers building apps with OpenStack, I wholeheartedly support the efforts of the API Working Group to reduce this problem in the future. [0] https://github.com/rackspace/pyrax Fast-forward to last September, when I joined IBM with one mandate: work full-time on OpenStack by contributing as much as possible to upstream OpenStack, and becoming involved in the community. Since then I've re-immersed myself in Nova, focusing mainly on the effort to clean up the Scheduler interfaces. So I believe that this history gives me a unique perspective on OpenStack development: where it came from, and where it needs to go to move forward. I mentioned improving the experience of developers working *with* OpenStack; I'd also like to improve the experience of developers working *on* OpenStack. To that end, I agree wholeheartedly with Thierry's call to step out of the way [1]. There is a lot of energy across the many projects, and the TC should do everything it can to help make that energy effective without stifling it. I spoke about the potential for OpenStack to change the world in an interview I gave for Rackspace last year [2], and the last thing I would want to see is someone with an idea get discouraged because there were too many hoops to jump through to make it happen on OpenStack. [1] http://ttx.re/stepping-out-of-the-way.html [2] https://youtu.be/0QRkzMW3OdA?t=115 One place, though, where I think the TC can inject itself is to help alleviate the lack of core reviewers, particularly in Nova, which I discussed in [3]. Having so few cores for the number of patches being created is simply not sustainable, and just wishing for things to get better isn't cutting it. And I do consider it a technical problem, since the main barrier isn't lack of interested people; it's that it's currently too difficult for most people to learn enough about all the various parts of the project necessary to achieve core status. [3] http://blog.leafe.com/the-core-deficiency I sort of wish that there was the questionnaire format like we had in the last TC election, so I could be sure to give everyone a clear view on where I see things on different issues. If you have any such concerns, please feel free to respond to this email (and to those of the other candidates!), and I will be happy to answer any questions. Thanks for your consideration, -- Ed Leafe __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Elizabeth Krumbach Joseph || Lyz || pleia2 __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Manila] Two nominations for Manila Core Reviewer Team
+1 Clinton Knight From: Ben Swartzlander b...@swartzlander.orgmailto:b...@swartzlander.org Reply-To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.orgmailto:openstack-dev@lists.openstack.org Date: Wednesday, April 22, 2015 at 2:23 PM To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.orgmailto:openstack-dev@lists.openstack.org Subject: [openstack-dev] [Manila] Two nominations for Manila Core Reviewer Team I would like to nominate Thomas Bechtold to join the Manila core reviewer team. Thomas has been contributing to Manila for close to 6 months and has provided a good number of quality code reviews in addition to a substantial amount of contributions. Thomas brings both Oslo experience as well as a packager/distro perspective which is especially helpful as Manila starts to get used in more production scenarios. I would also like to nominate Mark Sturdevant. He has also been active in the community for about 6 months and has a similar history of code reviews. Mark is the maintainer of the HP driver and would add vendor diversity to the core team. -Ben Swartzlander Manila PTL __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [horizon] Angular in Liberty
Hi everyone! In Kilo we made some very nice progress with adopting AngularJS in Horizon. The launch instance workflow is available as a beta level feature that can be optionally enabled now in Kilo [1]. Together with the identity panel, detail pages, and magic search workstreams we were able to encounter a well rounded set of challenges. These have helped us to start establishing a number of base components and concepts. Of course, there is more work to be done. In Liberty we want to solidify the components, concepts, and patterns to give us a solid foundation for mainstream AngularJS development across Horizon. There is plenty of work to go around and we would like to enable everybody to be able to contribute. Based on the many discussions and learnings that we had in the Kilo angular scrums, I¹ve started the following etherpad to help organize some thoughts going into Liberty. I hope this also provides some insight for those that haven¹t been able to attend the virtual scrums during Kilo. Thanks to all the participants of the angular scrums for the input you have already had on the ether pad. https://etherpad.openstack.org/p/horizon-liberty-angular Finally, I will send out a message to operators, but we are looking for feedback on launch instance [1][2]. There are already a few improvements being discussed, but additionally we would appreciate functional testing in various deployment environments. If you have a fun environment to test this against, please do! [1] Launch Instance in Kilo: https://youtu.be/e6MYAUzZZag [2] https://blueprints.launchpad.net/horizon/+spec/launch-instance-redesign Thank you, Travis __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] TC Candidacy
confirmed On Wed, Apr 22, 2015 at 11:19 AM, Zane Bitter zbit...@redhat.com wrote: Hello Stackists, I'd like to announce my candidacy for the OpenStack Technical Committee. I'm running because I don't think that the diversity of perspectives amongst TC members reflects the diversity of our community. We're fortunate to have a few people whose brilliance often transcends the scope of their day-to-day focus, but I don't think that can outweigh the fact that (by my, arguable, count) 12 out of 13 are focused primarily on Nova and the projects (including cross-project efforts) that evolved directly out of it. When a group of people share a common vision and goal, they can pretty much always figure out a way to work together toward it. When they don't, they have to invent rules and structure and bureaucracy to keep everyone in line.[1] I... think we need to work more on the vision thing ;) Thierry calls it 'stepping out of the way', but I think of it as stepping up, out of the weeds, to look at the bigger picture. My hope is that in a decade or two, developers will prefer to write their new applications against Open Source implementations of open APIs - and not just to avoid lock-in, but because they'll be as good or better than proprietary alternatives. Linux has already made that a reality at the level of individual servers - and while offering refuge from proprietary Unixes (Unices?) for legacy applications was no doubt a critical (and lucrative) part of getting there, it's much more important in the long run that it's also the preferred platform for new development. Today a growing fraction of applications are bigger than a single server, and I believe that OpenStack represents our best opportunity to make sure that in the future open source cloud APIs will be the preferred choice too. The big tent is a great start - it allows a project to assure contributors that they are committed to truly open[2] governance long before it matures to the point where it could have been incubated under the previous process. And after all, the most powerful technologies we've developed are social, and we shouldn't be reluctant to use them. However, if only a small section in the middle of the tent is waterproof, then the big tent exists in name only. We have to make sure we continue to help and guide all of the projects as they grow and mature - getting them in the tent is only the first step. I am strongly opposed to the TC using the tagging system to identify use cases it thinks are important - the community can and will decide for itself what is important. (Of course I support using the tagging system to provide more information that the community can use to evaluate projects for themselves, and more long-form documentation of use cases where that is lacking.) I believe in abundance, not scarcity: when our community provides space for participants to work on problems they care about we don't weaken the existing projects, we actually strengthen the community by increasing its critical mass. (In other words, we may have a task-allocation problem, but we shouldn't mistake it for a project-allocation problem since it will require different solutions.) Right now we're missing a lot of fundamental building blocks - like user-configurable authorisation, and public-facing asynchronous messaging - that we need to allow workloads running in a cloud to interact with it. As a result, a lot of projects that should be tightly (small-i) integrated (but loosely coupled!) with each other are developing in an ad hoc manner, and a lot of technical problems are being solved over and over, often badly. The pre-big-tent regime gave an incentive to new projects not to work together, and we need to reverse that effect. The TC needs to boost the confidence of OpenStack developers to simplify things by taking dependencies on other projects where appropriate, and I think the best way to do that is to kick off an ongoing discussion about the vision for and design of OpenStack at a level above that of indivudual projects. (We've made a good start on cross-project co-ordination at a _lower_ level, like the API working group, but to do so at a higher level will require the TC's blessing.) In case you don't know me, I'm a core developer of Heat and I've been involved in OpenStack since the Heat project kicked off about 3 years ago. I'm also a former elected PTL, and I've been working closely with the TC since Heat applied for incubation back around the time of the Grizzly summit. I've attended most TC meetings for at least the past 18 months, so I'm already up to speed with what has been happening. Finally, I currently lead a team at Red Hat developing (upstream!) tools for updating OpenStack deployments - which is another way of saying that few people are more motivated to make deployment easier than I ;) Thanks for reading this far. Please make time to read the other
Re: [openstack-dev] [oslo] eventlet 0.17.3 is now fully Python 3 compatible
On Wed, Apr 22, 2015 at 3:15 AM, Victor Stinner vstin...@redhat.com wrote: Hi, It's moving fast. I'm currently working on porting remaining libraries to prepare my spec for nova. Great, I did't realize how close all the dependencies were. oslo.db -- looks like it is almost there I don't know the status of oslo.db support of Python 3. I guess that it already works on Python 3, it's just a matter of running tests with MySQL (MySQL-Python blocks again here). oslo.messaging -- same Two changes of my Python 3 changes were already merged last week. Three remaining Python 3 changes are almost there, they are mostly blocked by the freeze on requirements, but changes are already approved. The blocker is the bump of eventlet to 0.17.3: https://review.openstack.org/#/c/172132/ paste -- almost there I fixed that last night (!) with the release of Paste 2.0. It's the first Paste release since 2010. Paste 2.0 has an experimental support of Python 3. It should be enough for nova, according to my quick tests in my local nova repository where I fixed most obvious Python 3 issues. Ian Bicking allowed me to publish new versions of Paste. sqlalchemy-migrate -- almost there It already supports Python 3, so it doesn't block. suds -- with the suds fork shouldn't be too hard You should just switch to the fork. I contacted the author of suds-jurko: he plans to rename its component from suds-jurko to suds. So his fork would be simple drop-in which would not require any change in OpenStack except of requirements. websockify -- unknown It announces Python 3.3 and 3.4 support and has a py34 env in tox.ini. I didn't check yet if it supports Python 3, but since the project is active (last commit 12 hours ago), it's shouldn't be hard to fix them quickly. libvirt-python -- unknown Oh, I missed this dependency. It's my next target :-) mysql-python -- alternitive looks viable. Yes, it's possible to replace it with PyMySQL. Does anyone know the status of this switch? Based on there being two unknowns, and a lot of dependencies that are just almost there, and a few we may want to migrate off of, I was assuming addressing those issues would make it hard for us to make nova python3 compatible for Liberty. My plan for Liberty is not to support fully Python 3 in nova, but only start the port (well, continue to be exact). The idea is to fix syntax errors and obvious issues like dict.iteritems() = six.iteritems(dict), then more complex changes. Maybe it's possible to finish the port in a cycle, but it really depends on the time taken to merge patches. How invasive would the port to python3 be? Would it be easier to have a python3 migration sprint and rip the band aid off instead of dragging it out and having partial python3 support for a whole cycle? In general what is the least painful way to add python3 support for a very large python project? I started to port nova in my local repository. Some unit tests already pass. nova already uses six in many places, so it's not like we really start from scratch, the port is already an ingoing process. Victor __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [Neutron] A big tent home for Neutron backend code
Hello! A couple of things I've been working on lately are project governance issues as a TC member and also implementation of a new virtual networking alternative with a Neutron driver. So, naturally I started thinking about how the Neutron driver code fits in to OpenStack governance. There are basically two areas with a lot of movement related to this issue. 1) Project governance has moved to a big tent model [1]. The vast majority of projects that used to be in Stackforge are being folded in to a larger definition of the OpenStack project. Projects making this move meet the following criteria as being one of us: http://governance.openstack.org/reference/new-projects-requirements.html Official project teams are tracked in this file along with the git repos they are responsible for: http://git.openstack.org/cgit/openstack/governance/tree/reference/projects.yaml which is also reflected here: http://governance.openstack.org/reference/projects/ The TC has also been working through defining a system to help differentiate efforts by using a set of tags [4]. So far, we have tags describing the release handling for a repository, as well as a tag for team diversity. We've also had a lot of discussion about tags to help describe maturity, but that is still a work in progress. 2) In Neutron, some fairly significant good changes are being made to help scale the development process. Advanced services were split out into their own repos [2]. Most of the plugin and driver code has also been split out into repos [3]. In terms of project teams, the Neutron team is defined as owning the following repos: http://governance.openstack.org/reference/projects/neutron.html - openstack/neutron - openstack/neutron-fwaas - openstack/neutron-lbaas - openstack/neutron-vpnaas - openstack/neutron-specs - openstack/python-neutronclient The advanced services split is reflected by the fwaas, lbaas, and vpnaas repos. We also have a large set of repositories related to Neutron backend code: http://git.openstack.org/cgit/?q=stackforge%2Fnetworking - stackforge/networking-arista - stackforge/networking-bagpipe-l2 - stackforge/networking-bgpvpn - stackforge/networking-bigswitch - stackforge/networking-brocade - stackforge/networking-cisco - stackforge/networking-edge-vpn - stackforge/networking-hyperv - stackforge/networking-ibm - stackforge/networking-l2gw - stackforge/networking-midonet - stackforge/networking-mlnx - stackforge/networking-nec - stackforge/networking-odl - stackforge/networking-ofagent - stackforge/networking-ovn - stackforge/networking-ovs-dpdk - stackforge/networking-plumgrid - stackforge/networking-portforwarding - stackforge/networking-vsphere Note that not all of these are equivalent. This is just a list of stackforge/networking-*. In some cases there is a split between code in the Neutron tree and in this repo. In those cases, a shim is in the Neutron tree, but most of the code is in the external repo. It's also possible to have all of the code in the external repo. There's also a big range of maturity. Some are quite mature and are already used in production. networking-ovn as an example is quite new and being developed in parallel with OVN in the Open vSwitch project. So, my question is: Where should these repositories live in terms of OpenStack governance and project teams? Here are a few paths I think we could take, along with some of my initial thoughts on pros/cons. a) Adopt these as repositories under the Neutron project team. In this case, I would see them operating with their own review teams as they do today to avoid imposing additional load on the neutron-core or neutron-specs-core teams. However, by being a part of the Neutron team, the backend team would submit to oversight by the Neutron PTL. There are some other details to work out to ensure expectations are clearly set for everyone involved. If this is the path that makes sense, we can work through those as a next step. Pros: + Seems to be the most natural first choice Cons: - A lot of changes have been made precisely because Neutron has gotten so big. A single project team/PTL may not be able to effectively coordinate all of the additional work. Maybe the core Neutron project would be better off focusing on being a platform, and other project teams organize work on backends. b) Let each interested group define a new project team for their backend code. So, as an example, the group of people working on Neutron integration with OpenDaylight could propose a new project team that would be a projects.yaml entry that looks something like: Neutron-OpenDaylight: ptl: Some Person (ircnick) service: OpenDaylight Networking mission: To implement Neutron support for the OpenDaylight project. url: ... projects: - repo: openstack/networking-odl Pros: + There's no additional load on the Neutron project team and PTL. Cons: - Having all of these efforts organized under
[openstack-dev] TC Candidacy
Hello Stackists, I'd like to announce my candidacy for the OpenStack Technical Committee. I'm running because I don't think that the diversity of perspectives amongst TC members reflects the diversity of our community. We're fortunate to have a few people whose brilliance often transcends the scope of their day-to-day focus, but I don't think that can outweigh the fact that (by my, arguable, count) 12 out of 13 are focused primarily on Nova and the projects (including cross-project efforts) that evolved directly out of it. When a group of people share a common vision and goal, they can pretty much always figure out a way to work together toward it. When they don't, they have to invent rules and structure and bureaucracy to keep everyone in line.[1] I... think we need to work more on the vision thing ;) Thierry calls it 'stepping out of the way', but I think of it as stepping up, out of the weeds, to look at the bigger picture. My hope is that in a decade or two, developers will prefer to write their new applications against Open Source implementations of open APIs - and not just to avoid lock-in, but because they'll be as good or better than proprietary alternatives. Linux has already made that a reality at the level of individual servers - and while offering refuge from proprietary Unixes (Unices?) for legacy applications was no doubt a critical (and lucrative) part of getting there, it's much more important in the long run that it's also the preferred platform for new development. Today a growing fraction of applications are bigger than a single server, and I believe that OpenStack represents our best opportunity to make sure that in the future open source cloud APIs will be the preferred choice too. The big tent is a great start - it allows a project to assure contributors that they are committed to truly open[2] governance long before it matures to the point where it could have been incubated under the previous process. And after all, the most powerful technologies we've developed are social, and we shouldn't be reluctant to use them. However, if only a small section in the middle of the tent is waterproof, then the big tent exists in name only. We have to make sure we continue to help and guide all of the projects as they grow and mature - getting them in the tent is only the first step. I am strongly opposed to the TC using the tagging system to identify use cases it thinks are important - the community can and will decide for itself what is important. (Of course I support using the tagging system to provide more information that the community can use to evaluate projects for themselves, and more long-form documentation of use cases where that is lacking.) I believe in abundance, not scarcity: when our community provides space for participants to work on problems they care about we don't weaken the existing projects, we actually strengthen the community by increasing its critical mass. (In other words, we may have a task-allocation problem, but we shouldn't mistake it for a project-allocation problem since it will require different solutions.) Right now we're missing a lot of fundamental building blocks - like user-configurable authorisation, and public-facing asynchronous messaging - that we need to allow workloads running in a cloud to interact with it. As a result, a lot of projects that should be tightly (small-i) integrated (but loosely coupled!) with each other are developing in an ad hoc manner, and a lot of technical problems are being solved over and over, often badly. The pre-big-tent regime gave an incentive to new projects not to work together, and we need to reverse that effect. The TC needs to boost the confidence of OpenStack developers to simplify things by taking dependencies on other projects where appropriate, and I think the best way to do that is to kick off an ongoing discussion about the vision for and design of OpenStack at a level above that of indivudual projects. (We've made a good start on cross-project co-ordination at a _lower_ level, like the API working group, but to do so at a higher level will require the TC's blessing.) In case you don't know me, I'm a core developer of Heat and I've been involved in OpenStack since the Heat project kicked off about 3 years ago. I'm also a former elected PTL, and I've been working closely with the TC since Heat applied for incubation back around the time of the Grizzly summit. I've attended most TC meetings for at least the past 18 months, so I'm already up to speed with what has been happening. Finally, I currently lead a team at Red Hat developing (upstream!) tools for updating OpenStack deployments - which is another way of saying that few people are more motivated to make deployment easier than I ;) Thanks for reading this far. Please make time to read the other candidate bios (especially Jay's), think about _your_ vision for OpenStack, engage with
Re: [openstack-dev] [all] Introducing the Cloud Service Federation project (cross-project design summit proposal)
On Apr 16, 2015 3:58 PM, Geoff Arnold ge...@geoffarnold.com wrote: Joe: you have identified many of the challenges of trying to work with multiple OpenStack clouds from different providers with different configurations, resources, etc. Nevertheless, people are doing it, and doing so successfully. (I know several teams that are running across multiple public and private clouds.) Doing so explicitly is very different then doing so implicitly. With your proposal, will the end consumer be aware of which underlying provider they are using? Your proposal here is pretty light on details on what this looks like for each persona involved (end user, reseller, cloud provider etc.) Packaging solutions like Docker may help with some of the low-level compatibility issues. This proposal is intended to remove one source of friction. There’s a lot more to be done. One interesting avenue for research is going to be the development of a virtual region metadata schema that will allow a tenant (or a broker) to determine the characteristics of virtual regions. (Such a model might be a useful complement to the RefStack work.) Geoff On Apr 16, 2015, at 3:00 PM, Joe Gordon joe.gord...@gmail.com wrote: On Thu, Apr 16, 2015 at 12:16 AM, Geoff Arnold ge...@geoffarnold.com wrote: I’ve discussed this with the Keystone team, especially the Reseller folks, but not as deeply as we need to. The biggest challenge that I see with doing this inside any existing project is the Aggregator system. It’s an independent deployment that doesn’t include any of the core OpenStack IaaS services - there’s no Nova, no networking (Nova or Neutron), no Glance, no Cinder. It’s just Horizon, Keystone, and a bunch of orchestration logic to wire up the virtual regions. Just assembling the bits into a deployable and testable system is going to be significantly different from a regular OpenStack cloud. Even though OpenStack is composed of relatively independent services, there’s an assumed context which affects just about everything. I really wouldn’t ask Keystone to take on the responsibility for such a thing. Better to build it in Stackforge, get some experience with it, and figure out where it lives later on. In spite of all that, we believe that this belongs in the “big tent” OpenStack, because it builds on existing OpenStack component services, and it’s value depends on interoperability. If you deploy the Virtual Region service as part of your OpenStack cloud, any Aggregator should be able to re-present your virtual regions to its users (subject to obvious security and operational policies). We’ve used the Reseller use case to describe the workflows, but there are a number of equally important use cases for this architecture. 'interoperability' is where I can see a lot of issues arising. If I am using a reseller with regions from two different providers that are configured even slightly differently, using the two regions interchangeably will become exceedingly difficult quickly. There are many cases where the same API when powered by different drivers and slightly different configurations result in very different end user behavior. A few example issues: * Glance images maintained by the cloud provider would be different across providers. * Policy files dictating what API calls a given user can use can differ across providers. * Network models. There is no single network model for OpenStack. * CPU performance. OpenStack has no way of saying 1VCPU in provider X is equivalent to 1.5 VCPUs under provider Y. * Config driver vs. metadata service. * Those are just a few issues I can think of off the top of my head but there are many many more. I can see this model working for only the simplest of use cases. Maintaining a cohesive experience across multiple providers who may not be working together is very difficult. But perhaps I am missing something. Geoff On Apr 15, 2015, at 10:03 PM, Miguel Angel Ajo Pelayo mangel...@redhat.com wrote: Sounds like a very interesting idea. Have you talked to the keystone folks?, I would do this work into the keystone project itself (just a separate daemon). This still looks like identity management (federated, but identity) I know the burden of working with a mainstream project could be higher, but benefits are also higher: it becomes more useful (it becomes automatically available for everyone) and also passes through the review process of the general keystone contributors, thus getting a more robust code. Best regards, Miguel On 16/4/2015, at 6:24, Geoff Arnold ge...@geoffarnold.com wrote: Yeah, we’ve taken account of: https://github.com/openstack/keystone-specs/blob/master/specs/juno/keystone-to-keystone-federation.rst http://blog.rodrigods.com/playing-with-keystone-to-keystone-federation/ http://docs.openstack.org/developer/keystone/configure_federation.html One of the use cases we’re wrestling with requires fairly strong anonymization: if user A
[openstack-dev] [Manila] 3rd party CI
I intend to introduce a requirement for 3rd party CI for Manila during the Liberty release. This shouldn't be a surprise to anyone because I've been talking about it, but in case anyone didn't hear about it, now you have. I expect that by the Liberty release date, every driver in the tree should have working/reporting vendor CI, similar to what the Cinder project requires [1]. In order to achieve this goal, I will be setting some deadlines at various points during the release, and I want the community to help define those goals and to agree that they're reasonable. I'm optimistic that this shouldn't be a terribly difficult process because nearly all the drivers in Manila are maintained by vendors who also have Cinder drivers and who have experience with OpenStack CI systems. However, if there's anyone who hasn't had to implement a CI system in the past, for Cinder or any other OpenStack project, then I can't stress enough how important it will be to get started on it early. Please contact me and let me know if CI systems are new to you so we can make sure to get you the help you'll need to be successful. I'm going to write up a draft plan before the Vancouver summit for CI-related deadlines and circulate it to be decided at one of the weekly IRC meetings (most likely the one after the design summit). [1] - https://wiki.openstack.org/wiki/Cinder/tested-3rdParty-drivers -Ben Swartzlander __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [oslo][policy][neutron] oslo.policy API is not powerful enough to switch Neutron to it
On Wed, 2015-04-22 at 08:49 -0400, Doug Hellmann wrote: That feature sounds like it could be useful outside of neutron, so let's see if we can come up with a new syntax to make it portable. Bonus points if the new syntax results in a proper DSL. I have been thinking that I should point people to the policies package[1] I built, and this DSL comment has finally pushed me over the edge :) It's a complete clean-room implementation with a completely different syntax than oslo.policy, so I don't know how interested anyone here would be in using it, but there it is. (I have it licensed under the GPL right now, but if anyone wants to use it, I'd be happy to relicense under Apache…) [1] https://pypi.python.org/pypi/policies -- Kevin L. Mitchell kevin.mitch...@rackspace.com Rackspace __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [Manila] Two nominations for Manila Core Reviewer Team
I would like to nominate Thomas Bechtold to join the Manila core reviewer team. Thomas has been contributing to Manila for close to 6 months and has provided a good number of quality code reviews in addition to a substantial amount of contributions. Thomas brings both Oslo experience as well as a packager/distro perspective which is especially helpful as Manila starts to get used in more production scenarios. I would also like to nominate Mark Sturdevant. He has also been active in the community for about 6 months and has a similar history of code reviews. Mark is the maintainer of the HP driver and would add vendor diversity to the core team. -Ben Swartzlander Manila PTL __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [all][api] 3 API Guidelines up for final review
Hi All, We have 3 API Guidelines that are ready for a final review. 1. Metadata guidelines document https://review.openstack.org/#/c/141229/ 2. Tagging guidelines https://review.openstack.org/#/c/155620/ 3. Guidelines on using date and time format https://review.openstack.org/#/c/159892/ If the API Working Group hasn’t received any further feedback, we’ll merge them on April 29. Thanks, Everett __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [puppet] Moving forward with puppet-keystone CI (beaker tests)
On Wed, Apr 22, 2015 at 02:03:13PM -0400, Emilien Macchi wrote: On 04/22/2015 11:53 AM, Spencer Krum wrote: Emillen, Do you see shelling out to openstackclient in the keystone test to verify that the keystone resources have been created? Do you see trying to hit the api from something like aviator? Ultimately I'd like to see us spin up an entire openstack in one test then hit it with tempest. * shell testing: yes because it's the way we wrote our providers. * aviator: no because we gave up some months ago in favor of using openstackclient. I'm not in favor of using aviator which would add yet another dependency and complexity. Maybe I'm wrong though. * tempest: well... tempest aims to test API features while we only want to check Keystone is actually running. I think serverspec + some shelling could help. Having tempest is (to me) overkill and could slow down our CI if something's wrong in Tempest. I kinda take issue with this comment, more likely than not when there are issues with running tempest it's either a config or service problem. That's not to say there aren't tempest bugs, I just don't think tempest breaking your CI should be a primary concern. (especially how often it's used for this exact purpose) If it did break something it's not like fixing it is a complicated procedure. We've been running tempest as the gating tests for some time so we've got some experience fixing issues with fixing the test suite when things really go haywire. I actually think running tempest against against a deployment using the puppet modules would probably be very useful. It'll let you test that everything comes together and actually works. It's used for gating devstack commits to verify basically the same thing. I'm also interested in doing this because I think it'll help us improve tempest by having direct feedback loop from running it against a non-devstack environment. That's something I'd really like to have because it's something I think we're missing right now. If you need any help with setting something like this up, I'm definitely available to give a hand with it. It may be possible to use a very narrow version of tempest to validate just keystone. Like, only a small set of tests that we would run with testr? So this is the intent of the smoke tag in tempest, to provide a small set of tests to do a quick sanity check that a deployment is working. Unfortunately right now the tag doesn't do that because it got conflated as part of the neutron testing bring up. We just need to spend some time sitting down and rationalizing the use of the tag to provide this. Also, if you want to just test service specific testing tempest already does service tagging of tests. So you can run a subset by using the service name as the filter. (ie identity for tests which talk to keystone or compute for nova tests) -Matt Treinish On Wed, Apr 22, 2015 at 7:51 AM, Emilien Macchi emil...@redhat.com mailto:emil...@redhat.com wrote: Hi, Some important work is being done on Keystone v3 API support in puppet-keystone. We've clearly seen there is a lack of review and I think we all worry about breaking something. Spencer I are working on beaker tests lately and the jobs are non-voting for now. I propose: * to review (and eventually merge) the beaker-tests patches [1] [2] for Keystone openstacklib. * to patch project-config [3] to make vote Beaker jobs in Puppet OpenStack gate for puppet-keystone puppet-openstacklib. Why voting? Because otherwise I'm not sure people will notice the failure and some patches will be merged while beaker is red. So we can have a good set of tests that will help us to detect some issues in the future. I don't think we will catch all mistakes we can do, but this is a good start. To vote this proposal, you can use the gerrit patches and let any feedback. Thanks, [1] puppet-keystone: https://review.openstack.org/#/c/155873/ [2] puppet-openstacklib: https://review.openstack.org/#/c/176098/ [3] project-config: https://review.openstack.org/176343 -- Emilien Macchi -- To unsubscribe from this group and stop receiving emails from it, send an email to puppet-openstack+unsubscr...@puppetlabs.com mailto:puppet-openstack%2bunsubscr...@puppetlabs.com. -- Spencer Krum (619)-980-7820 -- To unsubscribe from this group and stop receiving emails from it, send an email to puppet-openstack+unsubscr...@puppetlabs.com mailto:puppet-openstack+unsubscr...@puppetlabs.com. -- Emilien Macchi __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
[openstack-dev] [Manila] Mount automation using Zeroconf
Hello, Manila-philes. Back in Paris we started talking about Manila mount automation, whereby file shares could be automatically mounted on clients, and this will likely be a topic in Vancouver. So in order to have an informed discussion at the summit, I'd like to explore a few things beforehand. Besides brute force approaches like SSH or PsExec, one of the community suggestions was to use Zeroconf (aka Bonjour)[1]. Zeroconf sounds attractive on the surface, but it seems to have a number of limitations: * No standard way to specify local mount point * Additional setup required to work beyond the 'local' domain * Custom software needed on clients to mount advertised shares * Same issues with network connectivity as any other mount automation solution Does anyone have a clearer idea how Zeroconf might satisfy the need for Manila mount automation? Thanks, Clinton Knight Manila core team [1] http://en.wikipedia.org/wiki/Zero-configuration_networking __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [Swift] Kilo RC2 available
Hello everyone, Swift was last to RC1, but they are first in the RC2 race :) Due to release-critical issues spotted in RC1 testing, a new release candidate was created for Kilo. The 2.3.0 RC2 tarball is available for download at: https://launchpad.net/swift/kilo/2.3.0-rc2 Unless release-critical issues are found that warrant a release candidate respin, this tarball will be formally released as the Swift 2.3.0 final Kilo version on April 30. You are therefore strongly encouraged to test and validate this tarball ! Alternatively, you can directly test the stable/kilo branch at: https://github.com/openstack/swift/tree/stable/kilo If you find an issue that could be considered release-critical, please file it at: https://bugs.launchpad.net/swift/+filebug and tag it *kilo-rc-potential* to bring it to the release crew's attention. Regards, -- Thierry Carrez (ttx) __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Neutron] A big tent home for Neutron backend code
+1 I was in the middle of writing an email asking about the possibility of splitting out the reference implementation. you beat me to it. :) I also like the idea of having the PTL on top to help pass up/down ideas where code can be shared, benefiting everyone. Thanks, Kevin From: Kyle Mestery [mest...@mestery.com] Sent: Wednesday, April 22, 2015 12:30 PM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [Neutron] A big tent home for Neutron backend code On Wed, Apr 22, 2015 at 1:19 PM, Russell Bryant rbry...@redhat.commailto:rbry...@redhat.com wrote: Hello! A couple of things I've been working on lately are project governance issues as a TC member and also implementation of a new virtual networking alternative with a Neutron driver. So, naturally I started thinking about how the Neutron driver code fits in to OpenStack governance. Thanks for starting this conversation Russell. There are basically two areas with a lot of movement related to this issue. 1) Project governance has moved to a big tent model [1]. The vast majority of projects that used to be in Stackforge are being folded in to a larger definition of the OpenStack project. Projects making this move meet the following criteria as being one of us: http://governance.openstack.org/reference/new-projects-requirements.html Official project teams are tracked in this file along with the git repos they are responsible for: http://git.openstack.org/cgit/openstack/governance/tree/reference/projects.yaml which is also reflected here: http://governance.openstack.org/reference/projects/ The TC has also been working through defining a system to help differentiate efforts by using a set of tags [4]. So far, we have tags describing the release handling for a repository, as well as a tag for team diversity. We've also had a lot of discussion about tags to help describe maturity, but that is still a work in progress. 2) In Neutron, some fairly significant good changes are being made to help scale the development process. Advanced services were split out into their own repos [2]. Most of the plugin and driver code has also been split out into repos [3]. In terms of project teams, the Neutron team is defined as owning the following repos: http://governance.openstack.org/reference/projects/neutron.html - openstack/neutron - openstack/neutron-fwaas - openstack/neutron-lbaas - openstack/neutron-vpnaas - openstack/neutron-specs - openstack/python-neutronclient The advanced services split is reflected by the fwaas, lbaas, and vpnaas repos. We also have a large set of repositories related to Neutron backend code: http://git.openstack.org/cgit/?q=stackforge%2Fnetworking - stackforge/networking-arista - stackforge/networking-bagpipe-l2 - stackforge/networking-bgpvpn - stackforge/networking-bigswitch - stackforge/networking-brocade - stackforge/networking-cisco - stackforge/networking-edge-vpn - stackforge/networking-hyperv - stackforge/networking-ibm - stackforge/networking-l2gw - stackforge/networking-midonet - stackforge/networking-mlnx - stackforge/networking-nec - stackforge/networking-odl - stackforge/networking-ofagent - stackforge/networking-ovn - stackforge/networking-ovs-dpdk - stackforge/networking-plumgrid - stackforge/networking-portforwarding - stackforge/networking-vsphere Note that not all of these are equivalent. This is just a list of stackforge/networking-*. In some cases there is a split between code in the Neutron tree and in this repo. In those cases, a shim is in the Neutron tree, but most of the code is in the external repo. It's also possible to have all of the code in the external repo. There's also a big range of maturity. Some are quite mature and are already used in production. networking-ovn as an example is quite new and being developed in parallel with OVN in the Open vSwitch project. So, my question is: Where should these repositories live in terms of OpenStack governance and project teams? Here are a few paths I think we could take, along with some of my initial thoughts on pros/cons. a) Adopt these as repositories under the Neutron project team. In this case, I would see them operating with their own review teams as they do today to avoid imposing additional load on the neutron-core or neutron-specs-core teams. However, by being a part of the Neutron team, the backend team would submit to oversight by the Neutron PTL. Out of your options proposed, this seems like the most logical one to me. I don't really see this imposing a ton of strain on the existing core reviewer team, because we'll keep whatever core reviewer teams are already in the networking-foo projects. There are some other details to work out to ensure expectations are clearly set for everyone involved. If this is the path that makes sense, we can work through those as a next step. Pros: + Seems to be the most natural
Re: [openstack-dev] [Neutron] A big tent home for Neutron backend code
On Wed, Apr 22, 2015 at 1:19 PM, Russell Bryant rbry...@redhat.com wrote: Hello! A couple of things I've been working on lately are project governance issues as a TC member and also implementation of a new virtual networking alternative with a Neutron driver. So, naturally I started thinking about how the Neutron driver code fits in to OpenStack governance. Thanks for starting this conversation Russell. There are basically two areas with a lot of movement related to this issue. 1) Project governance has moved to a big tent model [1]. The vast majority of projects that used to be in Stackforge are being folded in to a larger definition of the OpenStack project. Projects making this move meet the following criteria as being one of us: http://governance.openstack.org/reference/new-projects-requirements.html Official project teams are tracked in this file along with the git repos they are responsible for: http://git.openstack.org/cgit/openstack/governance/tree/reference/projects.yaml which is also reflected here: http://governance.openstack.org/reference/projects/ The TC has also been working through defining a system to help differentiate efforts by using a set of tags [4]. So far, we have tags describing the release handling for a repository, as well as a tag for team diversity. We've also had a lot of discussion about tags to help describe maturity, but that is still a work in progress. 2) In Neutron, some fairly significant good changes are being made to help scale the development process. Advanced services were split out into their own repos [2]. Most of the plugin and driver code has also been split out into repos [3]. In terms of project teams, the Neutron team is defined as owning the following repos: http://governance.openstack.org/reference/projects/neutron.html - openstack/neutron - openstack/neutron-fwaas - openstack/neutron-lbaas - openstack/neutron-vpnaas - openstack/neutron-specs - openstack/python-neutronclient The advanced services split is reflected by the fwaas, lbaas, and vpnaas repos. We also have a large set of repositories related to Neutron backend code: http://git.openstack.org/cgit/?q=stackforge%2Fnetworking - stackforge/networking-arista - stackforge/networking-bagpipe-l2 - stackforge/networking-bgpvpn - stackforge/networking-bigswitch - stackforge/networking-brocade - stackforge/networking-cisco - stackforge/networking-edge-vpn - stackforge/networking-hyperv - stackforge/networking-ibm - stackforge/networking-l2gw - stackforge/networking-midonet - stackforge/networking-mlnx - stackforge/networking-nec - stackforge/networking-odl - stackforge/networking-ofagent - stackforge/networking-ovn - stackforge/networking-ovs-dpdk - stackforge/networking-plumgrid - stackforge/networking-portforwarding - stackforge/networking-vsphere Note that not all of these are equivalent. This is just a list of stackforge/networking-*. In some cases there is a split between code in the Neutron tree and in this repo. In those cases, a shim is in the Neutron tree, but most of the code is in the external repo. It's also possible to have all of the code in the external repo. There's also a big range of maturity. Some are quite mature and are already used in production. networking-ovn as an example is quite new and being developed in parallel with OVN in the Open vSwitch project. So, my question is: Where should these repositories live in terms of OpenStack governance and project teams? Here are a few paths I think we could take, along with some of my initial thoughts on pros/cons. a) Adopt these as repositories under the Neutron project team. In this case, I would see them operating with their own review teams as they do today to avoid imposing additional load on the neutron-core or neutron-specs-core teams. However, by being a part of the Neutron team, the backend team would submit to oversight by the Neutron PTL. Out of your options proposed, this seems like the most logical one to me. I don't really see this imposing a ton of strain on the existing core reviewer team, because we'll keep whatever core reviewer teams are already in the networking-foo projects. There are some other details to work out to ensure expectations are clearly set for everyone involved. If this is the path that makes sense, we can work through those as a next step. Pros: + Seems to be the most natural first choice Cons: - A lot of changes have been made precisely because Neutron has gotten so big. A single project team/PTL may not be able to effectively coordinate all of the additional work. Maybe the core Neutron project would be better off focusing on being a platform, and other project teams organize work on backends. It's interesting you mention neutron being a platform, because that's exactly where I want it to go in Liberty as well. To that end, I expect to
[openstack-dev] [Glance] [all] python-glanceclient release 0.17.1
The python-glanceclient release management team is pleased to announce: Release of python-glanceclient version 0.17.1 Please find the details related to the release at: https://launchpad.net/python-glanceclient/+milestone/0.17.1 Please report the issues through launchpad: https://bugs.launchpad.net/python-glanceclient Please note: 0.17.x series is the stable/kilo series of client-releases. Thanks, -Nikhil __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Policy rules are killed by the context admin check
On 4/22/2015 8:32 AM, Sylvain Bauza wrote: Hi, By discussing on a specific bug [1], I just discovered that the admin context check which was done at the DB level has been moved to the API level thanks to the api-policy-v3 blueprint [2] That behaviour still leads to a bug if the operator wants to change an endpoint policy by leaving it end-user but still continues to be denied because of that, as it will forbid any non-admin user to call the methods (even if authorize() grants the request) I consequently opened a bug [3] for this but I'm also concerned about the backportability of that and why it shouldn't fixed in v2.0 too. Releasing the check by removing it sounds an acceptable change, as it fixes a bug without changing the expected behaviour [4]. The impact of the change sounds also minimal with a very precise scope (ie. leave the policy rules work as they are expected) [5] Folks, thoughts ? -Sylvain [1] https://bugs.launchpad.net/nova/+bug/1447084 [2] https://review.openstack.org/#/q/project:openstack/nova+branch:master+topic:bp/v3-api-policy,n,z [3] https://bugs.launchpad.net/nova/+bug/1447164 [4] https://wiki.openstack.org/wiki/APIChangeGuidelines#Generally_Considered_OK Fixing a bug so that a request which resulted in an error response before is now successful [5] https://wiki.openstack.org/wiki/StableBranch#Stable_branch_policy __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev I don't disagree, see bug 1168488 from way back in grizzly. The only thing would be we'd have to make sure the default rule is admin for any v2 extensions which are enforcing an admin context today. -- Thanks, Matt Riedemann __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] TC Candidacy
My name is Maru Newby, and I am announcing my candidacy for the Technical Committee (TC) election. tl;dr; I'm a relative unknown outside of Neutron, but I've been helping to drive high-level change in that project. I would welcome the opportunity to bring my energy and dedication to OpenStack-wide concerns as a member of our TC. If you're willing to read any part of this email, I hope it would be 'Why vote for me?'. If not, and you are as frustrated with the status quo as I am, I hope you will consider voting for candidates that support the goal of building a more engaged TC focused on ensuring that participation in OpenStack becomes more enjoyable and sustainable. Thank you for reading, and make sure to vote! * Why vote for me? If elected, I'm going to be a squeaky wheel advocating for anything and everything that empowers our development community to deliver useful software into the hands of users. If this puts me at odds with those that want to focus on issues like 'What is OpenStack' or 'How can we systematize project culture to make it easier to communicate?', good! I believe that you, as a technical contributor to OpenStack, need stronger representation of your viewpoints on our TC, and I'm one of the strongest advocates you're likely to find in this community. I am currently employed full time upstream, and I am willing and able to devote the resources necessary to do justice to the role. I also intend to advocate for focusing the OpenStack community on ambitious, but achievable, goals. I believe this will require making hard choices about which use cases we can reasonably expect to support. To me, the idea that OpenStack can be all things to all people is dangerous. It's just as likely that in trying to do it all, we do little of it well, and risk irrelevance. I think our primary competition is and will continue to be proprietary cloud, and my biggest fear is that we become unable to compete on cost due to the operational complexity required to make everyone happy. We need to keep our collective eyes on the ball! To be clear, I want to be a part of a TC with as diverse a group of viewpoints as possible to ensure that we have to work hard to achieve consensus. If agreement is reached too easily, there is probably something wrong. My wanting to push a developer-centric agenda doesn't mean that I don't respect the need to address other issues. My voice would be one of many, and I recognize that consensus requires compromise. * Why not vote for me? There are candidates more deserving of your vote if you think our TC shouldn't have a role in providing leadership to the OpenStack community. If you believe that our TC is already sufficiently developer-focused, then I also don't deserve your vote. This is in no way to suggest that I want to see our TC become a top-down decision-making body. This is still an open source project, after all, and I know first-hand the complexities of the culture involved. I do think it appropriate, though, for an elected body with as much visibility as our TC to participate more actively in addressing the challenges we face. * Key concerns There are any number of issues facing OpenStack today, but the items on the shortlist that follows are the concerns that I would like to see the TC champion in the near-term: ** Scaling our development effort The stated goal of our TC of late has been to 'get out of the way'. I wholeheartedly support this intention, and would like to see it extended to how our TC approaches governance beyond project evaluation. I think our TC should be responsible for proposing what we want to achieve rather than in how we should achieve it. Outcomes, rather than process. I think project-level contributors are best equipped to solve the problems of implementation. Our TC can take responsibility for setting OpenStack-wide expectations and in facilitating - rather than managing - cross-project advancement towards these goals. More than ever, we need to be pulling in the same direction, and I think our TC is an obvious candidate to rally ourselves around. I hope we all recognize that we can't scale OpenStack if decisions are constantly gated on a small group of 'tastemakers'. Delegation of trust is required, whether at the level of our TC or the individual projects. I think we need to take our TC's recent momentum to the project level and find ways to increase delegation of responsibility. Nova and Neutron, for example, have had to face ever-increasing demands with the same small pool of core reviewers, and many of you know all-too-well the pain that has resulted. Core reviewers - like the pre-big tent TC - have inadvertently become micromanagers as their responsibilities have evolved beyond their capacity to meet them. The model of direct trust - in which each core needs to trust every other core - doesn't scale due to fundamental human limits. This is at odds with OpenStack's need to grow, and I want our TC to
Re: [openstack-dev] [Horizon] Outreachy call for mentor
I added my name to an openstack wiki page for this express purpose some time ago, apparently the wrong one, which I can find no reference to now. That said, I am willing to be a mentor for a Horizon focused intern, if inability to find the correct wiki pages isn't a limiting factor. David On Wed, Apr 22, 2015 at 6:31 PM, Victoria Martínez de la Cruz victo...@vmartinezdelacruz.com wrote: Hi all, Horizon folks, we have a really good Outreachy candidate interested in working with you. The internship is from May 25th to August 25th, and interns are expected to work full time on their projects. Being a mentor should not be time consuming, we expect interns to be able to do their tasks independently and to solve the blockers they might find with the help of the community. I will be mentoring an applicant this round and, if time is a problem, I offer to help with this applicant as well. The announcement of accepted participants is soon, so this is a real last minute call. Outreachy is an internship targeted to anyone who identifies as a woman and OpenStack has been participating on it for about two years now. We had really good participants and many important features had been added as part of this program. For more details on OpenStack participation on Outreachy, check out the OpenStack Outreachy wiki page [0] and the Outreachy official site [1]. If you decide you want to mentor, please reach me and I'll guide you through the process. Please, let me know if you have any doubt or concern. Thanks all, Victoria [0] https://wiki.openstack.org/wiki/Outreachy [1] https://wiki.gnome.org/Outreachy __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Required data migrations in Kilo, need Turbo Hipster tests updated
That is correct -- these are icehouse datasets which have been upgraded, but never had an juno run against them. It would be hard for turbo hipster to do anything else, as it doesn't actually run a cloud. We can explore ideas around how to run live upgrade code, but its probably a project to pursue separately. Sure, no complaints. I just came to the realization when thinking about this that the datasets probably need to be refreshed over time to keep us testing real things, which I think was the point of T-H in the first place. One quick option here is I can reach out and ask our dataset donors if they have more recent dumps they'd be willing to share. Yeah, that'd be sweet. Thanks! --Dan __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Horizon] Outreachy call for mentor
Hi Victoria, Count me in also. I'll go ahead and subscribe myself to the mailing list as well. From: Victoria Martínez de la Cruz victo...@vmartinezdelacruz.com To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Date: 04/22/2015 06:10 PM Subject:Re: [openstack-dev] [Horizon] Outreachy call for mentor Hi David, Thanks a lot for your quick response! I'll give you more details about the applicant in a follow up email or in IRC. Certainly having separate wikis is not practical sometimes, so Stefano has been working on centralize all the information wrt internships and mentoring opportunities in OpenStack. There is a new mailing list you can subscribe to http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-internships. Again, thanks. Victoria 2015-04-22 21:51 GMT-03:00 David Lyle dkly...@gmail.com: I added my name to an openstack wiki page for this express purpose some time ago, apparently the wrong one, which I can find no reference to now. That said, I am willing to be a mentor for a Horizon focused intern, if inability to find the correct wiki pages isn't a limiting factor. David On Wed, Apr 22, 2015 at 6:31 PM, Victoria Martínez de la Cruz victo...@vmartinezdelacruz.com wrote: Hi all, Horizon folks, we have a really good Outreachy candidate interested in working with you. The internship is from May 25th to August 25th, and interns are expected to work full time on their projects. Being a mentor should not be time consuming, we expect interns to be able to do their tasks independently and to solve the blockers they might find with the help of the community. I will be mentoring an applicant this round and, if time is a problem, I offer to help with this applicant as well. The announcement of accepted participants is soon, so this is a real last minute call. Outreachy is an internship targeted to anyone who identifies as a woman and OpenStack has been participating on it for about two years now. We had really good participants and many important features had been added as part of this program. For more details on OpenStack participation on Outreachy, check out the OpenStack Outreachy wiki page [0] and the Outreachy official site [1]. If you decide you want to mentor, please reach me and I'll guide you through the process. Please, let me know if you have any doubt or concern. Thanks all, Victoria [0] https://wiki.openstack.org/wiki/Outreachy [1] https://wiki.gnome.org/Outreachy __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] TC Candidacy
Hi everyone! I'd like to announce my own candidacy for the OpenStack Technical Committee. My TL/DR platform is: Represent Front-End Engineering. It's what I do, it's what I love, it's what I've been doing for the last 15 years, and it's what I want to keep doing for years to come. Would you like some details? Of course you would! *First: Represent Front-End Engineering on the TC.* To me, this means being an advocate to everyone who touches the things which people use to interact with OpenStack; CLI, Web UI, etc. From the engineers working on upstream projects such as Fuel, Refstack, Ironic-UI, Horizon, and StoryBoard, to the UI Developers downstream who are developing their own tools, I strongly feel this branch of our profession should be represented, and I would like to be that representative. *Second: Advocate ease-of-use across OpenStack.* I don't only mean pretty buttons - I also mean how easy the CLI clients are, how intuitive the API's are, and how easy it is to onboard and/or support your own engineering efforts. You can have all the feature support in the world, but if it's not easy to use, you're doomed out of the gate. *Third: Make JavaScript a first-class citizen.* Yep, _this_ can of worms! Between the projects mentioned above, it's pretty clear that JavaScript is here to stay. With that in mind there remain many problems with the tooling, and we need to be conscious of those shortcomings as we start to draft policies that support the needs of all stakeholders: OpenStack's components, Engineers both downstream and upstream, Package Maintainers, and most importantly Operators and their Customers. *My qualifications:* I've been active in the OpenStack community for about 18 months now, working on Monty Taylor's team here at HP on trying to get StoryBoard to the point where it can replace Launchpad. I'm more-or-less responsible for all things NodeJS, NPM, Selenium, and Browser on infra, basically everything you need to build and test a Web UI. I've recently landed supporting technologies (such as CORS) that will greatly assist in unleashing downstream UI developers post-liberty, and am trying to teach Infra how to publish javascript libraries much like it does for python. Also, I've got 15 years of experience as a UI engineer, and a scant year of experience as a UX researcher. Michael Krotscheck __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Neutron] A big tent home for Neutron backend code
Would it make sense to capture these projects as simply 'affiliated', ie. with a loose relationship to Neutron, because they use/integrate with Neutron in some form or another (e.g. having 3rd-party, extending-api, integrating-via-plugin-model, etc)? Then we could simply consider extending the projects.yaml to capture this new concept (for Neutron or any other project) once we defined its ontology. Thoughts? That seems interesting, but given the communities stated goals around Big Tent, it seems to me like affiliation or not, adding these under the Neutron tent, inside the larger OpenStack Bigger Tent, would be a good thing. Thanks, Kyle Thanks for clearing some of the questions I raised. I should stress the fact that I welcome the idea of finding a more sensible home for these projects in light of the big tent developments, but it seems like we're still pouring down the foundations. I'd rather get us to a point where the landscape is clear, and the dust settled. That would help us make a more informed decision compared to the one we can make right now. Cheers, Armando __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Neutron] A big tent home for Neutron backend code
Could you please also pay some attention on Cons of this ultimate splitting Kyle? I'm afraid it would hurt the user experiences. On the position of Dev, A naked Neutron without official built-in reference implementation probably has a more clear architecture. On the other side, users would be forced to make choice between a long list of backend implementations, which is very difficult for non-professionals. In most of the time, users need a off-the-shelf solution without paying much extra integration effort, and they have less interest to study which SDN controller is powerful and is better than others. Can we imagine Nova without KVM/Qemu virt driver, Cinder without Ceph/VKM volume driver [See Deployment Profiles section in 1a] ? Shall we really decide to make Neutron the only Openstack project which has not any in tree official implementation? I think the analogy here is between the agent reference implementation vs KVM or Ceph, rather than the plumbing that taps into the underlying technology. Nova doesn't build/package KVM as Cinder doesn't build/package Ceph. Neutron could rely on other open source solutions (ODL, OpenContrail, OVN, etc), and still be similar to the other projects. I think there's still room for clarifying what the split needs to be, but I have always seen Neutron as the exception rather than norm, where, for historic reasons, we had to build everything from the ground up for lack of viable open source solutions at the time the project was conceived. [1a] http://superuser.openstack.org/articles/openstack-user-survey-insights-november-2014 Here is my personal suggestion: decomposition decision needs some trade-off, remaining 2-3 mainstream opensource backends in tree [ML2 with OVSLB, based on the survey result of 1a above]. While we are progressing radically with architecture re-factoring, smooth experience and easy to adoption should also be cared. One thing which is worth bringing up in this context is the potential overlap between this implementations. I think having them all under the Neutron project would allow me as PTL and the rest of the team work to combine things when it makes sense. Kyle [1] http://www.faqs.org/rfcs/rfc1149.html b) Let each interested group define a new project team for their backend code. To be honest, I don't this is a scalable option. I'm involved with 2 of these networking-foo projects, and there is not enough participation so far to warrant an entirely new project, PTL and infra around it. This is just my opinion, but it's an opinion I've taken after having contributed to networking-odl and networking-ovn for the past 5 months. So, as an example, the group of people working on Neutron integration with OpenDaylight could propose a new project team that would be a projects.yaml entry that looks something like: Neutron-OpenDaylight: ptl: Some Person (ircnick) service: OpenDaylight Networking mission: To implement Neutron support for the OpenDaylight project. url: ... projects: - repo: openstack/networking-odl Pros: + There's no additional load on the Neutron project team and PTL. Cons: - Having all of these efforts organized under a single project team and PTL might be better for ensuring some level of collaboration and consistency. c) A single new project team could be created that is led by a single person to coordinate the sub-teams working on each repo. In this scenario, I could see the overall collaboration being around ensuring consistent approaches to development process, testing, documentation, and releases. That would be something like this (note that the project list would be limited to those who actually want to be included in the official project team and meet some set of inclusion criteria). Neutron-Backends: ptl: Some Person (ircnick) service: Networking Backends mission: To implement Neutron backend support for various networking technologies. url: ... projects: - openstack/networking-arista - openstack/networking-bagpipe-l2 - openstack/networking-bgpvpn - openstack/networking-bigswitch - openstack/networking-brocade - openstack/networking-cisco - openstack/networking-edge-vpn - openstack/networking-hyperv - openstack/networking-ibm - openstack/networking-l2gw - openstack/networking-midonet - openstack/networking-mlnx - openstack/networking-nec - openstack/networking-odl - openstack/networking-ofagent - openstack/networking-ovn - openstack/networking-ovs-dpdk - openstack/networking-plumgrid - openstack/networking-portforwarding - openstack/networking-vsphere Pros: + There's no additional load on the Neutron project team and PTL. + Avoids a proliferation of new project teams for each Neutron backend. + Puts efforts
Re: [openstack-dev] [nova] Policy rules are killed by the context admin check
2015-04-23 6:55 GMT+08:00 Matt Riedemann mrie...@linux.vnet.ibm.com: On 4/22/2015 8:32 AM, Sylvain Bauza wrote: Hi, By discussing on a specific bug [1], I just discovered that the admin context check which was done at the DB level has been moved to the API level thanks to the api-policy-v3 blueprint [2] That behaviour still leads to a bug if the operator wants to change an endpoint policy by leaving it end-user but still continues to be denied because of that, as it will forbid any non-admin user to call the methods (even if authorize() grants the request) I consequently opened a bug [3] for this but I'm also concerned about the backportability of that and why it shouldn't fixed in v2.0 too. Releasing the check by removing it sounds an acceptable change, as it fixes a bug without changing the expected behaviour [4]. The impact of the change sounds also minimal with a very precise scope (ie. leave the policy rules work as they are expected) [5] Folks, thoughts ? -Sylvain [1] https://bugs.launchpad.net/nova/+bug/1447084 [2] https://review.openstack.org/#/q/project:openstack/nova+branch:master+topic:bp/v3-api-policy,n,z [3] https://bugs.launchpad.net/nova/+bug/1447164 [4] https://wiki.openstack.org/wiki/APIChangeGuidelines#Generally_Considered_OK Fixing a bug so that a request which resulted in an error response before is now successful [5] https://wiki.openstack.org/wiki/StableBranch#Stable_branch_policy __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev I don't disagree, see bug 1168488 from way back in grizzly. The only thing would be we'd have to make sure the default rule is admin for any v2 extensions which are enforcing an admin context today. Agree, if we want to fix those for v2, we need make sure the default rule is admin. And do you mean [3] want to fix this for v2 both in Kilo and Liberty? For liberty, we can do that, but I think we will switch to v2.1 very soon. Not sure it is still worth to do that. For kilo, some of api is pretty easy to fix by just removing 'require_admin_context()'. But there still have many of policy patches didn't merged into the master yet. like: https://review.openstack.org/#/q/status:open+project:openstack/nova+branch:master+topic:bp/nova-api-policy-final-part,n,z https://review.openstack.org/#/q/status:open+project:openstack/nova+branch:master+topic:bp/v3-api-policy,n,z https://review.openstack.org/#/q/status:open+project:openstack/nova+branch:master+topic:remove_qutoa_hardcode_permission,n,z https://review.openstack.org/#/q/status:open+project:openstack/nova+branch:master+topic:remove_quotaclass_hardcode_permission,n,z Should we back-port them all? -- Thanks, Matt Riedemann __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [Horizon] Outreachy call for mentor
Hi all, Horizon folks, we have a really good Outreachy candidate interested in working with you. The internship is from May 25th to August 25th, and interns are expected to work full time on their projects. Being a mentor should not be time consuming, we expect interns to be able to do their tasks independently and to solve the blockers they might find with the help of the community. I will be mentoring an applicant this round and, if time is a problem, I offer to help with this applicant as well. The announcement of accepted participants is soon, so this is a real last minute call. Outreachy is an internship targeted to anyone who identifies as a woman and OpenStack has been participating on it for about two years now. We had really good participants and many important features had been added as part of this program. For more details on OpenStack participation on Outreachy, check out the OpenStack Outreachy wiki page [0] and the Outreachy official site [1]. If you decide you want to mentor, please reach me and I'll guide you through the process. Please, let me know if you have any doubt or concern. Thanks all, Victoria [0] https://wiki.openstack.org/wiki/Outreachy [1] https://wiki.gnome.org/Outreachy __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Neutron] A big tent home for Neutron backend code
On 22 April 2015 at 11:19, Russell Bryant rbry...@redhat.com wrote: Hello! A couple of things I've been working on lately are project governance issues as a TC member and also implementation of a new virtual networking alternative with a Neutron driver. So, naturally I started thinking about how the Neutron driver code fits in to OpenStack governance. There are basically two areas with a lot of movement related to this issue. 1) Project governance has moved to a big tent model [1]. The vast majority of projects that used to be in Stackforge are being folded in to a larger definition of the OpenStack project. Projects making this move meet the following criteria as being one of us: Is it sensible to assume that Stackforge is going away entirely at some point in the future, and we'll have a single namespace - OpenStack? http://governance.openstack.org/reference/new-projects-requirements.html Official project teams are tracked in this file along with the git repos they are responsible for: http://git.openstack.org/cgit/openstack/governance/tree/reference/projects.yaml which is also reflected here: http://governance.openstack.org/reference/projects/ The TC has also been working through defining a system to help differentiate efforts by using a set of tags [4]. So far, we have tags describing the release handling for a repository, as well as a tag for team diversity. We've also had a lot of discussion about tags to help describe maturity, but that is still a work in progress. 2) In Neutron, some fairly significant good changes are being made to help scale the development process. Advanced services were split out into their own repos [2]. Most of the plugin and driver code has also been split out into repos [3]. This is too still a work in progress. A lot has been achieved during the Kilo timeframe, but more is still to be done, like the 'lib-ification - if that's even a word' of Neutron [1a], the split of plugins out of the *aas drivers [2b], and the rest of the core-vendor-decomp (latest status update is available in [3b]) [1a] https://review.openstack.org/#/c/154736/ [2b] https://review.openstack.org/#/c/174619/ [3b] https://review.openstack.org/#/c/173549/ In terms of project teams, the Neutron team is defined as owning the following repos: http://governance.openstack.org/reference/projects/neutron.html - openstack/neutron - openstack/neutron-fwaas - openstack/neutron-lbaas - openstack/neutron-vpnaas - openstack/neutron-specs - openstack/python-neutronclient The advanced services split is reflected by the fwaas, lbaas, and vpnaas repos. We also have a large set of repositories related to Neutron backend code: http://git.openstack.org/cgit/?q=stackforge%2Fnetworking - stackforge/networking-arista - stackforge/networking-bagpipe-l2 - stackforge/networking-bgpvpn - stackforge/networking-bigswitch - stackforge/networking-brocade - stackforge/networking-cisco - stackforge/networking-edge-vpn - stackforge/networking-hyperv - stackforge/networking-ibm - stackforge/networking-l2gw - stackforge/networking-midonet - stackforge/networking-mlnx - stackforge/networking-nec - stackforge/networking-odl - stackforge/networking-ofagent - stackforge/networking-ovn - stackforge/networking-ovs-dpdk - stackforge/networking-plumgrid - stackforge/networking-portforwarding - stackforge/networking-vsphere Note that not all of these are equivalent. This is just a list of stackforge/networking-*. In some cases there is a split between code in the Neutron tree and in this repo. In those cases, a shim is in the Neutron tree, but most of the code is in the external repo. It's also possible to have all of the code in the external repo. There's also a big range of maturity. Some are quite mature and are already used in production. networking-ovn as an example is quite new and being developed in parallel with OVN in the Open vSwitch project. So, my question is: Where should these repositories live in terms of OpenStack governance and project teams? It's my understanding that StackForge projects are bound to the same governance model, or am I mistaken? Here are a few paths I think we could take, along with some of my initial thoughts on pros/cons. a) Adopt these as repositories under the Neutron project team. I fully understand how this is implemented in, can you elaborate? Would that translate into something like [4a]? The other concern I have is that the list is bound to change due to the WIP nature of the work we're going through, and I wouldn't want to freeze it just yet, as we can't. Would just renaming the existing repos from stackforge/* to openstack/* suffice? Do we ask people to submit the new ones to the openstack namespace from now on? Would that slow down their ability to decompose because the big tent is not there yet? [4a] https://review.openstack.org/#/c/175954 In
Re: [openstack-dev] [nova] Required data migrations in Kilo, need Turbo Hipster tests updated
On Thu, Apr 23, 2015 at 1:31 AM, Dan Smith d...@danplanet.com wrote: Sure, but for people doing continuous deployment, they clearly haven't ran the migrate_flavor_data (or if they have, they haven't filed any bugs about it not working[0]). Hence the usefulness of T-H here, right? The point of the migration check is to make sure that people _do_ run it before the leave kilo. Right now, they have nothing other than the big note in the release notes about doing it. Evidence seems to show that they're not seeing it, which is exactly why we need the check :) I also found what I believe to be a bug with the flavor migration code. I started working on a fix by my limited knowledge of nova's objects has hindered me. Any thoughts on the legitimacy of the bug would be helpful though: https://bugs.launchpad.net/nova/+bug/1447132 . Basically for some of the datasets that turbo-hipster uses there are no entries in the new instance_extra table stopping any flavor migration from actually running. Then in your change (174480) you check the metadata table instead of the extras table causing the migration to fail even though migrate_flavor_data thinks there is nothing to do. I don't think this has anything to do with the objects, it's probably just a result of my lack of sqlalchemy-fu. Sounds like you weren't close to a fix, so I'll try to cook something up. So, a question here: These data sets were captured at some point in time, right? Does that mean that they were from say Icehouse era and have had nothing done to them since? Meaning, are there data sets that actually had juno or kilo running on them? This instance_extra thing would be the case for any instance that hasn't been touched in a long time, so it's legit. However, as we move to more online migration of data, I do wonder if taking an ancient dataset, doing schema migrations forward to current and then expecting it to work is really reflective of reality. That is correct -- these are icehouse datasets which have been upgraded, but never had an juno run against them. It would be hard for turbo hipster to do anything else, as it doesn't actually run a cloud. We can explore ideas around how to run live upgrade code, but its probably a project to pursue separately. One quick option here is I can reach out and ask our dataset donors if they have more recent dumps they'd be willing to share. Can you shed some light on what is really going on? I think it's worth noting that your change (174480) will require operators (particularly continuous deployers) to run migrate_flavor_data and given the difficulties I've found I'm not sure it's ready to be ran. Right, that's the point. If we encounter bugs against real datasets with migrate_flavor_data then deployers will be unable to upgrade until migrate_flavor_data is fixed. This may make things awkward if a deployer updates their codebase and can't run the migrations. Clearly they'll have to roll-back the changes. This is the scenario turbo-hipster is meant to catch - migrations that don't work on real datasets. Right, that's why we're holding until we make sure that it works. If migrate_flavor_data is broken a backport into Kilo will be needed so that if Liberty requires all the flavor migrations to be finished they can indeed be ran before upgrading to Liberty. This may give reason not to block on having the flavors migrated until the M-release but I realise that has other undersiable consequences (ie high code maintenance). Backports to fix this are fine IMHO, and just like any other bug found in actual running of things. It's too bad that none of our continuous deployment people seem to have found this yet, but that's a not uncommon occurrence. Obviously if we hit something that makes it too painful to get right in kilo, then we can punt for another cycle. Thanks! Michael -- Rackspace Australia __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Murano] python-openstackclient support
+1 for the idea though not sure on priority of this since we have so many way more important things to implement in Kilo. I'd say that would be a great contribution if we find someone willing to contribute it :) Sincerely yours, Stan Lagun Principal Software Engineer @ Mirantis sla...@mirantis.com On Thu, Apr 23, 2015 at 1:55 AM, Kirill Zaitsev kzait...@mirantis.com wrote: Since python-openstackclient is now a part of openstack — I guess it would be a good idea to support it in murano. It has setuptools-based plugin system, and it should be fairly easy to add murano commands as plugins to it. BTW, It’s based on cliff and has a terrific completion support (which is basically why I started looking into the issue in the first place =)) What do you think, is it a good idea? -- Kirill Zaitsev Sent with Airmail __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] TC Candidacy
Hi folks, I’d like to announce my candidacy for the TC elections. Here is a list of the main directions I would like to concentrate on as a TC member: * Continue to grow the OpenStack long-term vision and inclusive approach that comes with The Big Tent. We should not just be more inclusive, but also provide new OpenStack projects with fine-grained directions for improvements, recommendations and cross-project coordination. * Strengthen the voice of the platform projects (Heat, Sahara, Murano, etc) on the Technical Committee. * Bring additional focus and attention to OpenStack end users of all levels, especially application developers who are the most active cloud users. * Help OpenStack community newcomers better understand the process of contributing, learning the upstream tooling, and finding answers to their common questions. * Allow more self-governance and automation of common tasks like adding new source repositories without TC pre-approval. A few words about me. I have been actively involved in OpenStack community for the last 2.5 years. I’m PTL of OpenStack Data Processing service (“Sahara”) from its very beginning and a core team member of the OpenStack Infrastructure team. Before OpenStack, I was involved in other open source projects including Apache/Twitter Storm, Kotlin and Big Data solutions. I’m currently leading the Sahara team and an internal OpenStack Infra clone initiative at Mirantis. Thanks. -- Sincerely yours, Sergey Lukjanov Sahara Technical Lead (OpenStack Data Processing) Principal Software Engineer Mirantis Inc. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] TC Candidacy
confirmed On Wed, Apr 22, 2015 at 5:05 PM, Sergey Lukjanov slukja...@mirantis.com wrote: Hi folks, I'd like to announce my candidacy for the TC elections. Here is a list of the main directions I would like to concentrate on as a TC member: * Continue to grow the OpenStack long-term vision and inclusive approach that comes with The Big Tent. We should not just be more inclusive, but also provide new OpenStack projects with fine-grained directions for improvements, recommendations and cross-project coordination. * Strengthen the voice of the platform projects (Heat, Sahara, Murano, etc) on the Technical Committee. * Bring additional focus and attention to OpenStack end users of all levels, especially application developers who are the most active cloud users. * Help OpenStack community newcomers better understand the process of contributing, learning the upstream tooling, and finding answers to their common questions. * Allow more self-governance and automation of common tasks like adding new source repositories without TC pre-approval. A few words about me. I have been actively involved in OpenStack community for the last 2.5 years. I'm PTL of OpenStack Data Processing service (Sahara) from its very beginning and a core team member of the OpenStack Infrastructure team. Before OpenStack, I was involved in other open source projects including Apache/Twitter Storm, Kotlin and Big Data solutions. I'm currently leading the Sahara team and an internal OpenStack Infra clone initiative at Mirantis. Thanks. -- Sincerely yours, Sergey Lukjanov Sahara Technical Lead (OpenStack Data Processing) Principal Software Engineer Mirantis Inc. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Elizabeth Krumbach Joseph || Lyz || pleia2 __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] TC Candidacy
confirmed On Wed, Apr 22, 2015 at 4:17 PM, Maru Newby ma...@redhat.com wrote: My name is Maru Newby, and I am announcing my candidacy for the Technical Committee (TC) election. tl;dr; I'm a relative unknown outside of Neutron, but I've been helping to drive high-level change in that project. I would welcome the opportunity to bring my energy and dedication to OpenStack-wide concerns as a member of our TC. If you're willing to read any part of this email, I hope it would be 'Why vote for me?'. If not, and you are as frustrated with the status quo as I am, I hope you will consider voting for candidates that support the goal of building a more engaged TC focused on ensuring that participation in OpenStack becomes more enjoyable and sustainable. Thank you for reading, and make sure to vote! * Why vote for me? If elected, I'm going to be a squeaky wheel advocating for anything and everything that empowers our development community to deliver useful software into the hands of users. If this puts me at odds with those that want to focus on issues like 'What is OpenStack' or 'How can we systematize project culture to make it easier to communicate?', good! I believe that you, as a technical contributor to OpenStack, need stronger representation of your viewpoints on our TC, and I'm one of the strongest advocates you're likely to find in this community. I am currently employed full time upstream, and I am willing and able to devote the resources necessary to do justice to the role. I also intend to advocate for focusing the OpenStack community on ambitious, but achievable, goals. I believe this will require making hard choices about which use cases we can reasonably expect to support. To me, the idea that OpenStack can be all things to all people is dangerous. It's just as likely that in trying to do it all, we do little of it well, and risk irrelevance. I think our primary competition is and will continue to be proprietary cloud, and my biggest fear is that we become unable to compete on cost due to the operational complexity required to make everyone happy. We need to keep our collective eyes on the ball! To be clear, I want to be a part of a TC with as diverse a group of viewpoints as possible to ensure that we have to work hard to achieve consensus. If agreement is reached too easily, there is probably something wrong. My wanting to push a developer-centric agenda doesn't mean that I don't respect the need to address other issues. My voice would be one of many, and I recognize that consensus requires compromise. * Why not vote for me? There are candidates more deserving of your vote if you think our TC shouldn't have a role in providing leadership to the OpenStack community. If you believe that our TC is already sufficiently developer-focused, then I also don't deserve your vote. This is in no way to suggest that I want to see our TC become a top-down decision-making body. This is still an open source project, after all, and I know first-hand the complexities of the culture involved. I do think it appropriate, though, for an elected body with as much visibility as our TC to participate more actively in addressing the challenges we face. * Key concerns There are any number of issues facing OpenStack today, but the items on the shortlist that follows are the concerns that I would like to see the TC champion in the near-term: ** Scaling our development effort The stated goal of our TC of late has been to 'get out of the way'. I wholeheartedly support this intention, and would like to see it extended to how our TC approaches governance beyond project evaluation. I think our TC should be responsible for proposing what we want to achieve rather than in how we should achieve it. Outcomes, rather than process. I think project-level contributors are best equipped to solve the problems of implementation. Our TC can take responsibility for setting OpenStack-wide expectations and in facilitating - rather than managing - cross-project advancement towards these goals. More than ever, we need to be pulling in the same direction, and I think our TC is an obvious candidate to rally ourselves around. I hope we all recognize that we can't scale OpenStack if decisions are constantly gated on a small group of 'tastemakers'. Delegation of trust is required, whether at the level of our TC or the individual projects. I think we need to take our TC's recent momentum to the project level and find ways to increase delegation of responsibility. Nova and Neutron, for example, have had to face ever-increasing demands with the same small pool of core reviewers, and many of you know all-too-well the pain that has resulted. Core reviewers - like the pre-big tent TC - have inadvertently become micromanagers as their responsibilities have evolved beyond their capacity to meet them. The
Re: [openstack-dev] TC Candidacy
confirmed On Wed, Apr 22, 2015 at 5:56 PM, David Lyle dkly...@gmail.com wrote: I'm announcing my candidacy for the Technical Committee elections. I have been contributing to OpenStack since Grizzly primarily in Horizon. I have also had the privilege to serve as Horizon PTL since Icehouse. Why I'm running: I believe there should be broader representation on the TC. We are growing the OpenStack ecosystem. Let's make sure horizontal teams and diverse parts of the ecosystem are represented more directly. I understand concerns of scaling were the reason for moving from the TC made up of all PTLs (I question that assertion), but the sacrifice so far has been diversity. I feel current TC members are exceptionally capable and take a broad viewpoint, but there are limits of how well that works. Let's represent broader swaths of our ecosystem in the technical leadership. I think growing the OpenStack ecosystem is fantastic. As a developer and the PTL of a project that tries to span across most of that ecosystem it also worries me a bit too. I think we need to focus on how the newer and older parts of our ecosystem work together. How do we manage all the horizontal needs this introduces without going to the extremes of just scaling existing horizontal efforts because that won't work. And pushing all horizontal work on the individual projects is not appropriate because that yields chaos. Finally, I believe the TC needs to be more active in guiding overall direction of OpenStack and problem resolution. I'm not suggesting a dictatorship of course. But let's set a direction, overall release goals for OpenStack and help enable and drive them. I'm really proud to be a part of the OpenStack developer community, but I think we're facing some real challenges. We need to address some primary issues or this community will struggle to remain the vibrant, supportive place it is now. Thank you, David __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Elizabeth Krumbach Joseph || Lyz || pleia2 __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Neutron] A big tent home for Neutron backend code
On Wed, Apr 22, 2015 at 7:44 PM, Armando M. arma...@gmail.com wrote: On 22 April 2015 at 11:19, Russell Bryant rbry...@redhat.com wrote: Hello! A couple of things I've been working on lately are project governance issues as a TC member and also implementation of a new virtual networking alternative with a Neutron driver. So, naturally I started thinking about how the Neutron driver code fits in to OpenStack governance. There are basically two areas with a lot of movement related to this issue. 1) Project governance has moved to a big tent model [1]. The vast majority of projects that used to be in Stackforge are being folded in to a larger definition of the OpenStack project. Projects making this move meet the following criteria as being one of us: Is it sensible to assume that Stackforge is going away entirely at some point in the future, and we'll have a single namespace - OpenStack? IMHO, I'm not sure what the StackForge designation means anymore now that we have the Big Tent. I obviously also don't have the authority to make the call on when it goes away however. http://governance.openstack.org/reference/new-projects-requirements.html Official project teams are tracked in this file along with the git repos they are responsible for: http://git.openstack.org/cgit/openstack/governance/tree/reference/projects.yaml which is also reflected here: http://governance.openstack.org/reference/projects/ The TC has also been working through defining a system to help differentiate efforts by using a set of tags [4]. So far, we have tags describing the release handling for a repository, as well as a tag for team diversity. We've also had a lot of discussion about tags to help describe maturity, but that is still a work in progress. 2) In Neutron, some fairly significant good changes are being made to help scale the development process. Advanced services were split out into their own repos [2]. Most of the plugin and driver code has also been split out into repos [3]. This is too still a work in progress. A lot has been achieved during the Kilo timeframe, but more is still to be done, like the 'lib-ification - if that's even a word' of Neutron [1a], the split of plugins out of the *aas drivers [2b], and the rest of the core-vendor-decomp (latest status update is available in [3b]) Don't forget about the split out of the tree of the reference implementation, defined here: https://review.openstack.org/#/c/176501/ [1a] https://review.openstack.org/#/c/154736/ [2b] https://review.openstack.org/#/c/174619/ [3b] https://review.openstack.org/#/c/173549/ In terms of project teams, the Neutron team is defined as owning the following repos: http://governance.openstack.org/reference/projects/neutron.html - openstack/neutron - openstack/neutron-fwaas - openstack/neutron-lbaas - openstack/neutron-vpnaas - openstack/neutron-specs - openstack/python-neutronclient The advanced services split is reflected by the fwaas, lbaas, and vpnaas repos. We also have a large set of repositories related to Neutron backend code: http://git.openstack.org/cgit/?q=stackforge%2Fnetworking - stackforge/networking-arista - stackforge/networking-bagpipe-l2 - stackforge/networking-bgpvpn - stackforge/networking-bigswitch - stackforge/networking-brocade - stackforge/networking-cisco - stackforge/networking-edge-vpn - stackforge/networking-hyperv - stackforge/networking-ibm - stackforge/networking-l2gw - stackforge/networking-midonet - stackforge/networking-mlnx - stackforge/networking-nec - stackforge/networking-odl - stackforge/networking-ofagent - stackforge/networking-ovn - stackforge/networking-ovs-dpdk - stackforge/networking-plumgrid - stackforge/networking-portforwarding - stackforge/networking-vsphere Note that not all of these are equivalent. This is just a list of stackforge/networking-*. In some cases there is a split between code in the Neutron tree and in this repo. In those cases, a shim is in the Neutron tree, but most of the code is in the external repo. It's also possible to have all of the code in the external repo. There's also a big range of maturity. Some are quite mature and are already used in production. networking-ovn as an example is quite new and being developed in parallel with OVN in the Open vSwitch project. So, my question is: Where should these repositories live in terms of OpenStack governance and project teams? It's my understanding that StackForge projects are bound to the same governance model, or am I mistaken? I'm not sure they are. Here are a few paths I think we could take, along with some of my initial thoughts on pros/cons. a) Adopt these as repositories under the Neutron project team. I fully understand how this is implemented in, can you elaborate? Would that translate into something like [4a]? The other concern I have is that the
Re: [openstack-dev] [nova] Required data migrations in Kilo, need Turbo Hipster tests updated
On 4/23/15 1:31 AM, Dan Smith wrote: Sure, but for people doing continuous deployment, they clearly haven't ran the migrate_flavor_data (or if they have, they haven't filed any bugs about it not working[0]). Hence the usefulness of T-H here, right? The point of the migration check is to make sure that people _do_ run it before the leave kilo. Right now, they have nothing other than the big note in the release notes about doing it. Evidence seems to show that they're not seeing it, which is exactly why we need the check :) I also found what I believe to be a bug with the flavor migration code. I started working on a fix by my limited knowledge of nova's objects has hindered me. Any thoughts on the legitimacy of the bug would be helpful though: https://bugs.launchpad.net/nova/+bug/1447132 . Basically for some of the datasets that turbo-hipster uses there are no entries in the new instance_extra table stopping any flavor migration from actually running. Then in your change (174480) you check the metadata table instead of the extras table causing the migration to fail even though migrate_flavor_data thinks there is nothing to do. I don't think this has anything to do with the objects, it's probably just a result of my lack of sqlalchemy-fu. Sounds like you weren't close to a fix, so I'll try to cook something up. Yeah it was my sqlalchemy-fu letting me down too. However I mentioned the objects because I was down a rabbit hole trying to figure out all of the code surrounding loading flavours from either system-metadata or extras. If I selected all the instance_type_id's from the system-metadata table and used those uuid's to load the object with something like: instance = objects.Instance.get_by_uuid( context, instance_uuid, expected_attrs=['system_metadata', 'flavor']) The tests would fail at that point when trying to read in the flavor as json. http://paste.openstack.org/show/205158/ Basically without digging further it seems like I should be able to load an instance by uuid regardless of the state of my flavor(s). Since this fails it seems like there is something wrong with the flavor handling on the objects. So, a question here: These data sets were captured at some point in time, right? Does that mean that they were from say Icehouse era and have had nothing done to them since? Meaning, are there data sets that actually had juno or kilo running on them? This instance_extra thing would be the case for any instance that hasn't been touched in a long time, so it's legit. However, as we move to more online migration of data, I do wonder if taking an ancient dataset, doing schema migrations forward to current and then expecting it to work is really reflective of reality. Can you shed some light on what is really going on? As mikal has followed up, that's correct. We have intended to refresh our datasets and will try and get some more recent ones, but testing the old ones has also proven useful. Another interesting thing is that migrate_flavor_data avoids migrating instances that are in the middle of an operation. The snapshot of databases we have include instances in this state. Since turbo-hipster is just testing against a snapshot in time there is no way for those instances to leave their working state and hence migrate_flavor_data can never finish (every run will leave instances undone). This therefore blocks the migrations from ever finishing. I don't think this is a real world problem though, so once we get this migration closer to merging we might have to force the instances to be migrated in our datasets. In fact, that could be a useful feature so I wrote it here: https://review.openstack.org/#/c/176574/ Cheers, Josh I think it's worth noting that your change (174480) will require operators (particularly continuous deployers) to run migrate_flavor_data and given the difficulties I've found I'm not sure it's ready to be ran. Right, that's the point. If we encounter bugs against real datasets with migrate_flavor_data then deployers will be unable to upgrade until migrate_flavor_data is fixed. This may make things awkward if a deployer updates their codebase and can't run the migrations. Clearly they'll have to roll-back the changes. This is the scenario turbo-hipster is meant to catch - migrations that don't work on real datasets. Right, that's why we're holding until we make sure that it works. If migrate_flavor_data is broken a backport into Kilo will be needed so that if Liberty requires all the flavor migrations to be finished they can indeed be ran before upgrading to Liberty. This may give reason not to block on having the flavors migrated until the M-release but I realise that has other undersiable consequences (ie high code maintenance). Backports to fix this are fine IMHO, and just like any other bug found in actual running of things. It's too bad that none of our continuous deployment people seem to have found
Re: [openstack-dev] [Neutron] A big tent home for Neutron backend code
On Thu, Apr 23, 2015 at 3:30 AM, Kyle Mestery mest...@mestery.com wrote: On Wed, Apr 22, 2015 at 1:19 PM, Russell Bryant rbry...@redhat.com wrote: Hello! A couple of things I've been working on lately are project governance issues as a TC member and also implementation of a new virtual networking alternative with a Neutron driver. So, naturally I started thinking about how the Neutron driver code fits in to OpenStack governance. Thanks for starting this conversation Russell. There are basically two areas with a lot of movement related to this issue. 1) Project governance has moved to a big tent model [1]. The vast majority of projects that used to be in Stackforge are being folded in to a larger definition of the OpenStack project. Projects making this move meet the following criteria as being one of us: http://governance.openstack.org/reference/new-projects-requirements.html Official project teams are tracked in this file along with the git repos they are responsible for: http://git.openstack.org/cgit/openstack/governance/tree/reference/projects.yaml which is also reflected here: http://governance.openstack.org/reference/projects/ The TC has also been working through defining a system to help differentiate efforts by using a set of tags [4]. So far, we have tags describing the release handling for a repository, as well as a tag for team diversity. We've also had a lot of discussion about tags to help describe maturity, but that is still a work in progress. 2) In Neutron, some fairly significant good changes are being made to help scale the development process. Advanced services were split out into their own repos [2]. Most of the plugin and driver code has also been split out into repos [3]. In terms of project teams, the Neutron team is defined as owning the following repos: http://governance.openstack.org/reference/projects/neutron.html - openstack/neutron - openstack/neutron-fwaas - openstack/neutron-lbaas - openstack/neutron-vpnaas - openstack/neutron-specs - openstack/python-neutronclient The advanced services split is reflected by the fwaas, lbaas, and vpnaas repos. We also have a large set of repositories related to Neutron backend code: http://git.openstack.org/cgit/?q=stackforge%2Fnetworking - stackforge/networking-arista - stackforge/networking-bagpipe-l2 - stackforge/networking-bgpvpn - stackforge/networking-bigswitch - stackforge/networking-brocade - stackforge/networking-cisco - stackforge/networking-edge-vpn - stackforge/networking-hyperv - stackforge/networking-ibm - stackforge/networking-l2gw - stackforge/networking-midonet - stackforge/networking-mlnx - stackforge/networking-nec - stackforge/networking-odl - stackforge/networking-ofagent - stackforge/networking-ovn - stackforge/networking-ovs-dpdk - stackforge/networking-plumgrid - stackforge/networking-portforwarding - stackforge/networking-vsphere Note that not all of these are equivalent. This is just a list of stackforge/networking-*. In some cases there is a split between code in the Neutron tree and in this repo. In those cases, a shim is in the Neutron tree, but most of the code is in the external repo. It's also possible to have all of the code in the external repo. There's also a big range of maturity. Some are quite mature and are already used in production. networking-ovn as an example is quite new and being developed in parallel with OVN in the Open vSwitch project. So, my question is: Where should these repositories live in terms of OpenStack governance and project teams? Here are a few paths I think we could take, along with some of my initial thoughts on pros/cons. a) Adopt these as repositories under the Neutron project team. In this case, I would see them operating with their own review teams as they do today to avoid imposing additional load on the neutron-core or neutron-specs-core teams. However, by being a part of the Neutron team, the backend team would submit to oversight by the Neutron PTL. Out of your options proposed, this seems like the most logical one to me. I don't really see this imposing a ton of strain on the existing core reviewer team, because we'll keep whatever core reviewer teams are already in the networking-foo projects. There are some other details to work out to ensure expectations are clearly set for everyone involved. If this is the path that makes sense, we can work through those as a next step. Pros: + Seems to be the most natural first choice Cons: - A lot of changes have been made precisely because Neutron has gotten so big. A single project team/PTL may not be able to effectively coordinate all of the additional work. Maybe the core Neutron project would be better off focusing on being a platform, and other project teams organize work on backends. It's interesting you mention neutron being a platform, because
[openstack-dev] TC Candidacy
I'm announcing my candidacy for the Technical Committee elections. I have been contributing to OpenStack since Grizzly primarily in Horizon. I have also had the privilege to serve as Horizon PTL since Icehouse. Why I'm running: I believe there should be broader representation on the TC. We are growing the OpenStack ecosystem. Let's make sure horizontal teams and diverse parts of the ecosystem are represented more directly. I understand concerns of scaling were the reason for moving from the TC made up of all PTLs (I question that assertion), but the sacrifice so far has been diversity. I feel current TC members are exceptionally capable and take a broad viewpoint, but there are limits of how well that works. Let's represent broader swaths of our ecosystem in the technical leadership. I think growing the OpenStack ecosystem is fantastic. As a developer and the PTL of a project that tries to span across most of that ecosystem it also worries me a bit too. I think we need to focus on how the newer and older parts of our ecosystem work together. How do we manage all the horizontal needs this introduces without going to the extremes of just scaling existing horizontal efforts because that won't work. And pushing all horizontal work on the individual projects is not appropriate because that yields chaos. Finally, I believe the TC needs to be more active in guiding overall direction of OpenStack and problem resolution. I'm not suggesting a dictatorship of course. But let's set a direction, overall release goals for OpenStack and help enable and drive them. I'm really proud to be a part of the OpenStack developer community, but I think we're facing some real challenges. We need to address some primary issues or this community will struggle to remain the vibrant, supportive place it is now. Thank you, David __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [QA] Meeting Thursday April 23rd at 17:00 UTC
Hi everyone, Just a quick reminder that the weekly OpenStack QA team IRC meeting will be tomorrow Thursday, April 23rd at 17:00 UTC in the #openstack-meeting channel. The agenda for tomorrow's meeting can be found here: https://wiki.openstack.org/wiki/Meetings/QATeamMeeting Anyone is welcome to add an item to the agenda. To help people figure out what time 17:00 UTC is in other timezones tomorrow's meeting will be at: 13:00 EDT 02:00 JST 02:30 ACST 19:00 CEST 12:00 CDT 10:00 PDT -Matt Treinish pgpVQp5eLdkdX.pgp Description: PGP signature __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Horizon] Outreachy call for mentor
Hi David, Thanks a lot for your quick response! I'll give you more details about the applicant in a follow up email or in IRC. Certainly having separate wikis is not practical sometimes, so Stefano has been working on centralize all the information wrt internships and mentoring opportunities in OpenStack. There is a new mailing list you can subscribe to http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-internships. Again, thanks. Victoria 2015-04-22 21:51 GMT-03:00 David Lyle dkly...@gmail.com: I added my name to an openstack wiki page for this express purpose some time ago, apparently the wrong one, which I can find no reference to now. That said, I am willing to be a mentor for a Horizon focused intern, if inability to find the correct wiki pages isn't a limiting factor. David On Wed, Apr 22, 2015 at 6:31 PM, Victoria Martínez de la Cruz victo...@vmartinezdelacruz.com wrote: Hi all, Horizon folks, we have a really good Outreachy candidate interested in working with you. The internship is from May 25th to August 25th, and interns are expected to work full time on their projects. Being a mentor should not be time consuming, we expect interns to be able to do their tasks independently and to solve the blockers they might find with the help of the community. I will be mentoring an applicant this round and, if time is a problem, I offer to help with this applicant as well. The announcement of accepted participants is soon, so this is a real last minute call. Outreachy is an internship targeted to anyone who identifies as a woman and OpenStack has been participating on it for about two years now. We had really good participants and many important features had been added as part of this program. For more details on OpenStack participation on Outreachy, check out the OpenStack Outreachy wiki page [0] and the Outreachy official site [1]. If you decide you want to mentor, please reach me and I'll guide you through the process. Please, let me know if you have any doubt or concern. Thanks all, Victoria [0] https://wiki.openstack.org/wiki/Outreachy [1] https://wiki.gnome.org/Outreachy __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] TC candidacy
All- I'd like to announce my candidacy to continue serving on the Technical Committee. Platform — OpenStack is a growing community comprised of many parts and we we must view ourselves as one unit. As a TC member, I will continue to place the interests of the larger community over those of those of individual projects. There are several key areas I'd like to see the TC focus: Development The Technical Committee should serve as a high level forum to facilitate defining cross project technical and procedural requirements. While many of our programs share commonalities, there are still too many differences in policies and technical decisions. The addition of cross project specifications is a step towards unifying the development process and design standards within our community. As we accept more projects under the new governance model, the TC should work to facilitate developer mobility across projects by promoting best practices. Increased mobility will strengthen our community as it helps to prevent silos. Reviews are an another area the TC should focus on during the upcoming cycle. The TC should review and work with projects to improve the contributor and reviewer experience when contributing to projects both big and small. Cross Project Communication Our codebase has grown significantly over the years and a contributor must invest significant time to understand and follow every project; however many contributors have limited time must choose a subset of projects to direct their focus. As a result, it becomes important to employ cross project communication to explain major technical decisions, share solutions to project challenges that might be widely applicable, and leverage our collective experience. The TC should seek new ways to facilitate cross project communication that will enable the community to craft improvements to the interfaces between projects as there will be greater familiarity between across the boundary. Unified Experience For OpenStack to be successful we must strive to provide a unified experience for both users and deployers. During the previous cycles we have made progress in this area, but there is still work to be completed. When visiting user groups, a common thread during discussion is the installation experience. The TC should identify common installation configurations and work with projects to optimize installation and upgrades. Equally important is the users. The TC should work to promote and support projects such the OpenStack client and SDK which aim to provide users with tools that are well maintained, documented and easy to use. Our community has invested significant effort to improve this experience and this should remain a focus going forward. About Me —— I have served on the Technical Committee for last two years and I am a former PTL. Believing that OpenStack is a unified community, I have contributed code and reviews to many of our projects since Sent from my iPad We have built a very special open community through the contributions of many. These contributions have powered our phenomenal growth and I'm excited about our future! Thanks, mark __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] Barbican : Dependency of pyenv configuration in Barbican.sh script
Hi All,
I would like to know the reason behind the dependency of the pyenv virtual
environment and pyenv in the barbican.sh script.
Ideally in the production environment , barbican would run on standalone
virtual box with a particular python version .I feel that their dependecies
needs to be removed from the script.
Was able to stand up the barbican instance without configuring pyenv and
pyenv-virtualenv dependencies by modifying the barbican script ,
installing few additional packages and exporting the python path to PATH
variable
Please find the change in barbican.sh script for installation and starting
of the script below :
VENV_DIR=${VIRTUAL_ENV:-`pyenv prefix`} - *This line needs to be removed *
uwsgi --master --emperor $CONFIG_DIR/vassals* -H* *$VENV_DIR - The
**$VENV_DIR
variable need to be removed as an argument and -H as an option.*
The barbican script has been tied to $VENV_DIR variable which is dependent
on the pyenv for python configuration.Hence the barbican.sh script needs
to be modified to remove *$VENV_DIR variable *by configuring python path
in PATH variable.
On doing this , we can avoid the sourcing the pyenv and pyenv-virtualenv
packages and its dependices on Barbican script.
Any help would be highly appreciated and also would like to know opinion
from the openstack group on the changes indicated
Thanks in advance
*Thanks and Regards,*
*Asha Seshagiri*
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Barbican : Dependency of pyenv configuration in Barbican.sh script
Hello Asha,
The barbican.sh script was originally intended to be a convenient way to boot
up a Barbican instance locally to quickly start evaluating its API and
functionality.
It was not intended to be used as a production script, deferring instead to
deployments utilizing packages such as RDO RPMs and so forth for that purpose.
That said, changes to that script have been discussed, including removing pyenv
and uWSGI as dependencies, hence such changes would be good to consider.
I'd also note that a solution based on this recently added script [1] might be
in order.
Thanks,
John
[1] https://github.com/openstack/barbican/blob/master/bin/barbican-api
From: Asha Seshagiri asha.seshag...@gmail.commailto:asha.seshag...@gmail.com
Date: Wednesday, April 22, 2015 at 4:57 PM
To: openstack-dev
openstack-dev@lists.openstack.orgmailto:openstack-dev@lists.openstack.org
Cc: John Wood john.w...@rackspace.commailto:john.w...@rackspace.com,
Reller, Nathan S.
nathan.rel...@jhuapl.edumailto:nathan.rel...@jhuapl.edu, Douglas Mendizabal
douglas.mendiza...@rackspace.commailto:douglas.mendiza...@rackspace.com,
Paul Kehrer paul.keh...@rackspace.commailto:paul.keh...@rackspace.com, Adam
Harwell adam.harw...@rackspace.commailto:adam.harw...@rackspace.com, Alexis
Lee alex...@hp.commailto:alex...@hp.com,
nut...@gmail.commailto:nut...@gmail.com
nut...@gmail.commailto:nut...@gmail.com
Subject: Barbican : Dependency of pyenv configuration in Barbican.sh script
Hi All,
I would like to know the reason behind the dependency of the pyenv virtual
environment and pyenv in the barbican.sh script.
Ideally in the production environment , barbican would run on standalone
virtual box with a particular python version .I feel that their dependecies
needs to be removed from the script.
Was able to stand up the barbican instance without configuring pyenv and
pyenv-virtualenv dependencies by modifying the barbican script , installing
few additional packages and exporting the python path to PATH variable
Please find the change in barbican.sh script for installation and starting of
the script below :
VENV_DIR=${VIRTUAL_ENV:-`pyenv prefix`} - This line needs to be removed
uwsgi --master --emperor $CONFIG_DIR/vassals -H $VENV_DIR - The $VENV_DIR
variable need to be removed as an argument and -H as an option.
The barbican script has been tied to $VENV_DIR variable which is dependent on
the pyenv for python configuration.Hence the barbican.sh script needs to be
modified to remove $VENV_DIR variable by configuring python path in PATH
variable.
On doing this , we can avoid the sourcing the pyenv and pyenv-virtualenv
packages and its dependices on Barbican script.
Any help would be highly appreciated and also would like to know opinion from
the openstack group on the changes indicated
Thanks in advance
Thanks and Regards,
Asha Seshagiri
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [neutron][QoS] service-plugin or not discussion
On 22 April 2015 at 06:02, Miguel Angel Ajo Pelayo mangel...@redhat.com wrote: Hi everybody, In the latest QoS meeting, one of the topics was a discussion about how to implement QoS [1] either as in core, or as a service plugin, in, or out-tree. My apologies if I was unable to join, the meeting clashed with another one I was supposed to attend. It’s my feeling, and Mathieu’s that it looks more like a core feature, as we’re talking of port properties that we define at high level, and most plugins (QoS capable) may want to implement at dataplane/controlplane level, and also that it’s something requiring a good amount of review. In the other hand Irena and Sean were more concerned about having a good separation of concerns (I agree actually with that part), and being able to do quicker iterations on a separate stackforge repo. Perhaps we're trying to address the issue at the wrong time. Once a reasonable agreement has been reached on the data model, and the API, whether we're going with a service plugin or core etc should be an implementation detail. I think the crux of the matter is the data plane integration. From a management and control standpoint it should be fairly trivial to expose/implement the API and business logic via a service plugin and, and some of you suggested, integrate with the core via callbacks. However, I am pretty sure there will be preliminary work necessary to integrate the server with the agent fabric (when there is one) so that is no longer a pain. Extending what the agent can do the way we did so far (e.g. by adding extra payloads/messages, mixin etc) is not sustainable, and incredibly brittle. Since we didn’t seem to find an agreement, and I’m probably missing some details, I’d like to loop in our core developers and PTL to provide an opinion on this. [1] http://eavesdrop.openstack.org/meetings/neutron_qos/2015/neutron_qos.2015-04-21-14.03.log.html#l-192 Thanks for your patience, Miguel Angel Ajo __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] TC candidacy
confirmed On Wed, Apr 22, 2015 at 1:13 PM, Armando M. arma...@gmail.com wrote: I would like to announce my candidacy for the OpenStack Technical Committee. I will try to be brief and to the point: I have been involved in OpenStack since the early days of the Austin release; I have worked on (perhaps) the two most prolific projects in OpenStack (Nova, and Neutron) and a few others aimed at ensuring their quality (Tempest, DevStack, and the infra ones). I have been mainly a developer, but also a deployer, distributor, user, tech writer, you name it. Under these points of views I have seen friction in dealing with OpenStack increase over time, and I want to do something about it. I have never run for the TC before, or any other position of 'influence'. I came to the realization that doing something about it by being on the fringes could only get me so far. My main goal is to empower the developer, allowing him/her to go at the pace he/she is comfortable with, whilst preserving the quality of software being produced: I believe we have many bottlenecks and inefficiencies in the way we develop and consume OpenStack technologies. In Neutron I, with the help of others, have tried to identify these and proactively did something about it: decomposing the codebase in core vs plugins and moving away from the project reliance on Tempest to ensure stability are examples to name a few. At the same time decentralizing and increasing speed of development can impair coherence and cohesion of the overall solution and that is why I think a seat in the TC would give me enough exposure to help preserve, and strive to achieve these qualities in the software we build. The OpenStack ecosystem is incredibly diverse and yet we need each technology developed under its umbrella to share a lot more than the four Opens. The definition of shared methodologies, practices and guidelines can help us achieve that, but most importantly a shared roadmap that can let us peek into the future of OpenStack as a whole rather than a fragmented collection of services that work poorly together. Thanks for reading. Regards, Armando __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Elizabeth Krumbach Joseph || Lyz || pleia2 __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Neutron] A big tent home for Neutron backend code
I prefer option a) as well. It does seem like most of the projects would really see no change at all other than being officially sanctioned as an Openstack project with some kind of tag. There's also the possibility the PTL may request some changes to improve the bigger picture of the Networking/Neutron project. Other than that it sounds like nothing much would change. Is this to solve the whole issue regarding projects graduating to become openstack projects? If so, it sounds like those issues might just be shuffled to the decision of whether a project should graduate to a better tag. I'm sure this has come up in the tags discussions, and its a bit out of scope of this email, but it's just a concern I have. Thanks, Brandon From: Fox, Kevin M kevin@pnnl.gov Sent: Wednesday, April 22, 2015 2:49 PM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [Neutron] A big tent home for Neutron backend code +1 I was in the middle of writing an email asking about the possibility of splitting out the reference implementation. you beat me to it. :) I also like the idea of having the PTL on top to help pass up/down ideas where code can be shared, benefiting everyone. Thanks, Kevin From: Kyle Mestery [mest...@mestery.com] Sent: Wednesday, April 22, 2015 12:30 PM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [Neutron] A big tent home for Neutron backend code On Wed, Apr 22, 2015 at 1:19 PM, Russell Bryant rbry...@redhat.commailto:rbry...@redhat.com wrote: Hello! A couple of things I've been working on lately are project governance issues as a TC member and also implementation of a new virtual networking alternative with a Neutron driver. So, naturally I started thinking about how the Neutron driver code fits in to OpenStack governance. Thanks for starting this conversation Russell. There are basically two areas with a lot of movement related to this issue. 1) Project governance has moved to a big tent model [1]. The vast majority of projects that used to be in Stackforge are being folded in to a larger definition of the OpenStack project. Projects making this move meet the following criteria as being one of us: http://governance.openstack.org/reference/new-projects-requirements.html Official project teams are tracked in this file along with the git repos they are responsible for: http://git.openstack.org/cgit/openstack/governance/tree/reference/projects.yaml which is also reflected here: http://governance.openstack.org/reference/projects/ The TC has also been working through defining a system to help differentiate efforts by using a set of tags [4]. So far, we have tags describing the release handling for a repository, as well as a tag for team diversity. We've also had a lot of discussion about tags to help describe maturity, but that is still a work in progress. 2) In Neutron, some fairly significant good changes are being made to help scale the development process. Advanced services were split out into their own repos [2]. Most of the plugin and driver code has also been split out into repos [3]. In terms of project teams, the Neutron team is defined as owning the following repos: http://governance.openstack.org/reference/projects/neutron.html - openstack/neutron - openstack/neutron-fwaas - openstack/neutron-lbaas - openstack/neutron-vpnaas - openstack/neutron-specs - openstack/python-neutronclient The advanced services split is reflected by the fwaas, lbaas, and vpnaas repos. We also have a large set of repositories related to Neutron backend code: http://git.openstack.org/cgit/?q=stackforge%2Fnetworking - stackforge/networking-arista - stackforge/networking-bagpipe-l2 - stackforge/networking-bgpvpn - stackforge/networking-bigswitch - stackforge/networking-brocade - stackforge/networking-cisco - stackforge/networking-edge-vpn - stackforge/networking-hyperv - stackforge/networking-ibm - stackforge/networking-l2gw - stackforge/networking-midonet - stackforge/networking-mlnx - stackforge/networking-nec - stackforge/networking-odl - stackforge/networking-ofagent - stackforge/networking-ovn - stackforge/networking-ovs-dpdk - stackforge/networking-plumgrid - stackforge/networking-portforwarding - stackforge/networking-vsphere Note that not all of these are equivalent. This is just a list of stackforge/networking-*. In some cases there is a split between code in the Neutron tree and in this repo. In those cases, a shim is in the Neutron tree, but most of the code is in the external repo. It's also possible to have all of the code in the external repo. There's also a big range of maturity. Some are quite mature and are already used in production. networking-ovn as an example is quite new and being developed in parallel with OVN in the Open vSwitch project. So, my question is:
Re: [openstack-dev] [Nova][Neutron] Cross-project coordination
On 04/22/2015 04:33 PM, Kyle Mestery wrote: On Wed, Apr 22, 2015 at 3:28 PM, Sean M. Collins s...@coreitpro.com mailto:s...@coreitpro.com wrote: Hi, Kyle Mestery has asked me to serve as a cross-project liaison between Neutron and Nova. So, I'd like to say hello, and direct everyone towards an etherpad that I have created. https://etherpad.openstack.org/p/nova-neutron The etherpad can serve as a way to collect items that should be discussed between the projects. I will be attending the Nova main meetings to field Neutron questions, or at least provide who on the Neutron side would know the answer to a question. This is a big job, and I'd like to see if there is a volunteer on the Nova side who would be interested in also helping this effort, who could do the reverse (Sit in on Neutron meetings, field Nova questions). Thank you, and I look forward to working with everyone! Sean, thank you so much for volunteering to take on this incredibly important job. I'm hoping we can get someone from the nova side to work hand-in-hand with you and ensure we continue to drive issues related to both nova and neutron to conclusion in a way which benefits are mutual users! Agreed. I'm hoping that someone in the Nova community -- note, this does not need to be a Nova core contributor -- can step up to the plate and serve in this critical role. Best, -jay __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [Murano] python-openstackclient support
Since python-openstackclient is now a part of openstack — I guess it would be a good idea to support it in murano. It has setuptools-based plugin system, and it should be fairly easy to add murano commands as plugins to it. BTW, It’s based on cliff and has a terrific completion support (which is basically why I started looking into the issue in the first place =)) What do you think, is it a good idea? -- Kirill Zaitsev Sent with Airmail__ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Manila] Two nominations for Manila Core Reviewer Team
+1 to both. From: Ben Swartzlander [mailto:b...@swartzlander.org] Sent: Wednesday, April 22, 2015 2:23 PM To: OpenStack Development Mailing List (not for usage questions) Subject: [openstack-dev] [Manila] Two nominations for Manila Core Reviewer Team I would like to nominate Thomas Bechtold to join the Manila core reviewer team. Thomas has been contributing to Manila for close to 6 months and has provided a good number of quality code reviews in addition to a substantial amount of contributions. Thomas brings both Oslo experience as well as a packager/distro perspective which is especially helpful as Manila starts to get used in more production scenarios. I would also like to nominate Mark Sturdevant. He has also been active in the community for about 6 months and has a similar history of code reviews. Mark is the maintainer of the HP driver and would add vendor diversity to the core team. -Ben Swartzlander Manila PTL __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [oslo][policy][neutron] oslo.policy API is not powerful enough to switch Neutron to it
On 22 April 2015 at 14:49, Doug Hellmann d...@doughellmann.com wrote: Excerpts from Ihar Hrachyshka's message of 2015-04-22 12:33:52 +0200: On 04/22/2015 05:01 AM, Doug Hellmann wrote: Excerpts from Ihar Hrachyshka's message of 2015-04-17 14:45:58 +0200: Hi, tl;dr neutron has special semantics for policy targets that relies on private symbols from oslo.policy, and it's impossible to introduce this semantics into oslo.policy itself due to backwards compatibility concerns, meaning we need to expose some more symbols as part of public API for the library to facilitate neutron switch to it. === oslo.policy was graduated during Kilo [1]. Neutron considered the switch to it [2], but failed to achieve it because some library symbols that were originally public (or at least looked like public) in policy.py from oslo-incubator, became private in oslo.policy. Specifically, Neutron policy code [3] relies on the following symbols that are now hidden inside oslo_policy._checks (note the underscore in the name of the module that suggests we cannot use the module directly): - - RoleCheck - - RuleCheck - - AndCheck I'm not sure I followed all of the rest of what you wrote below, but it seems like this is the real problem. We are pretty aggressive about making things private when we release a new library, because it's easier to make them public later if we need to than it is to make public things private. In this case, it looks like we made some symbols private even though they were already being used, and that seems like a mistake on our part. So, if we make those public, would that address the issues with neutron adopting the library? Yes, that would help. I will also check Adam's suggestion, maybe it will allow us to avoid exposing RoleCheck. Keeping stuff private is less of a priority if we can demonstrate that having it public makes it more useful. Those symbols are used for the following matters: (all the relevant neutron code is in neutron/policy.py) 1. debug logging in case policy does not authorize an action (RuleCheck, AndCheck) [log_rule_list] 2. filling in admin context with admin roles (RoleCheck, RuleCheck, AndCheck/OrCheck internals) [get_admin_roles] 3. aggregating core, attribute and subattribute policies (RuleCheck, AndCheck) [_prepare_check] == 1. debug logging in case policy does not authorize an action == Neutron logs rules that failed to match if policy module does not authorize an action. Not sure whether Neutron developers really want to have those debug logs, and whether we cannot just kill them to avoid this specific usage of private symbols; though it also seems that we could easily use __str__ that is present for all types of Checks instead. So it does not look like a blocker for the switch. == 2. filling in admin context with admin roles == Admin context object is filled with .roles attribute that is a list of roles considered granting admin permissions [4]. The attribute would then be used by plugins that would like to do explicit policy checks. As per Salvatore, this attribute can probably be dropped now that all plugins and services don't rely on it (Salvatore mentioned lbaas mixins as the ones that previously relied on it, but are now not doing it since service split from neutron tree (?)). The problem with dropping the .roles attribute from context object in Liberty is that we, as a responsible upstream with lots of plugins maintained out-of-tree (see the ongoing vendor decomposition effort) would need to support the attribute while it's marked as deprecated for at least one cycle, meaning that if we don't get those oslo.policy internals we rely on in Liberty, we would need to postpone the switch till Mizzle, or rely on private symbols during the switch (while a new release of oslo.policy can easily break us). (BTW the code to extract admin roles is not really robust and has bugs, f.e. it does not handle AndChecks that could be used in context_is_admin. In theory, 'and' syntax would mean that both roles are needed to claim someone is an admin, while the code to extract admin roles handles 'and' the same way as 'or'. For the deprecation time being, we may need to document this limitation.) == 3. aggregating core, attribute and subattribute policies == That's the most interesting issue. For oslo.policy, policies are described as target: rule, where rule is interpreted as per registered checks, while target is opaque to the library. Neutron extended the syntax for target as: target[:attribute[:subattribute]]. This extension of the syntax is a bit more problematic. We should talk about a way to fold that into the library so it can be used consistently across projects. FTR,
Re: [openstack-dev] [nova] Policy rules are killed by the context admin check
On Wednesday, April 22, 2015, Matt Riedemann mrie...@linux.vnet.ibm.com wrote: On 4/22/2015 8:32 AM, Sylvain Bauza wrote: Hi, By discussing on a specific bug [1], I just discovered that the admin context check which was done at the DB level has been moved to the API level thanks to the api-policy-v3 blueprint [2] That behaviour still leads to a bug if the operator wants to change an endpoint policy by leaving it end-user but still continues to be denied because of that, as it will forbid any non-admin user to call the methods (even if authorize() grants the request) I consequently opened a bug [3] for this but I'm also concerned about the backportability of that and why it shouldn't fixed in v2.0 too. Releasing the check by removing it sounds an acceptable change, as it fixes a bug without changing the expected behaviour [4]. The impact of the change sounds also minimal with a very precise scope (ie. leave the policy rules work as they are expected) [5] Folks, thoughts ? -Sylvain [1] https://bugs.launchpad.net/nova/+bug/1447084 [2] https://review.openstack.org/#/q/project:openstack/nova+branch:master+topic:bp/v3-api-policy,n,z [3] https://bugs.launchpad.net/nova/+bug/1447164 [4] https://wiki.openstack.org/wiki/APIChangeGuidelines#Generally_Considered_OK Fixing a bug so that a request which resulted in an error response before is now successful [5] https://wiki.openstack.org/wiki/StableBranch#Stable_branch_policy __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev I don't disagree, see bug 1168488 from way back in grizzly. The only thing would be we'd have to make sure the default rule is admin for any v2 extensions which are enforcing an admin context today. This sounds like a sane approach. --Morgan -- Thanks, Matt Riedemann __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [oslo][policy][neutron] oslo.policy API is not powerful enough to switch Neutron to it
Excerpts from Salvatore Orlando's message of 2015-04-22 23:10:01 +0200: On 22 April 2015 at 14:49, Doug Hellmann d...@doughellmann.com wrote: Excerpts from Ihar Hrachyshka's message of 2015-04-22 12:33:52 +0200: On 04/22/2015 05:01 AM, Doug Hellmann wrote: Excerpts from Ihar Hrachyshka's message of 2015-04-17 14:45:58 +0200: Hi, tl;dr neutron has special semantics for policy targets that relies on private symbols from oslo.policy, and it's impossible to introduce this semantics into oslo.policy itself due to backwards compatibility concerns, meaning we need to expose some more symbols as part of public API for the library to facilitate neutron switch to it. === oslo.policy was graduated during Kilo [1]. Neutron considered the switch to it [2], but failed to achieve it because some library symbols that were originally public (or at least looked like public) in policy.py from oslo-incubator, became private in oslo.policy. Specifically, Neutron policy code [3] relies on the following symbols that are now hidden inside oslo_policy._checks (note the underscore in the name of the module that suggests we cannot use the module directly): - - RoleCheck - - RuleCheck - - AndCheck I'm not sure I followed all of the rest of what you wrote below, but it seems like this is the real problem. We are pretty aggressive about making things private when we release a new library, because it's easier to make them public later if we need to than it is to make public things private. In this case, it looks like we made some symbols private even though they were already being used, and that seems like a mistake on our part. So, if we make those public, would that address the issues with neutron adopting the library? Yes, that would help. I will also check Adam's suggestion, maybe it will allow us to avoid exposing RoleCheck. Keeping stuff private is less of a priority if we can demonstrate that having it public makes it more useful. Those symbols are used for the following matters: (all the relevant neutron code is in neutron/policy.py) 1. debug logging in case policy does not authorize an action (RuleCheck, AndCheck) [log_rule_list] 2. filling in admin context with admin roles (RoleCheck, RuleCheck, AndCheck/OrCheck internals) [get_admin_roles] 3. aggregating core, attribute and subattribute policies (RuleCheck, AndCheck) [_prepare_check] == 1. debug logging in case policy does not authorize an action == Neutron logs rules that failed to match if policy module does not authorize an action. Not sure whether Neutron developers really want to have those debug logs, and whether we cannot just kill them to avoid this specific usage of private symbols; though it also seems that we could easily use __str__ that is present for all types of Checks instead. So it does not look like a blocker for the switch. == 2. filling in admin context with admin roles == Admin context object is filled with .roles attribute that is a list of roles considered granting admin permissions [4]. The attribute would then be used by plugins that would like to do explicit policy checks. As per Salvatore, this attribute can probably be dropped now that all plugins and services don't rely on it (Salvatore mentioned lbaas mixins as the ones that previously relied on it, but are now not doing it since service split from neutron tree (?)). The problem with dropping the .roles attribute from context object in Liberty is that we, as a responsible upstream with lots of plugins maintained out-of-tree (see the ongoing vendor decomposition effort) would need to support the attribute while it's marked as deprecated for at least one cycle, meaning that if we don't get those oslo.policy internals we rely on in Liberty, we would need to postpone the switch till Mizzle, or rely on private symbols during the switch (while a new release of oslo.policy can easily break us). (BTW the code to extract admin roles is not really robust and has bugs, f.e. it does not handle AndChecks that could be used in context_is_admin. In theory, 'and' syntax would mean that both roles are needed to claim someone is an admin, while the code to extract admin roles handles 'and' the same way as 'or'. For the deprecation time being, we may need to document this limitation.) == 3. aggregating core, attribute and subattribute policies == That's the most interesting issue. For oslo.policy, policies are described as target: rule, where rule is interpreted as per registered checks, while target is opaque to the library. Neutron extended the syntax for target as:
Re: [openstack-dev] Barbican : Dependency of pyenv configuration in Barbican.sh script
Thanks a lot John for your response.
I appreciate for your time and effort in answering the queries and also
pointing to the latest changes which you been always doing :)
Thanks and Regards,
Asha Seshagiri
On Wed, Apr 22, 2015 at 6:09 PM, John Wood john.w...@rackspace.com wrote:
Hello Asha,
The barbican.sh script was originally intended to be a convenient way to
boot up a Barbican instance locally to quickly start evaluating its API and
functionality.
It was not intended to be used as a production script, deferring instead
to deployments utilizing packages such as RDO RPMs and so forth for that
purpose.
That said, changes to that script have been discussed, including
removing pyenv and uWSGI as dependencies, hence such changes would be good
to consider.
I’d also note that a solution based on this recently added script [1]
might be in order.
Thanks,
John
[1] https://github.com/openstack/barbican/blob/master/bin/barbican-api
From: Asha Seshagiri asha.seshag...@gmail.com
Date: Wednesday, April 22, 2015 at 4:57 PM
To: openstack-dev openstack-dev@lists.openstack.org
Cc: John Wood john.w...@rackspace.com, Reller, Nathan S.
nathan.rel...@jhuapl.edu, Douglas Mendizabal
douglas.mendiza...@rackspace.com, Paul Kehrer paul.keh...@rackspace.com,
Adam Harwell adam.harw...@rackspace.com, Alexis Lee alex...@hp.com,
nut...@gmail.com nut...@gmail.com
Subject: Barbican : Dependency of pyenv configuration in Barbican.sh
script
Hi All,
I would like to know the reason behind the dependency of the pyenv
virtual environment and pyenv in the barbican.sh script.
Ideally in the production environment , barbican would run on standalone
virtual box with a particular python version .I feel that their dependecies
needs to be removed from the script.
Was able to stand up the barbican instance without configuring pyenv and
pyenv-virtualenv dependencies by modifying the barbican script ,
installing few additional packages and exporting the python path to PATH
variable
Please find the change in barbican.sh script for installation and starting
of the script below :
VENV_DIR=${VIRTUAL_ENV:-`pyenv prefix`} - *This line needs to be
removed *
uwsgi --master --emperor $CONFIG_DIR/vassals* -H* *$VENV_DIR - The
**$VENV_DIR
variable need to be removed as an argument and -H as an option.*
The barbican script has been tied to $VENV_DIR variable which is
dependent on the pyenv for python configuration.Hence the barbican.sh
script needs to be modified to remove *$VENV_DIR variable *by
configuring python path in PATH variable.
On doing this , we can avoid the sourcing the pyenv and pyenv-virtualenv
packages and its dependices on Barbican script.
Any help would be highly appreciated and also would like to know opinion
from the openstack group on the changes indicated
Thanks in advance
*Thanks and Regards,*
*Asha Seshagiri*
--
*Thanks and Regards,*
*Asha Seshagiri*
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] TC Candidacy
Announcing my candidacy for the TC. I would bring an operator's perspective (ie, operator, user, super-user and dev) to the Technical Committee. I've been involved in OpenStack for four years. I gave talks at San Diego, Atlanta, Paris, and soon Vancouver as a contributor, community organizer, and operator. I would consent to serve a single term. -dave __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] Required data migrations in Kilo, need Turbo Hipster tests updated
On 4/23/15 1:16 PM, Dan Smith wrote: If I selected all the instance_type_id's from the system-metadata table and used those uuid's to load the object with something like: instance = objects.Instance.get_by_uuid( context, instance_uuid, expected_attrs=['system_metadata', 'flavor']) The tests would fail at that point when trying to read in the flavor as json. http://paste.openstack.org/show/205158/ Basically without digging further it seems like I should be able to load an instance by uuid regardless of the state of my flavor(s). Since this fails it seems like there is something wrong with the flavor handling on the objects. You should. The above is a reasonable result to get without the fix to ensure that we create instance_extra records for instances missing it. Do you still see the above with https://review.openstack.org/#/c/176387/ applied? That change works on the dataset. However I was referring to the db/object api (which I have no real knowledge of) that it should be able to get_by_uuid unmigrated instances and in my case I got the traceback given in that paste. It's possible I'm just using the API incorrectly. Another interesting thing is that migrate_flavor_data avoids migrating instances that are in the middle of an operation. The snapshot of databases we have include instances in this state. Since turbo-hipster is just testing against a snapshot in time there is no way for those instances to leave their working state and hence migrate_flavor_data can never finish (every run will leave instances undone). This therefore blocks the migrations from ever finishing. Ah, yeah, that's interesting, but I think it's a restriction we have to make for sanity. Oh yes, we want that restriction, but a way around it for instances that may be stuck or just testing purposes could be useful. Cheers, Josh --Dan __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] TC Candidacy
I would like to announce my candidacy for the OpenStack Technical Committee Many of you many not know me, because I am not a very active contributor. First a bit about myself. I am a Platform Architect working for Cisco in Israel and have been involved with OpenStack for the past two years. I was and (and still am a very active member of the virtualization community [1] and have been for a good number of years. I was honored to contribute and participate in the OpenStack Architecture Design Guide [2]. I am not new to writing but this was a new experience for me. One that I thoroughly enjoyed and would love to do again, if presented with the opportunity. These are the following reasons I think I can make a difference as part of the TC #1 Diversity The vast majority of the TC members are all long standing and well known members of the OpenStack community. Many of them have been core-developers, PTL's, reviewers. All of them have one thing in common they are developers and people who take pride and joy in contributing code to the OpenStack projects. I too have contributed code to the OpenStack projects - but not on the scale that any of the other candidates have. nowhere near close. And I am not ashamed of that fact. I am not a developer. Yes, I dabble in code (not as much as I would like), but I am deep down in my heart, an Operations guy. I like to get things to work, but not only that they work, but that they are stable, robust, and will provide a sound infrastructure (so that I do not get paged at 03.00 every night because something fell over and died). When all of the members of a committee are from a single 'stream' then it is natural to look at things in a certain way. But that is not the only way to look at things, there are other perspectives that need to be considered and that perspective (in my humble opinion) is missing. #2 Focus There has been a lengthy discussion over the past few months about what OpenStack is and what it is not. What should be part of OpenStack and again what should not. I cannot say that I fully agree with all the decisions that have been made, although I do fully agree with the direction in which OpenStack is going and how the TC has been driving that. I feel that the as part of the TC I will help the community focus on building a tightly integrated suite of software (there is no way you can call OpenStack a product) that will work, and work well [3]. #3 Acceptance of others It is no secret that i think that I have spoken my mind[4], more than once on how I find it extremely hard for someone who is not a developer to help and make OpenStack better. I have tried to lower that hurdle in a number of ways [5] to make it easier for 'non-dev's' to starting 'chipping in'. But it seems that I was not successful. Even with regards to myself. We are all members of the community. But there are several levels. Even in this election only ''Individual Members who committed a change to a repository under ’any’ of the official OpenStack Project Teams (as defined above) over the last two 6-month release cycles are automatically considered ATC'' and they are the only ones that can vote. There are other ways to contribute. People that report bugs contribute. People that write blog posts contribute. People that evangelize in User groups and speak at conferences, meetings etc. - they also contribute. It is not all about code. Yes it is hard to quantify. Yes it is hard to measure. But that does not mean it should not be considered. If elected to serve as part of the TC - this is something I would like pursue - something that can make the OpenStack community not only a developer community - but a community of all those who care about it. I would like to leave you with one last thought. I thought long and hard about putting up myself as a candidate. I do think that I have a fresh perspective to bring to the table, a different way of looking at things and a lot of passion that can help OpenStack achieve what it set out to do. ``Our goal is to produce the ubiquitous Open Source cloud computing platform that will meet the needs of public and private cloud providers regardless of size, by being simple to implement and massively scalable.`` I would like to wish all the candidates the best of luck. -- Best Regards, Maish Saidel-Keesing [1] http://vsphere-land.com/news/top-vblog-2015-full-results.html [2] http://technodrone.blogspot.com/2014/08/the-openstack-architecture-design-guide.html [3] http://technodrone.blogspot.com/2014/08/to-innovate-or-to-stabilize-that-is.html [4] http://technodrone.blogspot.com/2014/09/an-open-letter-to-openstack-foundation.html [5] http://technodrone.blogspot.com/2014/11/start-contributing-to-openstack-easy.html __ OpenStack Development Mailing List (not for usage questions) Unsubscribe:
[openstack-dev] [neutron] [docs] Networking Guide Doc Day - April 23rd
Hello Folks, Just a kind reminder! I would like to invite all available contributors to help us to complete the OpenStack Networking Guide. We are having a Networking Doc Day on April 23rd in order to review the current guide and make a big push on its content. We will use the following IRC channel starting at 16:00UTC: #openstack-sprint We have prepared an etherpad to coordinate who is doing what: https://etherpad.openstack.org/p/networking-guide All the expected content is being described in the TOC: https://wiki.openstack.org/wiki/NetworkingGuide/TOC Information for Doc contributors in here: https://wiki.openstack.org/wiki/Documentation/HowTo#Edit_OpenStack_RST_and.2For_DocBook_documentation There are so many ways you can contribute: * Assign to yourself one of the available chapters * Review the current content and open bugs if needed * Review the existing gerrit commit if you are familiar with the information * Be available on IRC to answer some questions about configuration details or functionality * Cheering at the contributors! Do not hesitate in contacting me for any questions. Cheers! Edgar Magana IRC: emagana emag...@gmail.commailto:emag...@gmail.com edgar.mag...@workday.commailto:edgar.mag...@workday.com __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] About Sahara EDP New Ideas for Liberty
On Wed, 2015-04-22 at 12:36 +, Chen, Ken wrote: o more complex workflows (job dependencies, DAGs, etc. Do we rely on Oozie, or something else? Huichun is now figuring this. I am not whether you guys already have some detail ideas about this? If needed we can contribute some effort. If no details are ready, we can help draw a draft version first. I just made a note on the pad https://etherpad.openstack.org/p/sahara-liberty-proposed-sessions Maybe the right approach here is to develop a mapping notation that can be expressed as a JSON object (like the proposed job interface mapping). If we can develop an abstract way to describe relationships between jobs, then the individual EDP engines can implement it. For the Oozie EDP engine, maybe it uses Oozie features in workflows. For Spark, or Storm, maybe it uses some existing opensource coordinator or one is written. The key idea would be to make job coordination part of the EDP engine, with a well defined set of objects to describe the relationships. What do you think? Just a rough idea. Maybe there is a better way. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all][code quality] Voting coverage job (-1 if coverage get worse after patch)
On Sat, Apr 18, 2015 at 9:30 PM, Boris Pavlovic bo...@pavlovic.me wrote: Code coverage is one of the very important metric of overall code quality especially in case of Python. It's quite important to ensure that code is covered fully with well written unit tests. One of the nice thing is coverage job. I really like the idea of adding the coverage job everywhere so that developers can view the results be using a link in Gerrit. I think this would make it easier for many. I don't like the idea of checking that coverage is increased. There are many issues with that. The two biggest one for me are: 1. It will either lead to people doing things to game the system or overuse of the #no-coverage-check tag you mentioned. 2. It really doesn't tell you too much. A core developer should really be looking at the tested use cases to see if they are all there. Line coverage and even branch coverage won't tell you that. -- David blog: http://www.traceback.org twitter: http://twitter.com/dstanek www: http://dstanek.com __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [puppet] Moving forward with puppet-keystone CI (beaker tests)
On 04/22/2015 10:51 AM, Emilien Macchi wrote: Hi, Some important work is being done on Keystone v3 API support in puppet-keystone. We've clearly seen there is a lack of review and I think we all worry about breaking something. Spencer I are working on beaker tests lately and the jobs are non-voting for now. I propose: * to review (and eventually merge) the beaker-tests patches [1] [2] for Keystone openstacklib. * to patch project-config [3] to make vote Beaker jobs in Puppet OpenStack gate for puppet-keystone puppet-openstacklib. Why voting? Because otherwise I'm not sure people will notice the failure and some patches will be merged while beaker is red. So we can have a good set of tests that will help us to detect some issues in the future. I don't think we will catch all mistakes we can do, but this is a good start. To vote this proposal, you can use the gerrit patches and let any feedback. Thanks, [1] puppet-keystone: https://review.openstack.org/#/c/155873/ [2] puppet-openstacklib: https://review.openstack.org/#/c/176098/ [3] project-config: https://review.openstack.org/176343 I added Keystone-core (13 People) to each of these reviews. If things are critical, and concert Keystone, we really should be notified so someone on the Keystone side can voice an opinion. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] TC candidacy
I would like to announce my candidacy for the OpenStack Technical Committee. I will try to be brief and to the point: I have been involved in OpenStack since the early days of the Austin release; I have worked on (perhaps) the two most prolific projects in OpenStack (Nova, and Neutron) and a few others aimed at ensuring their quality (Tempest, DevStack, and the infra ones). I have been mainly a developer, but also a deployer, distributor, user, tech writer, you name it. Under these points of views I have seen friction in dealing with OpenStack increase over time, and I want to do something about it. I have never run for the TC before, or any other position of 'influence'. I came to the realization that doing something about it by being on the fringes could only get me so far. My main goal is to empower the developer, allowing him/her to go at the pace he/she is comfortable with, whilst preserving the quality of software being produced: I believe we have many bottlenecks and inefficiencies in the way we develop and consume OpenStack technologies. In Neutron I, with the help of others, have tried to identify these and proactively did something about it: decomposing the codebase in core vs plugins and moving away from the project reliance on Tempest to ensure stability are examples to name a few. At the same time decentralizing and increasing speed of development can impair coherence and cohesion of the overall solution and that is why I think a seat in the TC would give me enough exposure to help preserve, and strive to achieve these qualities in the software we build. The OpenStack ecosystem is incredibly diverse and yet we need each technology developed under its umbrella to share a lot more than the four Opens. The definition of shared methodologies, practices and guidelines can help us achieve that, but most importantly a shared roadmap that can let us peek into the future of OpenStack as a whole rather than a fragmented collection of services that work poorly together. Thanks for reading. Regards, Armando __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [Nova][Neutron] Cross-project coordination
Hi, Kyle Mestery has asked me to serve as a cross-project liaison between Neutron and Nova. So, I'd like to say hello, and direct everyone towards an etherpad that I have created. https://etherpad.openstack.org/p/nova-neutron The etherpad can serve as a way to collect items that should be discussed between the projects. I will be attending the Nova main meetings to field Neutron questions, or at least provide who on the Neutron side would know the answer to a question. This is a big job, and I'd like to see if there is a volunteer on the Nova side who would be interested in also helping this effort, who could do the reverse (Sit in on Neutron meetings, field Nova questions). Thank you, and I look forward to working with everyone! -- Sean M. Collins __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [heat][novaclient] heat gate is broken because of new novaclient
On Wed, Apr 22, 2015 at 11:49 PM, Sean Dague s...@dague.net wrote: On 04/22/2015 07:07 AM, Sean Dague wrote: On 04/22/2015 04:09 AM, Angus Salkeld wrote: Hi Can we please have a new release of novaclient (after the below fix)? Heat's unit tests pass fine with: python-novaclient (2.23.0) but python-novaclient 2.24.0 introduces this bug: https://bugs.launchpad.net/python-novaclient/+bug/1437244 We still need this patch in: https://review.openstack.org/176228 We should also update requirements to exclude 2.24.0 I've got an alternate fix here - https://review.openstack.org/#/c/176252/ I was -1 for a long time on the complex repr stuff in the introducing patch, and I think that's just going to get us into other problems down the road. The repr fall back for Resource is totally fine, and we should just use that. python-novaclient 2.24.1 has been released with https://review.openstack.org/#/c/176252/ in place. Hopefully that fixes things for Heat. Thanks Sean! Shouldn't we also exclude 2.24.0 from requirements? -Angus -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova][Neutron] Cross-project coordination
On Wed, Apr 22, 2015 at 3:28 PM, Sean M. Collins s...@coreitpro.com wrote: Hi, Kyle Mestery has asked me to serve as a cross-project liaison between Neutron and Nova. So, I'd like to say hello, and direct everyone towards an etherpad that I have created. https://etherpad.openstack.org/p/nova-neutron The etherpad can serve as a way to collect items that should be discussed between the projects. I will be attending the Nova main meetings to field Neutron questions, or at least provide who on the Neutron side would know the answer to a question. This is a big job, and I'd like to see if there is a volunteer on the Nova side who would be interested in also helping this effort, who could do the reverse (Sit in on Neutron meetings, field Nova questions). Thank you, and I look forward to working with everyone! Sean, thank you so much for volunteering to take on this incredibly important job. I'm hoping we can get someone from the nova side to work hand-in-hand with you and ensure we continue to drive issues related to both nova and neutron to conclusion in a way which benefits are mutual users! Thanks! Kyle -- Sean M. Collins __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [neutron] neutron DB upgrade
But maybe some component depend on another one. And it would be difficult to test all the components combination. /Yalei From: Guo, Ruijing [mailto:ruijing@intel.com] Sent: Wednesday, April 22, 2015 10:23 AM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [neutron] neutron DB upgrade Thanks for your detail explanation. I agree with you that we still use DB upgrade when fresh installation. Yes. It happened to me that unused vendor DB upgrade failure causes neutron DB upgrade failure. I have little concerns about adding whole DB upgrade in one directory alembic_migrations/versions. It is difficult for devops to debug/workaround the issue. I suggest to separate according to components or enforce file name as Revision_component_desctiption.py. If I don’t use the component and DB for that component upgrade fails, I can safely delete *component*. Thanks, -Ruijing From: Salvatore Orlando [mailto:sorla...@nicira.com] Sent: Tuesday, April 21, 2015 9:04 PM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [neutron] neutron DB upgrade On 21 April 2015 at 14:25, Guo, Ruijing ruijing@intel.commailto:ruijing@intel.com wrote: Somebody can help me to understand why neutron DB is needed upgrade even in refresh installation unlike other components. From my understanding, DB upgrade is risky and needed when upgrade. Alembic is a fairly reliable tool for managing DB upgrades. Also there are enough tests in place to ensure the sanity of each db migration and that the DB schema is always in sync with business logic's data model. Release A should have schema A and Release B should have schema B. This will make really difficult doing testing during the development cycle. While all interim migrations added during the release cycle can be squashed into a single migration provided at release time, this action will probably transform a relatively safe process into a risky one, as there will be little time to react to issues introduced by that single migration. Only upgrade A to B need to upgrade DB. This is what happens for most users. However there still are a few which more or less closely follow trunk - that is to say they're not tied to any specific release. Also, this does not apply to developers and CI (which is ultimately one of the principal tools we use to ensure what we deliver is not completely rubbish). In the same time, can we separate neutron DB as vendor DB non-vendor DB? We had vendor or plugin specific migrations until Juno. When he had these kind of conditional migrations doing DB upgrades was very risky. DB schema management has become a lot simpler and safer since we unified the schema. However, as a part of the vendor plugin split out there will be eventually a next step where all vendor-specific tables are moved into their own dbs. Are tables for plugins which are not ML2 causing you any problem? 1. For that case, we can upgrade vendor DB if we use some vendor scenario. 2. we already separate vendor code from neutron code base. Thanks, -Ruijing __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribehttp://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Ironic] [TripleO] Aborting in(tro)spection process in discoverd
Hi, Thank you for putting it up Dmitry. As I wrote into the blueprint, if there are requests to implement API aborting introspection from operators, we should introduce this feature as API, I think. But if we just want to use this feature as debug, we had better not to introduce it as API. And, instead of introducing as API, I suppose implement only client library and call it from shell script or something like tool. Best Regards, Yuiko Takada 2015-04-21 20:42 GMT+09:00 Dmitry Tantsur dtant...@redhat.com: Hi folks. Recently I got several requests to implement API aborting introspection in discoverd. This is useful mostly when debugging, when you know that introspection has failed, and you want to stop it right now. I've put a blueprint https://blueprints.launchpad.net/ironic-discoverd/+spec/abort-introspection to track it. Such API would cause discoverd to set error state for the node immediately and issue a power off request for it. The technical side is not a big deal. What I'm worried about is whether we want to introduce such a feature at all. Some Ironic processes (deploy, in-band cleaning) are async as well, they also may hang, and IIUC we don't have means of aborting them. Does debugging justify introducing new API? This looks somewhat similar to our discussion about breaking locks in Ironic: it might be useful, but it's an API which we'll support and which can be easily misused. But with introspection the only case where lack of this feature can't be easily worked around is when people want to recreate a node. I've been arguing for some time that recreating a node is not a good way to solve problems. In other cases one can just power off the node via Ironic API and restart the introspection again afterwards. What do you think? Cheers, Dmitry __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [neutron] [docs] Networking Guide Doc Day - April 23rd
Yatin, Please, feel free to add it. Even better if you can help us with the proper documentation. Thanks, Edgar From: yatin kumbhare yatinkumbh...@gmail.commailto:yatinkumbh...@gmail.com Reply-To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.orgmailto:openstack-dev@lists.openstack.org Date: Tuesday, April 21, 2015 at 1:28 AM To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.orgmailto:openstack-dev@lists.openstack.org Subject: Re: [openstack-dev] [neutron] [docs] Networking Guide Doc Day - April 23rd Nice idea. Looked at the TOC page, pretty much covering variants of neutron nuts bolts. Shall we include DNS into TOC somewhere? Regards, Yatin On Tue, Apr 21, 2015 at 10:22 AM, Guo, Ruijing ruijing@intel.commailto:ruijing@intel.com wrote: Hi, Edgar Some docs are still in private branch. Can we move to public branch for reviewing submitting patches? Thanks, -Ruijing From: Edgar Magana [mailto:edgar.mag...@workday.commailto:edgar.mag...@workday.com] Sent: Saturday, April 18, 2015 2:25 AM To: OpenStack Development Mailing List (not for usage questions) Subject: [openstack-dev] [neutron] [docs] Networking Guide Doc Day - April 23rd Hello Folks, I would like to invite all available contributors to help us to complete the OpenStack Networking Guide. We are having a Networking Doc Day on April 23rd in order to review the current guide and make a big push on its content. Let's use both the Neutron and Docs IRC channels: #openstack-neutron #openstack-doc All the expected content is being described in the TOC: https://wiki.openstack.org/wiki/NetworkingGuide/TOC Information for Doc contributors in here: https://wiki.openstack.org/wiki/Documentation/HowTo#Edit_OpenStack_RST_and.2For_DocBook_documentation We have prepared an etherpad to coordinate who is doing what: https://etherpad.openstack.org/p/networking-guide There are so many ways you can contribute: * Assign to yourself one of the available chapters * Review the current content and open bugs if needed * Review the existing gerrit commit if you are familiar with the information * Be available on IRC to answer some questions about configuration details or functionality * Cheering at the contributors! Do not hesitate in contacting me for any questions. Cheers! Edgar Magana IRC: emagana emag...@gmail.commailto:emag...@gmail.com edgar.mag...@workday.commailto:edgar.mag...@workday.com __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribehttp://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] TC Candidacy
Hi, I'd like to announce my candidacy for re-election to the TC. About Me I am the PTL for the OpenStack Infrastructure Program, which I have been helping to build for nearly four years. I have also served on the TC since the Icehouse cycle. I am responsible for a significant portion of our project infrastructure and developer workflow including setting up gerrit and helping to write git-review. All of that is to say that I've given a lot of thought and action to helping scale OpenStack to the number of projects and developers it has today. I also wrote zuul, nodepool, and devstack-gate to make sure that we are able to test that the different componensts of OpenStack work together as a cohesive whole. A good deal of my technical work is focused on achieving that and ensuring that all of the projects that make up OpenStack have the ability to test themselves in such a complex system. Throughout my time working on OpenStack I have always put the needs of the whole project first, above those of any individual contributor, organization, or program. I also believe in the values we have established as a project: open source, design, development, and community. To that end, I have worked hard to ensure that the project infrastructure is run just like an OpenStack project, using the same tools and processes, and I think we've succeeding in creating one of the most open operational project infrastructures ever. My Platform === I am very excited about the big tent. The infrastructure team has been involved in operating stackforge for some time, and so the big tent idea seems like a natural progression to me. We have a lot of folks who are participating in our community and it is time that we accept them in. At the same time we can strengthen the core of our project by acknowledging that there are a lot of components that can be a part of OpenStack, but not all of them need to be deployed in every installation. And so the layered approach helps us make sense of how a system should be constructed. As part of the move into the big tent, all of the cross-project efforts will need to change the way they operate to accomodate the scale we are dealing with. Most of that work is well underway, but the TC itself will need to change as well. Just as any other horizontal effort, the TC will need to provide the tools and processes for projects to be effective members of our community on their own. Part of the motivation for adopting the big tent strategy is to get the TC out of the business of doing detailed review of projects so that it can provide technical leadership for OpenStack as a whole. I believe we have made a great start on the work that is needed to build the big tent. There is still more work that needs to be done, I would like to continue to help the TC evolve into its new role and so I would appreciate your vote. Thanks for your consideration, Jim __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
