Re: [openstack-dev] [trove][all][tc] A proposal to rearchitect Trove

On 06/22/2017 11:59 AM, Fox, Kevin M wrote: My $0.02. That view of dependencies is why Kubernetes development is outpacing OpenStacks and some users are leaving IMO. Not trying to be mean here but trying to shine some light on this issue. Kubernetes at its core has essentially something

Re: [openstack-dev] [trove][all][tc] A proposal to rearchitect Trove

On 06/20/2017 11:45 AM, Jay Pipes wrote: Good discussion, Zane. Comments inline. On 06/20/2017 11:01 AM, Zane Bitter wrote: 2) The database VMs are created in a project belonging to the operator of the service. They're connected to the user's network through , and isolated from other

Re: [openstack-dev] [oslo.db] Stepping down from core

hey Roman - It was a huge pleasure working w/ you on oslo.db!I hope we can collaborate again soon. - mike On 06/11/2017 10:32 AM, Roman Podoliaka wrote: Hi all, I recently changed job and hasn't been able to devote as much time to oslo.db as it is expected from a core reviewer. I'm

Re: [openstack-dev] [all] etcd3 as base service - update

/c528539879e824b8e6d5654292a85ccbee6dcf89/keystone/conf/fernet_tokens.py#L44-L54 [1] https://launchpad.net/bugs/1649616 On Thu, Jun 8, 2017 at 11:37 AM, Mike Bayer <mba...@redhat.com <mailto:mba...@redhat.com> <mailto:mba...@redhat.com <mailto:mba...@redh

Re: [openstack-dev] [all] etcd3 as base service - update

On 06/08/2017 04:24 PM, Julien Danjou wrote: On Thu, Jun 08 2017, Mike Bayer wrote: So I wouldn't be surprised if new / existing openstack applications express some gravitational pull towards using it as their own datastore as well. I'll be trying to hang onto the etcd3 track as much

Re: [openstack-dev] [all] etcd3 as base service - update

hing. that was a terrible architectural decision, with apologies to whoever made it.if you're putting some kind of "we create an infinite, rapidly growing, turns-to-garbage-in-30-seconds" kind of data in a database, removing that data robustly and ASAP needs to be part of the proc

Re: [openstack-dev] [all] etcd3 as base service - update

On 06/08/2017 12:47 AM, Joshua Harlow wrote: So just out of curiosity, but do people really even know what etcd is good for? I am thinking that there should be some guidance from folks in the community as to where etcd should be used and where it shouldn't (otherwise we just all end up in a

Re: [openstack-dev] [Keystone] Cockroachdb for Keystone Multi-master

On 05/30/2017 09:06 PM, Jay Pipes wrote: On 05/30/2017 05:07 PM, Clint Byrum wrote: Excerpts from Jay Pipes's message of 2017-05-30 14:52:01 -0400: Sorry for the delay in getting back on this... comments inline. On 05/18/2017 06:13 PM, Adrian Turjak wrote: Hello fellow OpenStackers, For

Re: [openstack-dev] [upgrades][skip-level][leapfrog] - RFC - Skipping releases when upgrading

On 05/26/2017 10:56 AM, Dan Smith wrote: As most of the upgrade issues center around database migrations, we discussed some of the potential pitfalls at length. One approach was to roll-up all DB migrations into a single repository and run all upgrades for a given project in one step. Another

Re: [openstack-dev] [tc] Active or passive role with our database layer

On 05/23/2017 03:16 PM, Edward Leafe wrote: On May 23, 2017, at 1:43 PM, Jay Pipes wrote: [1] Witness the join constructs in Golang in Kubernetes as they work around etcd not being a relational data store: Maybe it’s just me, but I found that Go code more

Re: [openstack-dev] [tc] Active or passive role with our database layer

On 05/23/2017 01:10 PM, Octave J. Orgeron wrote: Comments below.. On 5/21/2017 1:38 PM, Monty Taylor wrote: For example: An HA strategy using slave promotion and a VIP that points at the current write master paired with an application incorrectly configured to do such a thing can lead to

Re: [openstack-dev] [Keystone] Cockroachdb for Keystone Multi-master

On 05/22/2017 05:02 AM, Thierry Carrez wrote: Mike Bayer wrote: On 05/18/2017 06:13 PM, Adrian Turjak wrote: So, specifically in the realm of Keystone, since we are using sqlalchemy we already have Postgresql support, and since Cockroachdb does talk Postgres it shouldn't be too hard to back

Re: [openstack-dev] [nova] Boston Forum session recap - searchlight integration

On 05/22/2017 05:39 AM, Matthew Booth wrote: There are also a couple of optimisations to make which I won't bother with up front. Dan suggested in his CellsV2 talk that we would only query cells where the user actually has instances. If we find users tend to clump in a small number of

Re: [openstack-dev] [tc] Active or passive role with our database layer

On 05/21/2017 03:38 PM, Monty Taylor wrote: documentation on the sequence of steps the operator should take. In the "active" approach, we still document expectations, but we also validate them. If they are not what we expect but can be changed at runtime, we change them overriding

Re: [openstack-dev] [tc] revised Postgresql support status patch for governance

On 05/21/2017 03:51 PM, Monty Taylor wrote: So I don't see the problem of "consistent utf8 support" having much to do with whether or not we support Posgtresql - you of course need your "CREATE DATABASE" to include the utf8 charset like we do on MySQL, but that's it. That's where we stand

Re: [openstack-dev] [oslo] Can we stop global requirements update?

On 05/20/2017 12:04 PM, Julien Danjou wrote: On Fri, May 19 2017, Mike Bayer wrote: IMO that's a bug for them. Of course it's a bug. IIRC Mehdi tried to fix it without much success. I'm inspired to see that Keystone, Nova etc. are able to move between and eventlet backend and a mod_wsgi

Re: [openstack-dev] [Keystone] Cockroachdb for Keystone Multi-master

On 05/18/2017 06:13 PM, Adrian Turjak wrote: So, specifically in the realm of Keystone, since we are using sqlalchemy we already have Postgresql support, and since Cockroachdb does talk Postgres it shouldn't be too hard to back Keystone with it. At that stage you have a Keystone DB that could

Re: [openstack-dev] [oslo] can we make everyone drop eventlet? (was: Can we stop global requirements update?)

FTFY On 05/19/2017 03:58 PM, Joshua Harlow wrote: Mehdi Abaakouk wrote: Not really, I just put some comments on reviews and discus this on IRC. Since nobody except Telemetry have expressed/try to get rid of eventlet. Octavia is using cotyledon and they have gotten rid of eventlet. Didn't

Re: [openstack-dev] [oslo] Can we stop global requirements update?

On 05/19/2017 04:23 AM, Mehdi Abaakouk wrote: And some applications rely on implicit internal contract/behavior/assumption. IMO that's a bug for them.I'm inspired to see that Keystone, Nova etc. are able to move between and eventlet backend and a mod_wsgi backend.IMO eventlet is

Re: [openstack-dev] [nova] Boston Forum session recap - searchlight integration

On 05/19/2017 02:46 AM, joehuang wrote: Support sort and pagination together will be the biggest challenge: it's up to how many cells will be involved in the query, 3,5 may be OK, you can search each cells, and cached data. But how about 20, 50 or more, and how many data will be cached?

Re: [openstack-dev] [oslo] Can we stop global requirements update?

On 05/18/2017 02:37 PM, Julien Danjou wrote: On Thu, May 18 2017, Mike Bayer wrote: I'm not understanding this? do you mean this? In the long run, yes. Unfortunately, we're not happy with the way Oslo libraries are managed and too OpenStack centric. I've tried for the last couple of years

Re: [openstack-dev] [oslo] Can we stop global requirements update?

On 05/16/2017 05:42 AM, Julien Danjou wrote: On Wed, Apr 19 2017, Julien Danjou wrote: So Gnocchi gate is all broken (agan) because it depends on "pbr" and some new release of oslo.* depends on pbr!=2.1.0. Same things happened today with Babel. As far as Gnocchi is concerned, we're

Re: [openstack-dev] [tc] revised Postgresql support status patch for governance

On 05/17/2017 02:38 PM, Sean Dague wrote: Some of the concerns/feedback has been "please describe things that are harder by this being an abstraction", so examples are provided. so let's go through this list: - OpenStack services taking a more active role in managing the DBMS ,

Re: [openstack-dev] [Zun]Use 'uuid' instead of 'id' as object ident in data model

On 04/05/2017 11:02 AM, gordon chung wrote: On 05/04/17 09:00 AM, Monty Taylor wrote: Please do NOT use uuid as a primary key in MySQL: * UUID has 36 characters which makes it bulky. you can store it as a binary if space is a concern. this is highly inconvenient from a datadump /

Re: [openstack-dev] [Zun]Use 'uuid' instead of 'id' as object ident in data model

On 04/05/2017 11:00 AM, Monty Taylor wrote: On 04/05/2017 09:39 AM, Akihiro Motoki wrote: I noticed this thread by Monty's reply. Sorry for my late :( I think we need to think 'id' separately for API modeling and DB modeling. In the API perspective, one of the important things is that 'id'

Re: [openstack-dev] [oslo][requirements][all] requesting assistance to unblock SQLAlchemy 1.1 from requirements

On Wed, Mar 15, 2017 at 5:20 PM, Sean Dague <s...@dague.net> wrote: On 03/15/2017 10:38 AM, Mike Bayer wrote: On 03/15/2017 07:30 AM, Sean Dague wrote: The problem was the original patch kept a cap on SQLA, just moved it up to the next pre-release, not realizing the caps in general are

Re: [openstack-dev] [oslo][requirements][all] requesting assistance to unblock SQLAlchemy 1.1 from requirements

On 03/15/2017 07:30 AM, Sean Dague wrote: The problem was the original patch kept a cap on SQLA, just moved it up to the next pre-release, not realizing the caps in general are the concern by the requirements team. So instead of upping the cap, I just removed it entirely. (It also didn't help

[openstack-dev] [oslo][requirements][all] requesting assistance to unblock SQLAlchemy 1.1 from requirements

Hello all - As mentioned previously, SQLAlchemy 1.1 has now been released for about six months. My work now is on SQLAlchemy 1.2 which should hopefully see initial releases in late spring.SQLAlchemy 1.1 includes tons of features, bugfixes, and improvements, and in particular the most

Re: [openstack-dev] [neutron] Some findings while profiling instances boot

On 02/15/2017 12:46 PM, Daniel Alvarez Sanchez wrote: Also, while having a look at server profiling, around the 33% of the time was spent building SQL queries [1]. Mike Bayer went through this and suggested having a look at baked queries and also submitted a sketch of his proposal [2

Re: [openstack-dev] [oslo][oslo.db] MySQL Cluster support

to change the approach. I'm also going to create a spec and blueprint to cover the changes across the services. I'll post links once all of that is up for review. Thanks, Octave On 2/6/2017 7:53 AM, Mike Bayer wrote: On 02/03/2017 11:59 AM, Octave J. Orgeron wrote: Hi Mike, Comments below.. Than

Re: [openstack-dev] [oslo][oslo.db] MySQL Cluster support

On 02/03/2017 11:59 AM, Octave J. Orgeron wrote: Hi Mike, Comments below.. Thanks, Octave On 2/3/2017 7:41 AM, Mike Bayer wrote: On 02/02/2017 05:28 PM, Octave J. Orgeron wrote: That refers to the total length of the row. InnoDB has a limit of 65k and NDB is limited to 14k. A simple

Re: [openstack-dev] [oslo][oslo.db] MySQL Cluster support

eir tables. Those also have a lot of churn between releases where the schema changes more often. Other services like keystone, heat, and ironic are considerably easier to work with and have well laid out tables that don't change much. Thanks, Octave On 2/2/2017 1:25 PM, Mike Bayer wrote: On

Re: [openstack-dev] [oslo][oslo.db] MySQL Cluster support

er to work with and have well laid out tables that don't change much. Thanks, Octave On 2/2/2017 1:25 PM, Mike Bayer wrote: On 02/02/2017 02:52 PM, Mike Bayer wrote: But more critically I noticed you referred to altering the names of columns to suit NDB. How will this be accomplished? Changin

Re: [openstack-dev] [oslo][oslo.db] MySQL Cluster support

On 02/02/2017 02:52 PM, Mike Bayer wrote: But more critically I noticed you referred to altering the names of columns to suit NDB. How will this be accomplished? Changing a column name in an openstack application is no longer trivial, because online upgrades must be supported

Re: [openstack-dev] [oslo][oslo.db] MySQL Cluster support

On 02/02/2017 02:16 PM, Octave J. Orgeron wrote: Hi Doug, Comments below.. Thanks, Octave On 2/2/2017 11:27 AM, Doug Hellmann wrote: It sounds like part of the plan is to use the configuration setting to control how the migration scripts create tables. How will that work? Does each

Re: [openstack-dev] [nova][ceilometer][postgresql][gate][telemetry] PostgreSQL gate failure (again)

On 02/02/2017 11:42 AM, Sean Dague wrote: That's all fine and good, we just need to rewrite about 100,000 unit tests to do that. I'm totally cool with someone taking that task on, but making a decision about postgresql shouldn't be filibustered on rewriting all the unit tests in OpenStack

Re: [openstack-dev] [oslo][oslo.db] MySQL Cluster support

On 02/02/2017 10:25 AM, Monty Taylor wrote: On 02/01/2017 09:33 PM, Octave J. Orgeron wrote: Hi Folks, I'm working on adding support for MySQL Cluster to the core OpenStack services. This will enable the community to benefit from an active/active, auto-sharding, and scale-out MySQL database.

Re: [openstack-dev] [nova][ceilometer][postgresql][gate][telemetry] PostgreSQL gate failure (again)

On 02/01/2017 10:22 AM, Monty Taylor wrote: I personally continue to be of the opinion that without an explicit vocal and well-staffed champion, supporting postgres is more trouble than it is worth. The vast majority of OpenStack deployments are on MySQL - and what's more, the code is written

Re: [openstack-dev] [All projects that use Alembic] Absence of pk on alembic_version table

one! > > Guys in our team faced an issue when they try to run alembic migrations on > Galera with ENFORCING mode. [1] > > This was an issue with Alembic [2], which was quickly fixed by Mike Bayer > (many thanks!) and new version of alembic was reseal

Re: [openstack-dev] [networking-sfc] Intermittent database transaction issues, affecting the tempest gate

On 12/20/2016 06:50 PM, Cathy Zhang wrote: Hi Bernard, Thanks for the email. I will take a look at this. Xiaodong has been working on tempest test scripts. I will work with Xiaodong on this issue. I've added a comment to the issue which refers to upstream SQLAlchemy issue

Re: [openstack-dev] [nova][oslo][openstack-ansible] DB deadlocks, Mitaka, and You

On 10/19/2016 08:36 AM, Ian Cordasco wrote: Hey Kevin, So just looking at the pastes you have here, I'm inclined to believe this is actually a bug in oslo_db/sqlalchemy. If you follow the trace, there's a PyMySQL InternalError not being handled inside of sqlalchemy. I'm not sure if SQLAlchemy

Re: [openstack-dev] [cinder][db] lazy loading of an attribute impossible

On 09/30/2016 10:54 AM, Roman Podoliaka wrote: Michał, You are absolutely right: this exception is raised when you try to lazy-load instance attributes outside a Session scope. There is an obvious problem with that - instances do not communicate with a DB on their own - it's left up to

Re: [openstack-dev] [oslo.db] [release] opportunistic tests breaking randomly

On 09/21/2016 11:41 AM, Joshua Harlow wrote: I've seen something similar at https://review.openstack.org/#/c/316935/ Maybe its time we asked again why are we still using eventlet and do we need to anymore. What functionality of it are people actually taking advantage of? If it's supporting

Re: [openstack-dev] [oslo.db] [release] opportunistic tests breaking randomly

On 09/14/2016 11:05 PM, Mike Bayer wrote: Are *these* errors also new as of version 4.13.3 of oslo.db ? Because here I have more suspicion of one particular oslo.db change here. The version in question that has the changes to provisioning and anything really to do with this area

Re: [openstack-dev] [oslo.db] [release] opportunistic tests breaking randomly

here. fits much more with your initial description On 09/14/2016 10:48 PM, Mike Bayer wrote: On 09/14/2016 07:04 PM, Alan Pevec wrote: Olso.db 4.13.3 did hit the scene about the time this showed up. So I think we need to strongly consider blocking it and revisiting these issues post new

Re: [openstack-dev] [oslo.db] [release] opportunistic tests breaking randomly

On 09/14/2016 07:04 PM, Alan Pevec wrote: Olso.db 4.13.3 did hit the scene about the time this showed up. So I think we need to strongly consider blocking it and revisiting these issues post newton. So that means reverting all stable/newton changes, previous 4.13.x have been already blocked

Re: [openstack-dev] [oslo.db] [release] opportunistic tests breaking randomly

On 09/14/2016 11:08 AM, Mike Bayer wrote: On 09/14/2016 09:15 AM, Sean Dague wrote: I noticed the following issues happening quite often now in the opportunistic db tests for nova - http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C

Re: [openstack-dev] [oslo.db] [release] opportunistic tests breaking randomly

On 09/14/2016 09:15 AM, Sean Dague wrote: I noticed the following issues happening quite often now in the opportunistic db tests for nova - http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C%22sqlalchemy.exc.ResourceClosedError%5C%22 It looks like some race has

Re: [openstack-dev] [oslo] [telemetry] [requirements] [FFE] Oslo.db 4.13.2

On 09/07/2016 01:29 PM, Doug Hellmann wrote: Excerpts from Matthew Thode's message of 2016-09-07 09:11:58 -0500: On 09/07/2016 08:58 AM, Doug Hellmann wrote: Excerpts from Matthew Thode's message of 2016-09-07 08:21:50 -0500: https://review.openstack.org/366298 This is just a bump to

Re: [openstack-dev] [keystone][nova][neutron][all] Rolling upgrades: database triggers and oslo.versionedobjects

On 09/02/2016 01:53 PM, Doug Hellmann wrote: Excerpts from Thierry Carrez's message of 2016-09-02 12:15:33 +0200: Sean Dague wrote: Putting DB trigger failure analysis into the toolkit required to manage an upgrade failure is a really high bar for new ops. I agree with Sean: increasing the

Re: [openstack-dev] [oslo] [telemetry] Oslo.db 4.13.1 broke Gnocchi

is the failure here something that comes up in gnocchi's test suite? Could there be some way that oslo libraries run the test suites of all consuming projects before a patch and/or a release? (apologies if we already do this). On 09/02/2016 12:17 PM, Matthew Thode wrote: On 09/02/2016

Re: [openstack-dev] [oslo] [telemetry] Oslo.db 4.13.1 broke Gnocchi

On 09/02/2016 04:43 AM, Julien Danjou wrote: On Fri, Sep 02 2016, Julien Danjou wrote: I'll look into fixing that, though any help would be welcome. My attempt at: https://review.openstack.org/364767 I've augmented it with mapper-level SQLAlchemy API use.

Re: [openstack-dev] [keystone][nova][neutron][all] Rolling upgrades: database triggers and oslo.versionedobjects

On 09/01/2016 11:52 AM, Dan Smith wrote: The indirection service is really unrelated to this discussion, IMHO. If you take RPC out of the picture, all you have left is a direct-to-the-database facade to handle the fact that schema has expanded underneath you. As Clint (et al) have said --

Re: [openstack-dev] [keystone][nova][neutron][all] Rolling upgrades: database triggers and oslo.versionedobjects

On 09/01/2016 08:29 AM, Henry Nash wrote: From a purely keystone perspective, my gut feeling is that actually the trigger approach is likely to lead to a more robust, not less, solution - due to the fact that we solve the very specific problems of a given migration (i.e. need to keep column A

Re: [openstack-dev] [keystone][nova][neutron][all] Rolling upgrades: database triggers and oslo.versionedobjects

On 08/31/2016 06:18 PM, Monty Taylor wrote: I said this the other day in the IRC channel, and I'm going to say it again here. I'm going to do it as bluntly as I can - please keeping in mind that I respect all of the humans involved. I think this is a monstrously terrible idea. There are

Re: [openstack-dev] [oslo] pymysql change in error formatting has broken exception handing in oslo.db

On 08/31/2016 10:48 AM, Ihar Hrachyshka wrote: Unless we fix the bug in next pymysql, it’s not either/or but both will be needed, and also minimal oslo.db version bump. upstream issue: https://github.com/PyMySQL/PyMySQL/issues/507 PyMySQL tends to be very responsive to issues (plus I

[openstack-dev] [oslo] pymysql change in error formatting has broken exception handing in oslo.db

We need to decide how to handle this: https://review.openstack.org/#/c/362991/ Basically, PyMySQL normally raises an error message like this: (pymysql.err.IntegrityError) (1452, u'Cannot add or update a child row: a foreign key constraint fails (`vaceciqnzs`.`resource_entity`, CONSTRAINT

Re: [openstack-dev] [keystone][nova][neutron][all] Rolling upgrades: database triggers and oslo.versionedobjects

On 08/30/2016 08:04 PM, Clint Byrum wrote: My direct experience with this was MySQL 5.0 and 5.1. They worked as documented, and no I don't think they've changed much since then. When they were actually installed into the schema and up to date with the code that expected them, and the

Re: [openstack-dev] [keystone][nova][neutron][all] Rolling upgrades: database triggers and oslo.versionedobjects

On 08/30/2016 04:43 PM, Clint Byrum wrote: Correct, it is harder for development. Since the database server has all of the potential for the worst problems, being a stateful service, then I believe moving complexity _out_ of it, is generally an operational win, at the expense of some

Re: [openstack-dev] [keystone][nova][neutron][all] Rolling upgrades: database triggers and oslo.versionedobjects

On 08/30/2016 09:57 AM, Clint Byrum wrote: As someone else brought up, this is an unnecessarily bleak view of how database migrations work. We aren't talking about database migrations. We are talking about *online* database migrations, where we would like both the *old* and *new*

Re: [openstack-dev] [keystone][nova][neutron][all] Rolling upgrades: database triggers and oslo.versionedobjects

On 08/25/2016 01:13 PM, Steve Martinelli wrote: The keystone team is pursuing a trigger-based approach to support rolling, zero-downtime upgrades. The proposed operator experience is documented here: http://docs.openstack.org/developer/keystone/upgrading.html This differs from Nova and

Re: [openstack-dev] Let's drop the postgresql gate job

On 08/18/2016 11:00 AM, Matt Riedemann wrote: It's that time of year again to talk about killing this job, at least from the integrated gate (move it to experimental for people that care about postgresql, or make it gating on a smaller subset of projects like oslo.db). Running a full

Re: [openstack-dev] [Neutron][oslo.db] Inspecting sqlite db during unit tests

On 07/25/2016 12:55 PM, Carl Baldwin wrote: On Fri, Jul 22, 2016 at 8:47 AM, Mike Bayer <mba...@redhat.com <mailto:mba...@redhat.com>> wrote: On 07/22/2016 04:02 AM, Kevin Benton wrote: Now that we have switched to oslo.db for test provisioning the re

Re: [openstack-dev] [Neutron][oslo.db] Inspecting sqlite db during unit tests

On 07/22/2016 04:02 AM, Kevin Benton wrote: Now that we have switched to oslo.db for test provisioning the responsibility of choosing a location lands here: https://github.com/openstack/oslo.db/blob/a79479088029e4fa51def91cb36bc652356462b6/oslo_db/sqlalchemy/provision.py#L505 The problem is

Re: [openstack-dev] [Neutron][oslo.db] Inspecting sqlite db during unit tests

On 07/22/2016 04:02 AM, Kevin Benton wrote: Now that we have switched to oslo.db for test provisioning the responsibility of choosing a location lands here: https://github.com/openstack/oslo.db/blob/a79479088029e4fa51def91cb36bc652356462b6/oslo_db/sqlalchemy/provision.py#L505 The problem is

Re: [openstack-dev] [oslo.db] [CC neutron] CIDR overlap functionality and constraints

On 07/21/2016 02:43 PM, Carl Baldwin wrote: None of these operations are expected to be very contentious and performance hasn't really been a concern yet. If it were a big concern, I'd be very interested in the GiST index solution because, as I understand it, detecting overlap without that

[openstack-dev] [oslo.db] [CC neutron] CIDR overlap functionality and constraints

Oslo.db devs : We've developed a system by which CIDR math, such as that of detecting region overlaps, can be performed on a MySQL database within queries [1] [2]. This feature makes use of a custom stored function I helped to produce which provides functionality similar to that which

[openstack-dev] [oslo] [keystone] dogpile.cache 0.6.0 released

gpile.core directly (this should be very rare). Changelog for 0.6.0 is at: http://dogpilecache.readthedocs.io/en/latest/changelog.html#change-0.6.0 On 06/01/2016 04:54 PM, Mike Bayer wrote: Just a reminder, dogpile.cache is doing away with namespace packaging in version 0.6.0, due f

Re: [openstack-dev] [oslo] [keystone] rolling dogpile.core into dogpile.cache, removing namespace packaging (PLEASE REVIEW)

Just a reminder, dogpile.cache is doing away with namespace packaging in version 0.6.0, due for the end of this week or sometime next week. dogpile.core is being retired and left as-is. No changes should be needed by anyone using only dopgile.cache. On 05/30/2016 06:17 PM, Mike Bayer

[openstack-dev] [oslo] [keystone] rolling dogpile.core into dogpile.cache, removing namespace packaging (PLEASE REVIEW)

Hi all - Just a heads up what's happening for dogpile.cache, in version 0.6.0 we are rolling the functionality of the dogpile.core package into dogpile.cache itself, and retiring the use of namespace package naming for dogpile.cache. Towards retiring the use of namespace packaging, the

Re: [openstack-dev] [neutron] Seeing db lockout issues in neutron add_router_interface

On 05/10/2016 04:57 PM, Divya wrote: Hi, I am trying to run this rally test on stable/kilo https://github.com/openstack/rally/blob/master/samples/tasks/scenarios/neutron/create_and_delete_routers.json with concurrency 50 and iterations 2000. This test basically cretaes routers and

Re: [openstack-dev] [nova] Distributed Database

On 05/02/2016 01:48 PM, Clint Byrum wrote: FWIW, I agree with you. If you're going to use SQLAlchemy, use it to take advantage of the relational model. However, how is what you describe a win? Whether you use SELECT .. FOR UPDATE, or a stored procedure, the lock is not distributed, and

Re: [openstack-dev] [nova] Distributed Database

On 05/02/2016 07:38 AM, Matthieu Simonin wrote: As far as we understand the idea of an ORM is to hide the relational database with an Object oriented API. I actually disagree with that completely. The reason ORMs are so maligned is because of this misconception; developer attempts to

Re: [openstack-dev] [Fuel][MySQL][DLM][Oslo][DB][Trove][Galera][operators] Multi-master writes look OK, OCF RA and more things

On 04/30/2016 10:50 AM, Clint Byrum wrote: Excerpts from Roman Podoliaka's message of 2016-04-29 12:04:49 -0700: I'm curious why you think setting wsrep_sync_wait=1 wouldn't help. The exact example appears in the Galera documentation:

Re: [openstack-dev] [Fuel][MySQL][DLM][Oslo][DB][Trove][Galera][operators] Multi-master writes look OK, OCF RA and more things

On 04/30/2016 02:57 AM, bdobre...@mirantis.com wrote: Hi Roman. That's interesting, although’s hard to believe (there is no slave lag in galera multi master). I can only suggest us to create another jepsen test to verify exactly scenario you describe. As well as other OpenStack specific

Re: [openstack-dev] [nova] Distributed Database

On 04/28/2016 08:25 PM, Edward Leafe wrote: Your own tests showed that a single RDBMS instance doesn’t even break a sweat under your test loads. I don’t see why we need to shard it in the first place, especially if in doing so we add another layer of complexity and another dependency in order

Re: [openstack-dev] [nova] Distributed Database

On 04/28/2016 08:44 AM, Edward Leafe wrote: On Apr 24, 2016, at 3:28 PM, Robert Collins wrote: For instance, the things I think are essential for a distributed database based datastore: - good single-machine developer story. Must not need a physical cluster to

Re: [openstack-dev] [oslo.config] Encrypt the sensitive options

On 04/26/2016 09:32 AM, Daniel P. Berrange wrote: IMHO encrypting oslo config files is addressing the wrong problem. Rather than having sensitive passwords stored in the main config files, we should have them stored completely separately by a secure password manager of some kind. The config

Re: [openstack-dev] [nova] Distributed Database

On 04/22/2016 04:27 PM, Ed Leafe wrote: OK, so I know that Friday afternoons are usually the worst times to write a blog post and start an email discussion, and that the Friday immediately before a Summit is the absolute worst, but I did it anyway.

Re: [openstack-dev] Deleting a cluster in Sahara SQL/PyMYSQL Error

cluster, it get stuck on "validating", then tried deleting it but get stuck again on "deleting", and seeing that SQL error. Will submit a bug @ Launchpad. On 25 March 2016 at 01:56, Mike Bayer <mba...@redhat.com <mailto:mba...@redhat.com>>

Re: [openstack-dev] Deleting a cluster in Sahara SQL/PyMYSQL Error

Id recommend filing a bug in Launchpad against Sahara for that. Can you reproduce it ? On 03/23/2016 07:10 PM, Jerico Revote wrote: Hello, When trying to delete a cluster in sahara, I'm getting the following error: code 500 and message 'Internal Server Error' 2016-03-23 17:25:21.651

Re: [openstack-dev] [tricircle] Using in-memory database for unit tests

On 03/23/2016 01:33 AM, Vega Cai wrote: On 22 March 2016 at 12:09, Shinobu Kinjo > wrote: Thank you for your comment (inline for my message). On Tue, Mar 22, 2016 at 11:53 AM, Vega Cai

Re: [openstack-dev] [oslo] documentation on using the oslo.db opportunistic test feature

On 02/23/2016 12:20 PM, Sean Dague wrote: On 02/23/2016 11:29 AM, Mike Bayer wrote: On 02/22/2016 08:18 PM, Sean Dague wrote: On 02/22/2016 08:08 PM, Davanum Srinivas wrote: Sean, You need to set the env variable like so. See testenv:mysql-python for example

Re: [openstack-dev] [oslo] oslo.db reset session?

MySQLOpportunisticTest classes it has. On Tue, Feb 23, 2016 at 6:09 PM, Mike Bayer <mba...@redhat.com> wrote: On 02/23/2016 09:22 AM, Sean Dague wrote: With enginefascade working coming into projects, there seems to be some new bits around oslo.db global sessions. The effect of this on tests is a

Re: [openstack-dev] [oslo] documentation on using the oslo.db opportunistic test feature

On 02/22/2016 08:18 PM, Sean Dague wrote: On 02/22/2016 08:08 PM, Davanum Srinivas wrote: Sean, You need to set the env variable like so. See testenv:mysql-python for example OS_TEST_DBAPI_ADMIN_CONNECTION=mysql://openstack_citest:openstack_citest@localhost Thanks, Dims [1]

Re: [openstack-dev] [oslo] documentation on using the oslo.db opportunistic test feature

On 02/22/2016 08:08 PM, Davanum Srinivas wrote: Sean, You need to set the env variable like so. See testenv:mysql-python for example OS_TEST_DBAPI_ADMIN_CONNECTION=mysql://openstack_citest:openstack_citest@localhost you should not need to set this if you're using the default URL. The

Re: [openstack-dev] [oslo] documentation on using the oslo.db opportunistic test feature

On 02/22/2016 08:02 PM, Sean Dague wrote: Before migrating into oslo.db the opportunistic testing for database backends was pretty simple. Create an openstack_citest@openstack_citest pw:openstack_citest and you could get tests running on mysql. This no longer seems to be the case. this is

Re: [openstack-dev] [oslo] oslo.db reset session?

On 02/23/2016 09:22 AM, Sean Dague wrote: With enginefascade working coming into projects, there seems to be some new bits around oslo.db global sessions. The effect of this on tests is a little problematic. Because it builds global state which couples between tests. I've got a review to use

Re: [openstack-dev] [nova] nova-compute blocking main thread under heavy disk IO

On 02/22/2016 11:30 AM, Chris Friesen wrote: On 02/22/2016 11:17 AM, Jay Pipes wrote: On 02/22/2016 10:43 AM, Chris Friesen wrote: Hi all, We've recently run into some interesting behaviour that I thought I should bring up to see if we want to do anything about it. Basically the problem

Re: [openstack-dev] [all] Please do *not* use git (and specifically "git log") when generating the docs

On 02/18/2016 04:39 PM, Dolph Mathews wrote: On Thu, Feb 18, 2016 at 11:17 AM, Thomas Goirand > wrote: Hi, I've seen Reno doing it, then some more. It's time that I raise the issue globally in this list before the epidemic spreads to the

Re: [openstack-dev] [oslo][oslo.versionedobjects] Is it possible to make changes to oslo repos?

On 01/28/2016 01:52 PM, Doug Hellmann wrote: > Excerpts from Hayes, Graham's message of 2016-01-28 17:01:09 +: >> Recently I tried started to use oslo.versionedobjects for a project. >> >> After playing around with it for a while, I noticed I could set "this is >> not a uuid" as the value of

Re: [openstack-dev] [neutron] InvalidRequestError: This session is in 'prepared' state; no further SQL can be emitted within this transaction.

On 01/11/2016 03:58 AM, Koteswar wrote: > Hi All, > > > > In my mechanism driver, I am reading/writing into sql db in a fixed > interval looping call. Sometimes I get the following error when I stop > and start neutron server > > InvalidRequestError: This session is in 'prepared' state; no

Re: [openstack-dev] [nova] [all] Excessively high greenlet default + excessively low connection pool defaults leads to connection pool latency, timeout errors, idle database connections / workers

On 01/11/2016 05:39 AM, Radomir Dopieralski wrote: > On 01/08/2016 09:51 PM, Mike Bayer wrote: >> >> >> On 01/08/2016 04:44 AM, Radomir Dopieralski wrote: >>> On 01/07/2016 05:55 PM, Mike Bayer wrote: >>> >>>> but also even if you're under

Re: [openstack-dev] [nova] [all] Excessively high greenlet default + excessively low connection pool defaults leads to connection pool latency, timeout errors, idle database connections / workers

On 01/08/2016 04:44 AM, Radomir Dopieralski wrote: > On 01/07/2016 05:55 PM, Mike Bayer wrote: > >> but also even if you're under something like >> mod_wsgi, you can spawn a child process or worker thread regardless. >> You always have a Python interpreter running a

Re: [openstack-dev] [nova] [all] Excessively high greenlet default + excessively low connection pool defaults leads to connection pool latency, timeout errors, idle database connections / workers

On 01/07/2016 11:02 AM, Sean Dague wrote: > On 01/07/2016 09:56 AM, Brant Knudson wrote: >> >> >> On Thu, Jan 7, 2016 at 6:39 AM, Clayton O'Neill > > wrote: >> >> On Thu, Jan 7, 2016 at 2:49 AM, Roman Podoliaka >>

Re: [openstack-dev] [nova] [all] Excessively high greenlet default + excessively low connection pool defaults leads to connection pool latency, timeout errors, idle database connections / workers

On 01/06/2016 09:11 AM, Roman Podoliaka wrote: > Hi Mike, > > Thank you for this brilliant analysis! We've been seeing such timeout > errors in downstream periodically and this is the first time someone > has analysed the root cause thoroughly. > > On Fri, Dec 18, 2015 at

Re: [openstack-dev] The command "neutron-db-manage" of 8.0.0~b1 fails

On 01/04/2016 06:59 AM, Ihar Hrachyshka wrote: > Martinx - ジェームズ wrote: > >> Guys, >> >> I'm trying to experiment Mitaka on Ubuntu Xenial, which already have >> beta version on its repositories, however, "neutron-db-manage" fails. >> >> Here is the output of it: >>

Re: [openstack-dev] The command "neutron-db-manage" of 8.0.0~b1 fails

On 01/03/2016 10:57 PM, Martinx - ジェームズ wrote: > On 4 January 2016 at 01:28, Mike Bayer <mba...@redhat.com> wrote: >> >> >> On 01/03/2016 05:15 PM, Martinx - ジェームズ wrote: >>> Guys, >>> >>> I'm trying to experiment Mitaka on Ubuntu Xenial, w

Re: [openstack-dev] The command "neutron-db-manage" of 8.0.0~b1 fails

On 01/03/2016 05:15 PM, Martinx - ジェームズ wrote: > Guys, > > I'm trying to experiment Mitaka on Ubuntu Xenial, which already have > beta version on its repositories, however, "neutron-db-manage" fails. > > Here is the output of it: > > http://paste.openstack.org/show/482920/ > > Any clue?

Re: [openstack-dev] Nova scheduler startup when database is not available

On 12/23/2015 01:32 PM, Jay Pipes wrote: > On 12/23/2015 12:27 PM, Lars Kellogg-Stedman wrote: >> I've been looking into the startup constraints involved when launching >> Nova services with systemd using Type=notify (which causes systemd to >> wait for an explicit notification from the service

  1   2   3   4   >