Re: [openstack-dev] [oslo] [telemetry] Oslo.db 4.13.1 broke Gnocchi

On 09/02/2016 04:43 AM, Julien Danjou wrote: On Fri, Sep 02 2016, Julien Danjou wrote: I'll look into fixing that, though any help would be welcome. My attempt at: https://review.openstack.org/364767 I've augmented it with mapper-level SQLAlchemy API use.

Re: [openstack-dev] [oslo] [telemetry] Oslo.db 4.13.1 broke Gnocchi

is the failure here something that comes up in gnocchi's test suite? Could there be some way that oslo libraries run the test suites of all consuming projects before a patch and/or a release? (apologies if we already do this). On 09/02/2016 12:17 PM, Matthew Thode wrote: On 09/02/2016

Re: [openstack-dev] [keystone][nova][neutron][all] Rolling upgrades: database triggers and oslo.versionedobjects

On 09/02/2016 01:53 PM, Doug Hellmann wrote: Excerpts from Thierry Carrez's message of 2016-09-02 12:15:33 +0200: Sean Dague wrote: Putting DB trigger failure analysis into the toolkit required to manage an upgrade failure is a really high bar for new ops. I agree with Sean: increasing the

Re: [openstack-dev] [keystone][nova][neutron][all] Rolling upgrades: database triggers and oslo.versionedobjects

On 09/01/2016 11:52 AM, Dan Smith wrote: The indirection service is really unrelated to this discussion, IMHO. If you take RPC out of the picture, all you have left is a direct-to-the-database facade to handle the fact that schema has expanded underneath you. As Clint (et al) have said --

Re: [openstack-dev] [keystone][nova][neutron][all] Rolling upgrades: database triggers and oslo.versionedobjects

On 08/30/2016 09:57 AM, Clint Byrum wrote: As someone else brought up, this is an unnecessarily bleak view of how database migrations work. We aren't talking about database migrations. We are talking about *online* database migrations, where we would like both the *old* and *new*

Re: [openstack-dev] [keystone][nova][neutron][all] Rolling upgrades: database triggers and oslo.versionedobjects

On 08/31/2016 06:18 PM, Monty Taylor wrote: I said this the other day in the IRC channel, and I'm going to say it again here. I'm going to do it as bluntly as I can - please keeping in mind that I respect all of the humans involved. I think this is a monstrously terrible idea. There are

Re: [openstack-dev] [keystone][nova][neutron][all] Rolling upgrades: database triggers and oslo.versionedobjects

On 09/01/2016 08:29 AM, Henry Nash wrote: From a purely keystone perspective, my gut feeling is that actually the trigger approach is likely to lead to a more robust, not less, solution - due to the fact that we solve the very specific problems of a given migration (i.e. need to keep column A

Re: [openstack-dev] [keystone][nova][neutron][all] Rolling upgrades: database triggers and oslo.versionedobjects

On 08/30/2016 04:43 PM, Clint Byrum wrote: Correct, it is harder for development. Since the database server has all of the potential for the worst problems, being a stateful service, then I believe moving complexity _out_ of it, is generally an operational win, at the expense of some

Re: [openstack-dev] [keystone][nova][neutron][all] Rolling upgrades: database triggers and oslo.versionedobjects

On 08/30/2016 08:04 PM, Clint Byrum wrote: My direct experience with this was MySQL 5.0 and 5.1. They worked as documented, and no I don't think they've changed much since then. When they were actually installed into the schema and up to date with the code that expected them, and the

[openstack-dev] [oslo] pymysql change in error formatting has broken exception handing in oslo.db

We need to decide how to handle this: https://review.openstack.org/#/c/362991/ Basically, PyMySQL normally raises an error message like this: (pymysql.err.IntegrityError) (1452, u'Cannot add or update a child row: a foreign key constraint fails (`vaceciqnzs`.`resource_entity`, CONSTRAINT

Re: [openstack-dev] [oslo] pymysql change in error formatting has broken exception handing in oslo.db

On 08/31/2016 10:48 AM, Ihar Hrachyshka wrote: Unless we fix the bug in next pymysql, it’s not either/or but both will be needed, and also minimal oslo.db version bump. upstream issue: https://github.com/PyMySQL/PyMySQL/issues/507 PyMySQL tends to be very responsive to issues (plus I

Re: [openstack-dev] [oslo.db] [release] opportunistic tests breaking randomly

On 09/14/2016 11:08 AM, Mike Bayer wrote: On 09/14/2016 09:15 AM, Sean Dague wrote: I noticed the following issues happening quite often now in the opportunistic db tests for nova - http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C

Re: [openstack-dev] [oslo.db] [release] opportunistic tests breaking randomly

On 09/14/2016 09:15 AM, Sean Dague wrote: I noticed the following issues happening quite often now in the opportunistic db tests for nova - http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C%22sqlalchemy.exc.ResourceClosedError%5C%22 It looks like some race has

Re: [openstack-dev] [oslo.db] [release] opportunistic tests breaking randomly

On 09/14/2016 07:04 PM, Alan Pevec wrote: Olso.db 4.13.3 did hit the scene about the time this showed up. So I think we need to strongly consider blocking it and revisiting these issues post newton. So that means reverting all stable/newton changes, previous 4.13.x have been already blocked

Re: [openstack-dev] [oslo.db] [release] opportunistic tests breaking randomly

here. fits much more with your initial description On 09/14/2016 10:48 PM, Mike Bayer wrote: On 09/14/2016 07:04 PM, Alan Pevec wrote: Olso.db 4.13.3 did hit the scene about the time this showed up. So I think we need to strongly consider blocking it and revisiting these issues post new

Re: [openstack-dev] [oslo.db] [release] opportunistic tests breaking randomly

On 09/14/2016 11:05 PM, Mike Bayer wrote: Are *these* errors also new as of version 4.13.3 of oslo.db ? Because here I have more suspicion of one particular oslo.db change here. The version in question that has the changes to provisioning and anything really to do with this area

Re: [openstack-dev] [oslo.db] [release] opportunistic tests breaking randomly

On 09/21/2016 11:41 AM, Joshua Harlow wrote: I've seen something similar at https://review.openstack.org/#/c/316935/ Maybe its time we asked again why are we still using eventlet and do we need to anymore. What functionality of it are people actually taking advantage of? If it's supporting

Re: [openstack-dev] [oslo] [telemetry] [requirements] [FFE] Oslo.db 4.13.2

On 09/07/2016 01:29 PM, Doug Hellmann wrote: Excerpts from Matthew Thode's message of 2016-09-07 09:11:58 -0500: On 09/07/2016 08:58 AM, Doug Hellmann wrote: Excerpts from Matthew Thode's message of 2016-09-07 08:21:50 -0500: https://review.openstack.org/366298 This is just a bump to

Re: [openstack-dev] [keystone][nova][neutron][all] Rolling upgrades: database triggers and oslo.versionedobjects

On 08/25/2016 01:13 PM, Steve Martinelli wrote: The keystone team is pursuing a trigger-based approach to support rolling, zero-downtime upgrades. The proposed operator experience is documented here: http://docs.openstack.org/developer/keystone/upgrading.html This differs from Nova and

Re: [openstack-dev] [cinder][db] lazy loading of an attribute impossible

On 09/30/2016 10:54 AM, Roman Podoliaka wrote: Michał, You are absolutely right: this exception is raised when you try to lazy-load instance attributes outside a Session scope. There is an obvious problem with that - instances do not communicate with a DB on their own - it's left up to

Re: [openstack-dev] [nova][oslo][openstack-ansible] DB deadlocks, Mitaka, and You

On 10/19/2016 08:36 AM, Ian Cordasco wrote: Hey Kevin, So just looking at the pastes you have here, I'm inclined to believe this is actually a bug in oslo_db/sqlalchemy. If you follow the trace, there's a PyMySQL InternalError not being handled inside of sqlalchemy. I'm not sure if SQLAlchemy

Re: [openstack-dev] [networking-sfc] Intermittent database transaction issues, affecting the tempest gate

On 12/20/2016 06:50 PM, Cathy Zhang wrote: Hi Bernard, Thanks for the email. I will take a look at this. Xiaodong has been working on tempest test scripts. I will work with Xiaodong on this issue. I've added a comment to the issue which refers to upstream SQLAlchemy issue

[openstack-dev] [oslo][requirements][all] requesting assistance to unblock SQLAlchemy 1.1 from requirements

Hello all - As mentioned previously, SQLAlchemy 1.1 has now been released for about six months. My work now is on SQLAlchemy 1.2 which should hopefully see initial releases in late spring.SQLAlchemy 1.1 includes tons of features, bugfixes, and improvements, and in particular the most

Re: [openstack-dev] [oslo][requirements][all] requesting assistance to unblock SQLAlchemy 1.1 from requirements

On 03/15/2017 07:30 AM, Sean Dague wrote: The problem was the original patch kept a cap on SQLA, just moved it up to the next pre-release, not realizing the caps in general are the concern by the requirements team. So instead of upping the cap, I just removed it entirely. (It also didn't help

Re: [openstack-dev] [oslo][requirements][all] requesting assistance to unblock SQLAlchemy 1.1 from requirements

On Wed, Mar 15, 2017 at 5:20 PM, Sean Dague <s...@dague.net> wrote: On 03/15/2017 10:38 AM, Mike Bayer wrote: On 03/15/2017 07:30 AM, Sean Dague wrote: The problem was the original patch kept a cap on SQLA, just moved it up to the next pre-release, not realizing the caps in general are

Re: [openstack-dev] [Zun]Use 'uuid' instead of 'id' as object ident in data model

On 04/05/2017 11:02 AM, gordon chung wrote: On 05/04/17 09:00 AM, Monty Taylor wrote: Please do NOT use uuid as a primary key in MySQL: * UUID has 36 characters which makes it bulky. you can store it as a binary if space is a concern. this is highly inconvenient from a datadump /

Re: [openstack-dev] [Zun]Use 'uuid' instead of 'id' as object ident in data model

On 04/05/2017 11:00 AM, Monty Taylor wrote: On 04/05/2017 09:39 AM, Akihiro Motoki wrote: I noticed this thread by Monty's reply. Sorry for my late :( I think we need to think 'id' separately for API modeling and DB modeling. In the API perspective, one of the important things is that 'id'

Re: [openstack-dev] [all] etcd3 as base service - update

/c528539879e824b8e6d5654292a85ccbee6dcf89/keystone/conf/fernet_tokens.py#L44-L54 [1] https://launchpad.net/bugs/1649616 On Thu, Jun 8, 2017 at 11:37 AM, Mike Bayer <mba...@redhat.com <mailto:mba...@redhat.com> <mailto:mba...@redhat.com <mailto:mba...@redh

Re: [openstack-dev] [oslo.db] Stepping down from core

hey Roman - It was a huge pleasure working w/ you on oslo.db!I hope we can collaborate again soon. - mike On 06/11/2017 10:32 AM, Roman Podoliaka wrote: Hi all, I recently changed job and hasn't been able to devote as much time to oslo.db as it is expected from a core reviewer. I'm

Re: [openstack-dev] [trove][all][tc] A proposal to rearchitect Trove

On 06/20/2017 11:45 AM, Jay Pipes wrote: Good discussion, Zane. Comments inline. On 06/20/2017 11:01 AM, Zane Bitter wrote: 2) The database VMs are created in a project belonging to the operator of the service. They're connected to the user's network through , and isolated from other

Re: [openstack-dev] [trove][all][tc] A proposal to rearchitect Trove

On 06/22/2017 11:59 AM, Fox, Kevin M wrote: My $0.02. That view of dependencies is why Kubernetes development is outpacing OpenStacks and some users are leaving IMO. Not trying to be mean here but trying to shine some light on this issue. Kubernetes at its core has essentially something

Re: [openstack-dev] [tc] revised Postgresql support status patch for governance

On 05/17/2017 02:38 PM, Sean Dague wrote: Some of the concerns/feedback has been "please describe things that are harder by this being an abstraction", so examples are provided. so let's go through this list: - OpenStack services taking a more active role in managing the DBMS ,

Re: [openstack-dev] [oslo] Can we stop global requirements update?

On 05/16/2017 05:42 AM, Julien Danjou wrote: On Wed, Apr 19 2017, Julien Danjou wrote: So Gnocchi gate is all broken (agan) because it depends on "pbr" and some new release of oslo.* depends on pbr!=2.1.0. Same things happened today with Babel. As far as Gnocchi is concerned, we're

Re: [openstack-dev] [tc] Active or passive role with our database layer

On 05/23/2017 01:10 PM, Octave J. Orgeron wrote: Comments below.. On 5/21/2017 1:38 PM, Monty Taylor wrote: For example: An HA strategy using slave promotion and a VIP that points at the current write master paired with an application incorrectly configured to do such a thing can lead to

Re: [openstack-dev] [tc] Active or passive role with our database layer

On 05/23/2017 03:16 PM, Edward Leafe wrote: On May 23, 2017, at 1:43 PM, Jay Pipes wrote: [1] Witness the join constructs in Golang in Kubernetes as they work around etcd not being a relational data store: Maybe it’s just me, but I found that Go code more

Re: [openstack-dev] [upgrades][skip-level][leapfrog] - RFC - Skipping releases when upgrading

On 05/26/2017 10:56 AM, Dan Smith wrote: As most of the upgrade issues center around database migrations, we discussed some of the potential pitfalls at length. One approach was to roll-up all DB migrations into a single repository and run all upgrades for a given project in one step. Another

Re: [openstack-dev] [nova] Boston Forum session recap - searchlight integration

On 05/22/2017 05:39 AM, Matthew Booth wrote: There are also a couple of optimisations to make which I won't bother with up front. Dan suggested in his CellsV2 talk that we would only query cells where the user actually has instances. If we find users tend to clump in a small number of

Re: [openstack-dev] [Keystone] Cockroachdb for Keystone Multi-master

On 05/22/2017 05:02 AM, Thierry Carrez wrote: Mike Bayer wrote: On 05/18/2017 06:13 PM, Adrian Turjak wrote: So, specifically in the realm of Keystone, since we are using sqlalchemy we already have Postgresql support, and since Cockroachdb does talk Postgres it shouldn't be too hard to back

Re: [openstack-dev] [oslo] Can we stop global requirements update?

On 05/18/2017 02:37 PM, Julien Danjou wrote: On Thu, May 18 2017, Mike Bayer wrote: I'm not understanding this? do you mean this? In the long run, yes. Unfortunately, we're not happy with the way Oslo libraries are managed and too OpenStack centric. I've tried for the last couple of years

Re: [openstack-dev] [oslo] Can we stop global requirements update?

On 05/19/2017 04:23 AM, Mehdi Abaakouk wrote: And some applications rely on implicit internal contract/behavior/assumption. IMO that's a bug for them.I'm inspired to see that Keystone, Nova etc. are able to move between and eventlet backend and a mod_wsgi backend.IMO eventlet is

Re: [openstack-dev] [oslo] can we make everyone drop eventlet? (was: Can we stop global requirements update?)

FTFY On 05/19/2017 03:58 PM, Joshua Harlow wrote: Mehdi Abaakouk wrote: Not really, I just put some comments on reviews and discus this on IRC. Since nobody except Telemetry have expressed/try to get rid of eventlet. Octavia is using cotyledon and they have gotten rid of eventlet. Didn't

Re: [openstack-dev] [Keystone] Cockroachdb for Keystone Multi-master

On 05/18/2017 06:13 PM, Adrian Turjak wrote: So, specifically in the realm of Keystone, since we are using sqlalchemy we already have Postgresql support, and since Cockroachdb does talk Postgres it shouldn't be too hard to back Keystone with it. At that stage you have a Keystone DB that could

Re: [openstack-dev] [nova] Boston Forum session recap - searchlight integration

On 05/19/2017 02:46 AM, joehuang wrote: Support sort and pagination together will be the biggest challenge: it's up to how many cells will be involved in the query, 3,5 may be OK, you can search each cells, and cached data. But how about 20, 50 or more, and how many data will be cached?

Re: [openstack-dev] [Keystone] Cockroachdb for Keystone Multi-master

On 05/30/2017 09:06 PM, Jay Pipes wrote: On 05/30/2017 05:07 PM, Clint Byrum wrote: Excerpts from Jay Pipes's message of 2017-05-30 14:52:01 -0400: Sorry for the delay in getting back on this... comments inline. On 05/18/2017 06:13 PM, Adrian Turjak wrote: Hello fellow OpenStackers, For

Re: [openstack-dev] [all] etcd3 as base service - update

On 06/08/2017 12:47 AM, Joshua Harlow wrote: So just out of curiosity, but do people really even know what etcd is good for? I am thinking that there should be some guidance from folks in the community as to where etcd should be used and where it shouldn't (otherwise we just all end up in a

Re: [openstack-dev] [all] etcd3 as base service - update

hing. that was a terrible architectural decision, with apologies to whoever made it.if you're putting some kind of "we create an infinite, rapidly growing, turns-to-garbage-in-30-seconds" kind of data in a database, removing that data robustly and ASAP needs to be part of the proc

Re: [openstack-dev] [all] etcd3 as base service - update

On 06/08/2017 04:24 PM, Julien Danjou wrote: On Thu, Jun 08 2017, Mike Bayer wrote: So I wouldn't be surprised if new / existing openstack applications express some gravitational pull towards using it as their own datastore as well. I'll be trying to hang onto the etcd3 track as much

Re: [openstack-dev] [oslo] Can we stop global requirements update?

On 05/20/2017 12:04 PM, Julien Danjou wrote: On Fri, May 19 2017, Mike Bayer wrote: IMO that's a bug for them. Of course it's a bug. IIRC Mehdi tried to fix it without much success. I'm inspired to see that Keystone, Nova etc. are able to move between and eventlet backend and a mod_wsgi

Re: [openstack-dev] [tc] revised Postgresql support status patch for governance

On 05/21/2017 03:51 PM, Monty Taylor wrote: So I don't see the problem of "consistent utf8 support" having much to do with whether or not we support Posgtresql - you of course need your "CREATE DATABASE" to include the utf8 charset like we do on MySQL, but that's it. That's where we stand

Re: [openstack-dev] [tc] Active or passive role with our database layer

On 05/21/2017 03:38 PM, Monty Taylor wrote: documentation on the sequence of steps the operator should take. In the "active" approach, we still document expectations, but we also validate them. If they are not what we expect but can be changed at runtime, we change them overriding

<    1   2   3   4