Re: [openstack-dev] [stable][all] Revisiting the 6 month release cycle

2015-02-25 Thread Johannes Erdfelt
On Tue, Feb 24, 2015, Jeremy Stanley fu...@yuggoth.org wrote: On 2015-02-24 10:00:51 -0800 (-0800), Johannes Erdfelt wrote: [...] Recently, I have spent a lot more time waiting on reviews than I have spent writing the actual code. That's awesome, assuming what you mean here is that you've

Re: [openstack-dev] [stable][all] Revisiting the 6 month release cycle

2015-02-24 Thread Johannes Erdfelt
On Mon, Feb 23, 2015, Joe Gordon joe.gord...@gmail.com wrote: What this actually means: - Stop approving blueprints for specific stable releases, instead just approve them and target them to milestones. - Milestones stop becoming Kilo-1, Kilo-2, Kilo-3 etc. and just become

Re: [openstack-dev] [stable][all] Revisiting the 6 month release cycle

2015-02-24 Thread Johannes Erdfelt
On Tue, Feb 24, 2015, Thierry Carrez thie...@openstack.org wrote: Agree on the pain of maintaining milestone plans though, which is why I propose we get rid of most of it in Liberty. That will actually be discussed at the cross-project meeting today:

Re: [openstack-dev] [nova] The strange case of osapi_compute_unique_server_name_scope

2015-02-20 Thread Johannes Erdfelt
On Thu, Feb 19, 2015, Matthew Booth mbo...@redhat.com wrote: Assuming this configurability is required, is there any way we can instead use it to control a unique constraint in the db at service startup? This would be something akin to a db migration. How do we manage those? Ignoring if this

Re: [openstack-dev] [all][tc] SQL Schema Downgrades and Related Issues

2015-01-29 Thread Johannes Erdfelt
On Thu, Jan 29, 2015, Morgan Fainberg morgan.fainb...@gmail.com wrote: The concept that there is a utility that can (and in many cases willfully) cause permanent, and in some cases irrevocable, data loss from a simple command line interface sounds crazy when I try and explain it to someone.

Re: [openstack-dev] [oslo.db] PyMySQL review

2015-01-28 Thread Johannes Erdfelt
On Wed, Jan 28, 2015, Clint Byrum cl...@fewbar.com wrote: As is often the case with threading, a reason to avoid using it is that libraries often aren't able or willing to assert thread safety. That said, one way to fix that, is to fix those libraries that we do want to use, to be thread

Re: [openstack-dev] [oslo.db] PyMySQL review

2015-01-28 Thread Johannes Erdfelt
On Wed, Jan 28, 2015, Mike Bayer mba...@redhat.com wrote: I can envision turning this driver into a total monster, adding C-speedups where needed but without getting in the way of async patching, adding new APIs for explicit async, and everything else. However, I’ve no idea what the developers

Re: [openstack-dev] [oslo.db] PyMySQL review

2015-01-28 Thread Johannes Erdfelt
On Wed, Jan 28, 2015, Vishvananda Ishaya vishvana...@gmail.com wrote: On Jan 28, 2015, at 4:03 PM, Doug Hellmann d...@doughellmann.com wrote: I hope someone who was around at the time will chime in with more detail about why green threads were deemed better than regular threads, and I look

Re: [openstack-dev] design question : green thread model

2015-01-28 Thread Johannes Erdfelt
On Wed, Jan 28, 2015, murali reddy muralimmre...@gmail.com wrote: I am trying to understand how a nova component can be run parallely on a host. From the developer reference documentation it seems to indicate that all the openstack services use green thread model of threading. Is it the only

Re: [openstack-dev] design question : green thread model

2015-01-28 Thread Johannes Erdfelt
On Wed, Jan 28, 2015, murali reddy muralimmre...@gmail.com wrote: On hosts with multi-core processors, it does not seem optimal to run a single service instance with just green thread. I understand that on controller node, we can run one or more nova services but still it does not seem to

Re: [openstack-dev] [nova] Adding new features to Kilo and future releases - DB upgrades

2015-01-26 Thread Johannes Erdfelt
On Thu, Jan 22, 2015, Kekane, Abhishek abhishek.kek...@nttdata.com wrote: With online schema changes/No downtime DB upgrades things would be much lot easier for OpenStack deployments. Big kudos to Johannes who initiated this feature. But as a service provider, I'm curious to understand what is

Re: [openstack-dev] [hacking] proposed rules drop for 1.0

2014-12-09 Thread Johannes Erdfelt
On Tue, Dec 09, 2014, Sean Dague s...@dague.net wrote: This check should run on any version of python and give the same results. It does not, because it queries python to know what's in stdlib vs. not. Just to underscore that it's difficult to get right, I found out recently that hacking

Re: [openstack-dev] [hacking] proposed rules drop for 1.0

2014-12-09 Thread Johannes Erdfelt
On Tue, Dec 09, 2014, Sean Dague s...@dague.net wrote: I'd like to propose that for hacking 1.0 we drop 2 groups of rules entirely. 1 - the entire H8* group. This doesn't function on python code, it functions on git commit message, which makes it tough to run locally. It also would be a

Re: [openstack-dev] [Nova] Spring cleaning nova-core

2014-12-07 Thread Johannes Erdfelt
On Mon, Dec 08, 2014, Michael Still mi...@stillhq.com wrote: There are other things happening behind the scenes as well -- we have a veto process for current cores when we propose a new core. It has been made clear to me that several current core members believe we have reached the maximum

Re: [openstack-dev] [Nova] sqlalchemy-migrate vs alembic for new database

2014-12-05 Thread Johannes Erdfelt
On Fri, Dec 05, 2014, Andrew Laski andrew.la...@rackspace.com wrote: The cells v2 effort is going to be introducing a new database into Nova. This has been an opportunity to rethink and approach a few things differently, including how we should handle migrations. There have been discussions

Re: [openstack-dev] [all][oslo][db][docs] RFC: drop support for libpq 9.1

2014-10-06 Thread Johannes Erdfelt
On Mon, Oct 06, 2014, Ihar Hrachyshka ihrac...@redhat.com wrote: Maybe it is indeed wasteful, I don't have numbers; though the fact is we don't allow any migrations for databases with any non utf8 tables as of [1]. The code was copied in multiple projects (Nova, Glance among other things).

Re: [openstack-dev] [nova] formally distinguish server desired state from actual state?

2014-10-01 Thread Johannes Erdfelt
On Wed, Oct 01, 2014, Chris Friesen chris.frie...@windriver.com wrote: Currently in nova we have the vm_state, which according to the code comments is supposed to represent a VM's current stable (not transition) state, or what the customer expect the VM to be. However, we then added in an

Re: [openstack-dev] [neutron][all] switch from mysqldb to another eventlet aware mysql client

2014-09-12 Thread Johannes Erdfelt
On Fri, Sep 12, 2014, Doug Hellmann d...@doughellmann.com wrote: I don’t think we will want to retroactively change the migration scripts (that’s not something we generally like to do), We don't allow semantic changes to migration scripts since people who have already run it won't get those

[openstack-dev] [Nova] [Spec freeze exception] Online Schema Changes

2014-07-18 Thread Johannes Erdfelt
I'm requestion a spec freeze exception for online schema changes. https://review.openstack.org/102545 This work is being done to try to minimize the downtime as part of upgrades. Database migrations have historically been a source of long periods of downtime. The spec is an attempt to start

Re: [openstack-dev] [nova] fair standards for all hypervisor drivers

2014-07-17 Thread Johannes Erdfelt
On Thu, Jul 17, 2014, Daniel P. Berrange berra...@redhat.com wrote: On Wed, Jul 16, 2014 at 09:44:55AM -0700, Johannes Erdfelt wrote: So that means the libvirt driver will be a mix of tested and untested features, but only the tested code paths will be enabled by default? The gate

Re: [openstack-dev] [nova] fair standards for all hypervisor drivers

2014-07-17 Thread Johannes Erdfelt
On Thu, Jul 17, 2014, Russell Bryant rbry...@redhat.com wrote: On 07/17/2014 02:31 PM, Johannes Erdfelt wrote: It kind of helps. It's still implicit in that you need to look at what features are enabled at what version and determine if it is being tested. But the behavior is still

Re: [openstack-dev] [nova] fair standards for all hypervisor drivers

2014-07-16 Thread Johannes Erdfelt
On Wed, Jul 16, 2014, Mark McLoughlin mar...@redhat.com wrote: No, there are features or code paths of the libvirt 1.2.5+ driver that aren't as well tested as the class A designation implies. And we have a proposal to make sure these aren't used by default:

Re: [openstack-dev] [nova][vmware] Convert to rescue by adding the rescue image and booting from it

2014-07-14 Thread Johannes Erdfelt
On Mon, Jul 14, 2014, Daniel P. Berrange berra...@redhat.com wrote: I think that I'd probably say there is an expectation that the rescue image will be different from the primary image the OS was booted from. So every image would now need a corresponding rescue image? JE

Re: [openstack-dev] [nova] Do any hyperviors allow disk reduction as part of resize ?

2014-06-13 Thread Johannes Erdfelt
On Fri, Jun 13, 2014, Russell Bryant rbry...@redhat.com wrote: On 06/13/2014 09:22 AM, Day, Phil wrote: I guess the question I’m really asking here is: “Since we know resize down won’t work in all cases, and the failure if it does occur will be hard for the user to detect, should we just

Re: [openstack-dev] [nova] Do any hyperviors allow disk reduction as part of resize ?

2014-06-13 Thread Johannes Erdfelt
On Fri, Jun 13, 2014, Andrew Laski andrew.la...@rackspace.com wrote: On 06/13/2014 10:53 AM, Johannes Erdfelt wrote: On Fri, Jun 13, 2014, Russell Bryant rbry...@redhat.com wrote: On 06/13/2014 09:22 AM, Day, Phil wrote: I guess the question I’m really asking here is: “Since we know resize

Re: [openstack-dev] Promoting healing script to scheme migration script?

2014-06-09 Thread Johannes Erdfelt
On Mon, Jun 09, 2014, Jakub Libosvar libos...@redhat.com wrote: I'd like to get some opinions on following idea: Because currently we have (thanks to Ann) WIP of healing script capable of changing database scheme by comparing tables in the database to models in current codebase, I started to

Re: [openstack-dev] Gate and Skipped Tests

2014-05-23 Thread Johannes Erdfelt
On Fri, May 23, 2014, Rick Harris rconradhar...@gmail.com wrote: On Thu, May 22, 2014 at 7:31 PM, Johannes Erdfelt johan...@erdfelt.comwrote: I noticed recently that some tests are being skipped in the Nova gate. Some will always be skipped, but others are conditional. I'd like to hear

[openstack-dev] Gate and Skipped Tests

2014-05-22 Thread Johannes Erdfelt
I noticed recently that some tests are being skipped in the Nova gate. Some will always be skipped, but others are conditional. In particular the ZooKeeper driver tests are being skipped because an underlying python module is missing. It seems to me that we should want no tests to be

Re: [openstack-dev] [oslo] Logging exceptions and Python 3

2014-05-21 Thread Johannes Erdfelt
On Wed, May 21, 2014, John Dennis jden...@redhat.com wrote: But that's a bug in the logging implementation. Are we supposed to write perverse code just to avoid coding mistakes in other modules? Why not get the fundamental problem fixed? It has been fixed, by making Python 3 :) This is a

Re: [openstack-dev] [Neutron] Consistency between models and migrations

2014-05-19 Thread Johannes Erdfelt
On Tue, May 20, 2014, Collins, Sean sean_colli...@cable.comcast.com wrote: I've been looking at two reviews that Ann Kamyshnikova has proposed https://review.openstack.org/#/c/82073/ https://review.openstack.org/#/c/80518/ I think the changes are fundamentally a Good Thing™ - they

Re: [openstack-dev] [oslo] Logging exceptions and Python 3

2014-05-17 Thread Johannes Erdfelt
On Fri, May 16, 2014, Victor Stinner victor.stin...@enovance.com wrote: See my documentation: https://wiki.openstack.org/wiki/Python3#logging_module_and_format_exceptions six.text_type(exc): always use Unicode. It may raise unicode error depending on the exception, be careful. Example of

Re: [openstack-dev] [oslo] Logging exceptions and Python 3

2014-05-17 Thread Johannes Erdfelt
On Fri, May 16, 2014, Igor Kalnitsky ikalnit...@mirantis.com wrote: unicode(exc) (or six.text_type(exc)) works for all exceptions, built-in or custom. That's too much of a statement. Sometimes exceptions implement their own __str__ / __unitcode__ methods, that return too many rubbish

Re: [openstack-dev] [oslo] Logging exceptions and Python 3

2014-05-16 Thread Johannes Erdfelt
On Fri, May 16, 2014, Igor Kalnitsky ikalnit...@mirantis.com wrote: According to http://legacy.python.org/dev/peps/pep-0352/ the message attribute of BaseException is deprecated since Python 2.6 and was dropped in Python 3.0. Some projects have custom exception hierarchy, with strictly

Re: [openstack-dev] [oslo] Logging exceptions and Python 3

2014-05-16 Thread Johannes Erdfelt
On Thu, May 15, 2014, Victor Stinner victor.stin...@enovance.com wrote: I'm trying to define some rules to port OpenStack code to Python 3. I just added a section in the Port Python 2 code to Python 3 about formatting exceptions and the logging module:

Re: [openstack-dev] [all][db][performance] Proposal: Get rid of soft deletion (step by step)

2014-03-12 Thread Johannes Erdfelt
On Wed, Mar 12, 2014, CARVER, PAUL pc2...@att.com wrote: I have personally witnessed someone (honestly, not me) select Terminate Instance when they meant Reboot Instance and that mistake is way too easy. I'm not sure if it was a brain mistake or mere slip of the mouse, but it's enough to make

Re: [openstack-dev] [all][db][performance] Proposal: Get rid of soft deletion (step by step)

2014-03-11 Thread Johannes Erdfelt
On Tue, Mar 11, 2014, Mike Wilson geekinu...@gmail.com wrote: Undeleting things is an important use case in my opinion. We do this in our environment on a regular basis. In that light I'm not sure that it would be appropriate just to log the deletion and git rid of the row. I would like to see

Re: [openstack-dev] [all][db][performance] Proposal: Get rid of soft deletion (step by step)

2014-03-10 Thread Johannes Erdfelt
On Mon, Mar 10, 2014, Joshua Harlow harlo...@yahoo-inc.com wrote: Sounds like a good idea to me. I generally think this is a good idea too. I've never understood why we treat the DB as a LOG (keeping deleted == 0 records around) when we should just use a LOG (or similar system) to begin with

Re: [openstack-dev] [nova] changing old migrations is verboten

2013-11-01 Thread Johannes Erdfelt
On Fri, Nov 01, 2013, Sean Dague s...@dague.net wrote: It's trading one source of bugs for another. I'd love to say we can have our cake and eat it to, but we really can't. And I very much fall on the side of getting migrations is hard, updating past migrations without ever forking the

Re: [openstack-dev] [nova] changing old migrations is verboten

2013-10-31 Thread Johannes Erdfelt
On Thu, Oct 31, 2013, Sean Dague s...@dague.net wrote: So there is a series of patches starting with - https://review.openstack.org/#/c/53417/ that go back and radically change existing migration files. This is really a no-no, unless there is a critical bug fix that absolutely requires it.

Re: [openstack-dev] Does DB schema hygiene warrant long migrations?

2013-10-24 Thread Johannes Erdfelt
On Fri, Oct 25, 2013, Michael Still mi...@stillhq.com wrote: Because I am a grumpy old man I have just -2'ed https://review.openstack.org/#/c/39685/ and I wanted to explain my rationale. Mostly I am hoping for a consensus to form -- if I am wrong then I'll happy remove my vote from this patch.

Re: [openstack-dev] Does DB schema hygiene warrant long migrations?

2013-10-24 Thread Johannes Erdfelt
On Fri, Oct 25, 2013, Michael Still mi...@stillhq.com wrote: On Fri, Oct 25, 2013 at 8:19 AM, Johannes Erdfelt johan...@erdfelt.com wrote: On Fri, Oct 25, 2013, Michael Still mi...@stillhq.com wrote: However, when I run it with medium sized (30 million instances) databases, the change

Re: [openstack-dev] [Nova] Frustrations with review wait times

2013-08-28 Thread Johannes Erdfelt
On Wed, Aug 28, 2013, Russell Bryant rbry...@redhat.com wrote: On 08/28/2013 12:25 PM, Davanum Srinivas wrote: If each little group had at least one active Nova core member, i think it would speed things up way faster IMHO. Agreed, in theory. However, we should not add someone just for

Re: [openstack-dev] [Glance] Replacing Glance DB code to Oslo DB code.

2013-08-20 Thread Johannes Erdfelt
On Tue, Aug 20, 2013, Flavio Percoco fla...@redhat.com wrote: There are a couple of things that would worry me about an hypothetic support for NoSQL but I guess one that I'd consider very critical is migrations. Some could argue asking whether we'd really need them or not - when talking about

Re: [openstack-dev] Work around DB in OpenStack (Oslo, Nova, Cinder, Glance)

2013-07-03 Thread Johannes Erdfelt
On Wed, Jul 03, 2013, Michael Still mi...@stillhq.com wrote: On Wed, Jul 3, 2013 at 3:50 AM, Boris Pavlovic bo...@pavlovic.me wrote: Question: Why we should put in oslo slqlalchemy-migrate monkey patches, when we are planing to switch to alembic? Answer: If we don’t put in oslo

Re: [openstack-dev] [oslo.config] Config files overriding CLI: The path of most surprise.

2013-07-01 Thread Johannes Erdfelt
On Mon, Jul 01, 2013, Clint Byrum cl...@fewbar.com wrote: I am writing today to challenge that notion, and also to suggest that even if that is the case, it is inappropriate to have oslo.config operate in such a profoundly different manner than basically any other config library or system