Drop. It is wasting cycles, and not something we should use in production. Migrations specific to SQLPlus are the most time consuming work-arounds we have. SQLPlus does not suit our development approach.


On 02/03/2015 01:32 PM, Georgy Okrokvertskhov wrote:
I think we should switch to clean migration path. We do have production installations but we can handle initial db uprgade case by case for customers. It is better to fix this issue now when we have few customers rather then doing later at larger scale.

Thanks
Georgy

On Tue, Feb 3, 2015 at 9:05 AM, Mike Bayer <mba...@redhat.com <mailto:mba...@redhat.com>> wrote:



    Andrew Pashkin <apash...@mirantis.com
    <mailto:apash...@mirantis.com>> wrote:

    > Mike Bayer wrote:
    >> there’s always a naming convention in place; all databases other than
    >> SQLite produce them on the fly if you don’t specify one.  The
    purpose
    >> of the Alembic/SQLAlchemy naming_convention feature is so that you
    >> have *one* naming convention, rather than N unpredictable
    conventions.
>> I’m not sure if you’re arguing the feature should not be used. IMHO
    >> it should definitely be used for an application that is deploying
    >> cross-database.  Otherwise you have no choice but to hardcode the
    >> naming conventions of each target database individually in all
    cases
    >> that you need to refer to them.
    > You can't just bring SA/Alembic "naming conventions" into the
    project,
    > because they will collide with auto-generated constraint names.

    I was proposing a way to fix this for the murano project which
    only appears to have four migrations so far, but with the
    assumption that there are existing production environments which
    cannot do a full rebuild.

    >
    > So you need to hardcode reverese-engineered constrants names
    into the
    > old migrations and then add new migration that renames constraint
    > according with "naming conventions”.
    > OR you need to drop old
    > migrations, and create new one with "naming conventions" - that will
    > be backward incompatible, but cleaner.


    My proposal was to essentially do both strategies.  Build out
    fully clean migrations from the start, but also add an additional
    “conditional” migration that will repair a Postgresql / MySQL
    database that is already at the head, and is detected as having
    the older naming convention.  Because openstack does not appear to
    use offline migrations, this would be doable, though not
    necessarily worth it.

    If Murano can afford to just restart with clean migrations and has
    no production deployments yet which would be disrupted by a full
    rebuild, then sure, just do this.





    >
    > On 03.02.2015 18:32, Mike Bayer wrote:
    >> Andrew Pashkin <apash...@mirantis.com
    <mailto:apash...@mirantis.com>> wrote:
    >>
    >>> Mike Bayer wrote:
    >>>> The patch seems to hardcode the conventions for MySQL and
    Postgresql.
    >>>> The first thought I had was that in order to remove the
    dependence
    >>>> on them here, you’d need to instead simply turn off the
    >>>> “naming_convention” in the MetaData if you detect that you’re
    on one
    >>>> of those two databases. That would be a safer idea than trying to
    >>>> hardcode these conventions (and would also work for other kinds
    >>>> of backends).
    >>> With your solution it is still will be necessary for developers
    >>> to guess constraints names when writing new migrations. And it
    will
    >>> be even harder, because they will need also to handle case of
    >>> "naming conventions”.
    >>
    >> there’s always a naming convention in place; all databases
    other than SQLite produce them on the fly if you don’t specify
    one.  The purpose of the Alembic/SQLAlchemy naming_convention
    feature is so that you have *one* naming convention, rather than N
    unpredictable conventions.   I’m not sure if you’re arguing the
    feature should not be used.  IMHO it should definitely be used for
    an application that is deploying cross-database.  Otherwise you
    have no choice but to hardcode the naming conventions of each
    target database individually in all cases that you need to refer
    to them.
    >>
    >>
    >>
    >>
    >>> Mike Bayer wrote:
    >>>> However, it’s probably worthwhile to introduce a migration
    that does
    >>>> in fact rename existing constraints on MySQL and Postgresql.
    >>> Yes, that's what I want to do in case of the first solution.
    >>>
    >>> Mike Bayer wrote:
    >>>>> Another possible solution is to drop all current migrations and
    >>>>> introduce new one with correct names.
    >>>> you definitely shouldn’t need to do that.
    >>> Why?
    >>>
    >>> On 30.01.2015 22:00, Mike Bayer wrote:
    >>>> Andrew Pashkin <apash...@mirantis.com
    <mailto:apash...@mirantis.com>> wrote:
    >>>>
    >>>>> Working on this issue I encountered another problem.
    >>>>>
    >>>>> Most indices in the project has no names and because of that,
    >>>>> developer must reverse-engineer them in every migration.
    >>>>> Read about that also here [1].
    >>>>>
    >>>>> SQLAlchemy and Alembic provide feature for generation constraint
    >>>>> names by pattern, specifically to resolve that kind of
    issues [1].
    >>>>>
    >>>>> I decided to introduce usage of this feature in Murano.
    >>>>>
    >>>>> I've implemented solution that preserves backward-compatibility
    >>>>> for migration and allows to rename all constraints according
    >>>>> to patterns safely [2]. With it user, that have already
    deployed Murano
    >>>>> will be able to upgrade to new version of Murano without issues.
    >>>>>
    >>>>> There are downsides in this solution:
    >>>>> - It assumes that all versions of Postgres and MySQL uses the
    >>>>> same patterns for constraints names generation.
    >>>>> - It is hard to implement a test for this solution and it
    will be slow.
    >>>>> Because there is need to reproduce such situation when user
    has old
    >>>>> versions of migrations applied, and then tries to upgrade.
    >>>>
    >>>> The patch seems to hardcode the conventions for MySQL and
    Postgresql.   The first thought I had was that in order to remove
    the dependence on them here, you’d need to instead simply turn off
    the “naming_convention” in the MetaData if you detect that you’re
    on one of those two databases.   That would be a safer idea than
    trying to hardcode these conventions (and would also work for
    other kinds of backends).
    >>>>
    >>>> However, I’m not actually sure that you even need special
    behavior for these two backends.  If an operator runs these
    migrations on a clean database, then the constraints are generated
    with the consistent names on all backends.   if a target database
    already has these schema constructs present, then these migrations
    are never run; it doesn’t matter that they have the right or wrong
    names already.
    >>>>
    >>>> I suppose then that the fear is that some PG/MySQL databases
    will have constraints that are named in one convention, and others
    will have constraints using the native conventions.    However,
    the case now is that all deployments are using native conventions,
    and being able to DROP these constraints is already not very
    feasible unless you again were willing to hardcode those naming
    conventions up forward.    The constraints in these initial
    migrations, assuming you don’t regenerate them, might just need to
    be left alone, and the project proceeds in the future with a
    consistent convention.
    >>>>
    >>>> However, it’s probably worthwhile to introduce a migration
    that does in fact rename existing constraints on MySQL and
    Postgresql.  This would be a migration script that emits DROP
    CONSTRAINT and CREATE CONSTRAINT for all the above constraints
    that have an old name and a new name.  The script would need to
    check the backend, as you’re doing now, in order to run, and yes
    it would hardcode the names of those conventions, but at least it
    would just be a one-time run against only currently deployed
    databases.   Since your migrations are run “live”, the script can
    make itself a “conditional” run by checking for the “old” names
    and skipping those that don’t exist.
    >>>>
    >>>>> Another possible solution is to drop all current migrations and
    >>>>> introduce new one with correct names.
    >>>>
    >>>> you definitely shouldn’t need to do that.
    >>>>
    >>>>
    >>>>> This brings us to new problem - migrations and models are
    out of sync
    >>>>> right now in multiple places - there are different field
    types in
    >>>>> migrations and models, migrations introduces indices that is
    absent
    >>>>> in models, etc.
    >>>>>
    >>>>> And this solution has great downside - it is not
    backward-compatible,
    >>>>> so all old users will lost their data.
    >>>>>
    >>>>> We (Murano team) should decide, what solution we want to use.
    >>>>>
    >>>>>
    >>>>> [1]
    >>>>>
    
http://alembic.readthedocs.org/en/latest/naming.html#tutorial-constraint-names
    >>>>> [2] https://review.openstack.org/150818
    >>>>>
    >>>>> --
    >>>>> With kind regards, Andrew Pashkin.
    >>>>> cell phone - +7 (985) 898 57 59
    <tel:%2B7%20%28985%29%20898%2057%2059>
    >>>>> Skype - waves_in_fluids
    >>>>> e-mail - apash...@mirantis.com <mailto:apash...@mirantis.com>
    >>>>>
    >>>>>
    __________________________________________________________________________
    >>>>> OpenStack Development Mailing List (not for usage questions)
    >>>>> Unsubscribe:
    openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
    <http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe>
    >>>>>
    http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
    >>>>
    >>>>
    __________________________________________________________________________
    >>>> OpenStack Development Mailing List (not for usage questions)
    >>>> Unsubscribe:
    openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
    <http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe>
    >>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
    >>>
    >>> --
    >>> With kind regards, Andrew Pashkin.
    >>> cell phone - +7 (985) 898 57 59
    <tel:%2B7%20%28985%29%20898%2057%2059>
    >>> Skype - waves_in_fluids
    >>> e-mail - apash...@mirantis.com <mailto:apash...@mirantis.com>
    >>>
    >>>
    __________________________________________________________________________
    >>> OpenStack Development Mailing List (not for usage questions)
    >>> Unsubscribe:
    openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
    <http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe>
    >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
    >>
    >>
    __________________________________________________________________________
    >> OpenStack Development Mailing List (not for usage questions)
    >> Unsubscribe:
    openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
    <http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe>
    >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
    >
    > --
    > With kind regards, Andrew Pashkin.
    > cell phone - +7 (985) 898 57 59
    <tel:%2B7%20%28985%29%20898%2057%2059>
    > Skype - waves_in_fluids
    > e-mail - apash...@mirantis.com <mailto:apash...@mirantis.com>
    >
    >
    __________________________________________________________________________
    > OpenStack Development Mailing List (not for usage questions)
    > Unsubscribe:
    openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
    <http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe>
    > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

    __________________________________________________________________________
    OpenStack Development Mailing List (not for usage questions)
    Unsubscribe:
    openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
    <http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe>
    http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




--
Georgy Okrokvertskhov
Architect,
OpenStack Platform Products,
Mirantis
http://www.mirantis.com <http://www.mirantis.com/>
Tel. +1 650 963 9828
Mob. +1 650 996 3284


__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Reply via email to