Re: merging old versions
>> On Thu, Jun 20, 2019 at 2:37 AM Chris Withers > <mailto:ch...@withers.org>> wrote: >> >> Hi All, >> >> I have some versions that make use of the third party package I no >> longer use, how do I collapse down alembic revisions that have >> already >> been executed everywhere? >> >> I found >> https://stackoverflow.com/questions/34491914/alembic-how-to-merge-all-revision-files-to-one-file >> >> but that doesn't feel right... On 20/06/2019 19:00, Mike Bayer wrote: I think the basic idea is to create a database and codebase in the state of the target revision. Then autogenerate a migration from nothing to that revision - just like you would do when starting to use alembic from an existing schema. From there you can change the slug on it so that it works as the down_revision of later migrations and clear out the old unused migrations that you're replacing. - Michael I think that's what the stackoverflow answer says too. Yep. At the moment that might be the most expedient approach. Okay, TBH here the case is more that I have a couple of migrations that use a package I want to remove as a dependency (sqlalchemy-searchable, which is really good, fwiw, just turns out "like" querying is actually going to work better for my needs), but I think I can just remove those bits once all the migrations are run through... However I can see that Alembic might benefit from having a special option to autogenerate a model into Python code assuming no database to start with. Feel free to propose though Alembic is suffering from lack of contributors right now. Gotcha, I'll come with a PR rather just asking for something, but I'm suffering from the same problem you are ;-) thanks as always, Chris -- You received this message because you are subscribed to the Google Groups "sqlalchemy-alembic" group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy-alembic+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/sqlalchemy-alembic/b8fe3ab2-042a-a789-07a4-f874ef8c34eb%40withers.org. For more options, visit https://groups.google.com/d/optout.
merging old versions
Hi All, I have some versions that make use of the third party package I no longer use, how do I collapse down alembic revisions that have already been executed everywhere? I found https://stackoverflow.com/questions/34491914/alembic-how-to-merge-all-revision-files-to-one-file but that doesn't feel right... Chris -- You received this message because you are subscribed to the Google Groups "sqlalchemy-alembic" group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy-alembic+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/sqlalchemy-alembic/933cd6aa-0e35-8716-3725-56947157103b%40withers.org. For more options, visit https://groups.google.com/d/optout.
Re: unit testing migration code
Has anyone done anything like an equivalent of the following but for Alembic? https://github.com/plumdog/django_migration_testcase cheers, Chris On 01/12/2017 15:06, Mike Bayer wrote: it's kind of a PITA but in Openstack we have fixtures which actually run all the alembic (or sqlalchemy-migrate) migrations on a new database. Some of the more ambitious projects even write unit tests in between each migration that use inspect() to check that the database state is what's expected. so to do things like that, you need a fixture which can: 1. create a new database (you probably need to produce a randomized name for concurrency) 2. invoke alembic to each revision individually (you can do this through alembic API: http://alembic.zzzcomputing.com/en/latest/api/commands.html 3. have a dispatch which can call upon test cases linked to that rev, like "def test_aabbccddee_does_thing_one()" 4. drops the database and...that's how you do it ! On Thu, Nov 30, 2017 at 1:54 PM, Chris Withers <ch...@simplistix.co.uk> wrote: Hi All, How would I add test coverage for this sort of code? https://coveralls.io/builds/14408741/source?filename=mortar_mixins%2Fmigrations.py cheers, Chris -- You received this message because you are subscribed to the Google Groups "sqlalchemy-alembic" group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy-alembic+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "sqlalchemy-alembic" group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy-alembic+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: adding an auto increment column to an existing table
On 17/01/2017 15:07, mike bayer wrote: Because there's no data in a brand new table, the server default isn't needed to create the not-null column. No needed, but it is created, isn't that the point of autoincrement=True? for postgresql, autoincrement=True means that if the column is marked primary_key=True it will use the datatype SERIAL for that column, which in PG does mean the sequence is generated and added as the server side default. Would you expect a SERIAL on Postgres to end up being an inteeger with a default of a sequence when viewed in psql? What happens with autoincrement on a non-primary-key column? My experiences suggests it does nothing... http://stackoverflow.com/a/19262262/216229 implies the same works for Alembic, but it didn't work at all for me. So I was wondering if that answer was wrong, or if I was doing something wrong. Still wondering if this answer is wrong or if I'm doing something wrong... that SO answer is showing op.create_table(). your example was just for op.add_column(). super different. I have to admit that, at the column level, that's surprising to me. Where can I see the differences between a column created as part of create_table() verus as part of an add_column()? cheers, Chris -- You received this message because you are subscribed to the Google Groups "sqlalchemy-alembic" group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy-alembic+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: adding an auto increment column to an existing table
Okay, so this worked:
op.execute(CreateSequence(Sequence("observation_id_seq")))
op.add_column('observation', sa.Column(
'id', sa.Integer(), nullable=False,
server_default=sa.text("nextval('observation_id_seq'::regclass)")
))
op.drop_constraint('observation_pkey', table_name='observation')
op.create_primary_key('observation_pkey', 'observation', ['id'])
...but how come my original attempt didn't?
cheers,
Chris
On 10/01/2017 08:03, Chris Withers wrote:
So, I screwed up and realised I really want an auto-incrementing integer
as the primary key for a bunch of tables.
I've changed my models, got all the tests passing and now I need to get
the migrations done, I have:
op.add_column('observation',
sa.Column('id', sa.Integer(), nullable=False,
autoincrement=True))
op.drop_constraint('observation_pkey', table_name='observation')
op.create_primary_key('observation_pkey', 'observation', ['id'])
According to this answer, this should work:
http://stackoverflow.com/a/19262262/216229
...but when running the migration, I get:
sqlalchemy.exc.IntegrityError: (psycopg2.IntegrityError) column "id"
contains null values
[SQL: 'ALTER TABLE observation ADD COLUMN id INTEGER NOT NULL']
...so what am I doing wrong?
cheers,
Chris
--
You received this message because you are subscribed to the Google Groups
"sqlalchemy-alembic" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to sqlalchemy-alembic+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
adding an auto increment column to an existing table
So, I screwed up and realised I really want an auto-incrementing integer
as the primary key for a bunch of tables.
I've changed my models, got all the tests passing and now I need to get
the migrations done, I have:
op.add_column('observation',
sa.Column('id', sa.Integer(), nullable=False,
autoincrement=True))
op.drop_constraint('observation_pkey', table_name='observation')
op.create_primary_key('observation_pkey', 'observation', ['id'])
According to this answer, this should work:
http://stackoverflow.com/a/19262262/216229
...but when running the migration, I get:
sqlalchemy.exc.IntegrityError: (psycopg2.IntegrityError) column "id"
contains null values
[SQL: 'ALTER TABLE observation ADD COLUMN id INTEGER NOT NULL']
...so what am I doing wrong?
cheers,
Chris
--
You received this message because you are subscribed to the Google Groups
"sqlalchemy-alembic" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to sqlalchemy-alembic+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: patterns for automated tests of migrations
Hi Tom, This is certainly interesting, but even more heavyweight than I was looking for ;-) I love the idea of really making sure the schema that results from running all the migrations is the same as the one from doing a create_all with your metadata. Does alembic have anything like django's squashmigrations command yet? Anyway, what I'm interested in is testing migrations in the presence of data, particularly migrations that have to do "fun stuff" with existing rows to create new rows. Those feel more like unit tests to me - "run a bunch of individual scenarios", rather than your excellent but integration-y "run them all and make sure they build the expected schema". Thoughts? Chris On 23/11/2016 09:20, Tom Lazar wrote: hi chris, here’s how we do it in all our projects: https://github.com/pyfidelity/rest-seed/blob/master/backend/backrest/tests/test_migrations.py basically, our migrations start with an empty database, so we run them, dump the resulting SQL, then create a new database using the `metadata.create_all` feature and then normalize the results and compare. if there is a mismatch the test fails. if you want to write more elaborate tests involving actual data, this should be a good starting point, though HTH, tom On 23 Nov 2016, at 10:15, Chris Withers <ch...@simplistix.co.uk> wrote: Hi All, How do you go about writing automated tests for a migration? I don't often do this, but when migrations involve data, I prefer to but now I don't know how ;-) There's nothing in the docs, right? (or am I missing something obvious) cheers, Chris -- You received this message because you are subscribed to the Google Groups "sqlalchemy-alembic" group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy-alembic+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "sqlalchemy-alembic" group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy-alembic+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
