Re: [ADMIN] Postgres Backup Utility
On Jan 20, 2011, at 10:42 AM, Bradley Holbrook wrote: Martin French is right though, ask your developers to write down all their SQL struct changes and they look at you funny... and being a developer myself I'd look at me funny. Well, it's what I do and it's trivial. Just don't type DDL directly into a psql session, edit it in your favorite text editor, in a document called updates-pending.sql, and copy paste into psql. Sometimes a tiny bit of *process* can actually help productivity ;-) -- Scott Ribe scott_r...@elevated-dev.com http://www.elevated-dev.com/ (303) 722-0567 voice -- Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-admin
Re: [ADMIN] Postgres Backup Utility
Having been a C/C++ developer many years before being a DBA, and having written ITIL software; How is migrating structure from a Development database to a test database whilst maintaining test data backwards? Besides, the OP was asking how to diff to databases and create ddl, not asking for us to comment on why he's doing it. Personally, I'd rather not go trawling through what can only be described as hundreds of thousands of lines of PostgreSQL log to find THE RIGHT DDL statements. To me that's surely a recipe for disaster, and certainly not a role I would expect to complete as a DBA. Yes the Developers should've kept the DDL for the changes, however; I know from experience that this isn't always the case, and have found it necessary to Diff databases on a regular basis for one reason or another. Each time I've done something like this I have resulted to coding shell scripts, or C programs that are bespoke to accomplish this, as I've yet to find a tool that does this to my satisfaction levels. -Original Message- From: Scott Marlowe [mailto:scott.marl...@gmail.com] Sent: 20 January 2011 04:41 To: French, Martin Cc: Bradley Holbrook; pgsql-admin@postgresql.org Subject: Re: [ADMIN] Postgres Backup Utility On Wed, Jan 19, 2011 at 12:12 AM, French, Martin fren...@cromwell.co.uk wrote: Ok, you say that you cannot drop and recreate, so you need to do this via alter statements only? That's obviously going to complicate matters, as a straight dump, drop, recreate, restore would be the fastest and by far simplest method. So, Ideally, you'll need to do a table def comparison over the two databases, and generate the necessary sql to amend the tables in test accordingly? Sorry but this is exactly backwards from good procedure. What you do it have your developers checkin the SQL code they used to alter the tables / add rows in the test database, and you apply that to testing / QA / staging / production. Any other way is a recipe for both disaster and DBA burnout. To make it easier, you can always turn on ddl logging on the developer's databases and then just trawl the logs for what they did. Still easier than trying to compare schemas and figure out what's different and how to write the SQL to make it happen. ___ This email is intended for the named recipient. The information contained in it is confidential. You should not copy it for any purposes, nor disclose its contents to any other party. If you received this email in error, please notify the sender immediately via email, and delete it from your computer. Any views or opinions presented are solely those of the author and do not necessarily represent those of the company. PCI Compliancy: Please note, we do not send or wish to receive banking, credit or debit card information by email or any other form of communication. Cromwell Tools Limited, PO Box 14, 65 Chartwell Drive Wigston, Leicester LE18 1AT. Tel 0116 2888000 Registered in England and Wales, Reg No 00986161 VAT GB 115 5713 87 900 __ -- Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-admin
Re: [ADMIN] Postgres Backup Utility
On Thu, Jan 20, 2011 at 5:00 AM, French, Martin fren...@cromwell.co.uk wrote: Having been a C/C++ developer many years before being a DBA, and having written ITIL software; How is migrating structure from a Development database to a test database whilst maintaining test data backwards? It's not. doing by running diffs on ddl / schema is. This is what the OP asked for: I need to be able to quickly apply the structure updates from a development database to a testing database, and do selective data updates (like on lookup tables, but not content tables). Besides, the OP was asking how to diff to databases and create ddl, not asking for us to comment on why he's doing it. Personally, I'd rather Re-read what he asked for. He did not ask how to run diffs of schema, he asked how to update it with changes from a test database. The best way to do this is to check in changes to the database and lookup tables so the DBA can then apply them when needed to the various other databases. But hey, feel free to answer a question the OP didn't ask with an answer he doesn't need. -- Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-admin
Re: [ADMIN] Postgres Backup Utility
On Thu, Jan 20, 2011 at 5:00 AM, French, Martin fren...@cromwell.co.uk wrote: Personally, I'd rather not go trawling through what can only be described as hundreds of thousands of lines of PostgreSQL log to find THE RIGHT DDL statements. Oh that's easy. Grep out the statements that start with alter etc, ptu them all in a big email, send them to all the developers and tell them to let you know which ones need applying to QA / Staging / etc. Can't get your DDL to the DBA? Your code won't work. You break the build because of that, you get to fix it. -- Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-admin
Re: [ADMIN] Postgres Backup Utility
On Thu, Jan 20, 2011 at 8:46 AM, Scott Marlowe scott.marl...@gmail.com wrote: On Thu, Jan 20, 2011 at 5:00 AM, French, Martin fren...@cromwell.co.uk wrote: Personally, I'd rather not go trawling through what can only be described as hundreds of thousands of lines of PostgreSQL log to find THE RIGHT DDL statements. Oh that's easy. Grep out the statements that start with alter etc, ptu them all in a big email, send them to all the developers and tell them to let you know which ones need applying to QA / Staging / etc. Can't get your DDL to the DBA? Your code won't work. You break the build because of that, you get to fix it. Note that I also mentioned turning on logging JUST ddl statements. Assumiing you're not logging EVERYTHING in development (and why would you) you should only have a few dozen lines of log to grep, and little or none of it will be things other than DDL. -- To understand recursion, one must first understand recursion. -- Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-admin
Re: [ADMIN] Postgres Backup Utility
Thanks Scott... a couple comments. Our developers never decide what goes to where... they just happily plumb away on the development db until we're ready to take our product to testing (at regular intervals), once QA is passed, we wish to apply these to live. We have several diff tools and sync tools, but they take forever (especially the ones that only go one schema at a time). The DDL Logging sounds like a sufficient solution, can it be configured to only record create and alter commands (or create or replace commands on functions or updates on sequences, etc)? I'd likely write a script to have this emailed to me at the end of every day. I'm going to google DDL logging (never heard of it), but any good resources off the top of your head? Martin French is right though, ask your developers to write down all their SQL struct changes and they look at you funny... and being a developer myself I'd look at me funny. If you forget just once you're screwed into a day sifting through tables and code. -Original Message- From: Scott Marlowe [mailto:scott.marl...@gmail.com] Sent: January-20-11 7:40 AM To: French, Martin Cc: Bradley Holbrook; pgsql-admin@postgresql.org Subject: Re: [ADMIN] Postgres Backup Utility On Thu, Jan 20, 2011 at 5:00 AM, French, Martin fren...@cromwell.co.uk wrote: Having been a C/C++ developer many years before being a DBA, and having written ITIL software; How is migrating structure from a Development database to a test database whilst maintaining test data backwards? It's not. doing by running diffs on ddl / schema is. This is what the OP asked for: I need to be able to quickly apply the structure updates from a development database to a testing database, and do selective data updates (like on lookup tables, but not content tables). Besides, the OP was asking how to diff to databases and create ddl, not asking for us to comment on why he's doing it. Personally, I'd rather Re-read what he asked for. He did not ask how to run diffs of schema, he asked how to update it with changes from a test database. The best way to do this is to check in changes to the database and lookup tables so the DBA can then apply them when needed to the various other databases. But hey, feel free to answer a question the OP didn't ask with an answer he doesn't need. -- Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-admin
Re: [ADMIN] Postgres Backup Utility
On Thu, Jan 20, 2011 at 10:42 AM, Bradley Holbrook operations_brad...@servillian.ca wrote: Thanks Scott... a couple comments. Our developers never decide what goes to where... they just happily plumb away on the development db until we're ready to take our product to testing (at regular intervals), once QA is passed, we wish to apply these to live. We have several diff tools and sync tools, but they take forever (especially the ones that only go one schema at a time). The DDL Logging sounds like a sufficient solution, can it be configured to only record create and alter commands (or create or replace commands on functions or updates on sequences, etc)? I'd likely write a script to have this emailed to me at the end of every day. I'm going to google DDL logging (never heard of it), but any good resources off the top of your head? It's basically logging anything that changes the structure of the database. It would be easy enough to grep out what you do and don't want later. Martin French is right though, ask your developers to write down all their SQL struct changes and they look at you funny... and being a developer myself I'd look at me funny. If you forget just once you're screwed into a day sifting through tables and code. I've worked in three different shops now as a dev-dba and sysadmin, and in all three, all DDL changes had to be committed and / or handed over to the DBAs. period. Look funny all they want, they either give up the DDL or their code doesn't get pushed off dev servers onto anything else. At the very least they should be able to tell you which tables changed to go with which code changes, or you're not sure what code you can and can't push. I get both of your point on this, but it's a discipline issue that needs sorting out with the developers if you want to have reproduceable ddl changes in all your systems that match the code changes. -- Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-admin
Re: [ADMIN] Postgres Backup Utility
Followup, note that you can set the log_statement='ddl' for an entire pg cluster, for a single database, or for a single user, if that helps. logging ddl does not log dml, or data changes, just structural changes. -- Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-admin
Re: [ADMIN] Postgres Backup Utility
On Thu, Jan 20, 2011 at 10:52 AM, Scott Marlowe scott.marl...@gmail.com wrote: Followup, note that you can set the log_statement='ddl' for an entire pg cluster, for a single database, or for a single user, if that helps. logging ddl does not log dml, or data changes, just structural changes. also also wik, you can use the log_line_prefix to make it so that you can grep out what you need easier too. #log_line_prefix = '' # special values: # %u = user name # %d = database name # %r = remote host and port # %h = remote host # %p = process ID # %t = timestamp without milliseconds # %m = timestamp with milliseconds # %i = command tag # %c = session ID # %l = session line number # %s = session start timestamp # %v = virtual transaction ID # %x = transaction ID (0 if none) # %q = stop here in non-session #processes # %% = '%' # e.g. '%u%%%d ' -- Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-admin
Re: [ADMIN] Postgres Backup Utility
-Original Message- From: Scott Marlowe [mailto:scott.marl...@gmail.com] Sent: Thursday, January 20, 2011 12:51 PM To: Bradley Holbrook Cc: French, Martin; pgsql-admin@postgresql.org Subject: Re: Postgres Backup Utility On Thu, Jan 20, 2011 at 10:42 AM, Bradley Holbrook operations_brad...@servillian.ca wrote: Thanks Scott... a couple comments. Our developers never decide what goes to where... they just happily plumb away on the development db until we're ready to take our product to testing (at regular intervals), once QA is passed, we wish to apply these to live. We have several diff tools and sync tools, but they take forever (especially the ones that only go one schema at a time). The DDL Logging sounds like a sufficient solution, can it be configured to only record create and alter commands (or create or replace commands on functions or updates on sequences, etc)? I'd likely write a script to have this emailed to me at the end of every day. I'm going to google DDL logging (never heard of it), but any good resources off the top of your head? It's basically logging anything that changes the structure of the database. It would be easy enough to grep out what you do and don't want later. Martin French is right though, ask your developers to write down all their SQL struct changes and they look at you funny... and being a developer myself I'd look at me funny. If you forget just once you're screwed into a day sifting through tables and code. I've worked in three different shops now as a dev-dba and sysadmin, and in all three, all DDL changes had to be committed and / or handed over to the DBAs. period. Look funny all they want, they either give up the DDL or their code doesn't get pushed off dev servers onto anything else. At the very least they should be able to tell you which tables changed to go with which code changes, or you're not sure what code you can and can't push. I get both of your point on this, but it's a discipline issue that needs sorting out with the developers if you want to have reproduceable ddl changes in all your systems that match the code changes. Completely agree with Scott. Only want to add that in this kind of development environment: development/test/production - Source code versioning software is absolute necessity (there are many: CSV, SourceSafe, Perforce, ... - pick your choice). Regards, Igor Neyman -- Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-admin
Re: [ADMIN] Postgres Backup Utility
On Thu, Jan 20, 2011 at 10:42 AM, Bradley Holbrook operations_brad...@servillian.ca wrote: Thanks Scott... a couple comments. Our developers never decide what goes to where... they just happily plumb away on the development db until we're ready to take our product to testing (at regular intervals), once QA is passed, we wish to apply these to live. We have several diff tools and sync tools, but they take forever (especially the ones that only go one schema at a time). The DDL Logging sounds like a sufficient solution, can it be configured to only record create and alter commands (or create or replace commands on functions or updates on sequences, etc)? I'd likely write a script to have this emailed to me at the end of every day. I'm going to google DDL logging (never heard of it), but any good resources off the top of your head? It's basically logging anything that changes the structure of the database. It would be easy enough to grep out what you do and don't want later. Martin French is right though, ask your developers to write down all their SQL struct changes and they look at you funny... and being a developer myself I'd look at me funny. If you forget just once you're screwed into a day sifting through tables and code. I've worked in three different shops now as a dev-dba and sysadmin, and in all three, all DDL changes had to be committed and / or handed over to the DBAs. period. Look funny all they want, they either give up the DDL or their code doesn't get pushed off dev servers onto anything else. At the very least they should be able to tell you which tables changed to go with which code changes, or you're not sure what code you can and can't push. I get both of your point on this, but it's a discipline issue that needs sorting out with the developers if you want to have reproduceable ddl changes in all your systems that match the code changes. Completely agree with Scott. Me too. The idea that a developer can change the schema without telling anyone is laughable. The idea that someone else has to reverse engineer schema changes is ludicrous. We have a strict procedure that every schema change has to be accompanied by a script that applies the changes. It has to be checked into Subversion along with all the other code. Make a new rule: Developers have to document schema changes. Institute a zero-tolerance policy for omissions. If a developer can create the DDL to make the change, surely it's not too much trouble to spend another minute adding it to a file and checking that file in. Craig -- Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-admin
Re: [ADMIN] Postgres Backup Utility
On Jan 20, 2011, at 12:42 PM, Bradley Holbrook wrote: Our developers never decide what goes to where... they just happily plumb away on the development db until we're ready to take our product to testing (at regular intervals), once QA is passed, we wish to apply these to live. We have several diff tools and sync tools, but they take forever (especially the ones that only go one schema at a time). Using a diff tool, how would you know if a column was dropped and another one added versus simply renaming a column? Won't that kind of difference matter when you want to apply the changes to your production database? John DeSoi, Ph.D. -- Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-admin
Re: [ADMIN] Postgres Backup Utility
On Thu, Jan 20, 2011 at 6:08 PM, John DeSoi de...@pgedit.com wrote: On Jan 20, 2011, at 12:42 PM, Bradley Holbrook wrote: Our developers never decide what goes to where... they just happily plumb away on the development db until we're ready to take our product to testing (at regular intervals), once QA is passed, we wish to apply these to live. We have several diff tools and sync tools, but they take forever (especially the ones that only go one schema at a time). Using a diff tool, how would you know if a column was dropped and another one added versus simply renaming a column? Won't that kind of difference matter when you want to apply the changes to your production database? Yikes! That could be bad. Drop column xyz. Hope that data wasn't important. To see if the column was renamed or something you'd have to examine its rows to see if they matched a previous version. I think that at the least I'd trawl the logs with ddl logging turned on to get changes. But seriously, I'd just lay down the law. You want changes in the dbs outside development, you give me, the DBA, your DDL statements. No statements, no code push. -- Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-admin
Re: [ADMIN] Postgres Backup Utility
Brad, Google for SQL Power Architect, download it, and try a schema comparison. That might get you a ways down to road to what you want. Bob Lunney --- On Wed, 1/19/11, French, Martin fren...@cromwell.co.uk wrote: From: French, Martin fren...@cromwell.co.uk Subject: Re: [ADMIN] Postgres Backup Utility To: Bradley Holbrook operations_brad...@servillian.ca Cc: pgsql-admin@postgresql.org Date: Wednesday, January 19, 2011, 2:12 AM Ok, you say that you cannot drop and recreate, so you need to do this via alter statements only? That’s obviously going to complicate matters, as a straight dump, drop, recreate, restore would be the fastest and by far simplest method. So, Ideally, you’ll need to do a table def comparison over the two databases, and generate the necessary sql to amend the tables in test accordingly? Querying the pg_catalog/information_schema over the two db’s should give you the table ddl from which you can diff, and then generate the alter statements from the results. Cheers Martin From: Bradley Holbrook [mailto:operations_brad...@servillian.ca] Sent: 18 January 2011 16:57 To: French, Martin Cc: pgsql-admin@postgresql.org Subject: RE: [ADMIN] Postgres Backup Utility Well, I can’t just go dropping and recreating tables… it needs to create the correct alter statements if existing tables and or functions already exist. Secondly, when I’m finished changing the structure, I need to be able to select the list of tables that will have content updates. Using a script might be more work maintaining then it’s worth. I have a backup utility that can do the job, but 3 tedious steps per schema, that only work about 10% of the time (and no batching options so that I can create a list of actions and run the list). From: French, Martin [mailto:fren...@cromwell.co.uk] Sent: January-18-11 5:47 AM To: Bradley Holbrook; pgsql-admin@postgresql.org Subject: RE: [ADMIN] Postgres Backup Utility I’m assuming that this needs to be tightly controlled and as such a replication tool is out of the question? In that case; The first thing to pop into my head here would be to use either use shell scripting, or to use the pg API and write a c program to handle it. I remember doing something very similar with Oracle a few years back. Cheers Martin From: pgsql-admin-ow...@postgresql.org [mailto:pgsql-admin-ow...@postgresql.org] On Behalf Of Bradley Holbrook Sent: 18 January 2011 00:08 To: pgsql-admin@postgresql.org Subject: [ADMIN] Postgres Backup Utility Hello! First day on the new mailing list as I have need of some expert’s advice. I need to be able to quickly apply the structure updates from a development database to a testing database, and do selective data updates (like on lookup tables, but not content tables). Any help would be appreciated! Brad ___ This email is intended for the named recipient. The information contained in it is confidential. You should not copy it for any purposes, nor disclose its contents to any other party. If you received this email in error, please notify the sender immediately via email, and delete it from your computer. Any views or opinions presented are solely those of the author and do not necessarily represent those of the company. PCI Compliancy: Please note, we do not send or wish to receive banking, credit or debit card information by email or any other form of communication. Cromwell Tools Limited, PO Box 14, 65 Chartwell Drive Wigston, Leicester LE18 1AT. Tel 0116 2888000 Registered in England and Wales, Reg No 00986161 VAT GB 115 5713 87 900 __ ___ This email is intended for the named recipient. The information contained in it is confidential. You should not copy it for any purposes, nor disclose its contents to any other party. If you received this email in error, please notify the sender immediately via email, and delete it from your computer. Any views or opinions presented are solely those of the author and do not necessarily represent those of the company. PCI Compliancy: Please note, we do not send or wish to receive banking, credit or debit card information by email or any other form of communication. Cromwell Tools Limited, PO Box 14, 65 Chartwell Drive Wigston, Leicester LE18 1AT. Tel 0116 2888000 Registered in England and Wales, Reg No 00986161 VAT GB 115 5713 87 900 __
Re: [ADMIN] Postgres Backup Utility
On Wed, Jan 19, 2011 at 12:12 AM, French, Martin fren...@cromwell.co.uk wrote: Ok, you say that you cannot drop and recreate, so you need to do this via alter statements only? That’s obviously going to complicate matters, as a straight dump, drop, recreate, restore would be the fastest and by far simplest method. So, Ideally, you’ll need to do a table def comparison over the two databases, and generate the necessary sql to amend the tables in test accordingly? Sorry but this is exactly backwards from good procedure. What you do it have your developers checkin the SQL code they used to alter the tables / add rows in the test database, and you apply that to testing / QA / staging / production. Any other way is a recipe for both disaster and DBA burnout. To make it easier, you can always turn on ddl logging on the developer's databases and then just trawl the logs for what they did. Still easier than trying to compare schemas and figure out what's different and how to write the SQL to make it happen. -- Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-admin
Re: [ADMIN] Postgres Backup Utility
I'm assuming that this needs to be tightly controlled and as such a replication tool is out of the question? In that case; The first thing to pop into my head here would be to use either use shell scripting, or to use the pg API and write a c program to handle it. I remember doing something very similar with Oracle a few years back. Cheers Martin From: pgsql-admin-ow...@postgresql.org [mailto:pgsql-admin-ow...@postgresql.org] On Behalf Of Bradley Holbrook Sent: 18 January 2011 00:08 To: pgsql-admin@postgresql.org Subject: [ADMIN] Postgres Backup Utility Hello! First day on the new mailing list as I have need of some expert's advice. I need to be able to quickly apply the structure updates from a development database to a testing database, and do selective data updates (like on lookup tables, but not content tables). Any help would be appreciated! Brad ___ This email is intended for the named recipient. The information contained in it is confidential. You should not copy it for any purposes, nor disclose its contents to any other party. If you received this email in error, please notify the sender immediately via email, and delete it from your computer. Any views or opinions presented are solely those of the author and do not necessarily represent those of the company. PCI Compliancy: Please note, we do not send or wish to receive banking, credit or debit card information by email or any other form of communication. Cromwell Tools Limited, PO Box 14, 65 Chartwell Drive Wigston, Leicester LE18 1AT. Tel 0116 2888000 Registered in England and Wales, Reg No 00986161 VAT GB 115 5713 87 900 __
Re: [ADMIN] Postgres Backup Utility
Well, I can't just go dropping and recreating tables. it needs to create the correct alter statements if existing tables and or functions already exist. Secondly, when I'm finished changing the structure, I need to be able to select the list of tables that will have content updates. Using a script might be more work maintaining then it's worth. I have a backup utility that can do the job, but 3 tedious steps per schema, that only work about 10% of the time (and no batching options so that I can create a list of actions and run the list). From: French, Martin [mailto:fren...@cromwell.co.uk] Sent: January-18-11 5:47 AM To: Bradley Holbrook; pgsql-admin@postgresql.org Subject: RE: [ADMIN] Postgres Backup Utility I'm assuming that this needs to be tightly controlled and as such a replication tool is out of the question? In that case; The first thing to pop into my head here would be to use either use shell scripting, or to use the pg API and write a c program to handle it. I remember doing something very similar with Oracle a few years back. Cheers Martin From: pgsql-admin-ow...@postgresql.org [mailto:pgsql-admin-ow...@postgresql.org] On Behalf Of Bradley Holbrook Sent: 18 January 2011 00:08 To: pgsql-admin@postgresql.org Subject: [ADMIN] Postgres Backup Utility Hello! First day on the new mailing list as I have need of some expert's advice. I need to be able to quickly apply the structure updates from a development database to a testing database, and do selective data updates (like on lookup tables, but not content tables). Any help would be appreciated! Brad ___ This email is intended for the named recipient. The information contained in it is confidential. You should not copy it for any purposes, nor disclose its contents to any other party. If you received this email in error, please notify the sender immediately via email, and delete it from your computer. Any views or opinions presented are solely those of the author and do not necessarily represent those of the company. PCI Compliancy: Please note, we do not send or wish to receive banking, credit or debit card information by email or any other form of communication. Cromwell Tools Limited, PO Box 14, 65 Chartwell Drive Wigston, Leicester LE18 1AT. Tel 0116 2888000 Registered in England and Wales, Reg No 00986161 VAT GB 115 5713 87 900 __
Re: [ADMIN] Postgres Backup Utility
Ok, you say that you cannot drop and recreate, so you need to do this via alter statements only? That's obviously going to complicate matters, as a straight dump, drop, recreate, restore would be the fastest and by far simplest method. So, Ideally, you'll need to do a table def comparison over the two databases, and generate the necessary sql to amend the tables in test accordingly? Querying the pg_catalog/information_schema over the two db's should give you the table ddl from which you can diff, and then generate the alter statements from the results. Cheers Martin From: Bradley Holbrook [mailto:operations_brad...@servillian.ca] Sent: 18 January 2011 16:57 To: French, Martin Cc: pgsql-admin@postgresql.org Subject: RE: [ADMIN] Postgres Backup Utility Well, I can't just go dropping and recreating tables... it needs to create the correct alter statements if existing tables and or functions already exist. Secondly, when I'm finished changing the structure, I need to be able to select the list of tables that will have content updates. Using a script might be more work maintaining then it's worth. I have a backup utility that can do the job, but 3 tedious steps per schema, that only work about 10% of the time (and no batching options so that I can create a list of actions and run the list). From: French, Martin [mailto:fren...@cromwell.co.uk] Sent: January-18-11 5:47 AM To: Bradley Holbrook; pgsql-admin@postgresql.org Subject: RE: [ADMIN] Postgres Backup Utility I'm assuming that this needs to be tightly controlled and as such a replication tool is out of the question? In that case; The first thing to pop into my head here would be to use either use shell scripting, or to use the pg API and write a c program to handle it. I remember doing something very similar with Oracle a few years back. Cheers Martin From: pgsql-admin-ow...@postgresql.org [mailto:pgsql-admin-ow...@postgresql.org] On Behalf Of Bradley Holbrook Sent: 18 January 2011 00:08 To: pgsql-admin@postgresql.org Subject: [ADMIN] Postgres Backup Utility Hello! First day on the new mailing list as I have need of some expert's advice. I need to be able to quickly apply the structure updates from a development database to a testing database, and do selective data updates (like on lookup tables, but not content tables). Any help would be appreciated! Brad ___ This email is intended for the named recipient. The information contained in it is confidential. You should not copy it for any purposes, nor disclose its contents to any other party. If you received this email in error, please notify the sender immediately via email, and delete it from your computer. Any views or opinions presented are solely those of the author and do not necessarily represent those of the company. PCI Compliancy: Please note, we do not send or wish to receive banking, credit or debit card information by email or any other form of communication. Cromwell Tools Limited, PO Box 14, 65 Chartwell Drive Wigston, Leicester LE18 1AT. Tel 0116 2888000 Registered in England and Wales, Reg No 00986161 VAT GB 115 5713 87 900 __ ___ This email is intended for the named recipient. The information contained in it is confidential. You should not copy it for any purposes, nor disclose its contents to any other party. If you received this email in error, please notify the sender immediately via email, and delete it from your computer. Any views or opinions presented are solely those of the author and do not necessarily represent those of the company. PCI Compliancy: Please note, we do not send or wish to receive banking, credit or debit card information by email or any other form of communication. Cromwell Tools Limited, PO Box 14, 65 Chartwell Drive Wigston, Leicester LE18 1AT. Tel 0116 2888000 Registered in England and Wales, Reg No 00986161 VAT GB 115 5713 87 900 __
[ADMIN] Postgres Backup Utility
Hello! First day on the new mailing list as I have need of some expert's advice. I need to be able to quickly apply the structure updates from a development database to a testing database, and do selective data updates (like on lookup tables, but not content tables). Any help would be appreciated! Brad
