Re: [HACKERS] In-place upgrade: catalog side

2008-12-05 Thread Gregory Stark
Greg Smith [EMAIL PROTECTED] writes: I'm sorry, I think I misunderstood the original idea, what you're talking about makes a lot more sense now. You want to save the space of the dead column by replacing it with NULL, not remove it from the table definition. Not so much to save the space,

Re: [HACKERS] In-place upgrade: catalog side

2008-12-05 Thread Bruce Momjian
Greg Smith wrote: On Wed, 3 Dec 2008, Bruce Momjian wrote: As the author of the original shell script, which was in /contrib/pg_upgrade, I think the code has grown to the point where it should be reimplemented in something like Perl. Ah, there's the common ancestor I couldn't find.

Re: [HACKERS] In-place upgrade: catalog side

2008-12-04 Thread Zdenek Kotala
Greg Smith napsal(a): On Wed, 3 Dec 2008, Zdenek Kotala wrote: It works fine for 8.3-8.4 too, but I'm working on cleanup and fixing bugs. I hope that I will send updated version to community today. That would be great. It didn't feel like you were quite done with it yet. I'll be glad to

Re: [HACKERS] In-place upgrade: catalog side

2008-12-04 Thread Greg Smith
On Thu, 4 Dec 2008, Zdenek Kotala wrote: The TOAST problem is already addressed and script should handle it correctly. But I don't like it much, because it is kind of magic. I just read through the whole toast by chunk-end thread again and it does seem pretty complicated. What is the magic

Re: [HACKERS] In-place upgrade: catalog side

2008-12-04 Thread Gregory Stark
Greg Smith [EMAIL PROTECTED] writes: Here's a good example; that seems a perfect problem for somebody else to work on. I understand it now well enough to float ideas without even needing to see your code. Stop worring about it, I'll grab responsibility for making sure it gets done by

Re: [HACKERS] In-place upgrade: catalog side

2008-12-04 Thread Zdenek Kotala
Gregory Stark napsal(a): How about adding a special syntax for CREATE TABLE which indicates to include a dropped column in that position? Then pg_dump could have a -X option to include those columns as placeholders. Something like: CREATE TABLE foo ( col1 integer, NULL COLUMN, col2 integer

Re: [HACKERS] In-place upgrade: catalog side

2008-12-04 Thread Gregory Stark
Zdenek Kotala [EMAIL PROTECTED] writes: Gregory Stark napsal(a): How about adding a special syntax for CREATE TABLE which indicates to include a dropped column in that position? Then pg_dump could have a -X option to include those columns as placeholders. Something like: CREATE TABLE foo (

Re: [HACKERS] In-place upgrade: catalog side

2008-12-04 Thread Greg Smith
On Thu, 4 Dec 2008, Gregory Stark wrote: They all seem functional ideas. But it seems to me they're all ideas that would be appropriate if this was a pgfoundry add-on for existing releases. I was mainly trying to target things that would be achievable within the context of the existing shell

Re: [HACKERS] In-place upgrade: catalog side

2008-12-04 Thread Robert Haas
Not being familiar with the code, my assumption was that it would be possible to push all the tuples involved off to another page as if they'd been updated, with WAL logging and everything, similarly to the ideas that keep getting kicked around for creating extra space for header expansion.

Re: [HACKERS] In-place upgrade: catalog side

2008-12-04 Thread Gregory Stark
Greg Smith [EMAIL PROTECTED] writes: On Thu, 4 Dec 2008, Gregory Stark wrote: They all seem functional ideas. But it seems to me they're all ideas that would be appropriate if this was a pgfoundry add-on for existing releases. I was mainly trying to target things that would be achievable

Re: [HACKERS] In-place upgrade: catalog side

2008-12-04 Thread Greg Smith
On Wed, 3 Dec 2008, Bruce Momjian wrote: As the author of the original shell script, which was in /contrib/pg_upgrade, I think the code has grown to the point where it should be reimplemented in something like Perl. Ah, there's the common ancestor I couldn't find. Sheesh, you learn Perl

Re: [HACKERS] In-place upgrade: catalog side

2008-12-04 Thread Greg Smith
On Thu, 4 Dec 2008, Gregory Stark wrote: Incidentally I got this wrong in my previous email. If we're aiming at 8.4-8.5 as the first in-place update then we actually don't need this in 8.4 at all. I don't know about everybody else, but I haven't give up on putting together something that

Re: [HACKERS] In-place upgrade: catalog side

2008-12-03 Thread Bruce Momjian
Zdenek Kotala wrote: If you compare with pg_migrator, there is better handling of locale and I think vacuum freeze is used correctly. Also shuffling with tablespaces is little bit different (it should avoid to move data outside of mountpoint). But in principal the idea is same.

Re: [HACKERS] In-place upgrade: catalog side

2008-12-03 Thread Zdenek Kotala
Heikki Linnakangas napsal(a): Zdenek Kotala wrote: Greg Smith napsal(a): -There are 10 TODO items listed for the pg_migrator project, most or all of which look like should be squashed before this is really complete. Any chance somebody (Korry?) has an improved version of this floating around

Re: [HACKERS] In-place upgrade: catalog side

2008-12-03 Thread Greg Smith
On Wed, 3 Dec 2008, Zdenek Kotala wrote: It works fine for 8.3-8.4 too, but I'm working on cleanup and fixing bugs. I hope that I will send updated version to community today. That would be great. It didn't feel like you were quite done with it yet. I'll be glad to help test it out, just

[HACKERS] In-place upgrade: catalog side

2008-12-02 Thread Greg Smith
Since this whole in-place upgrade thing is going nowhere until there's also a good solution for ye olde The database cluster was initialized with CATALOG_VERSION_NO ... error, I spent some time today touring through what everybody else has done there and have some initial review commentary and

Re: [HACKERS] In-place upgrade: catalog side

2008-12-02 Thread Zdenek Kotala
Greg Smith napsal(a): The main thing you'll find there is a ksh script that handles most of the upgrade, presuming there's no page format changes. It looks like it was originally aimed at 8.1-8.2 upgrades (easy as long as you don't use INET/CIDR in your database) and still has some

Re: [HACKERS] In-place upgrade: catalog side

2008-12-02 Thread Heikki Linnakangas
Zdenek Kotala wrote: Greg Smith napsal(a): -There are 10 TODO items listed for the pg_migrator project, most or all of which look like should be squashed before this is really complete. Any chance somebody (Korry?) has an improved version of this floating around beyond what's in the pgfoundry