Re: [PATCHES] ALTER TABLE modifications
Rod Taylor kirjutas L, 08.11.2003 kell 18:55: A general re-organization of Alter Table. Node wise, it is a AlterTableStmt with a list of AlterTableCmds. The Cmds are the individual actions to be completed (Add constraint, drop constraint, add column, etc.) Processing is done in 2 phases. The first phase updates the system catalogs and creates a work queue for the table scan. The second phase is to conduct the actual table scan evaluating all constraints and other per tuple processing simultaneously, as required. This has no effect on single step operations, but has a large benefit for combinational logic where multiple table scans would otherwise be required. ... ALTER TABLE tab ALTER COLUMN col TYPE text TRANSFORM ...; Currently migrates indexes, check constraints, defaults, and the column definition to the new type with optional transform. If the tranform is not supplied, a standard assignment cast is attempted. Do you have special cases for type changes which don't need data transforms. I mean things like changing VARCHAR(10) to VARCHAR(20), dropping the NOT NULL constraint or changing CHECK A 3 to CHECK A 4. All these could be done with no data migration or extra checking. So how much of it should PG attemt to detect automatically and should there be NOSCAN option when progremmer knows better (changing CHECK ABS(A) 3 into CHECK 9 (A*A) ) Hannu ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [PATCHES] ALTER TABLE modifications
Rod, I tried the current patch on a RC2 release, and I noticed one undesirable side affect. Modifying a column moves it to the end. In high availability situations this would not be desirable, I would imagine it would break lots of code. Dave On Thu, 2003-11-13 at 11:35, Hannu Krosing wrote: Rod Taylor kirjutas N, 13.11.2003 kell 16:59: Can you please suggest a better term to use in place of TRANSFORM? Perhaps UPDATE WITH? or perhaps USING, based loosely on our use of USING in CREATE INDEX ? -- Hannu ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [PATCHES] ALTER TABLE modifications
On Fri, Nov 14, 2003 at 08:59:05AM -0500, Dave Cramer wrote: I tried the current patch on a RC2 release, and I noticed one undesirable side affect. Modifying a column moves it to the end. In high availability situations this would not be desirable, I would imagine it would break lots of code. This is expected. Doing otherwise would incur into a much bigger performance hit. Anyway, IMHO no code should use SELECT * in any case, which is the only scenario where one would expect physical column order to matter, isn't it? -- Alvaro Herrera (alvherre[a]dcc.uchile.cl) La primera ley de las demostraciones en vivo es: no trate de usar el sistema. Escriba un guión que no toque nada para no causar daños. (Jakob Nielsen) ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [PATCHES] ALTER TABLE modifications
Alvaro Herrera kirjutas R, 14.11.2003 kell 16:17: On Fri, Nov 14, 2003 at 08:59:05AM -0500, Dave Cramer wrote: I tried the current patch on a RC2 release, and I noticed one undesirable side affect. Modifying a column moves it to the end. In high availability situations this would not be desirable, I would imagine it would break lots of code. This is expected. Doing otherwise would incur into a much bigger performance hit. Not neccessarily, but it would make the patch much bigger ;) IIRC there was discussion about splitting colum numbers into physical and logical numbers at the time when DROP COLUMN was done. Anyway, IMHO no code should use SELECT * in any case, which is the only scenario where one would expect physical column order to matter, isn't it? and this could also break when just changing the column type. Hannu ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [PATCHES] ALTER TABLE modifications
Rod Taylor writes: The method is rename old column, add new column, move data across, move or reform dependencies, drop old column. I can do this by hand. If we have an explicit command to do it, then it needs to preserve the table schema. Else, this feature would be mostly useless and a certain source of complaints. -- Peter Eisentraut [EMAIL PROTECTED] ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [PATCHES] ALTER TABLE modifications
On Fri, 2003-11-14 at 09:57, Peter Eisentraut wrote: Rod Taylor writes: The method is rename old column, add new column, move data across, move or reform dependencies, drop old column. I can do this by hand. If we have an explicit command to do it, then it needs to preserve the table schema. Else, this feature would be mostly useless and a certain source of complaints. The method was agreed to on -hackers prior to any code having been written. Unless I'm mistaken, the method was suggested to me by other on -hackers. Can you give me an alternative on how to approach this problem without relying on another TODO item labelled ALTER TABLE .. POSITION? The point of the command isn't to accomplish anything magical, simply to make it easier. begin; Rename old column Add new column Copy data Recreate default Recreate NOT NULL constraint Recreate index 1 Recreate index 2 (multi-key) Recreate check constraint 1 Recreate check constraint 2 Recreate check constraint 3 Recreate foreign key 1 remove old column commit; vacuum full table (to get rid of 2x problem since the user was unable to do a fileswap) The above is by hand, the below is with this command: begin; Alter table .. type .. commit; If it becomes possible to change the order without having to goto an exceptional amount of work (that POSITION item) then it would only be obvious to attempt to preserve the position. ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: [PATCHES] ALTER TABLE modifications
OK, Here is another approach, that would retain column order. It will require that the table be locked while this proceeds, but I think this is a good idea anyway. lock table create newtable as select c1, c2, c3::newtype modify pg_class to point to the new filename modify existing pg_attribute for the column in question recreate indexes that exist on the column unlock table Dave On Fri, 2003-11-14 at 09:57, Peter Eisentraut wrote: Rod Taylor writes: The method is rename old column, add new column, move data across, move or reform dependencies, drop old column. I can do this by hand. If we have an explicit command to do it, then it needs to preserve the table schema. Else, this feature would be mostly useless and a certain source of complaints. ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [PATCHES] ALTER TABLE modifications
Rod Taylor [EMAIL PROTECTED] writes: On Fri, 2003-11-14 at 09:57, Peter Eisentraut wrote: I can do this by hand. If we have an explicit command to do it, then it needs to preserve the table schema. Else, this feature would be mostly useless and a certain source of complaints. The method was agreed to on -hackers prior to any code having been written. I believe the consensus was that automating what you could do by hand is still a step forward. It clearly would be better if we could relabel the logical column position after finishing the whole process, but I agree with Rod that that is an independent patch. Combining them into one mega-patch doesn't sound like good engineering. I guess the real question here is whether we would want to revert this capability if a patch to adjust logical column orderings doesn't appear before 7.5. My vote would be no, but apparently Peter's is yes. Any other opinions? regards, tom lane ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: [PATCHES] ALTER TABLE modifications
lock table create newtable as select c1, c2, c3::newtype modify pg_class to point to the new filename modify existing pg_attribute for the column in question recreate indexes that exist on the column unlock table I actually tried this to start with an ran into several dead-ends in trying to rebuild constraints, defaults, etc. In order to do this I think you would need to create a second pg_class entry and do a full table swap. By far, the easiest method to preserve order without writing a ton of additional code is to do something that is on the TODO list already, separate the visible position from physical position. Doing the above has lots of added benefits, where spending a ton of time on TYPE change has very few benefits. ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [PATCHES] ALTER TABLE modifications
Tom Lane writes: I believe the consensus was that automating what you could do by hand is still a step forward. I don't recall that, but if so, I would like to revisit that consensus. AFAICT, this patch does not buy us anything at all. It's just a different spelling of existing functionality. We have never done that before. It just makes the system harder to maintain and use. All commands should be reasonably independent, or at least add some functionality of their own. It clearly would be better if we could relabel the logical column position after finishing the whole process, but I agree with Rod that that is an independent patch. Combining them into one mega-patch doesn't sound like good engineering. Good engineering would be if the logical column number patch comes first. We cannot possibly leave this patch as is. People expect in-place column changes. Things will break left and right, users will complain all over the place if we offer a way to change a column, but yeah, by the way it changes the structure of the table as well. We've had these kinds of good idea/right direction/better than nothing approaches in areas like DROP COLUMN and CLUSTER already, and they were no good. Except in this case, better than nothing doesn't even apply, because there is already something. -- Peter Eisentraut [EMAIL PROTECTED] ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: [PATCHES] ALTER TABLE modifications
This is expected. Doing otherwise would incur into a much bigger performance hit. Anyway, IMHO no code should use SELECT * in any case, which is the only scenario where one would expect physical column order to matter, isn't it? Well, we can always bring back the old idea of a attlognum which is the logical order of the columns. Then we use that to expand * Chris ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [PATCHES] ALTER TABLE modifications
I guess the real question here is whether we would want to revert this capability if a patch to adjust logical column orderings doesn't appear before 7.5. My vote would be no, but apparently Peter's is yes. Any other opinions? The fact that it deals with the nightmare of dropping and recreating indexes and fk's makes it worth it to me. How about functions and views and rules that depend on it though? Chris ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [PATCHES] ALTER TABLE modifications
Peter Eisentraut kirjutas K, 12.11.2003 kell 21:02: Rod Taylor writes: ALTER TABLE tab ADD COLUMN col DEFAULT 3, ADD CHECK (anothercol 3); The above combinational syntax is commented out in gram.y. The support framework is used in both the above and below items, but arbitrary statements probably have some issues -- I've not tested enough to determine. If it is useful, it will be submitted at a later date. I think it's perfectly fine to write two separate ALTER TABLE statements. I guess the difference is that each pass (i.e. ALTER TABLE) needs to do another scan and copy of the table. Putting them in one statement allows all the alterations to be done in one pass. No need to introduce this nonstandard syntax. ALTER TABLE tab ALTER COLUMN col TYPE text TRANSFORM ...; Currently migrates indexes, check constraints, defaults, and the column definition to the new type with optional transform. If the tranform is not supplied, a standard assignment cast is attempted. Please don't use the term transform. It is used by the SQL standard for other purposes. Is the other use conflicting with this syntax ? I think we have preferred reusing existing keywords to adding new ones in the past. - Hannu ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [PATCHES] ALTER TABLE modifications
Hannu Krosing writes: Please don't use the term transform. It is used by the SQL standard for other purposes. Is the other use conflicting with this syntax ? I think we have preferred reusing existing keywords to adding new ones in the past. Maybe (although I don't agree). but we've never intentionally done terminology overloading. -- Peter Eisentraut [EMAIL PROTECTED] ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [PATCHES] ALTER TABLE modifications
On Tue, 2003-11-11 at 23:46, Tom Lane wrote: Bruce Momjian [EMAIL PROTECTED] writes: Is this to be applied to CVS HEAD? It sounded like large portions were still at the request-for-comment stage... It can be applied to -HEAD without breaking anything or backtracking in feature set (that I can see) but there is substantial change in the way things work so a pre-review is warranted and I'm sure changes will be requested. If I don't receive comments for corrections by the weekend, I'll finish off one of the missing components (foreign keys) and submit a replacement. ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [PATCHES] ALTER TABLE modifications
Rod Taylor writes: ALTER TABLE tab ADD COLUMN col DEFAULT 3, ADD CHECK (anothercol 3); The above combinational syntax is commented out in gram.y. The support framework is used in both the above and below items, but arbitrary statements probably have some issues -- I've not tested enough to determine. If it is useful, it will be submitted at a later date. I think it's perfectly fine to write two separate ALTER TABLE statements. No need to introduce this nonstandard syntax. ALTER TABLE tab ALTER COLUMN col TYPE text TRANSFORM ...; Currently migrates indexes, check constraints, defaults, and the column definition to the new type with optional transform. If the tranform is not supplied, a standard assignment cast is attempted. Please don't use the term transform. It is used by the SQL standard for other purposes. What kind of object would you put in place of the ... anyway? A function? What syntax do other databases use? -- Peter Eisentraut [EMAIL PROTECTED] ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send unregister YourEmailAddressHere to [EMAIL PROTECTED])
Re: [PATCHES] ALTER TABLE modifications
On Wed, 2003-11-12 at 14:02, Peter Eisentraut wrote: Rod Taylor writes: ALTER TABLE tab ADD COLUMN col DEFAULT 3, ADD CHECK (anothercol 3); I think it's perfectly fine to write two separate ALTER TABLE statements. No need to introduce this nonstandard syntax. Yes, it is certainly fine to do so, but much faster to do the above. The command shown executes nearly 40% faster than 2 independent statements in a single transaction -- the difference is even more significant with additional sub-commands. ALTER TABLE tab ALTER COLUMN col TYPE text TRANSFORM ...; Currently migrates indexes, check constraints, defaults, and the column definition to the new type with optional transform. If the tranform is not supplied, a standard assignment cast is attempted. Please don't use the term transform. It is used by the SQL standard for other purposes. What kind of object would you put in place of the ... anyway? A function? What syntax do other databases use? I've not found another database which allows this syntax. The suggestion of TRANSFORM was Toms and was a result of using an assignment cast by default. Do you have a better term I can use? http://groups.google.ca/groups?hl=enlr=ie=UTF-8oe=UTF-8frame=rightth=266b02a270a164aaseekm=1064805960.60248.24.camel%40jester#link4 The ... is an A_Expr which does not accept (among other things) subselects. CASE statements, equations, etc. work fine. CREATE TABLE tab (col int2); -- integer to boolean ALTER TABLE tab ALTER col TYPE boolean TRANSFORM CASE WHEN col = 1 THEN true ELSE false END; -- or say Bytes to MBytes (original column is int8) ALTER TABLE tab ALTER col TYPE integer TRANSFORM col / (1024 * 1024); ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [PATCHES] ALTER TABLE modifications
Bruce Momjian [EMAIL PROTECTED] writes: Is this to be applied to CVS HEAD? It sounded like large portions were still at the request-for-comment stage... regards, tom lane ---(end of broadcast)--- TIP 8: explain analyze is your friend