Re: [HACKERS] logical column ordering

2015-04-14 Thread Robert Haas
On Tue, Apr 14, 2015 at 2:38 PM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: As I said, I'm still writing the first pieces of this so I'm not sure what other ramifications it will have. If there are any thoughts, I would appreciate them. (Particularly useful input on whether it is

Re: [HACKERS] logical column ordering

2015-04-14 Thread Alvaro Herrera
I've been looking at this again. It has become apparent to me that what we're doing in parse analysis is wrong, and the underlying node representation is wrong too. Here's a different approach, which I hope will give better fruits. I'm still working on implementing the ideas here (and figuring

Re: [HACKERS] logical column ordering

2015-03-24 Thread Kevin Grittner
Robert Haas robertmh...@gmail.com wrote: On Thu, Mar 12, 2015 at 9:57 AM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: However, there's a difference between making a query silently given different results, and breaking it completely forcing the user to re-study how to write it. I think

Re: [HACKERS] logical column ordering

2015-03-23 Thread Andres Freund
On 2015-03-23 13:01:48 -0400, Robert Haas wrote: I'm a little confused as to the status of this patch. It's marked as Waiting on Author in the CommitFest application, and the last patch version was posted in December. I think it fairly can be marked as returned with feedback for now?

Re: [HACKERS] logical column ordering

2015-03-23 Thread Andres Freund
On 2015-03-23 14:19:50 -0300, Alvaro Herrera wrote: Andres Freund wrote: On 2015-03-23 13:01:48 -0400, Robert Haas wrote: I'm a little confused as to the status of this patch. It's marked as Waiting on Author in the CommitFest application, and the last patch version was posted in

Re: [HACKERS] logical column ordering

2015-03-23 Thread Robert Haas
On Thu, Mar 12, 2015 at 9:57 AM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: However, there's a difference between making a query silently given different results, and breaking it completely forcing the user to re-study how to write it. I think the latter is better. In that light we

Re: [HACKERS] logical column ordering

2015-03-23 Thread Tomas Vondra
On 23.3.2015 19:52, Peter Geoghegan wrote: On Mon, Mar 23, 2015 at 11:50 AM, Tomas Vondra tomas.von...@2ndquadrant.com wrote: Not sure how that's supposed to improve the situation? Also, when you change the status to 'stalled', it only makes it more difficult to identify why it was stalled

Re: [HACKERS] logical column ordering

2015-03-23 Thread Josh Berkus
On 03/23/2015 02:32 PM, Tomas Vondra wrote: Oh, right. Yes, tracking time since the last status change like this might be useful, although my experience is that many patches are stuck at some status yet there was a long discussion on the list ... Not sure if that counts as 'stalled'. Time

Re: [HACKERS] logical column ordering

2015-03-23 Thread Tomas Vondra
On 23.3.2015 18:30, Andres Freund wrote: I think it fairly can be marked as returned with feedback for now? That will eventually be the end result, yes. If it's time to do that now, or leave the patch in the CF and only bounce it at the end, I don't know. ... which means that no useful

Re: [HACKERS] logical column ordering

2015-03-23 Thread Tomas Vondra
On 23.3.2015 22:53, Jeff Janes wrote: On Mon, Mar 23, 2015 at 11:52 AM, Tomas Vondra Sorry, the 23/2 one is the one I meant. I got confused over which of the emails listed as having an attachment but no patch was the one that actually had a patch. (If the commitfest app can't correctly

Re: [HACKERS] logical column ordering

2015-03-23 Thread Jeff Janes
On Mon, Mar 23, 2015 at 10:01 AM, Robert Haas robertmh...@gmail.com wrote: On Thu, Mar 12, 2015 at 9:57 AM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: However, there's a difference between making a query silently given different results, and breaking it completely forcing the user to

Re: [HACKERS] logical column ordering

2015-03-23 Thread Peter Geoghegan
On Mon, Mar 23, 2015 at 10:01 AM, Robert Haas robertmh...@gmail.com wrote: I'm a little confused as to the status of this patch. It's marked as Waiting on Author in the CommitFest application, and the last patch version was posted in December. The fact that the new CommitFest application

Re: [HACKERS] logical column ordering

2015-03-23 Thread Alvaro Herrera
Andres Freund wrote: On 2015-03-23 13:01:48 -0400, Robert Haas wrote: I'm a little confused as to the status of this patch. It's marked as Waiting on Author in the CommitFest application, and the last patch version was posted in December. I think it fairly can be marked as returned with

Re: [HACKERS] logical column ordering

2015-03-23 Thread Tomas Vondra
Hi, On 23.3.2015 18:01, Robert Haas wrote: On Thu, Mar 12, 2015 at 9:57 AM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: However, there's a difference between making a query silently given different results, and breaking it completely forcing the user to re-study how to write it. I think

Re: [HACKERS] logical column ordering

2015-03-23 Thread Tomas Vondra
Hi, On 23.3.2015 18:07, Peter Geoghegan wrote: On Mon, Mar 23, 2015 at 10:01 AM, Robert Haas robertmh...@gmail.com wrote: I'm a little confused as to the status of this patch. It's marked as Waiting on Author in the CommitFest application, and the last patch version was posted in December.

Re: [HACKERS] logical column ordering

2015-03-23 Thread Peter Geoghegan
On Mon, Mar 23, 2015 at 11:50 AM, Tomas Vondra tomas.von...@2ndquadrant.com wrote: Not sure how that's supposed to improve the situation? Also, when you change the status to 'stalled', it only makes it more difficult to identify why it was stalled (was it waiting for author or a review?).

Re: [HACKERS] logical column ordering

2015-03-23 Thread Tomas Vondra
On 23.3.2015 18:08, Jeff Janes wrote: On Mon, Mar 23, 2015 at 10:01 AM, Robert Haas robertmh...@gmail.com mailto:robertmh...@gmail.com wrote: There was a patch here, which in the commit fest is hidden behind other non-attachments in the same email: Attachment (randomize.sql

Re: [HACKERS] logical column ordering

2015-03-13 Thread Jeff Janes
On Mon, Feb 23, 2015 at 3:09 PM, Tomas Vondra tomas.von...@2ndquadrant.com wrote: Hi, attached is the result of my first attempt to make the logical column ordering patch work. This touches a lot of code in the executor that is mostly new to me, so if you see something that looks like an

Re: [HACKERS] logical column ordering

2015-03-12 Thread Andres Freund
Hi, On 2015-03-11 22:16:52 -0400, Tom Lane wrote: I agree though that it's worth considering defining pg_attribute.attnum as the logical column position so as to minimize the effects on client-side code. I actually wonder if it'd not make more sense to define it as the physical column number.

Re: [HACKERS] logical column ordering

2015-03-12 Thread Alvaro Herrera
Tomas Vondra wrote: On 12.3.2015 03:16, Tom Lane wrote: I agree though that it's worth considering defining pg_attribute.attnum as the logical column position so as to minimize the effects on client-side code. I doubt there is much stuff client-side that cares about column creation

Re: [HACKERS] logical column ordering

2015-03-12 Thread Tom Lane
Alvaro Herrera alvhe...@2ndquadrant.com writes: However, there's a difference between making a query silently given different results, and breaking it completely forcing the user to re-study how to write it. I think the latter is better. In that light we should just drop attnum as a column

Re: [HACKERS] logical column ordering

2015-03-12 Thread Tomas Vondra
On 12.3.2015 14:17, Alvaro Herrera wrote: Tomas Vondra wrote: On 12.3.2015 03:16, Tom Lane wrote: I agree though that it's worth considering defining pg_attribute.attnum as the logical column position so as to minimize the effects on client-side code. I doubt there is much stuff

Re: [HACKERS] logical column ordering

2015-03-12 Thread Alvaro Herrera
Tomas Vondra wrote: On 12.3.2015 14:17, Alvaro Herrera wrote: Tomas Vondra wrote: On 12.3.2015 03:16, Tom Lane wrote: I agree though that it's worth considering defining pg_attribute.attnum as the logical column position so as to minimize the effects on client-side code. I doubt

Re: [HACKERS] logical column ordering

2015-03-12 Thread Peter Eisentraut
On 3/12/15 10:07 AM, Andres Freund wrote: I actually wonder if it'd not make more sense to define it as the physical column number. That'd reduce the invasiveness and risk of the patch considerably. Clearly, the number of places where attnum has to be changed to something else is not zero, and

Re: [HACKERS] logical column ordering

2015-03-12 Thread Peter Eisentraut
On 3/11/15 10:16 PM, Tom Lane wrote: I think using an OID would break more stuff than it fixes in dependency tracking; in particular you would now need an explicit dependency link from each column to its table, because the sub-object knowledge would no longer work. That might not be a bad

Re: [HACKERS] logical column ordering

2015-03-12 Thread Tomas Vondra
On 12.3.2015 03:16, Tom Lane wrote: Peter Eisentraut pete...@gmx.net writes: Side idea: Let attnum be the logical number, introduce attphysnum as the storage position, and add an oid to pg_attribute as the eternal identifier. That way you avoid breaking pretty much all user code that

Re: [HACKERS] logical column ordering

2015-03-11 Thread Peter Eisentraut
On 12/9/14 12:41 PM, Alvaro Herrera wrote: To recap, this is based on the idea of having three numbers for each attribute rather than a single attnum; the first of these is attnum (a number that uniquely identifies an attribute since its inception and may or may not have any relationship to

Re: [HACKERS] logical column ordering

2015-03-11 Thread Tom Lane
Peter Eisentraut pete...@gmx.net writes: Side idea: Let attnum be the logical number, introduce attphysnum as the storage position, and add an oid to pg_attribute as the eternal identifier. That way you avoid breaking pretty much all user code that looks at pg_attribute, which will probably

Re: [HACKERS] logical column ordering

2015-03-03 Thread Bruce Momjian
On Thu, Feb 26, 2015 at 01:55:44PM -0800, Josh Berkus wrote: On 02/26/2015 01:54 PM, Alvaro Herrera wrote: This patch decouples these three things so that they can changed freely -- but provides no user interface to do so. I think that trying to only decouple the thing we currently have in

Re: [HACKERS] logical column ordering

2015-03-03 Thread Jim Nasby
On 3/3/15 11:23 AM, Bruce Momjian wrote: On Thu, Feb 26, 2015 at 01:55:44PM -0800, Josh Berkus wrote: On 02/26/2015 01:54 PM, Alvaro Herrera wrote: This patch decouples these three things so that they can changed freely -- but provides no user interface to do so. I think that trying to only

Re: [HACKERS] logical column ordering

2015-03-03 Thread Bruce Momjian
On Tue, Mar 3, 2015 at 11:24:38AM -0600, Jim Nasby wrote: On 3/3/15 11:23 AM, Bruce Momjian wrote: On Thu, Feb 26, 2015 at 01:55:44PM -0800, Josh Berkus wrote: On 02/26/2015 01:54 PM, Alvaro Herrera wrote: This patch decouples these three things so that they can changed freely -- but

Re: [HACKERS] logical column ordering

2015-03-03 Thread Bruce Momjian
On Tue, Mar 3, 2015 at 11:41:20AM -0600, Jim Nasby wrote: Wouldn't it need attno info too, so all 3 orderings? Uh, what is the third ordering? Physical, logical, and ? It already gets information about dropped columns, if that is the third one. attnum; used in other catalogs to

Re: [HACKERS] logical column ordering

2015-03-03 Thread Jim Nasby
On 3/3/15 11:26 AM, Bruce Momjian wrote: On Tue, Mar 3, 2015 at 11:24:38AM -0600, Jim Nasby wrote: On 3/3/15 11:23 AM, Bruce Momjian wrote: On Thu, Feb 26, 2015 at 01:55:44PM -0800, Josh Berkus wrote: On 02/26/2015 01:54 PM, Alvaro Herrera wrote: This patch decouples these three things so

Re: [HACKERS] logical column ordering

2015-03-01 Thread Alvaro Herrera
Jim Nasby wrote: On 2/27/15 2:37 PM, Gavin Flower wrote: Might be an idea to allow lists of columns and their starting position. ALTER TABLE customer ALTER COLUMN (job_id, type_id, account_num) SET ORDER 3; I would certainly want something along those lines because it would be *way* less

Re: [HACKERS] logical column ordering

2015-03-01 Thread Alvaro Herrera
Jim Nasby wrote: On 2/27/15 2:49 PM, Alvaro Herrera wrote: Tomas Vondra wrote: 1) change the order of columns in SELECT * - display columns so that related ones grouped together (irrespectedly whether they were added later, etc.) FWIW, I find the ordering more important for

Re: [HACKERS] logical column ordering

2015-03-01 Thread Alvaro Herrera
Tomas Vondra wrote: We need an API for physical column reordering, even if it's just pg_ functions. The reason is that we want to enable people writing their own physical column re-ordering tools, so that our users can figure out for us what the best reordering algorithm is. I doubt

Re: [HACKERS] logical column ordering

2015-02-27 Thread Jim Nasby
On 2/27/15 2:37 PM, Gavin Flower wrote: Might be an idea to allow lists of columns and their starting position. ALTER TABLE customer ALTER COLUMN (job_id, type_id, account_num) SET ORDER 3; I would certainly want something along those lines because it would be *way* less verbose (and

Re: [HACKERS] logical column ordering

2015-02-27 Thread Gavin Flower
On 28/02/15 18:34, Jim Nasby wrote: On 2/27/15 2:49 PM, Alvaro Herrera wrote: Tomas Vondra wrote: 1) change the order of columns in SELECT * - display columns so that related ones grouped together (irrespectedly whether they were added later, etc.) FWIW, I find the ordering more

Re: [HACKERS] logical column ordering

2015-02-27 Thread Gavin Flower
On 28/02/15 12:21, Josh Berkus wrote: On 02/27/2015 03:06 PM, Tomas Vondra wrote: On 27.2.2015 23:48, Josh Berkus wrote: Actually, I'm going to go back on what I said. We need an API for physical column reordering, even if it's just pg_ functions. The reason is that we want to enable people

Re: [HACKERS] logical column ordering

2015-02-27 Thread Jim Nasby
On 2/27/15 2:49 PM, Alvaro Herrera wrote: Tomas Vondra wrote: 1) change the order of columns in SELECT * - display columns so that related ones grouped together (irrespectedly whether they were added later, etc.) FWIW, I find the ordering more important for things like \d than

Re: [HACKERS] logical column ordering

2015-02-27 Thread Matt Kelly
Even if it does fit in memory I suspect memory bandwidth is more important than clock cycles. http://people.freebsd.org/~lstewart/articles/cpumemory.pdf This paper is old but the ratios should still be pretty accurate. Main memory is 240 clock cycles away and L1d is only 3. If the

Re: [HACKERS] logical column ordering

2015-02-27 Thread Tomas Vondra
On 26.2.2015 23:36, Tom Lane wrote: Jim Nasby jim.na...@bluetreble.com writes: On 2/26/15 4:01 PM, Alvaro Herrera wrote: Josh Berkus wrote: Oh, I didn't realize there weren't commands to change the LCO. Without at least SQL syntax for LCO, I don't see why we'd take it; this sounds more like

Re: [HACKERS] logical column ordering

2015-02-27 Thread Tomas Vondra
On 27.2.2015 19:23, Alvaro Herrera wrote: Tomas Vondra wrote: Physical ordering is still determined by the CREATE TABLE command, In theory, you should be able to UPDATE attphysnum in pg_attribute too when the table is empty, and new tuples inserted would follow the specified ordering.

Re: [HACKERS] logical column ordering

2015-02-27 Thread Alvaro Herrera
Tomas Vondra wrote: On 26.2.2015 23:42, Kevin Grittner wrote: One use case is to be able to suppress default display of columns that are used for internal purposes. For example, incremental maintenance of materialized views will require storing a count(t) column, and sometimes state

Re: [HACKERS] logical column ordering

2015-02-27 Thread Tomas Vondra
On 26.2.2015 23:42, Kevin Grittner wrote: Tomas Vondra tomas.von...@2ndquadrant.com wrote: Over the time I've heard various use cases for this patch, but in most cases it was quite speculative. If you have an idea where this might be useful, can you explain it here, or maybe point me to a

Re: [HACKERS] logical column ordering

2015-02-27 Thread Alvaro Herrera
Tomas Vondra wrote: Physical ordering is still determined by the CREATE TABLE command, In theory, you should be able to UPDATE attphysnum in pg_attribute too when the table is empty, and new tuples inserted would follow the specified ordering. -- Álvaro Herrera

Re: [HACKERS] logical column ordering

2015-02-27 Thread Alvaro Herrera
Arthur Silva wrote: Sorry to intrude, I've been following this post and I was wondering if it would allow (in the currently planed form or in the future) a wider set of non-rewriting DDLs to Postgres. For example, drop a column without rewriting the table. Perhaps. But dropping a column

Re: [HACKERS] logical column ordering

2015-02-27 Thread Tomas Vondra
On 27.2.2015 20:34, Alvaro Herrera wrote: Tomas Vondra wrote: I think we could calls to the randomization functions into some of the regression tests (say 'create_tables.sql'), but that makes regression tests ... well, random, and I'm not convinced that's a good thing. Also, this makes

Re: [HACKERS] logical column ordering

2015-02-27 Thread Tomas Vondra
OK, so let me summarize here (the other posts in this subthread are discussing the user interface, not the use cases, so I'll respond here). There are two main use cases: 1) change the order of columns in SELECT * - display columns so that related ones grouped together (irrespectedly

Re: [HACKERS] logical column ordering

2015-02-27 Thread Tomas Vondra
On 27.2.2015 19:50, Alvaro Herrera wrote: Tomas Vondra wrote: On 26.2.2015 23:42, Kevin Grittner wrote: One use case is to be able to suppress default display of columns that are used for internal purposes. For example, incremental maintenance of materialized views will require storing a

Re: [HACKERS] logical column ordering

2015-02-27 Thread Tomas Vondra
On 26.2.2015 23:34, Andres Freund wrote: On 2015-02-26 16:16:54 -0600, Jim Nasby wrote: On 2/26/15 4:01 PM, Alvaro Herrera wrote: The reason for doing it this way is that changing the underlying architecture is really hard, without having to bear an endless hackers bike shed discussion about

Re: [HACKERS] logical column ordering

2015-02-27 Thread Gavin Flower
On 28/02/15 09:09, Josh Berkus wrote: Tomas, So for an API, 100% of the use cases I have for this feature would be satisfied by: ALTER TABLE __ ALTER COLUMN _ SET ORDER # and: ALTER TABLE _ ADD COLUMN colname coltype ORDER # If that's infeasible, a function would be less

Re: [HACKERS] logical column ordering

2015-02-27 Thread Tomas Vondra
On 27.2.2015 21:42, Josh Berkus wrote: On 02/27/2015 12:25 PM, Tomas Vondra wrote: On 27.2.2015 21:09, Josh Berkus wrote: Tomas, So for an API, 100% of the use cases I have for this feature would be satisfied by: ALTER TABLE __ ALTER COLUMN _ SET ORDER # and: ALTER TABLE _

Re: [HACKERS] logical column ordering

2015-02-27 Thread Alvaro Herrera
Tomas Vondra wrote: 1) change the order of columns in SELECT * - display columns so that related ones grouped together (irrespectedly whether they were added later, etc.) - keep columns synced with COPY - requires user interface (ALTER TABLE _ ALTER COLUMN _ SET ORDER _)

Re: [HACKERS] logical column ordering

2015-02-27 Thread Gavin Flower
On 28/02/15 09:49, Alvaro Herrera wrote: Tomas Vondra wrote: 1) change the order of columns in SELECT * - display columns so that related ones grouped together (irrespectedly whether they were added later, etc.) - keep columns synced with COPY - requires user interface

Re: [HACKERS] logical column ordering

2015-02-27 Thread Arthur Silva
On Fri, Feb 27, 2015 at 4:44 PM, Tomas Vondra tomas.von...@2ndquadrant.com wrote: On 27.2.2015 20:34, Alvaro Herrera wrote: Tomas Vondra wrote: I think we could calls to the randomization functions into some of the regression tests (say 'create_tables.sql'), but that makes regression

Re: [HACKERS] logical column ordering

2015-02-27 Thread Josh Berkus
On 02/27/2015 12:25 PM, Tomas Vondra wrote: On 27.2.2015 21:09, Josh Berkus wrote: Tomas, So for an API, 100% of the use cases I have for this feature would be satisfied by: ALTER TABLE __ ALTER COLUMN _ SET ORDER # and: ALTER TABLE _ ADD COLUMN colname coltype ORDER #

Re: [HACKERS] logical column ordering

2015-02-27 Thread Alvaro Herrera
Tomas Vondra wrote: I think we could calls to the randomization functions into some of the regression tests (say 'create_tables.sql'), but that makes regression tests ... well, random, and I'm not convinced that's a good thing. Also, this makes regression tests harder to think, because

Re: [HACKERS] logical column ordering

2015-02-27 Thread Alvaro Herrera
Tomas Vondra wrote: On 27.2.2015 20:34, Alvaro Herrera wrote: Tomas Vondra wrote: I think we could calls to the randomization functions into some of the regression tests (say 'create_tables.sql'), but that makes regression tests ... well, random, and I'm not convinced that's a good

Re: [HACKERS] logical column ordering

2015-02-27 Thread Arthur Silva
On Feb 27, 2015 5:02 PM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Arthur Silva wrote: Sorry to intrude, I've been following this post and I was wondering if it would allow (in the currently planed form or in the future) a wider set of non-rewriting DDLs to Postgres. For example,

Re: [HACKERS] logical column ordering

2015-02-27 Thread Josh Berkus
On 02/27/2015 12:48 PM, Tomas Vondra wrote: On 27.2.2015 21:42, Josh Berkus wrote: On 02/27/2015 12:25 PM, Tomas Vondra wrote: On 27.2.2015 21:09, Josh Berkus wrote: Tomas, So for an API, 100% of the use cases I have for this feature would be satisfied by: ALTER TABLE __ ALTER COLUMN

Re: [HACKERS] logical column ordering

2015-02-27 Thread Tomas Vondra
On 27.2.2015 23:48, Josh Berkus wrote: On 02/27/2015 12:48 PM, Tomas Vondra wrote: On 27.2.2015 21:42, Josh Berkus wrote: On 02/27/2015 12:25 PM, Tomas Vondra wrote: On 27.2.2015 21:09, Josh Berkus wrote: Tomas, So for an API, 100% of the use cases I have for this feature would be

Re: [HACKERS] logical column ordering

2015-02-27 Thread David G Johnston
Tomas Vondra-4 wrote But if we want to allow users to define this, I'd say let's make that part of CREATE TABLE, i.e. the order of columns defines logical order, and you use something like 'AFTER' to specify physical order. CREATE TABLE test ( a INT AFTER b,-- attlognum = 1,

Re: [HACKERS] logical column ordering

2015-02-27 Thread Josh Berkus
On 02/27/2015 03:06 PM, Tomas Vondra wrote: On 27.2.2015 23:48, Josh Berkus wrote: Actually, I'm going to go back on what I said. We need an API for physical column reordering, even if it's just pg_ functions. The reason is that we want to enable people writing their own physical column

Re: [HACKERS] logical column ordering

2015-02-27 Thread David G Johnston
David G Johnston wrote Tomas Vondra-4 wrote But if we want to allow users to define this, I'd say let's make that part of CREATE TABLE, i.e. the order of columns defines logical order, and you use something like 'AFTER' to specify physical order. CREATE TABLE test ( a INT

Re: [HACKERS] logical column ordering

2015-02-26 Thread Jim Nasby
On 2/26/15 4:34 PM, Andres Freund wrote: On 2015-02-26 16:16:54 -0600, Jim Nasby wrote: On 2/26/15 4:01 PM, Alvaro Herrera wrote: The reason for doing it this way is that changing the underlying architecture is really hard, without having to bear an endless hackers bike shed discussion about

Re: [HACKERS] logical column ordering

2015-02-26 Thread Jim Nasby
On 2/23/15 5:09 PM, Tomas Vondra wrote: Over the time I've heard various use cases for this patch, but in most cases it was quite speculative. If you have an idea where this might be useful, can you explain it here, or maybe point me to a place where it's described? For better or worse, table

Re: [HACKERS] logical column ordering

2015-02-26 Thread Tom Lane
Jim Nasby jim.na...@bluetreble.com writes: On 2/26/15 4:01 PM, Alvaro Herrera wrote: Josh Berkus wrote: Oh, I didn't realize there weren't commands to change the LCO. Without at least SQL syntax for LCO, I don't see why we'd take it; this sounds more like a WIP patch. The reason for doing

Re: [HACKERS] logical column ordering

2015-02-26 Thread Jim Nasby
On 2/26/15 4:01 PM, Alvaro Herrera wrote: Josh Berkus wrote: On 02/26/2015 01:54 PM, Alvaro Herrera wrote: This patch decouples these three things so that they can changed freely -- but provides no user interface to do so. I think that trying to only decouple the thing we currently have in

Re: [HACKERS] logical column ordering

2015-02-26 Thread Andres Freund
On 2015-02-26 16:16:54 -0600, Jim Nasby wrote: On 2/26/15 4:01 PM, Alvaro Herrera wrote: The reason for doing it this way is that changing the underlying architecture is really hard, without having to bear an endless hackers bike shed discussion about the best userland syntax to use. It seems

Re: [HACKERS] logical column ordering

2015-02-26 Thread Alvaro Herrera
Josh Berkus wrote: On 02/26/2015 01:54 PM, Alvaro Herrera wrote: This patch decouples these three things so that they can changed freely -- but provides no user interface to do so. I think that trying to only decouple the thing we currently have in two pieces, and then have a subsequent

Re: [HACKERS] logical column ordering

2015-02-26 Thread Kevin Grittner
Tomas Vondra tomas.von...@2ndquadrant.com wrote: Over the time I've heard various use cases for this patch, but in most cases it was quite speculative. If you have an idea where this might be useful, can you explain it here, or maybe point me to a place where it's described? One use case is

Re: [HACKERS] logical column ordering

2015-02-26 Thread David Steele
On 2/26/15 1:49 PM, Jim Nasby wrote: On 2/23/15 5:09 PM, Tomas Vondra wrote: Over the time I've heard various use cases for this patch, but in most cases it was quite speculative. If you have an idea where this might be useful, can you explain it here, or maybe point me to a place where it's

Re: [HACKERS] logical column ordering

2015-02-26 Thread Gavin Flower
On 27/02/15 14:08, David Steele wrote: [...] I agree with Jim's comments. I've generally followed column ordering that goes something like: 1) primary key 2) foreign keys 3) flags 4) other programmatic data fields (type, order, etc.) 5) non-programmatic data fields (name, description, etc.)

Re: [HACKERS] logical column ordering

2015-02-23 Thread Tomas Vondra
Hi, attached is the result of my first attempt to make the logical column ordering patch work. This touches a lot of code in the executor that is mostly new to me, so if you see something that looks like an obvious bug, it probably is (so let me know). improvements The main

Re: [HACKERS] logical column ordering

2015-01-24 Thread Tomas Vondra
On 20.1.2015 22:30, Alvaro Herrera wrote: I've decided to abandon this patch. I have spent too much time looking at it now. If anyone is interested in trying to study, I can provide the patches I came up with, explanations, and references to prior discussion -- feel free to ask. I'll take

Re: [HACKERS] logical column ordering

2015-01-20 Thread Alvaro Herrera
I've decided to abandon this patch. I have spent too much time looking at it now. If anyone is interested in trying to study, I can provide the patches I came up with, explanations, and references to prior discussion -- feel free to ask. My main motivation for this work is to enable a later

Re: [HACKERS] logical column ordering

2015-01-14 Thread Michael Paquier
On Sun, Jan 4, 2015 at 10:37 AM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: So I'm reworking my patch with that in mind. Switching to returned with feedback. Alvaro, feel free to add an entry to the next CF if you are planning to work on it again. -- Michael -- Sent via pgsql-hackers

Re: [HACKERS] logical column ordering

2015-01-03 Thread Andres Freund
On 2014-12-09 14:41:46 -0300, Alvaro Herrera wrote: So I've been updating my very old patch to allow logical and physical column reordering. Here's a WIP first cut for examination. Do you have a updated patch that has ripened further? The first thing where this matters is tuple descriptor

Re: [HACKERS] logical column ordering

2015-01-03 Thread Alvaro Herrera
Andres Freund wrote: On 2014-12-09 14:41:46 -0300, Alvaro Herrera wrote: So I've been updating my very old patch to allow logical and physical column reordering. Here's a WIP first cut for examination. Do you have a updated patch that has ripened further? Not yet. Phil was kind enough

Re: [HACKERS] logical column ordering

2014-12-12 Thread Andres Freund
On 2014-12-10 19:06:28 -0800, Josh Berkus wrote: On 12/10/2014 05:14 PM, Stephen Frost wrote: * Andres Freund (and...@2ndquadrant.com) wrote: But the scheduling of commits with regard to the 9.5 schedule actually opens a relevant question: When are we planning to release 9.5? Because

Re: [HACKERS] logical column ordering

2014-12-10 Thread Robert Haas
On Wed, Dec 10, 2014 at 12:17 AM, Tom Lane t...@sss.pgh.pa.us wrote: Alvaro Herrera alvhe...@2ndquadrant.com writes: Andrew Dunstan wrote: I seriously doubt it, although I could be wrong. Unless someone can show a significant performance gain from using physical order, which would be a bit of

Re: [HACKERS] logical column ordering

2014-12-10 Thread Alvaro Herrera
Robert Haas wrote: On Wed, Dec 10, 2014 at 12:17 AM, Tom Lane t...@sss.pgh.pa.us wrote: Alvaro Herrera alvhe...@2ndquadrant.com writes: Andrew Dunstan wrote: I seriously doubt it, although I could be wrong. Unless someone can show a significant performance gain from using physical order,

Re: [HACKERS] logical column ordering

2014-12-10 Thread Robert Haas
On Wed, Dec 10, 2014 at 9:25 AM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: FWIW I have no intention to add options for physical/logical ordering anywhere. All users will see is that tables will follow the same (logical) order everywhere. Just to be clear, I wasn't in any way attending to

Re: [HACKERS] logical column ordering

2014-12-10 Thread Stephen Frost
Robert, all, * Robert Haas (robertmh...@gmail.com) wrote: To the extent that I have any concern about the patch at this point, it's around stability. I would awfully rather see something like this get committed at the beginning of a development cycle than the end. I tend to agree with this;

Re: [HACKERS] logical column ordering

2014-12-10 Thread Robert Haas
On Wed, Dec 10, 2014 at 11:22 AM, Stephen Frost sfr...@snowman.net wrote: I'm not quite sure that I see how that's relevant. Bugs will happen, unfortunately, no matter how much review is done of a given patch. That isn't to say that we shouldn't do any review, but it's a trade-off. This

Re: [HACKERS] logical column ordering

2014-12-10 Thread Andres Freund
On 2014-12-10 12:06:11 -0500, Robert Haas wrote: Ultimately, I think this is mostly going to be a question of what Alvaro feels comfortable with; he's presumably going to have a better sense of where and to what extent there might be bugs lurking than any of the rest of us. But the point is

Re: [HACKERS] logical column ordering

2014-12-10 Thread Stephen Frost
* Andres Freund (and...@2ndquadrant.com) wrote: But the scheduling of commits with regard to the 9.5 schedule actually opens a relevant question: When are we planning to release 9.5? Because If we try ~ one year from now it's a whole different ballgame than if we try to go back to september.

Re: [HACKERS] logical column ordering

2014-12-10 Thread Josh Berkus
On 12/10/2014 05:14 PM, Stephen Frost wrote: * Andres Freund (and...@2ndquadrant.com) wrote: But the scheduling of commits with regard to the 9.5 schedule actually opens a relevant question: When are we planning to release 9.5? Because If we try ~ one year from now it's a whole different

Re: [HACKERS] logical column ordering

2014-12-10 Thread Josh Berkus
On 12/09/2014 09:11 PM, Tom Lane wrote: Josh Berkus j...@agliodbs.com writes: Question on COPY, though: there's reasons why people would want COPY to dump in either physical or logical order. If you're doing COPY to create CSV files for output, then you want the columns in logical order. If

Re: [HACKERS] logical column ordering

2014-12-09 Thread Jim Nasby
On 12/9/14, 11:41 AM, Alvaro Herrera wrote: I'm going to see about it while I get feedback on the rest of this patch; in particular, extra test cases that fail to work when columns have been moved around are welcome, so that I can add them to the regress test. What I have now is the basics I'm

Re: [HACKERS] logical column ordering

2014-12-09 Thread Josh Berkus
On 12/09/2014 09:41 AM, Alvaro Herrera wrote: The first thing where this matters is tuple descriptor expansion in parse analysis; at this stage, things such as * (in select *) are turned into a target list, which must be sorted according to attlognum. To achieve this I added a new routine to

Re: [HACKERS] logical column ordering

2014-12-09 Thread Andrew Dunstan
On 12/09/2014 06:19 PM, Josh Berkus wrote: On 12/09/2014 09:41 AM, Alvaro Herrera wrote: The first thing where this matters is tuple descriptor expansion in parse analysis; at this stage, things such as * (in select *) are turned into a target list, which must be sorted according to attlognum.

Re: [HACKERS] logical column ordering

2014-12-09 Thread Alvaro Herrera
Andrew Dunstan wrote: On 12/09/2014 06:19 PM, Josh Berkus wrote: On 12/09/2014 09:41 AM, Alvaro Herrera wrote: The first thing where this matters is tuple descriptor expansion in parse analysis; at this stage, things such as * (in select *) are turned into a target list, which must be

Re: [HACKERS] logical column ordering

2014-12-09 Thread Tom Lane
Josh Berkus j...@agliodbs.com writes: Question on COPY, though: there's reasons why people would want COPY to dump in either physical or logical order. If you're doing COPY to create CSV files for output, then you want the columns in logical order. If you're doing COPY for pg_dump, then you

Re: [HACKERS] logical column ordering

2014-12-09 Thread Tom Lane
Alvaro Herrera alvhe...@2ndquadrant.com writes: Andrew Dunstan wrote: I seriously doubt it, although I could be wrong. Unless someone can show a significant performance gain from using physical order, which would be a bit of a surprise to me, I would just stick with logical ordering as the