Re: [HACKERS] OUTER JOIN performance regression remains in 8.3beta4

2008-01-17 Thread Kevin Grittner
On Mon, Jan 7, 2008 at 9:01 AM, in message [EMAIL PROTECTED], Kevin Grittner [EMAIL PROTECTED] wrote: On Sun, Jan 6, 2008 at 7:20 PM, in message [EMAIL PROTECTED], Tom Lane [EMAIL PROTECTED] wrote: Kevin Grittner [EMAIL PROTECTED] writes: There was a serious performance regression in

Re: [HACKERS] OUTER JOIN performance regression remains in 8.3beta4

2008-01-17 Thread Kevin Grittner
On Thu, Jan 17, 2008 at 12:30 PM, in message [EMAIL PROTECTED], Tom Lane [EMAIL PROTECTED] wrote: Kevin Grittner [EMAIL PROTECTED] writes: I see this didn't make it into 8.3RC1. Will it be in the 8.3.0 release? You mean this patch?

Re: [HACKERS] OUTER JOIN performance regression remains in 8.3beta4

2008-01-17 Thread Tom Lane
Kevin Grittner [EMAIL PROTECTED] writes: I see this didn't make it into 8.3RC1. Will it be in the 8.3.0 release? You mean this patch? http://archives.postgresql.org/pgsql-committers/2008-01/msg00151.php regards, tom lane ---(end of

Re: [HACKERS] OUTER JOIN performance regression remains in 8.3beta4

2008-01-17 Thread Kevin Grittner
On Thu, Jan 17, 2008 at 1:35 PM, in message [EMAIL PROTECTED], Kevin Grittner [EMAIL PROTECTED] wrote: On Thu, Jan 17, 2008 at 12:30 PM, in message [EMAIL PROTECTED], Tom Lane [EMAIL PROTECTED] wrote: Kevin Grittner [EMAIL PROTECTED] writes: I see this didn't make it into 8.3RC1. Will it

Re: [HACKERS] OUTER JOIN performance regression remains in 8.3beta4

2008-01-09 Thread Gregory Stark
Alvaro Herrera [EMAIL PROTECTED] writes: Tom Lane wrote: Comparing the behavior of this to my patch for HEAD, I am coming to the conclusion that this is actually a *better* planning method than removing the redundant join conditions, even when they're truly rendundant! The reason emerges

Re: [HACKERS] OUTER JOIN performance regression remains in 8.3beta4

2008-01-09 Thread Tom Lane
Gregory Stark [EMAIL PROTECTED] writes: Alvaro Herrera [EMAIL PROTECTED] writes: Would it be a good idea to keep removing redundant clauses and rethink the preference for clauseful joins, going forward? I don't understand what's going on here. The planner is choosing one join order over

Re: [HACKERS] OUTER JOIN performance regression remains in 8.3beta4

2008-01-09 Thread Gregory Stark
Tom Lane [EMAIL PROTECTED] writes: As an example, consider t1 join t2 on (...) join t3 on (...) ... join t8 on (...) and for simplicity suppose that each ON condition relates the new table to the immediately preceding table, and that we can't derive any additional join conditions

Re: [HACKERS] OUTER JOIN performance regression remains in 8.3beta4

2008-01-09 Thread Tom Lane
Gregory Stark [EMAIL PROTECTED] writes: So if I write (along with some other joins): t1 join t2 on (t1.x=t2.x) where t1.x=3 I'll get a different result than if I write t1, t2 where t1.x=3 and t2.x=3 In 8.3 you won't, because those are in fact exactly equivalent (and the new EquivalenceClass

Re: [HACKERS] OUTER JOIN performance regression remains in 8.3beta4

2008-01-08 Thread Alvaro Herrera
Tom Lane wrote: Comparing the behavior of this to my patch for HEAD, I am coming to the conclusion that this is actually a *better* planning method than removing the redundant join conditions, even when they're truly rendundant! The reason emerges as soon as you look at cases involving more

Re: [HACKERS] OUTER JOIN performance regression remains in 8.3beta4

2008-01-08 Thread Tom Lane
I wrote: Haven't looked closely at how to fix 8.2, yet. After some study it seems that the simplest, most reliable fix for 8.2 is to dike out the code that removes redundant outer join conditions after propagating a constant across them. This gives the right answer in the cases of concern

Re: [HACKERS] OUTER JOIN performance regression remains in 8.3beta4

2008-01-08 Thread Tom Lane
Alvaro Herrera [EMAIL PROTECTED] writes: Would it be a good idea to keep removing redundant clauses and rethink the preference for clauseful joins, going forward? No --- it would create an exponential growth in planning time for large join problems, while not actually buying anything in the

Re: [HACKERS] OUTER JOIN performance regression remains in 8.3beta4

2008-01-07 Thread Kevin Grittner
On Sun, Jan 6, 2008 at 7:20 PM, in message [EMAIL PROTECTED], Tom Lane [EMAIL PROTECTED] wrote: Kevin Grittner [EMAIL PROTECTED] writes: There was a serious performance regression in OUTER JOIN planning going from 8.2.4 to 8.2.5. I know Tom came up with some patches to mitigate the issues

Re: [HACKERS] OUTER JOIN performance regression remains in 8.3beta4

2008-01-07 Thread Kevin Grittner
On Fri, Jan 4, 2008 at 6:46 PM, in message [EMAIL PROTECTED], Tom Lane [EMAIL PROTECTED] wrote: 8.2 (at least at branch tip, can't say for sure about earlier dot-releases) 8.2.4 and 8.2.5 both behave this way. f2 | f3 | f1 ++ 53 || 53 (1 row) which I claim is

Re: [HACKERS] OUTER JOIN performance regression remains in 8.3beta4

2008-01-06 Thread Tom Lane
Kevin Grittner [EMAIL PROTECTED] writes: There was a serious performance regression in OUTER JOIN planning going from 8.2.4 to 8.2.5. I know Tom came up with some patches to mitigate the issues in 8.2.5, but my testing shows that problems remain in 8.3beta4. Please try the attached proposed

Re: [HACKERS] OUTER JOIN performance regression remains in 8.3beta4

2008-01-05 Thread Tom Lane
I wrote: It's possible that we could teach 8.3 to propagate the constant and keep the join condition in cases like this; I think we actually can do this without too big a change. The main problem is that initsplan.c doesn't put the upper outer join's clause into the list of mergejoinable

[HACKERS] OUTER JOIN performance regression remains in 8.3beta4

2008-01-04 Thread Kevin Grittner
There was a serious performance regression in OUTER JOIN planning going from 8.2.4 to 8.2.5. I know Tom came up with some patches to mitigate the issues in 8.2.5, but my testing shows that problems remain in 8.3beta4. The query: SELECT CH.caseNo, CH.countyNo, CH.chargeNo,

Re: [HACKERS] OUTER JOIN performance regression remains in 8.3beta4

2008-01-04 Thread Kevin Grittner
On Fri, Jan 4, 2008 at 12:16 PM, in message [EMAIL PROTECTED], Kevin Grittner [EMAIL PROTECTED] wrote: problems remain in 8.3beta4. As I poked around at this, it started to seem familiar. I had previously posted about this query's performance under 8.2.4.

Re: [HACKERS] OUTER JOIN performance regression remains in 8.3beta4

2008-01-04 Thread Tom Lane
Kevin Grittner [EMAIL PROTECTED] writes: As I poked around at this, it started to seem familiar. I had previously posted about this query's performance under 8.2.4. http://archives.postgresql.org/pgsql-performance/2007-10/msg00087.php Well, that thread gave some of the missing details, such

Re: [HACKERS] OUTER JOIN performance regression remains in 8.3beta4

2008-01-04 Thread Kevin Grittner
On Fri, Jan 4, 2008 at 4:40 PM, in message [EMAIL PROTECTED], Tom Lane [EMAIL PROTECTED] wrote: what the heck is CountyNoT? bigbird=# \dD CountyNoT List of domains Schema | Name| Type | Modifier | Check +---+--+--+--- public

Re: [HACKERS] OUTER JOIN performance regression remains in 8.3beta4

2008-01-04 Thread Kevin Grittner
On Fri, Jan 4, 2008 at 4:29 PM, in message [EMAIL PROTECTED], Tom Lane [EMAIL PROTECTED] wrote: Can't do much with this without seeing the table and view definitions involved. Understood. It was while I was putting that together that it struck me as familiar. They are the same as in this

Re: [HACKERS] OUTER JOIN performance regression remains in 8.3beta4

2008-01-04 Thread Tom Lane
Kevin Grittner [EMAIL PROTECTED] writes: There was a serious performance regression in OUTER JOIN planning going from 8.2.4 to 8.2.5. I know Tom came up with some patches to mitigate the issues in 8.2.5, but my testing shows that problems remain in 8.3beta4. Can't do much with this without

Re: [HACKERS] OUTER JOIN performance regression remains in 8.3beta4

2008-01-04 Thread Kevin Grittner
On Fri, Jan 4, 2008 at 4:46 PM, in message [EMAIL PROTECTED], Tom Lane [EMAIL PROTECTED] wrote: nor mentions the data types involved. Schema | Name | Type| Modifier | Check +---+-+--+--- public |

Re: [HACKERS] OUTER JOIN performance regression remains in 8.3beta4

2008-01-04 Thread Tom Lane
Kevin Grittner [EMAIL PROTECTED] writes: On Fri, Jan 4, 2008 at 4:29 PM, in message [EMAIL PROTECTED], Tom Lane [EMAIL PROTECTED] wrote: Can't do much with this without seeing the table and view definitions involved. Understood. It was while I was putting that together that it struck me

Re: [HACKERS] OUTER JOIN performance regression remains in 8.3beta4

2008-01-04 Thread Kevin Grittner
On Fri, Jan 4, 2008 at 4:51 PM, in message [EMAIL PROTECTED], Kevin Grittner [EMAIL PROTECTED] wrote: keyEventSeqNo | integer | COALESCE( CASE WHEN d.eventType IS NOT NULL THEN d.keyEventSeqNo::smallint ELSE b.keyEventSeqNo::smallint

Re: [HACKERS] OUTER JOIN performance regression remains in 8.3beta4

2008-01-04 Thread Tom Lane
Hmm ... I think I've managed to invent a test case, and unfortunately for you, what it shows is that 8.2 is optimizing the query incorrectly. create table t1 (f1 int primary key); create table t2 (f2 int primary key); create table t3 (f3 int primary key); insert into t1 values(53); insert into t2

Re: [HACKERS] OUTER JOIN performance regression remains in 8.3beta4

2008-01-04 Thread Kevin Grittner
On Fri, Jan 4, 2008 at 5:45 PM, in message [EMAIL PROTECTED], Kevin Grittner [EMAIL PROTECTED] wrote: On Fri, Jan 4, 2008 at 4:51 PM, in message [EMAIL PROTECTED], Kevin Grittner [EMAIL PROTECTED] wrote: keyEventSeqNo | integer | COALESCE( CASE

Re: [HACKERS] OUTER JOIN performance regression remains in 8.3beta4

2008-01-04 Thread Tom Lane
I wrote: [ Pokes at older branches... ] Oh, that's interesting, 8.1 seems to do the right thing already! Seems that 8.1 does the right thing for the wrong reason :-(. Just like 8.2, it falsely concludes that the f3 = f1 clause can be deleted, but it fails to get rid of every copy of it. The