Re: [HACKERS] Query optimization problem

2010-07-28 Thread Dimitri Fontaine
Tom Lane t...@sss.pgh.pa.us writes: In the example, we do have d1.id and d2.basedon grouped in an equivalence class. So in principle you could substitute d1.id into the WHERE clause in place of d2.basedon, once you'd checked that it was being used with an operator that's compatible with the

Re: [HACKERS] merge command - GSoC progress

2010-07-28 Thread Boxuan Zhai
2010/7/28 Robert Haas robertmh...@gmail.com On Tue, Jul 27, 2010 at 1:04 AM, Boxuan Zhai bxzhai2...@gmail.com wrote: I have get a edition that the merge command can run. It accept the standard merge command and can do UPDATE, INSERT and DELETE actions now. But we cannot put additional

Re: [HACKERS] merge command - GSoC progress

2010-07-28 Thread Robert Haas
On Wed, Jul 28, 2010 at 6:08 AM, Boxuan Zhai bxzhai2...@gmail.com wrote: On Tue, Jul 27, 2010 at 1:04 AM, Boxuan Zhai bxzhai2...@gmail.com wrote: I have get a edition that the merge command can run. It accept the standard merge command and can do UPDATE, INSERT and DELETE actions now. But

Re: [HACKERS] PostGIS vs. PGXS in 9.0beta3

2010-07-28 Thread Mark Cave-Ayland
Andrew Dunstan wrote: The real problem has nothing to do with any of the analysis, as you say. It is this: they have an override file for PGXS and it uses $(mkinstalldirs) which we got rid of about a year ago. So apparently they haven't been testing much against any of our alphas or betas or

Re: [HACKERS] page corruption on 8.3+ that makes it to standby

2010-07-28 Thread Robert Haas
On Wed, Jul 28, 2010 at 12:23 AM, Jeff Davis pg...@j-davis.com wrote: On Tue, 2010-07-27 at 15:23 -0700, Jeff Davis wrote: On Tue, 2010-07-27 at 17:18 -0400, Robert Haas wrote: My first concern with that idea was that it may create an inconsistency between the primary and the standby. The

Re: [HACKERS] Query optimization problem

2010-07-28 Thread Robert Haas
On Wed, Jul 28, 2010 at 3:45 AM, Dimitri Fontaine dfonta...@hi-media.com wrote: Even if we understood how to direct the rewriting process, I'm really dubious that it would win often enough to justify the added planning time.  The particular problem here seems narrow enough that solving it on

Re: [HACKERS] Query optimization problem

2010-07-28 Thread Dimitri Fontaine
Robert Haas robertmh...@gmail.com writes: But here you want to have different paths for the same relation that generate *different output*, and the planner doesn't understand that concept. Sorry? I though what Equivalence Class provides is the proving that using this qualification or another

Re: [HACKERS] page corruption on 8.3+ that makes it to standby

2010-07-28 Thread Simon Riggs
On Tue, 2010-07-27 at 21:23 -0700, Jeff Davis wrote: Both potential fixes attached and both appear to work. fix1 -- Only call PageSetLSN/TLI inside log_newpage() and heap_xlog_newpage() if the page is not zeroed. fix2 -- Don't call log_newpage() at all if the page is not zeroed. Please

Re: [HACKERS] PostGIS vs. PGXS in 9.0beta3

2010-07-28 Thread Andrew Dunstan
Mark Cave-Ayland wrote: Andrew Dunstan wrote: The real problem has nothing to do with any of the analysis, as you say. It is this: they have an override file for PGXS and it uses $(mkinstalldirs) which we got rid of about a year ago. So apparently they haven't been testing much against any

Re: [HACKERS] Query optimization problem

2010-07-28 Thread Robert Haas
On Wed, Jul 28, 2010 at 6:55 AM, Dimitri Fontaine dfonta...@hi-media.com wrote: Robert Haas robertmh...@gmail.com writes:  But here you want to have different paths for the same relation that generate *different output*, and the planner doesn't understand that concept. Sorry? I though what

Re: [HACKERS] Query optimization problem

2010-07-28 Thread Yeb Havinga
Robert Haas wrote: On Wed, Jul 28, 2010 at 6:55 AM, Dimitri Fontaine dfonta...@hi-media.com wrote: Robert Haas robertmh...@gmail.com writes: But here you want to have different paths for the same relation that generate *different output*, and the planner doesn't understand that

Re: [HACKERS] PostGIS vs. PGXS in 9.0beta3

2010-07-28 Thread Mark Cave-Ayland
Andrew Dunstan wrote: No, the configure test is wrong. Here's what's in configure.ac: dnl Temporary hack until minimum PostgreSQL version is 8.5: dnl If PostgreSQL 8.5 is detected, trigger the inclusion of the new versioned PGXS targets PGXSOVERRIDE=0 if test !

Re: [HACKERS] page corruption on 8.3+ that makes it to standby

2010-07-28 Thread Robert Haas
On Wed, Jul 28, 2010 at 7:02 AM, Simon Riggs si...@2ndquadrant.com wrote: On Tue, 2010-07-27 at 21:23 -0700, Jeff Davis wrote: Both potential fixes attached and both appear to work. fix1 -- Only call PageSetLSN/TLI inside log_newpage() and heap_xlog_newpage() if the page is not zeroed.

Re: [HACKERS] Query optimization problem

2010-07-28 Thread Robert Haas
On Wed, Jul 28, 2010 at 7:24 AM, Yeb Havinga yebhavi...@gmail.com wrote: Sorry? I though what Equivalence Class provides is the proving that using this qualification or another will *not* affect the output. In a query like...  SELECT d1.ID, d2.ID  FROM DocPrimary d1   JOIN DocPrimary d2 ON

Re: [HACKERS] Query optimization problem

2010-07-28 Thread Dimitri Fontaine
Robert Haas robertmh...@gmail.com writes: SELECT d1.ID, d2.ID FROM DocPrimary d1 JOIN DocPrimary d2 ON d2.BasedOn=d1.ID WHERE (d1.ID=234409763) or (d2.ID=234409763) ...you're going to scan d1, scan d2, and then join the results. The scan of d1 is going to produce different results

[HACKERS] patch saved in commitfest application isn't actual now

2010-07-28 Thread Pavel Stehule
Hello, can you send a current version, please. I looked to git repository, but you did more changes. Thank you Pavel Stehule -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Query optimization problem

2010-07-28 Thread Yeb Havinga
Robert Haas wrote: On Wed, Jul 28, 2010 at 7:24 AM, Yeb Havinga yebhavi...@gmail.com wrote: Sorry? I though what Equivalence Class provides is the proving that using this qualification or another will *not* affect the output. In a query like... SELECT d1.ID, d2.ID FROM

Re: [HACKERS] ALTER TABLE SET STATISTICS requires AccessExclusiveLock

2010-07-28 Thread Peter Eisentraut
On tor, 2010-07-15 at 10:24 +0100, Simon Riggs wrote: Patch to reduce lock levels for ALTER TABLE CREATE TRIGGER CREATE RULE Tried this out, but $subject is still the case. The problem is that ATRewriteCatalogs() calls AlterTableCreateToastTable() based on what it thinks the subcommands

Re: [HACKERS] ALTER TABLE SET STATISTICS requires AccessExclusiveLock

2010-07-28 Thread Simon Riggs
On Wed, 2010-07-28 at 15:24 +0300, Peter Eisentraut wrote: On tor, 2010-07-15 at 10:24 +0100, Simon Riggs wrote: Patch to reduce lock levels for ALTER TABLE CREATE TRIGGER CREATE RULE Tried this out, but $subject is still the case. The problem is that ATRewriteCatalogs() calls

Re: [HACKERS] Query optimization problem

2010-07-28 Thread Tom Lane
Yeb Havinga yebhavi...@gmail.com writes: Robert Haas wrote: On Wed, Jul 28, 2010 at 7:24 AM, Yeb Havinga yebhavi...@gmail.com wrote: Wouldn't it be relatively easy, to rewrite the filter expression by adding expressions, instead of replacing constants, in the disjunctive case, so the example

Re: [HACKERS] Toward a column reorder solution

2010-07-28 Thread Joshua D. Drake
On Tue, 27 Jul 2010 19:55:18 -0700, David E. Wheeler da...@kineticode.com wrote: On Jul 27, 2010, at 3:01 PM, Joshua D. Drake wrote: Correct. We are also hoping to get some sponsorship for it. https://www.fossexperts.com/ Frigging copycat. Hah! I gave you kudos :P (you are in the FAQ)

Re: [HACKERS] ALTER TABLE SET STATISTICS requires AccessExclusiveLock

2010-07-28 Thread Simon Riggs
On Wed, 2010-07-28 at 15:24 +0300, Peter Eisentraut wrote: On tor, 2010-07-15 at 10:24 +0100, Simon Riggs wrote: Patch to reduce lock levels for ALTER TABLE CREATE TRIGGER CREATE RULE Tried this out, but $subject is still the case. The problem is that ATRewriteCatalogs() calls

Re: [HACKERS] ERROR: argument to pg_get_expr() must come from system catalogs

2010-07-28 Thread Dave Page
On Wed, Jul 28, 2010 at 4:54 PM, Bruce Momjian br...@momjian.us wrote: Are we basically leaving pgAdmin in this state until we come up with a fix and need a new minor release?  We pride ourselves in not introducing breakage in minor releases, but it has certainly happened in this case, and it

Re: [HACKERS] page corruption on 8.3+ that makes it to standby

2010-07-28 Thread Jeff Davis
On Wed, 2010-07-28 at 06:40 -0400, Robert Haas wrote: fix1 -- Only call PageSetLSN/TLI inside log_newpage() and heap_xlog_newpage() if the page is not zeroed. fix2 -- Don't call log_newpage() at all if the page is not zeroed. Please review. I don't have a strong opinion about which one

Re: [HACKERS] Toward a column reorder solution

2010-07-28 Thread David E. Wheeler
On Jul 28, 2010, at 7:57 AM, Joshua D. Drake wrote: Hah! I gave you kudos :P (you are in the FAQ) Ah, thanks. The link is missing a G: It's PGXN, not PXN. Best, David -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription:

Re: [HACKERS] [JDBC] Trouble with COPY IN

2010-07-28 Thread James William Pye
On Jul 25, 2010, at 8:01 AM, Kris Jurka wrote: The JDBC driver reads server messages for multiple reasons. One of them is indeed to do early failure detection. That's high quality. =) Another is to pickup NoticeResponse messages to avoid a network buffer deadlock. That's a good catch. I

Re: [HACKERS] Toward a column reorder solution

2010-07-28 Thread Joshua D. Drake
On Wed, 2010-07-28 at 09:30 -0700, David E. Wheeler wrote: On Jul 28, 2010, at 7:57 AM, Joshua D. Drake wrote: Hah! I gave you kudos :P (you are in the FAQ) Ah, thanks. The link is missing a G: It's PGXN, not PXN. Yeah that is already fixed, just waiting for cache to clear (on the

Re: [HACKERS] page corruption on 8.3+ that makes it to standby

2010-07-28 Thread Tom Lane
Jeff Davis pg...@j-davis.com writes: However, when Simon said We definitely shouldn't do anything that leaves standby different to primary. you said obviously. Fix2 can leave a difference between the two, because zeroed pages at the end of the heap file on the primary will not be sent to the

Re: [HACKERS] [JDBC] Trouble with COPY IN

2010-07-28 Thread Kris Jurka
On Wed, 28 Jul 2010, James William Pye wrote: hrm, I suppose a lazy way around that problem would be to suspend all client messages(client_min_messages) during COPY IN. Tho, I guess one would still have to contend with NotificationResponse, and ParameterStatus.. Technically you won't

Re: [HACKERS] page corruption on 8.3+ that makes it to standby

2010-07-28 Thread Jeff Davis
On Wed, 2010-07-28 at 12:36 -0400, Tom Lane wrote: Jeff Davis pg...@j-davis.com writes: However, when Simon said We definitely shouldn't do anything that leaves standby different to primary. you said obviously. Fix2 can leave a difference between the two, because zeroed pages at the end of

Re: [HACKERS] page corruption on 8.3+ that makes it to standby

2010-07-28 Thread Robert Haas
On Wed, Jul 28, 2010 at 12:36 PM, Tom Lane t...@sss.pgh.pa.us wrote: Jeff Davis pg...@j-davis.com writes: However, when Simon said We definitely shouldn't do anything that leaves standby different to primary. you said obviously. Fix2 can leave a difference between the two, because zeroed pages

Re: [HACKERS] page corruption on 8.3+ that makes it to standby

2010-07-28 Thread Jeff Davis
On Wed, 2010-07-28 at 13:18 -0400, Robert Haas wrote: In Jeff's original example, he crashes the database after extending the relation but before initializing and writing the new page. I believe that at that point no XLOG has been written yet, so the relation has been extended but there is no

Re: [HACKERS] page corruption on 8.3+ that makes it to standby

2010-07-28 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Wed, Jul 28, 2010 at 12:36 PM, Tom Lane t...@sss.pgh.pa.us wrote: I understand it, and I don't like it one bit.  I haven't caught up on this thread yet, but I think the only acceptable solution is one that leaves the slave in the *same* state as the

Re: [HACKERS] page corruption on 8.3+ that makes it to standby

2010-07-28 Thread Robert Haas
On Wed, Jul 28, 2010 at 2:21 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Wed, Jul 28, 2010 at 12:36 PM, Tom Lane t...@sss.pgh.pa.us wrote: I understand it, and I don't like it one bit.  I haven't caught up on this thread yet, but I think the only

Re: [HACKERS] reducing NUMERIC size for 9.1

2010-07-28 Thread Tom Lane
[ gradually catching up on email ] Robert Haas robertmh...@gmail.com writes: On Fri, Jul 16, 2010 at 2:39 PM, Tom Lane t...@sss.pgh.pa.us wrote: I don't like the way you did that either (specifically, not the kluge in NUMERIC_DIGITS()).  It would probably work better if you declared two

Re: [HACKERS] page corruption on 8.3+ that makes it to standby

2010-07-28 Thread Jeff Davis
On Wed, 2010-07-28 at 14:50 -0400, Robert Haas wrote: It seems like if log_newpage() were to set the LSN/TLI before calling XLogInsert() - or optionally not - then it wouldn't be necessary to set them also in heap_xlog_newpage(); the memcpy operation would by definition have copied the right

Re: [HACKERS] page corruption on 8.3+ that makes it to standby

2010-07-28 Thread Robert Haas
On Wed, Jul 28, 2010 at 3:08 PM, Jeff Davis pg...@j-davis.com wrote: On Wed, 2010-07-28 at 14:50 -0400, Robert Haas wrote: It seems like if log_newpage() were to set the LSN/TLI before calling XLogInsert() - or optionally not - then it wouldn't be necessary to set them also in

Re: [HACKERS] page corruption on 8.3+ that makes it to standby

2010-07-28 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Wed, Jul 28, 2010 at 2:21 PM, Tom Lane t...@sss.pgh.pa.us wrote: I've caught up on the thread now, and I think that fix2 (skip logging the page) is extremely dangerous and has little if anything in its favor. Why do you think that? They will be

Re: [HACKERS] page corruption on 8.3+ that makes it to standby

2010-07-28 Thread Tom Lane
I wrote: I think it is appropriate to be setting the LSN/TLI in the case of a page that's been constructed by the caller as part of the WAL-logged action, but doing so in copy_relation_data seems rather questionable. BTW, I thought of an argument that explains why that's sane: it marks the

Re: [HACKERS] page corruption on 8.3+ that makes it to standby

2010-07-28 Thread Robert Haas
On Wed, Jul 28, 2010 at 3:16 PM, Tom Lane t...@sss.pgh.pa.us wrote: (1) You're assuming that the page will be zeroes on the slave without having forced it to be so.  A really obvious case where this fails is where we're doing crash-and-restart on the master: a later action could have modified

Re: [HACKERS] [JDBC] Trouble with COPY IN

2010-07-28 Thread James William Pye
On Jul 28, 2010, at 9:53 AM, Kris Jurka wrote: Technically you won't get NotificationResponse until transaction end, so you don't need to worry about that mid copy. Ah, thanks for noting that. It would appear my original reading of the async section didn't get far enough beyond Frontends must

Re: [HACKERS] page corruption on 8.3+ that makes it to standby

2010-07-28 Thread Jeff Davis
On Wed, 2010-07-28 at 15:37 -0400, Tom Lane wrote: So nevermind that distraction. I'm back to thinking that fix1 is the way to go. Agreed. It's uncontroversial to have a simple guard against corrupting an uninitialized page, and uncontroversial is good for things that will be back-patched.

Re: [HACKERS] Patch to show individual statement latencies in pgbench output

2010-07-28 Thread Greg Smith
Finally got around to taking a longer look at your patch, sorry about the delay here. Patch itself seems to work on simple tests anyway (more on the one suspect bit below). You didn't show what the output looks like, so let's start with that because it is both kind of neat and not what I

Re: [HACKERS] do we need to postpone beta4?

2010-07-28 Thread Robert Haas
On Tue, Jul 27, 2010 at 4:09 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: Well, that's pretty much saying we won't release before September. Yup, that's what I think.  In fact I think September might be optimistic.  This is what happens when you fork early

Re: [HACKERS] string_to_array has to be stable?

2010-07-28 Thread Jeff Davis
On Tue, 2010-07-20 at 11:31 +0200, Pavel Stehule wrote: Hello I am working on to_array, to_string functions and I am looking on string_to_array function. I am surprised so this function is marked as immutable postgres=# select array_to_string(array[current_date],','); array_to_string

Re: [HACKERS] documentation for committing with git

2010-07-28 Thread Daniel Farina
On Wed, Jul 21, 2010 at 9:22 AM, Robert Haas robertmh...@gmail.com wrote: On the other hand, if you have technical corrections, or if you have suggestions on how to do the same things better (rather than suggestions on what to do differently), that would be greatly appreciated. Somewhere in

Re: [HACKERS] [GENERAL] Incorrect FTS result with GIN index

2010-07-28 Thread Tom Lane
Oleg Bartunov o...@sai.msu.su writes: you can download dump http://mira.sai.msu.su/~megera/tmp/search_tab.dump Hmm ... I'm not sure why you're failing to reproduce it, because it's falling over pretty easily for me. After poking at it for awhile, I am of the opinion that scanGetItem's handling

Re: [HACKERS] do we need to postpone beta4?

2010-07-28 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: So now we have the following issues remaining: * page corruption after moving tablespace * ExplainOnePlan handles snapshots differently than ProcessQuery * name and comment of XLogSetAsyncCommitLSN() should be changed * Documentation fails to build

Re: [HACKERS] string_to_array has to be stable?

2010-07-28 Thread Tom Lane
Jeff Davis pg...@j-davis.com writes: On Tue, 2010-07-20 at 11:31 +0200, Pavel Stehule wrote: I am working on to_array, to_string functions and I am looking on string_to_array function. I am surprised so this function is marked as immutable What's wrong with that? current_date is the part

Re: [HACKERS] string_to_array has to be stable?

2010-07-28 Thread Jeff Davis
On Wed, 2010-07-28 at 20:25 -0400, Tom Lane wrote: string_to_array() seems fine to me: it's a predictable transformation from text to text. However, I think that there really is an issue with array_to_string(), because that takes an anyarray and invokes the array element's type output

Re: [HACKERS] Patch to show individual statement latencies in pgbench output

2010-07-28 Thread Florian Pflug
On Jul29, 2010, at 00:48 , Greg Smith wrote: Finally got around to taking a longer look at your patch, sorry about the delay here. Patch itself seems to work on simple tests anyway (more on the one suspect bit below). You didn't show what the output looks like, so let's start with that