Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2012-03-01 Thread Shigeru Hanada
(2012/03/01 0:33), Tom Lane wrote: I don't think that creating such a dependency is acceptable. Even if we didn't mind the dependency, you said yourself that contrib/postgresql_fdw's validator will accept stuff that's not appropriate for dblink. Agreed. I think that these two contrib modules

Re: [HACKERS] 16-bit page checksums for 9.2

2012-03-01 Thread Robert Haas
On Wed, Feb 29, 2012 at 5:52 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Wed, Feb 29, 2012 at 4:34 PM, Tom Lane t...@sss.pgh.pa.us wrote: Easier for who?  I don't care for the idea of code that has to cope with two page formats, or before long N page

Re: [HACKERS] pg_upgrade --logfile option documentation

2012-03-01 Thread Robert Haas
On Wed, Feb 29, 2012 at 6:02 PM, Bruce Momjian br...@momjian.us wrote: Any ideas about improving the error reporting more generally, so that when reloading the dump fails, the user can easily see what went belly-up, even if they didn't use -l? The only idea I have is to write the psql log to

Re: [HACKERS] Parameterized-path cost comparisons need some work

2012-03-01 Thread Robert Haas
On Wed, Feb 29, 2012 at 6:01 PM, Tom Lane t...@sss.pgh.pa.us wrote: Well, my evidence is that a parameterized path should pretty much always include a paramaterized path somewhere in there - otherwise, what is parameterization doing for us? Well, yes, we know that much. I didn't write what I

[HACKERS] performance results on IBM POWER7

2012-03-01 Thread Robert Haas
IBM has provided the PostgreSQL community with access to a couple of IBM POWER7 machines through OSUOSL. Simon has access to one, carved up into a couple of LPARs, for replication work, and there's a buildfarm animal on there as well, I think; I have access to the other, for performance testing.

Re: [HACKERS] Collect frequency statistics for arrays

2012-03-01 Thread Alexander Korotkov
On Thu, Mar 1, 2012 at 1:19 AM, Alexander Korotkov aekorot...@gmail.comwrote: On Thu, Mar 1, 2012 at 1:09 AM, Tom Lane t...@sss.pgh.pa.us wrote: That seems like a pretty narrow, uncommon use-case. Also, to get accurate stats for such queries that way, you'd need really enormous histograms.

Re: [HACKERS] Collect frequency statistics for arrays

2012-03-01 Thread Robert Haas
On Wed, Feb 29, 2012 at 5:43 PM, Tom Lane t...@sss.pgh.pa.us wrote: Nathan Boley npbo...@gmail.com writes: On Wed, Feb 29, 2012 at 12:39 PM, Tom Lane t...@sss.pgh.pa.us wrote: I am starting to look at this patch now.  I'm wondering exactly why the decision was made to continue storing

Re: [HACKERS] Collect frequency statistics for arrays

2012-03-01 Thread Alvaro Herrera
Excerpts from Robert Haas's message of jue mar 01 12:00:08 -0300 2012: On Wed, Feb 29, 2012 at 5:43 PM, Tom Lane t...@sss.pgh.pa.us wrote: No, just that we'd no longer have statistics relevant to that, and would have to fall back on default selectivity assumptions.  Do you think that such

Re: [HACKERS] Speed dblink using alternate libpq tuple storage

2012-03-01 Thread Marko Kreen
On Tue, Feb 28, 2012 at 05:04:44PM +0900, Kyotaro HORIGUCHI wrote: There is still one EOF in v3 getAnotherTuple() - pqGetInt(tupnfields), please turn that one also to protocolerror. pqGetInt() returns EOF only when it wants additional reading from network if the parameter `bytes' is

Re: [HACKERS] performance results on IBM POWER7

2012-03-01 Thread Ants Aasma
On Thu, Mar 1, 2012 at 4:54 PM, Robert Haas robertmh...@gmail.com wrote: ... After that I think maybe some testing of the remaining CommitFest patches might be in order (though personally I'd like to wrap this CommitFest up fairly soon) to see if any of those improve things. Besides

[HACKERS] Allowing multi -t and adding -n to vacuumdb ?

2012-03-01 Thread Jehan-Guillaume (ioguix) de Rorthais
Hi, One of our customer send us a patch he wrote for his needs (on src/bin/scripts/vacuumdb.c, no doc were included). He's using one schema per application and would like to be able to run vacuumdb on each of them independently so he added the --schema|-n option and send us the patch. Reviewing

Re: [HACKERS] performance results on IBM POWER7

2012-03-01 Thread Robert Haas
On Thu, Mar 1, 2012 at 11:23 AM, Ants Aasma ants.aa...@eesti.ee wrote: On Thu, Mar 1, 2012 at 4:54 PM, Robert Haas robertmh...@gmail.com wrote: ... After that I think maybe some testing of the remaining CommitFest patches might be in order (though personally I'd like to wrap this CommitFest up

Re: [HACKERS] COPY with hints, rebirth

2012-03-01 Thread Simon Riggs
On Sun, Feb 26, 2012 at 7:16 PM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: On 24.02.2012 22:55, Simon Riggs wrote: A long time ago, in a galaxy far away, we discussed ways to speed up data loads/COPY. http://archives.postgresql.org/pgsql-hackers/2007-01/msg00470.php In

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2012-03-01 Thread Peter Eisentraut
On tor, 2012-03-01 at 20:56 +0900, Shigeru Hanada wrote: How about moving postgresql_fdw_validator into dblink, That's probably a good move. If this were C++, we might try to subclass this whole thing a bit, to avoid code duplication, but I don't see an easy way to do that here. with

Re: [HACKERS] incompatible pointer types with newer zlib

2012-03-01 Thread Peter Eisentraut
On fre, 2012-02-24 at 11:10 -0500, Tom Lane wrote: Peter Eisentraut pete...@gmx.net writes: On tor, 2012-02-23 at 10:17 -0500, Tom Lane wrote: void * seems entirely reasonable given the two different usages, but I would be happier if the patch added explicit casts whereever FH is set to

Re: [HACKERS] 16-bit page checksums for 9.2

2012-03-01 Thread Josh Berkus
And how would a DBA know that? We'd add a column to pg_class that tracks which page version is in use for each relation. So a relation can't have some pages in Version 9.2, and other pages in version 9.3? How will this work for 2TB tables? -- Josh Berkus PostgreSQL Experts Inc.

Re: [HACKERS] 16-bit page checksums for 9.2

2012-03-01 Thread Robert Haas
On Thu, Mar 1, 2012 at 12:42 PM, Josh Berkus j...@agliodbs.com wrote: And how would a DBA know that? We'd add a column to pg_class that tracks which page version is in use for each relation. So a relation can't have some pages in Version 9.2, and other pages in version 9.3?  How will this

Re: [HACKERS] 16-bit page checksums for 9.2

2012-03-01 Thread Josh Berkus
So a relation can't have some pages in Version 9.2, and other pages in version 9.3? How will this work for 2TB tables? Not very well, but better than Tom's proposal to require upgrading the entire cluster in a single off-line operation. Yes, but the result will be that anyone with a 2TB

Re: [HACKERS] review: CHECK FUNCTION statement

2012-03-01 Thread Alvaro Herrera
Why does CollectCheckedFunctions skip trigger functions? My only guess is that at one point the checker was not supposed to know how to check them, and a later version learned about it and this bit wasn't updated; but maybe there's another reason? -- Álvaro Herrera alvhe...@commandprompt.com

Re: [HACKERS] pg_upgrade --logfile option documentation

2012-03-01 Thread Peter Eisentraut
On ons, 2012-02-29 at 22:47 -0500, Bruce Momjian wrote: Hey, that's a good idea. I would always write the pg_dump output to a log file. If the dump succeeds, I remove the file, if not, I tell users to read the log file for details about the failure --- good idea. But we also need the server

Re: [HACKERS] COPY with hints, rebirth

2012-03-01 Thread Heikki Linnakangas
On 01.03.2012 18:40, Simon Riggs wrote: On Sun, Feb 26, 2012 at 7:16 PM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: On 24.02.2012 22:55, Simon Riggs wrote: What exactly does it do? Previously, we optimised COPY when it was loading data into a newly created table or a

Re: [HACKERS] 16-bit page checksums for 9.2

2012-03-01 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: So a relation can't have some pages in Version 9.2, and other pages in version 9.3?  How will this work for 2TB tables? Not very well, but better than Tom's proposal to require upgrading the entire cluster in a single off-line operation. WTF? That

Re: [HACKERS] Re: pg_stat_statements normalisation without invasive changes to the parser (was: Next steps on pg_stat_statements normalisation)

2012-03-01 Thread Peter Geoghegan
On 1 March 2012 00:48, Alvaro Herrera alvhe...@commandprompt.com wrote: I'm curious about the LeafNode stuff.  Is this something that could be done by expression_tree_walker?  I'm not completely familiar with it so maybe there's some showstopper such as some node tags not being supported, or

Re: [HACKERS] Collect frequency statistics for arrays

2012-03-01 Thread Tom Lane
Alvaro Herrera alvhe...@commandprompt.com writes: Excerpts from Robert Haas's message of jue mar 01 12:00:08 -0300 2012: On Wed, Feb 29, 2012 at 5:43 PM, Tom Lane t...@sss.pgh.pa.us wrote: I confess I am nervous about ripping this out. I am pretty sure we will get complaints about it.

Re: [HACKERS] Re: pg_stat_statements normalisation without invasive changes to the parser (was: Next steps on pg_stat_statements normalisation)

2012-03-01 Thread Daniel Farina
On Thu, Mar 1, 2012 at 1:27 PM, Peter Geoghegan pe...@2ndquadrant.com wrote: My expectation is that this feature will make life a lot easier for a lot of Postgres users. Yes. It's hard to overstate the apparent utility of this feature in the general category of visibility and profiling. --

Re: [HACKERS] Re: pg_stat_statements normalisation without invasive changes to the parser (was: Next steps on pg_stat_statements normalisation)

2012-03-01 Thread Josh Berkus
On 3/1/12 1:57 PM, Daniel Farina wrote: On Thu, Mar 1, 2012 at 1:27 PM, Peter Geoghegan pe...@2ndquadrant.com wrote: My expectation is that this feature will make life a lot easier for a lot of Postgres users. Yes. It's hard to overstate the apparent utility of this feature in the general

Re: [HACKERS] Collect frequency statistics for arrays

2012-03-01 Thread Alvaro Herrera
Excerpts from Tom Lane's message of jue mar 01 18:51:38 -0300 2012: Alvaro Herrera alvhe...@commandprompt.com writes: Excerpts from Robert Haas's message of jue mar 01 12:00:08 -0300 2012: On Wed, Feb 29, 2012 at 5:43 PM, Tom Lane t...@sss.pgh.pa.us wrote: I confess I am nervous about

Re: [HACKERS] Allowing multi -t and adding -n to vacuumdb ?

2012-03-01 Thread Tom Lane
Jehan-Guillaume (ioguix) de Rorthais j...@dalibo.com writes: One of our customer send us a patch he wrote for his needs (on src/bin/scripts/vacuumdb.c, no doc were included). He's using one schema per application and would like to be able to run vacuumdb on each of them independently so he

Re: [HACKERS] Collect frequency statistics for arrays

2012-03-01 Thread Nathan Boley
What about MCV's? Will those be removed as well? Sure.  Those seem even less useful. Ya, this will destroy the performance of several queries without some heavy tweaking. Maybe this is bad design, but I've gotten in the habit of storing sequences as arrays and I commonly join on them. I

Re: [HACKERS] Collect frequency statistics for arrays

2012-03-01 Thread Tom Lane
Nathan Boley npbo...@gmail.com writes: Maybe this is bad design, but I've gotten in the habit of storing sequences as arrays and I commonly join on them. I looked through my code this morning, and I only have one 'range' query ( of the form described up-thread ), but there are tons of the form

Re: [HACKERS] Caching for stable expressions with constant arguments v6

2012-03-01 Thread Jaime Casanova
On Sat, Feb 4, 2012 at 5:40 AM, Marti Raudsepp ma...@juffo.org wrote: On Sat, Feb 4, 2012 at 09:49, Jaime Casanova ja...@2ndquadrant.com wrote: i little review... Thanks! By the way, you should update to the v7 patch. just tried it and it fail when initializing on make check creating

Re: [HACKERS] Allowing multi -t and adding -n to vacuumdb ?

2012-03-01 Thread Kevin Grittner
Tom Lane t...@sss.pgh.pa.us wrote: Why isn't your customer using autovacuum? If there are concrete reasons why that doesn't get the job done for him, it would be more useful in the long run to work on fixing that. FWIW, we're using autovacuum here, at slightly more aggressive settings from

Re: [HACKERS] Collect frequency statistics for arrays

2012-03-01 Thread Tom Lane
Alvaro Herrera alvhe...@commandprompt.com writes: Excerpts from Tom Lane's message of jue mar 01 18:51:38 -0300 2012: How would we make it optional? There's noplace I can think of to stick such a knob ... Uhm, attoptions? Oh, I had forgotten we had that mechanism already. Yeah, that might

Re: [HACKERS] Collect frequency statistics for arrays

2012-03-01 Thread Nathan Boley
[ sorry Tom, reply all this time... ] What do you mean by storing sequences as arrays? So, a simple example is, for transcripts ( sequences of DNA that are turned into proteins ), we store each of the connected components as an array of the form: exon_type in [1,6] splice_type = [1,3] and

Re: [HACKERS] 16-bit page checksums for 9.2

2012-03-01 Thread Robert Haas
On Thu, Mar 1, 2012 at 4:08 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: So a relation can't have some pages in Version 9.2, and other pages in version 9.3?  How will this work for 2TB tables? Not very well, but better than Tom's proposal to require

Re: [HACKERS] 16-bit page checksums for 9.2

2012-03-01 Thread Alvaro Herrera
Excerpts from Robert Haas's message of jue mar 01 21:23:06 -0300 2012: On Thu, Mar 1, 2012 at 4:08 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: So a relation can't have some pages in Version 9.2, and other pages in version 9.3?  How will this work for

Re: [HACKERS] review of: collation for (expr)

2012-03-01 Thread Jaime Casanova
On Thu, Jan 19, 2012 at 1:50 PM, Peter Eisentraut pete...@gmx.net wrote: On tor, 2012-01-12 at 21:25 -0800, probabble wrote: Compiling on Ubuntu 10.04 LTS AMD64 on a GoGrid virtual machine from 2012-01-12 checkout. Bison upgraded to v2.5, and downgraded to v2.4.1 Make process for both

Re: [HACKERS] 16-bit page checksums for 9.2

2012-03-01 Thread Tom Lane
Alvaro Herrera alvhe...@commandprompt.com writes: Excerpts from Robert Haas's message of jue mar 01 21:23:06 -0300 2012: and that, further, you were arguing that we should not support multiple page versions. I don't think we need to support multiple page versions, if multiple means 2.

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2012-03-01 Thread Shigeru Hanada
2012/3/2 Peter Eisentraut pete...@gmx.net: with renaming to dblink_fdw_validator? Well, it's not the validator of the dblink_fdw, so maybe something like basic_postgresql_fdw_validator. -1 for same reason. It's not the validator of basic_postgresql_fdw. Using fdw in the name of validator

Re: [HACKERS] 16-bit page checksums for 9.2

2012-03-01 Thread Robert Haas
On Thu, Mar 1, 2012 at 8:32 PM, Tom Lane t...@sss.pgh.pa.us wrote: Alvaro Herrera alvhe...@commandprompt.com writes: Excerpts from Robert Haas's message of jue mar 01 21:23:06 -0300 2012: and that, further, you were arguing that we should not support multiple page versions. I don't think we

Re: [HACKERS] pg_upgrade --logfile option documentation

2012-03-01 Thread Bruce Momjian
On Thu, Mar 01, 2012 at 08:45:26AM -0500, Robert Haas wrote: On Wed, Feb 29, 2012 at 6:02 PM, Bruce Momjian br...@momjian.us wrote: Any ideas about improving the error reporting more generally, so that when reloading the dump fails, the user can easily see what went belly-up, even if they

Re: [HACKERS] pg_upgrade --logfile option documentation

2012-03-01 Thread Bruce Momjian
On Thu, Mar 01, 2012 at 10:17:04PM +0200, Peter Eisentraut wrote: On ons, 2012-02-29 at 22:47 -0500, Bruce Momjian wrote: Hey, that's a good idea. I would always write the pg_dump output to a log file. If the dump succeeds, I remove the file, if not, I tell users to read the log file for

Re: [HACKERS] 16-bit page checksums for 9.2

2012-03-01 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: One thing I'm not too sure about is how to extend the page format to handle optional features. For example, suppose we want to add 2 bytes to the page header for a checksum (or 4 bytes, or any other number). Ideally, I'd like to not use up those 2

Re: [HACKERS] Re: pg_stat_statements normalisation without invasive changes to the parser (was: Next steps on pg_stat_statements normalisation)

2012-03-01 Thread Peter Geoghegan
On 1 March 2012 22:09, Josh Berkus j...@agliodbs.com wrote: On 3/1/12 1:57 PM, Daniel Farina wrote: On Thu, Mar 1, 2012 at 1:27 PM, Peter Geoghegan pe...@2ndquadrant.com wrote: My expectation is that this feature will make life a lot easier for a lot of Postgres users. Yes.  It's hard to

Re: [HACKERS] 16-bit page checksums for 9.2

2012-03-01 Thread Robert Haas
On Thu, Mar 1, 2012 at 9:11 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: One thing I'm not too sure about is how to extend the page format to handle optional features.  For example, suppose we want to add 2 bytes to the page header for a checksum (or 4

[HACKERS] Canceling ROLLBACK statement

2012-03-01 Thread Pavan Deolasee
Hi All, While working on something in XC, I hit upon an assertion failure. While this is in XC code, I believe there can be a way of reproducing this on vanilla PostgreSQL as well. I could not do so even after several tries, unless I add some code or run it through debugger. Here is the theory

Re: [HACKERS] Canceling ROLLBACK statement

2012-03-01 Thread Tom Lane
Pavan Deolasee pavan.deola...@gmail.com writes: If client is running a ROLLBACK statement and sends a statement cancel signal to the server and if the cancel signal gets processed after the transaction AbortTransaction() is completed, but before CleanupTransaction() is called, I think the