Re: [HACKERS] Canceling ROLLBACK statement

2012-03-01 Thread Tom Lane
Pavan Deolasee writes: > If client is running a ROLLBACK statement and sends a statement cancel > signal to the server and if the cancel signal gets processed after the > transaction AbortTransaction() is completed, but before > CleanupTransaction() is called, I think the server may try to ABORT t

[HACKERS] Canceling ROLLBACK statement

2012-03-01 Thread Pavan Deolasee
Hi All, While working on something in XC, I hit upon an assertion failure. While this is in XC code, I believe there can be a way of reproducing this on vanilla PostgreSQL as well. I could not do so even after several tries, unless I add some code or run it through debugger. Here is the theory any

Re: [HACKERS] 16-bit page checksums for 9.2

2012-03-01 Thread Robert Haas
On Thu, Mar 1, 2012 at 9:11 PM, Tom Lane wrote: > Robert Haas writes: >> One thing I'm not too sure about is how to extend the page format to >> handle optional features.  For example, suppose we want to add 2 bytes >> to the page header for a checksum (or 4 bytes, or any other number). >> Ideall

Re: [HACKERS] Re: pg_stat_statements normalisation without invasive changes to the parser (was: Next steps on pg_stat_statements normalisation)

2012-03-01 Thread Peter Geoghegan
On 1 March 2012 22:09, Josh Berkus wrote: > On 3/1/12 1:57 PM, Daniel Farina wrote: >> On Thu, Mar 1, 2012 at 1:27 PM, Peter Geoghegan >> wrote: >>> My expectation is that this feature will make life a lot >>> easier for a lot of Postgres users. >> >> Yes.  It's hard to overstate the apparent ut

Re: [HACKERS] 16-bit page checksums for 9.2

2012-03-01 Thread Tom Lane
Robert Haas writes: > One thing I'm not too sure about is how to extend the page format to > handle optional features. For example, suppose we want to add 2 bytes > to the page header for a checksum (or 4 bytes, or any other number). > Ideally, I'd like to not use up those 2 bytes on pages that a

Re: [HACKERS] pg_upgrade --logfile option documentation

2012-03-01 Thread Bruce Momjian
On Thu, Mar 01, 2012 at 10:17:04PM +0200, Peter Eisentraut wrote: > On ons, 2012-02-29 at 22:47 -0500, Bruce Momjian wrote: > > Hey, that's a good idea. I would always write the pg_dump output to a > > log file. If the dump succeeds, I remove the file, if not, I tell > > users to read the log fil

Re: [HACKERS] pg_upgrade --logfile option documentation

2012-03-01 Thread Bruce Momjian
On Thu, Mar 01, 2012 at 08:45:26AM -0500, Robert Haas wrote: > On Wed, Feb 29, 2012 at 6:02 PM, Bruce Momjian wrote: > >> Any ideas about improving the error reporting more generally, so that > >> when reloading the dump fails, the user can easily see what went > >> belly-up, even if they didn't u

Re: [HACKERS] 16-bit page checksums for 9.2

2012-03-01 Thread Robert Haas
On Thu, Mar 1, 2012 at 8:32 PM, Tom Lane wrote: > Alvaro Herrera writes: >> Excerpts from Robert Haas's message of jue mar 01 21:23:06 -0300 2012: >>> and that, further, you were arguing that we should not support >>> multiple page versions. > >> I don't think we need to support multiple page ver

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2012-03-01 Thread Shigeru Hanada
2012/3/2 Peter Eisentraut : >> with renaming to dblink_fdw_validator? > > Well, it's not the validator of the dblink_fdw, so maybe something like > basic_postgresql_fdw_validator. -1 for same reason. It's not the validator of basic_postgresql_fdw. Using "fdw" in the name of validator which does

Re: [HACKERS] 16-bit page checksums for 9.2

2012-03-01 Thread Tom Lane
Alvaro Herrera writes: > Excerpts from Robert Haas's message of jue mar 01 21:23:06 -0300 2012: >> and that, further, you were arguing that we should not support >> multiple page versions. > I don't think we need to support multiple page versions, if multiple > means > 2. That's exactly the poin

Re: [HACKERS] review of: collation for (expr)

2012-03-01 Thread Jaime Casanova
On Thu, Jan 19, 2012 at 1:50 PM, Peter Eisentraut wrote: > On tor, 2012-01-12 at 21:25 -0800, probabble wrote: >> Compiling on Ubuntu 10.04 LTS AMD64 on a GoGrid virtual machine from >> 2012-01-12 checkout. >> >> Bison upgraded to v2.5, and downgraded to v2.4.1 >> >> Make process for both versions

Re: [HACKERS] 16-bit page checksums for 9.2

2012-03-01 Thread Alvaro Herrera
Excerpts from Robert Haas's message of jue mar 01 21:23:06 -0300 2012: > On Thu, Mar 1, 2012 at 4:08 PM, Tom Lane wrote: > > Robert Haas writes: > >>> So a relation can't have some pages in Version 9.2, and other pages in > >>> version 9.3?  How will this work for 2TB tables? > > > >> Not very w

Re: [HACKERS] 16-bit page checksums for 9.2

2012-03-01 Thread Robert Haas
On Thu, Mar 1, 2012 at 4:08 PM, Tom Lane wrote: > Robert Haas writes: >>> So a relation can't have some pages in Version 9.2, and other pages in >>> version 9.3?  How will this work for 2TB tables? > >> Not very well, but better than Tom's proposal to require upgrading the >> entire cluster in a

Re: [HACKERS] Collect frequency statistics for arrays

2012-03-01 Thread Nathan Boley
[ sorry Tom, reply all this time... ] > What do you mean by "storing sequences as arrays"? So, a simple example is, for transcripts ( sequences of DNA that are turned into proteins ), we store each of the connected components as an array of the form: exon_type in [1,6] splice_type = [1,3] and t

Re: [HACKERS] Collect frequency statistics for arrays

2012-03-01 Thread Tom Lane
Alvaro Herrera writes: > Excerpts from Tom Lane's message of jue mar 01 18:51:38 -0300 2012: >> How would we make it optional? There's noplace I can think of to stick >> such a knob ... > Uhm, attoptions? Oh, I had forgotten we had that mechanism already. Yeah, that might work. I'm a bit temp

Re: [HACKERS] Allowing multi "-t" and adding "-n" to vacuumdb ?

2012-03-01 Thread Kevin Grittner
Tom Lane wrote: > Why isn't your customer using autovacuum? If there are concrete > reasons why that doesn't get the job done for him, it would be > more useful in the long run to work on fixing that. FWIW, we're using autovacuum here, at slightly more aggressive settings from the default, an

Re: [HACKERS] Caching for stable expressions with constant arguments v6

2012-03-01 Thread Jaime Casanova
On Sat, Feb 4, 2012 at 5:40 AM, Marti Raudsepp wrote: > On Sat, Feb 4, 2012 at 09:49, Jaime Casanova wrote: >> i little review... > > Thanks! By the way, you should update to the v7 patch. > just tried it and it fail when initializing on make check """ creating information schema ... TRAP: Faile

Re: [HACKERS] Collect frequency statistics for arrays

2012-03-01 Thread Tom Lane
Nathan Boley writes: > Maybe this is bad design, but I've gotten in the habit of storing > sequences as arrays and I commonly join on them. I looked through my > code this morning, and I only have one 'range' query ( of the form > described up-thread ), but there are tons of the form > SELECT att

Re: [HACKERS] Collect frequency statistics for arrays

2012-03-01 Thread Nathan Boley
>> What about MCV's? Will those be removed as well? > > Sure.  Those seem even less useful. Ya, this will destroy the performance of several queries without some heavy tweaking. Maybe this is bad design, but I've gotten in the habit of storing sequences as arrays and I commonly join on them. I lo

Re: [HACKERS] Allowing multi "-t" and adding "-n" to vacuumdb ?

2012-03-01 Thread Tom Lane
"Jehan-Guillaume (ioguix) de Rorthais" writes: > One of our customer send us a patch he wrote for his needs (on > "src/bin/scripts/vacuumdb.c", no doc were included). > He's using one schema per application and would like to be able to run > vacuumdb on each of them independently so he added the

Re: [HACKERS] Collect frequency statistics for arrays

2012-03-01 Thread Alvaro Herrera
Excerpts from Tom Lane's message of jue mar 01 18:51:38 -0300 2012: > > Alvaro Herrera writes: > > Excerpts from Robert Haas's message of jue mar 01 12:00:08 -0300 2012: > >> On Wed, Feb 29, 2012 at 5:43 PM, Tom Lane wrote: > >> I confess I am nervous about ripping this out. I am pretty sure w

Re: [HACKERS] Re: pg_stat_statements normalisation without invasive changes to the parser (was: Next steps on pg_stat_statements normalisation)

2012-03-01 Thread Josh Berkus
On 3/1/12 1:57 PM, Daniel Farina wrote: > On Thu, Mar 1, 2012 at 1:27 PM, Peter Geoghegan wrote: >> My expectation is that this feature will make life a lot >> easier for a lot of Postgres users. > > Yes. It's hard to overstate the apparent utility of this feature in > the general category of vi

Re: [HACKERS] Re: pg_stat_statements normalisation without invasive changes to the parser (was: Next steps on pg_stat_statements normalisation)

2012-03-01 Thread Daniel Farina
On Thu, Mar 1, 2012 at 1:27 PM, Peter Geoghegan wrote: > My expectation is that this feature will make life a lot > easier for a lot of Postgres users. Yes. It's hard to overstate the apparent utility of this feature in the general category of visibility and profiling. -- fdr -- Sent via pgs

Re: [HACKERS] Collect frequency statistics for arrays

2012-03-01 Thread Tom Lane
Alvaro Herrera writes: > Excerpts from Robert Haas's message of jue mar 01 12:00:08 -0300 2012: >> On Wed, Feb 29, 2012 at 5:43 PM, Tom Lane wrote: >> I confess I am nervous about ripping this out. I am pretty sure we >> will get complaints about it. Performance optimizations that benefit >> gr

Re: [HACKERS] Re: pg_stat_statements normalisation without invasive changes to the parser (was: Next steps on pg_stat_statements normalisation)

2012-03-01 Thread Peter Geoghegan
On 1 March 2012 00:48, Alvaro Herrera wrote: > I'm curious about the LeafNode stuff.  Is this something that could be > done by expression_tree_walker?  I'm not completely familiar with it so > maybe there's some showstopper such as some node tags not being > supported, or maybe it just doesn't he

Re: [HACKERS] 16-bit page checksums for 9.2

2012-03-01 Thread Tom Lane
Robert Haas writes: >> So a relation can't have some pages in Version 9.2, and other pages in >> version 9.3?  How will this work for 2TB tables? > Not very well, but better than Tom's proposal to require upgrading the > entire cluster in a single off-line operation. WTF? That was most certainl

Re: [HACKERS] COPY with hints, rebirth

2012-03-01 Thread Heikki Linnakangas
On 01.03.2012 18:40, Simon Riggs wrote: On Sun, Feb 26, 2012 at 7:16 PM, Heikki Linnakangas wrote: On 24.02.2012 22:55, Simon Riggs wrote: What exactly does it do? Previously, we optimised COPY when it was loading data into a newly created table or a freshly truncated table. This patch exten

Re: [HACKERS] pg_upgrade --logfile option documentation

2012-03-01 Thread Peter Eisentraut
On ons, 2012-02-29 at 22:47 -0500, Bruce Momjian wrote: > Hey, that's a good idea. I would always write the pg_dump output to a > log file. If the dump succeeds, I remove the file, if not, I tell > users to read the log file for details about the failure --- good > idea. But we also need the ser

Re: [HACKERS] review: CHECK FUNCTION statement

2012-03-01 Thread Alvaro Herrera
Why does CollectCheckedFunctions skip trigger functions? My only guess is that at one point the checker was not supposed to know how to check them, and a later version learned about it and this bit wasn't updated; but maybe there's another reason? -- Álvaro Herrera The PostgreSQL Company - Co

Re: [HACKERS] 16-bit page checksums for 9.2

2012-03-01 Thread Josh Berkus
>> So a relation can't have some pages in Version 9.2, and other pages in >> version 9.3? How will this work for 2TB tables? > > Not very well, but better than Tom's proposal to require upgrading the > entire cluster in a single off-line operation. Yes, but the result will be that anyone with a

Re: [HACKERS] 16-bit page checksums for 9.2

2012-03-01 Thread Robert Haas
On Thu, Mar 1, 2012 at 12:42 PM, Josh Berkus wrote: >>> And how would a DBA know that? >> >> We'd add a column to pg_class that tracks which page version is in use >> for each relation. > > So a relation can't have some pages in Version 9.2, and other pages in > version 9.3?  How will this work fo

Re: [HACKERS] 16-bit page checksums for 9.2

2012-03-01 Thread Josh Berkus
>> And how would a DBA know that? > > We'd add a column to pg_class that tracks which page version is in use > for each relation. So a relation can't have some pages in Version 9.2, and other pages in version 9.3? How will this work for 2TB tables? -- Josh Berkus PostgreSQL Experts Inc. http:

Re: [HACKERS] incompatible pointer types with newer zlib

2012-03-01 Thread Peter Eisentraut
On fre, 2012-02-24 at 11:10 -0500, Tom Lane wrote: > Peter Eisentraut writes: > > On tor, 2012-02-23 at 10:17 -0500, Tom Lane wrote: > >> void * seems entirely reasonable given the two different usages, but > >> I would be happier if the patch added explicit casts whereever FH is > >> set to or us

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2012-03-01 Thread Peter Eisentraut
On tor, 2012-03-01 at 20:56 +0900, Shigeru Hanada wrote: > How about moving postgresql_fdw_validator into dblink, That's probably a good move. If this were C++, we might try to subclass this whole thing a bit, to avoid code duplication, but I don't see an easy way to do that here. > with renami

Re: [HACKERS] COPY with hints, rebirth

2012-03-01 Thread Simon Riggs
On Sun, Feb 26, 2012 at 7:16 PM, Heikki Linnakangas wrote: > On 24.02.2012 22:55, Simon Riggs wrote: >> >> A long time ago, in a galaxy far away, we discussed ways to speed up >> data loads/COPY. >> http://archives.postgresql.org/pgsql-hackers/2007-01/msg00470.php >> >> In particular, the idea tha

Re: [HACKERS] performance results on IBM POWER7

2012-03-01 Thread Robert Haas
On Thu, Mar 1, 2012 at 11:23 AM, Ants Aasma wrote: > On Thu, Mar 1, 2012 at 4:54 PM, Robert Haas wrote: >> ... After that I think maybe some testing of the >> remaining CommitFest patches might be in order (though personally I'd >> like to wrap this CommitFest up fairly soon) to see if any of tho

[HACKERS] Allowing multi "-t" and adding "-n" to vacuumdb ?

2012-03-01 Thread Jehan-Guillaume (ioguix) de Rorthais
Hi, One of our customer send us a patch he wrote for his needs (on "src/bin/scripts/vacuumdb.c", no doc were included). He's using one schema per application and would like to be able to run vacuumdb on each of them independently so he added the "--schema|-n" option and send us the patch. Review

Re: [HACKERS] performance results on IBM POWER7

2012-03-01 Thread Ants Aasma
On Thu, Mar 1, 2012 at 4:54 PM, Robert Haas wrote: > ... After that I think maybe some testing of the > remaining CommitFest patches might be in order (though personally I'd > like to wrap this CommitFest up fairly soon) to see if any of those > improve things. Besides performance testing, could

Re: [HACKERS] Speed dblink using alternate libpq tuple storage

2012-03-01 Thread Marko Kreen
On Tue, Feb 28, 2012 at 05:04:44PM +0900, Kyotaro HORIGUCHI wrote: > > There is still one EOF in v3 getAnotherTuple() - > > pqGetInt(tupnfields), please turn that one also to > > protocolerror. > > pqGetInt() returns EOF only when it wants additional reading from > network if the parameter `bytes'

Re: [HACKERS] Collect frequency statistics for arrays

2012-03-01 Thread Alvaro Herrera
Excerpts from Robert Haas's message of jue mar 01 12:00:08 -0300 2012: > On Wed, Feb 29, 2012 at 5:43 PM, Tom Lane wrote: > > No, just that we'd no longer have statistics relevant to that, and would > > have to fall back on default selectivity assumptions.  Do you think that > > such application

Re: [HACKERS] Collect frequency statistics for arrays

2012-03-01 Thread Robert Haas
On Wed, Feb 29, 2012 at 5:43 PM, Tom Lane wrote: > Nathan Boley writes: >> On Wed, Feb 29, 2012 at 12:39 PM, Tom Lane wrote: >>> I am starting to look at this patch now.  I'm wondering exactly why the >>> decision was made to continue storing btree-style statistics for arrays, >>> in addition to

Re: [HACKERS] Collect frequency statistics for arrays

2012-03-01 Thread Alexander Korotkov
On Thu, Mar 1, 2012 at 1:19 AM, Alexander Korotkov wrote: > On Thu, Mar 1, 2012 at 1:09 AM, Tom Lane wrote: > >> That seems like a pretty narrow, uncommon use-case. Also, to get >> accurate stats for such queries that way, you'd need really enormous >> histograms. I doubt that the existing para

[HACKERS] performance results on IBM POWER7

2012-03-01 Thread Robert Haas
IBM has provided the PostgreSQL community with access to a couple of IBM POWER7 machines through OSUOSL. Simon has access to one, carved up into a couple of LPARs, for replication work, and there's a buildfarm animal on there as well, I think; I have access to the other, for performance testing.

Re: [HACKERS] Parameterized-path cost comparisons need some work

2012-03-01 Thread Robert Haas
On Wed, Feb 29, 2012 at 6:01 PM, Tom Lane wrote: >> Well, my "evidence" is that a parameterized path should pretty much >> always include a paramaterized path somewhere in there - otherwise, >> what is parameterization doing for us? > > Well, yes, we know that much. I didn't write what I meant to

Re: [HACKERS] pg_upgrade --logfile option documentation

2012-03-01 Thread Robert Haas
On Wed, Feb 29, 2012 at 6:02 PM, Bruce Momjian wrote: >> Any ideas about improving the error reporting more generally, so that >> when reloading the dump fails, the user can easily see what went >> belly-up, even if they didn't use -l? > > The only idea I have is to write the psql log to a tempora

Re: [HACKERS] 16-bit page checksums for 9.2

2012-03-01 Thread Robert Haas
On Wed, Feb 29, 2012 at 5:52 PM, Tom Lane wrote: > Robert Haas writes: >> On Wed, Feb 29, 2012 at 4:34 PM, Tom Lane wrote: >>> Easier for who?  I don't care for the idea of code that has to cope with >>> two page formats, or before long N page formats, because if we don't >>> have some mechanism

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2012-03-01 Thread Shigeru Hanada
(2012/03/01 0:33), Tom Lane wrote: > I don't think that creating such a dependency is acceptable. > Even if we didn't mind the dependency, you said yourself that > contrib/postgresql_fdw's validator will accept stuff that's not > appropriate for dblink. Agreed. I think that these two contrib modu