Re: [HACKERS] pg_dump --serializable-deferrable

2011-05-10 Thread Kevin Grittner
Peter Eisentraut wrote: > The name of this new option is a bit of a mouthful, and it mixes in > an otherwise standardized term (deferrable, as in constraints) with > transaction isolation. Wouldn't something like > --wait-for-serializable be clearer (and shorter)? I see it's not mentioned in t

Re: [HACKERS] time-delayed standbys

2011-05-10 Thread Fujii Masao
On Sat, May 7, 2011 at 10:48 PM, Robert Haas wrote: > I was able to reproduce something very like this in unpatched master, > just by letting recovery pause at a named restore point, and then > resuming it. I was able to reproduce the same problem even in 9.0. When the standby reaches the recover

Re: [HACKERS] the big picture for index-only scans

2011-05-10 Thread Jesper Krogh
On 2011-05-11 01:54, Greg Stark wrote: To be fair about 3/4 of them were actually complaining about the lack of some global materialized cache of the aggregate value. Covering index-only scans are only going to be a linear speedup no matter how large the factor it's not going to turn select count

Re: [HACKERS] PGC_S_DEFAULT is inadequate

2011-05-10 Thread Peter Eisentraut
On tis, 2011-05-10 at 22:20 -0400, Tom Lane wrote: > My conclusion about all this is that we really need to invent another > GucSource value falling between PGC_S_DEFAULT and PGC_S_ENV_VAR, > called perhaps PGC_S_DYNAMIC_DEFAULT, for the purpose of denoting > values that are defaults in terms of th

[HACKERS] pg_dump --serializable-deferrable

2011-05-10 Thread Peter Eisentraut
The name of this new option is a bit of a mouthful, and it mixes in an otherwise standardized term (deferrable, as in constraints) with transaction isolation. Wouldn't something like --wait-for-serializable be clearer (and shorter)? -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgres

Re: [HACKERS] Windows build issues

2011-05-10 Thread Brar Piening
On Wed, 11 May 2011 06:15:08 +0200, Brar Piening wrote: I've updated the patch once again to reflect the fixes to pgbison.bat in my alternative pgbison.pl Actually not pgbison but pgflex -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscrip

Re: [HACKERS] Windows build issues

2011-05-10 Thread Brar Piening
On Wed, 06 Apr 2011 22:07:52 +0200, Brar Piening wrote: On Wed, 06 Apr 2011 20:04:37 +0200, Brar Piening wrote: It's not ready yet but I'm prepared to get back to it as soon as there's some serious interest. I've updated the patch once again to reflect the fixes to pgbison.bat in my alt

Re: [HACKERS] the big picture for index-only scans

2011-05-10 Thread Bruce Momjian
Tom Lane wrote: > Bruce Momjian writes: > > Greg Stark wrote: > >> Putting aside the politics questions, count(*) is an interesting case > >> -- it exposes some of the unanswered questions about index-only scans. > >> > >> The reason "select count(*)" might win would be because we could pick > >>

Re: [HACKERS] VARIANT / ANYTYPE datatype

2011-05-10 Thread Pavel Stehule
2011/5/11 Joseph Adams : > On Tue, May 10, 2011 at 10:29 PM, Joseph Adams > wrote: >> The VARIANT type, or similar, would be useful for the JSON data type >> I've been intermittently working on, as it would allow us to create a >> function like this: >> >>    from_json(JSON) returns VARIANT > > Th

Re: [HACKERS] VARIANT / ANYTYPE datatype

2011-05-10 Thread Joseph Adams
On Tue, May 10, 2011 at 10:29 PM, Joseph Adams wrote: > The VARIANT type, or similar, would be useful for the JSON data type > I've been intermittently working on, as it would allow us to create a > function like this: > >    from_json(JSON) returns VARIANT This occurred to me: if PostgreSQL func

Re: [HACKERS] VARIANT / ANYTYPE datatype

2011-05-10 Thread Joseph Adams
It seems to me a reasonable way to implement VARIANT would be to have a data type called VARIANT that stores an OID of the inner type at the beginning, followed by the binary data. When you say pg_typeof(somevariant), you'll get 'variant'. Instead, you'd use a function like this: variant_typ

Re: [HACKERS] the big picture for index-only scans

2011-05-10 Thread Tom Lane
Bruce Momjian writes: > Greg Stark wrote: >> Putting aside the politics questions, count(*) is an interesting case >> -- it exposes some of the unanswered questions about index-only scans. >> >> The reason "select count(*)" might win would be because we could pick >> any index and do an index sca

[HACKERS] PGC_S_DEFAULT is inadequate

2011-05-10 Thread Tom Lane
I believe I've sussed the reason for the recent reports of Windows builds crashing when asked to process 'infinity'::timestamp. It's a bit tedious, so bear with me: 1. The immediate cause is that datebsearch() is being called with a NULL pointer and zero count, ie, the powerup default values of t

Re: [HACKERS] the big picture for index-only scans

2011-05-10 Thread Bruce Momjian
Greg Stark wrote: > On Wed, May 11, 2011 at 1:47 AM, Bruce Momjian wrote: > > Isn't speeding up COUNT(*) a sufficient case because it will not have to > > touch the heap in many cases? > > Putting aside the politics questions, count(*) is an interesting case > -- it exposes some of the unanswered

Re: [HACKERS] the big picture for index-only scans

2011-05-10 Thread Bruce Momjian
Robert Haas wrote: > So, what do we need in order to find our way to index-only scans? > > 1. The visibility map needs to be crash-safe. The basic idea of > index-only scans is that, instead of checking the heap to find out > whether each tuple is visible, we first check the visibility map. If >

Re: [HACKERS] the big picture for index-only scans

2011-05-10 Thread Bruce Momjian
Greg Stark wrote: > On Wed, May 11, 2011 at 1:47 AM, Bruce Momjian wrote: > > Isn't speeding up COUNT(*) a sufficient case because it will not have to > > touch the heap in many cases? > > Putting aside the politics questions, count(*) is an interesting case > -- it exposes some of the unanswered

Re: [HACKERS] Why not install pgstattuple by default?

2011-05-10 Thread Bruce Momjian
Tom Lane wrote: > Bruce Momjian writes: > > Tom Lane wrote: > >> here are the sizes of the built RPMs from my last build for Fedora: > >> > >> -rw-r--r--. 1 tgl tgl 3839458 Apr 18 10:50 > >> postgresql-9.0.4-1.fc13.x86_64.rpm > >> -rw-r--r--. 1 tgl tgl 490788 Apr 18 10:50 > >> postgresql-con

Re: [HACKERS] the big picture for index-only scans

2011-05-10 Thread Greg Stark
On Wed, May 11, 2011 at 1:47 AM, Bruce Momjian wrote: > Isn't speeding up COUNT(*) a sufficient case because it will not have to > touch the heap in many cases? Putting aside the politics questions, count(*) is an interesting case -- it exposes some of the unanswered questions about index-only sc

Re: [HACKERS] the big picture for index-only scans

2011-05-10 Thread Bruce Momjian
Greg Stark wrote: > On a separate note though, Simon, I don't know what you mean by "we > normally start with a problem". It's an free software project and > people are free to work on whatever interests them whether that's > because it solves a problem they have, helps a client who's paying > them

Re: [HACKERS] the big picture for index-only scans

2011-05-10 Thread Bruce Momjian
Robert Haas wrote: > >> Any thoughts welcome. ?Incidentally, if anyone else feels like working > >> on this, feel free to let me know and I'm happy to step away, from all > >> of it or from whatever part someone else wants to tackle. ?I'm mostly > >> working on this because it's something that I th

Re: [HACKERS] postgresql.conf error checking strategy

2011-05-10 Thread Bruce Momjian
Robert Haas wrote: > On Mon, May 9, 2011 at 11:10 AM, Tom Lane wrote: > > Robert Haas writes: > >> On Sun, May 8, 2011 at 1:04 AM, Tom Lane wrote: > >>> Yes, definitely. ?Perhaps summarize as "rethink how we handle partially > >>> correct postgresql.conf files". ?Or maybe Robert sees it as "reth

Re: [HACKERS] the big picture for index-only scans

2011-05-10 Thread Greg Stark
On Wed, May 11, 2011 at 12:14 AM, Kevin Grittner wrote: > The problem is that there are regular and fairly frequent complaints > on the list about queries which run slower than people expect > To be fair about 3/4 of them were actually complaining about the lack of some global materialized cache

Re: [HACKERS] Re: [BUGS] BUG #5957: createdb with description and md5 auth forces to provide password twice

2011-05-10 Thread Bruce Momjian
Bruce Momjian wrote: > Tom Lane wrote: > > Bruce Momjian writes: > > > Prior to PG 8.2, this was necessary to put the comment on the database, > > > but now that we have the shared comment/description table > > > pg_shdescription, this is not necessary. > > > > > Do we need createdb to be able to

Re: [HACKERS] the big picture for index-only scans

2011-05-10 Thread Kevin Grittner
Simon Riggs wrote: > Normally, others come forward with the why? when? questions and it > feels like there's a bit of groupthink going on here. This looks > to me like its being approached like it was a feature, but it > looks to me like a possible optimisation, so suggest we treat it > that way

Re: [HACKERS] Infinity bsearch crash on Windows

2011-05-10 Thread Tom Lane
Greg Smith writes: > A 9.1Beta1 test report from Richard Broersma (and confirmed on another > system by Mark Watson) showed up pgsql-testers this week at > http://archives.postgresql.org/pgsql-testers/2011-05/msg0.php with > the following test crashing his Windows server every time: > SELE

Re: [HACKERS] Feature proposal: distinguish each PostgreSQL instance in the event log

2011-05-10 Thread Tom Lane
"MauMau" writes: >> "MauMau" writes: >>> I've encountered one problem on Windows. I need to support running all of >>> my >>> products on one host simultaneously. Plus, I need to log messages in >>> syslog/event log. On Linux, I can distinguish the messages of one product >>> and those of other

Re: [HACKERS] Infinity bsearch crash on Windows

2011-05-10 Thread Peter Geoghegan
On 10 May 2011 23:02, Greg Smith wrote: > Why crash there only on Windows?  Was the problem actually introduced above > this part of the code?  These are all questions I have no answer for. I don't find it at all surprising that there's a memory corruption bug that only manifests itself on Window

Re: [HACKERS] Feature proposal: distinguish each PostgreSQL instance in the event log

2011-05-10 Thread MauMau
From: "Tom Lane" "MauMau" writes: I've encountered one problem on Windows. I need to support running all of my products on one host simultaneously. Plus, I need to log messages in syslog/event log. On Linux, I can distinguish the messages of one product and those of other products by setting

Re: [HACKERS] the big picture for index-only scans

2011-05-10 Thread Simon Riggs
On Tue, May 10, 2011 at 8:35 PM, Kevin Grittner wrote: > Simon Riggs wrote: >> Kevin Grittner wrote: >> ... but I share Simon's desire to see some proof before anything gets committed. >>> >>> And we agree there.  In fact, I can't think of anyone in the >>> community who doesn't want t

Re: [HACKERS] VARIANT / ANYTYPE datatype

2011-05-10 Thread Eric McKeeth
On Tue, May 10, 2011 at 3:57 PM, Simon Riggs wrote: > On Wed, May 4, 2011 at 5:58 PM, Alvaro Herrera > wrote: > > > A customer came to us with this request: a way to store "any" data in a > > column. We've gone back and forth trying to determine reasonable > > implementation restrictions, safet

Re: [HACKERS] Backpatching of "Teach the regular expression functions to do case-insensitive matching"

2011-05-10 Thread Tom Lane
Peter Eisentraut writes: > I'm all for more test suites, but we should make them as widely > accessible and accessed as possible so that they get maintained. Yeah. My preference would really be to push something like collate.linux.utf8 into the standard regression tests, but we'd first have to g

[HACKERS] Infinity bsearch crash on Windows

2011-05-10 Thread Greg Smith
A 9.1Beta1 test report from Richard Broersma (and confirmed on another system by Mark Watson) showed up pgsql-testers this week at http://archives.postgresql.org/pgsql-testers/2011-05/msg0.php with the following test crashing his Windows server every time: SELECT 'INFINITY'::TIMESTAMP; Th

Re: [HACKERS] VARIANT / ANYTYPE datatype

2011-05-10 Thread Simon Riggs
On Wed, May 4, 2011 at 5:58 PM, Alvaro Herrera wrote: > A customer came to us with this request: a way to store "any" data in a > column.  We've gone back and forth trying to determine reasonable > implementation restrictions, safety and useful semantics for them. > I note that this has been requ

Re: [HACKERS] Why not install pgstattuple by default?

2011-05-10 Thread Tom Lane
Bruce Momjian writes: > Tom Lane wrote: >> here are the sizes of the built RPMs from my last build for Fedora: >> >> -rw-r--r--. 1 tgl tgl 3839458 Apr 18 10:50 >> postgresql-9.0.4-1.fc13.x86_64.rpm >> -rw-r--r--. 1 tgl tgl 490788 Apr 18 10:50 >> postgresql-contrib-9.0.4-1.fc13.x86_64.rpm >>

Re: [HACKERS] Backpatching of "Teach the regular expression functions to do case-insensitive matching"

2011-05-10 Thread Peter Eisentraut
On tis, 2011-05-10 at 15:48 -0400, Robert Haas wrote: > On Tue, May 10, 2011 at 3:09 PM, Peter Eisentraut wrote: > > Well, the result of "people don't always run them" is the rest of > > src/test/. How much of that stuff even works anymore? > > I don't know. But I'm not sure I see your point.

Re: [HACKERS] Backpatching of "Teach the regular expression functions to do case-insensitive matching"

2011-05-10 Thread Peter Eisentraut
On tis, 2011-05-10 at 15:17 -0400, Tom Lane wrote: > Well, that would be great, but the "someone" is not going to be me; > I don't do Windows. Yeah, me neither. At least not for this release. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscrip

Re: [HACKERS] Why not install pgstattuple by default?

2011-05-10 Thread Bruce Momjian
Tom Lane wrote: > Christopher Browne writes: > > But people are evidently still setting packaging policies based on how > > things were back in 7.3, even though that perhaps isn't necessary > > anymore. > > FWIW, once you get past the client versus server distinction, I think > most subpackaging

Re: [HACKERS] VARIANT / ANYTYPE datatype

2011-05-10 Thread Darren Duncan
Alvaro Herrera wrote: Excerpts from Bruce Momjian's message of mar may 10 16:21:36 -0400 2011: Darren Duncan wrote: To follow-up, an additional feature that would be useful and resembles union types is the variant where you could declare a union type first and then separately other types could

Re: [HACKERS] VARIANT / ANYTYPE datatype

2011-05-10 Thread Alvaro Herrera
Excerpts from Bruce Momjian's message of mar may 10 16:21:36 -0400 2011: > Darren Duncan wrote: > > To follow-up, an additional feature that would be useful and resembles > > union > > types is the variant where you could declare a union type first and then > > separately other types could decla

Re: [HACKERS] VARIANT / ANYTYPE datatype

2011-05-10 Thread Bruce Momjian
Darren Duncan wrote: > To follow-up, an additional feature that would be useful and resembles union > types is the variant where you could declare a union type first and then > separately other types could declare they are a member of the union. I'm > talking about loosely what mixins or type-r

Re: [HACKERS] Backpatching of "Teach the regular expression functions to do case-insensitive matching"

2011-05-10 Thread Robert Haas
On Tue, May 10, 2011 at 3:09 PM, Peter Eisentraut wrote: > On mån, 2011-05-09 at 10:56 -0400, Robert Haas wrote: >> I'm just shooting from the hip here, but maybe we could have a >> separate (probably smaller) set of tests that are only designed to >> work in a limited range of locales and/or enco

Re: [HACKERS] Formatting Curmudgeons WAS: MMAP Buffers

2011-05-10 Thread Robert Haas
On Tue, May 10, 2011 at 3:09 PM, Greg Stark wrote: > The thing is, I think things are much better now than they were three > or four years ago. At the time the project had grown much faster than > the existing stable of developers and the rate at which patches were > being submitted was much great

Re: [HACKERS] the big picture for index-only scans

2011-05-10 Thread Kevin Grittner
Simon Riggs wrote: > Kevin Grittner wrote: > >>> ... but I share Simon's desire to see some proof before anything >>> gets committed. >> >> And we agree there. In fact, I can't think of anyone in the >> community who doesn't want to see that for *any* purported >> performance enhancement. > >

Re: [HACKERS] Backpatching of "Teach the regular expression functions to do case-insensitive matching"

2011-05-10 Thread Tom Lane
Peter Eisentraut writes: > On mån, 2011-05-09 at 12:42 -0400, Tom Lane wrote: >> The problem we'd have is that there's no way (at present) to make such >> a test pass on every platform. Windows has its own set of locale names >> (which initdb fails to install as collations anyway) and we also ha

Re: [HACKERS] Backpatching of "Teach the regular expression functions to do case-insensitive matching"

2011-05-10 Thread Peter Eisentraut
On mån, 2011-05-09 at 12:42 -0400, Tom Lane wrote: > The problem we'd have is that there's no way (at present) to make such > a test pass on every platform. Windows has its own set of locale names > (which initdb fails to install as collations anyway) and we also have > the problem that OS X can b

Re: [HACKERS] Formatting Curmudgeons WAS: MMAP Buffers

2011-05-10 Thread Josh Berkus
> The thing is, I think things are much better now than they were three > or four years ago. Oh, no question. If you read above in this thread, I'm not really proposing a change in the current process, just documenting the current process. Right now there's a gap between how sumbitters expect t

Re: [HACKERS] Backpatching of "Teach the regular expression functions to do case-insensitive matching"

2011-05-10 Thread Peter Eisentraut
On mån, 2011-05-09 at 10:56 -0400, Robert Haas wrote: > I'm just shooting from the hip here, but maybe we could have a > separate (probably smaller) set of tests that are only designed to > work in a limited range of locales and/or encodings. I'm really > pleased that we now have the src/test/isol

Re: [HACKERS] Formatting Curmudgeons WAS: MMAP Buffers

2011-05-10 Thread Greg Stark
On Tue, May 10, 2011 at 6:54 PM, Josh Berkus wrote: > Of course, there are always idiots who won't learn anything no matter > how good our process is.  But if the whole submission process is > perceived to be fair and understandible, those will be a tiny minority. The thing is, I think things are

Re: [HACKERS] the big picture for index-only scans

2011-05-10 Thread Simon Riggs
On Tue, May 10, 2011 at 6:25 PM, Kevin Grittner wrote: >> ... but I share Simon's desire to see some proof before anything >> gets committed. > > And we agree there.  In fact, I can't think of anyone in the > community who doesn't want to see that for *any* purported > performance enhancement. I

Re: [HACKERS] 4.1beta1: ANYARRAY disallowed for DOMAIN types which happen to be arrays

2011-05-10 Thread Tom Lane
Robert Haas writes: > On Tue, May 10, 2011 at 1:52 PM, Tom Lane wrote: >> Robert Haas writes: >>> On Mon, May 9, 2011 at 11:32 PM, Tom Lane wrote: So we basically had three alternatives to make it better: * downcast to the array type, which would possibly silently

Re: [HACKERS] the big picture for index-only scans

2011-05-10 Thread Greg Stark
On Tue, May 10, 2011 at 5:53 PM, Tom Lane wrote: > It's already the case that we'll flip over to a bitmap indexscan, > and thus get rid of most/all of the "random" page accesses, in > situations where this is likely to be a big win.  Pointing to the > performance difference in databases that don't

Re: [HACKERS] Server Programming Interface underspecified in 4.1beta1

2011-05-10 Thread Kevin Grittner
"J. Greg Davidson" wrote: > I would like to be able to program to a C or C++ SPI > which is clean, complete and type-safe. I am good at > reading API documentation in C or C++ and would be happy > to review any proposed improvements. I want to second Andrew's post, and emphasize that such sug

Re: [HACKERS] Collation mega-cleanups

2011-05-10 Thread Tom Lane
"Ross J. Reedstrom" writes: > So perhaps it was more of the "This code is less ready than I thought > it was, but now that I've spent the time understanding it and the > problem, the shortest way out is forward". Yeah, exactly. By the time I really understood how incomplete the collation patch w

Re: [HACKERS] Process wakeups when idle and power consumption

2011-05-10 Thread Peter Geoghegan
On 10 May 2011 17:43, Heikki Linnakangas wrote: > It should be an anonymous pipe that's inherited by the child process by > rather than a named pipe. Otherwise seems fine to me, as far as this proof > of concept program goes. Alright, thanks. I'll use an anonymous pipe in the patch itself. --

Re: [HACKERS] collateral benefits of a crash-safe visibility map

2011-05-10 Thread Simon Riggs
On Tue, May 10, 2011 at 6:08 PM, Tom Lane wrote: > Robert Haas writes: >> On Tue, May 10, 2011 at 12:57 PM, Simon Riggs wrote: >>> Hmmm, do we really need to WAL log freezing? > >> That might solve the relfrozenxid problem - set the bits in the heap, >> sync the heap, then update relfrozenxid on

Re: [HACKERS] Formatting Curmudgeons WAS: MMAP Buffers

2011-05-10 Thread Josh Berkus
All, > Part of the trouble is in the question. Having a patch rejected is not > really a problem; it's something you should learn from. I know it can be > annoying. I get annoyed when it happens to me too. But I try to get over > it as quickly as possible, and either fix the patch, or find another

Re: [HACKERS] 4.1beta1: ANYARRAY disallowed for DOMAIN types which happen to be arrays

2011-05-10 Thread Robert Haas
On Tue, May 10, 2011 at 1:52 PM, Tom Lane wrote: > Robert Haas writes: >> On Mon, May 9, 2011 at 11:32 PM, Tom Lane wrote: >>> So we basically had three alternatives to make it better: >>>        * downcast to the array type, which would possibly silently >>>          break applications that wer

Re: [HACKERS] 4.1beta1: ANYARRAY disallowed for DOMAIN types which happen to be arrays

2011-05-10 Thread Tom Lane
Robert Haas writes: > On Mon, May 9, 2011 at 11:32 PM, Tom Lane wrote: >> So we basically had three alternatives to make it better: >>* downcast to the array type, which would possibly silently >> break applications that were relying on the function result >> being consi

Re: [HACKERS] collateral benefits of a crash-safe visibility map

2011-05-10 Thread Robert Haas
On Tue, May 10, 2011 at 1:49 PM, Simon Riggs wrote: > On Tue, May 10, 2011 at 6:02 PM, Robert Haas wrote: >> On Tue, May 10, 2011 at 12:57 PM, Simon Riggs wrote: >>> Hmmm, do we really need to WAL log freezing? >>> >>> Can we break down freezing into a 2 stage process, so that we can have >>> fi

Re: [HACKERS] collateral benefits of a crash-safe visibility map

2011-05-10 Thread Simon Riggs
On Tue, May 10, 2011 at 6:02 PM, Robert Haas wrote: > On Tue, May 10, 2011 at 12:57 PM, Simon Riggs wrote: >> Hmmm, do we really need to WAL log freezing? >> >> Can we break down freezing into a 2 stage process, so that we can have >> first stage as a lossy operation and a second stage that is WA

Re: [HACKERS] 4.1beta1: ANYARRAY disallowed for DOMAIN types which happen to be arrays

2011-05-10 Thread Robert Haas
On Mon, May 9, 2011 at 11:32 PM, Tom Lane wrote: > "J. Greg Davidson" writes: >>       * Tighten casting checks for domains based on arrays (Tom Lane) > >>         When a domain is based on an array type,..., such a domain type >>         is no longer allowed to match an anyarray parameter of a >

Re: [HACKERS] Collation mega-cleanups

2011-05-10 Thread Ross J. Reedstrom
On Tue, May 10, 2011 at 07:21:16PM +0200, Andres Freund wrote: > On Tuesday, May 10, 2011 07:08:23 PM Ross J. Reedstrom wrote: > > On Mon, May 09, 2011 at 03:57:12PM -0400, Robert Haas wrote: > > > On Mon, May 9, 2011 at 2:58 PM, Bruce Momjian wrote: > > > > Tom this collation stuff has seen more

Re: [HACKERS] 4.1beta1: ANYARRAY disallowed for DOMAIN types which happen to be arrays

2011-05-10 Thread J. Greg Davidson
Given: CREATE DOMAIN int_array AS int[]; The operator [] works fine in 4.1beta1: SELECT (ARRAY[1,2,3]::int_array)[1]; proving that int_array is an array type with element type int. It is inconsistent that other array functions and operators don't work. On Mon, 2011-05-09 at 23:32 -0400, T

Re: [HACKERS] collateral benefits of a crash-safe visibility map

2011-05-10 Thread Merlin Moncure
On Tue, May 10, 2011 at 9:47 AM, Robert Haas wrote: > On Tue, May 10, 2011 at 9:59 AM, Merlin Moncure wrote: >> no, that wasn't my intent at all, except in the sense of wondering if >> a crash-safe visibility map provides a route of displacing a lot of >> hint bit i/o and by extension, making alt

Re: [HACKERS] Formatting Curmudgeons WAS: MMAP Buffers

2011-05-10 Thread Greg Smith
Heikki Linnakangas wrote: Well, my first patch was two-phase commit. And I had never even used PostgreSQL before I dived into the source tree and started to work on that Well, everyone knows you're awesome. A small percentage of the people who write patches end up having the combination of ba

Re: [HACKERS] hint bit cache v5

2011-05-10 Thread Merlin Moncure
On Tue, May 10, 2011 at 11:59 AM, Simon Riggs wrote: > On Mon, May 9, 2011 at 5:12 PM, Merlin Moncure wrote: > >> I'd like to know if this is a strategy that merits further work...If >> anybody has time/interest that is.  It's getting close to the point >> where I can just post it to the commit f

Re: [HACKERS] the big picture for index-only scans

2011-05-10 Thread Kevin Grittner
Tom Lane wrote: > "Kevin Grittner" writes: >> Simon Riggs wrote: >>> This topic has been discussed many times, yet I have never seen >>> an assessment that explains WHY we would want to do index-only >>> scans. > >> In databases with this feature, it's not too unusual for a query >> which uses

Re: [HACKERS] collateral benefits of a crash-safe visibility map

2011-05-10 Thread Heikki Linnakangas
On 10.05.2011 17:47, Robert Haas wrote: On Tue, May 10, 2011 at 9:59 AM, Merlin Moncure wrote: no, that wasn't my intent at all, except in the sense of wondering if a crash-safe visibility map provides a route of displacing a lot of hint bit i/o and by extension, making alternative approaches o

Re: [HACKERS] Collation mega-cleanups

2011-05-10 Thread Andres Freund
On Tuesday, May 10, 2011 07:08:23 PM Ross J. Reedstrom wrote: > On Mon, May 09, 2011 at 03:57:12PM -0400, Robert Haas wrote: > > On Mon, May 9, 2011 at 2:58 PM, Bruce Momjian wrote: > > > Tom this collation stuff has seen more post-feature-commit cleanups > > > than I think any patch I remember.

Re: [HACKERS] the big picture for index-only scans

2011-05-10 Thread Simon Riggs
On Tue, May 10, 2011 at 5:17 PM, Kevin Grittner wrote: > Simon Riggs wrote: > >> This topic has been discussed many times, yet I have never seen an >> assessment that explains WHY we would want to do index-only scans. > > In databases with this feature, it's not too unusual for a query > which us

Re: [HACKERS] collateral benefits of a crash-safe visibility map

2011-05-10 Thread Tom Lane
Robert Haas writes: > On Tue, May 10, 2011 at 12:57 PM, Simon Riggs wrote: >> Hmmm, do we really need to WAL log freezing? > That might solve the relfrozenxid problem - set the bits in the heap, > sync the heap, then update relfrozenxid once the heap is guaranteed > safely on disk - but it again

Re: [HACKERS] Collation mega-cleanups

2011-05-10 Thread Ross J. Reedstrom
On Mon, May 09, 2011 at 03:57:12PM -0400, Robert Haas wrote: > On Mon, May 9, 2011 at 2:58 PM, Bruce Momjian wrote: > > Tom this collation stuff has seen more post-feature-commit cleanups than > > I think any patch I remember.  Is there anything we can learn from this? > > How about "don't commit

Re: [HACKERS] the big picture for index-only scans

2011-05-10 Thread Robert Haas
On Tue, May 10, 2011 at 12:53 PM, Tom Lane wrote: > "Kevin Grittner" writes: >> Simon Riggs wrote: >>> This topic has been discussed many times, yet I have never seen an >>> assessment that explains WHY we would want to do index-only scans. > >> In databases with this feature, it's not too unusu

Re: [HACKERS] crash-safe visibility map, take five

2011-05-10 Thread Robert Haas
On Tue, May 10, 2011 at 1:00 PM, Tom Lane wrote: > Robert Haas writes: >> On Tue, May 10, 2011 at 11:34 AM, Jesper Krogh wrote: >>> Or what is the downside for keeping it across IO? Will it block other >>> processes trying to read it? > >> Heikki might be in a better position to comment on that

Re: [HACKERS] collateral benefits of a crash-safe visibility map

2011-05-10 Thread Robert Haas
On Tue, May 10, 2011 at 12:57 PM, Simon Riggs wrote: > Hmmm, do we really need to WAL log freezing? > > Can we break down freezing into a 2 stage process, so that we can have > first stage as a lossy operation and a second stage that is WAL > logged? That might solve the relfrozenxid problem - se

Re: [HACKERS] crash-safe visibility map, take five

2011-05-10 Thread Tom Lane
Robert Haas writes: > On Tue, May 10, 2011 at 11:34 AM, Jesper Krogh wrote: >> Or what is the downside for keeping it across IO? Will it block other >> processes trying to read it? > Heikki might be in a better position to comment on that than I am, > since he wrote the existing code. But I thi

Re: [HACKERS] hint bit cache v5

2011-05-10 Thread Simon Riggs
On Mon, May 9, 2011 at 5:12 PM, Merlin Moncure wrote: > I'd like to know if this is a strategy that merits further work...If > anybody has time/interest that is.  It's getting close to the point > where I can just post it to the commit fest for review.  In > particular, I'm concerned if Tom's ear

Re: [HACKERS] collateral benefits of a crash-safe visibility map

2011-05-10 Thread Simon Riggs
On Tue, May 10, 2011 at 3:47 PM, Robert Haas wrote: > To address the first problem, what we've talked about doing is > something along the line of freezing the tuples at the time we mark > the page all-visible, so we don't have to go back and do it again > later.  Unfortunately, it's not quite th

Re: [HACKERS] the big picture for index-only scans

2011-05-10 Thread Tom Lane
"Kevin Grittner" writes: > Simon Riggs wrote: >> This topic has been discussed many times, yet I have never seen an >> assessment that explains WHY we would want to do index-only scans. > In databases with this feature, it's not too unusual for a query > which uses just an index to run one or m

Re: [HACKERS] crash-safe visibility map, take five

2011-05-10 Thread Robert Haas
On Tue, May 10, 2011 at 11:34 AM, Jesper Krogh wrote: > On 2011-05-10 14:48, Robert Haas wrote: >> >> We could avoid all of this complexity - and the possibility of pinning >> the visibility map page needlessly - by locking the heap buffer first >> and then pinning the visibility map page if the h

Re: [HACKERS] the big picture for index-only scans

2011-05-10 Thread Robert Haas
On Tue, May 10, 2011 at 11:27 AM, Cédric Villemain wrote: > 2011/5/10 Robert Haas : >> On Tue, May 10, 2011 at 10:58 AM, Cédric Villemain >> wrote: >>> ANALYZE can do the stats job for 'free' on the pages it collects >>> anyway. So that looks like a good idea. >>> I believe the really lazy vacuum

Re: [HACKERS] Process wakeups when idle and power consumption

2011-05-10 Thread Heikki Linnakangas
On 10.05.2011 14:39, Peter Geoghegan wrote: Attached is win32 implementation of the "named pipe trick". It consists of a Visual Studio 2008 solution that contains two projects, named_pipe_trick (which represents the postmaster) and auxiliary_backend (which represents each auxiliary process). I s

Re: [HACKERS] Process wakeups when idle and power consumption

2011-05-10 Thread Tom Lane
Simon Riggs writes: > I've got a feeling that things will go easier if we have a separate > connection for the feedback channel. > Yes, two connections, one in either direction. > That would make everything simple, nice one way connections. It would > also mean we could stream at higher data rat

Re: [HACKERS] the big picture for index-only scans

2011-05-10 Thread Kevin Grittner
Simon Riggs wrote: > This topic has been discussed many times, yet I have never seen an > assessment that explains WHY we would want to do index-only scans. In databases with this feature, it's not too unusual for a query which uses just an index to run one or more orders of magnitude faster t

Re: [HACKERS] Process wakeups when idle and power consumption

2011-05-10 Thread Simon Riggs
On Tue, May 10, 2011 at 12:45 PM, Robert Haas wrote: > On Tue, May 10, 2011 at 5:14 AM, Peter Geoghegan > wrote: >> On 10 May 2011 09:45, Heikki Linnakangas >> wrote: >> >>> I think we need to refactor the function into something like: >>> >>> #define WL_LATCH_SET    1 >>> #define WL_SOCKET_REA

Re: [HACKERS] the big picture for index-only scans

2011-05-10 Thread Cédric Villemain
>> The temptation is high to estimate the cost of an "index_scan(only) + >> ordered(by ctid) table pages fetch if heap required". (this is what I >> understood from heikki suggestion 3-4. and it makes sense). It may be >> easier to implement both at once but I didn't find the branch in the >> Heikk

Re: [HACKERS] the big picture for index-only scans

2011-05-10 Thread Simon Riggs
On Tue, May 10, 2011 at 3:25 AM, Robert Haas wrote: > So, what do we need in order to find our way to index-only scans? > > 1. The visibility map needs to be crash-safe.  The basic idea of > index-only scans is that, instead of checking the heap to find out > whether each tuple is visible, we fir

Re: [HACKERS] crash-safe visibility map, take five

2011-05-10 Thread Jesper Krogh
On 2011-05-10 14:48, Robert Haas wrote: We could avoid all of this complexity - and the possibility of pinning the visibility map page needlessly - by locking the heap buffer first and then pinning the visibility map page if the heap page is all-visible. However, that would involve holding the l

Re: [HACKERS] the big picture for index-only scans

2011-05-10 Thread Cédric Villemain
2011/5/10 Robert Haas : > On Tue, May 10, 2011 at 10:58 AM, Cédric Villemain > wrote: >> ANALYZE can do the stats job for 'free' on the pages it collects >> anyway. So that looks like a good idea. >> I believe the really lazy vacuum is another topic; even if it will >> improve the performance of t

Re: [HACKERS] the big picture for index-only scans

2011-05-10 Thread Robert Haas
On Tue, May 10, 2011 at 10:58 AM, Cédric Villemain wrote: > ANALYZE can do the stats job for 'free' on the pages it collects > anyway. So that looks like a good idea. > I believe the really lazy vacuum is another topic; even if it will > improve the performance of the index only scan to have table

Re: [HACKERS] the big picture for index-only scans

2011-05-10 Thread Cédric Villemain
2011/5/10 Robert Haas : > So, what do we need in order to find our way to index-only scans? > > 3. Statistics.  I believe that in order to accurately estimate the > cost of an index-only scan, we're going to need to know the fraction > of tuples that are on pages whose visibility map bits are set.

Re: [HACKERS] improvements to pgtune

2011-05-10 Thread Bruce Momjian
FYI, I can help if you need javascript assistance. --- Greg Smith wrote: > Shiv wrote: > > So my exams are over now and am fully committed to the project in > > terms of time. I have started compiling a sort of personal to

[HACKERS] collateral benefits of a crash-safe visibility map

2011-05-10 Thread Robert Haas
On Tue, May 10, 2011 at 9:59 AM, Merlin Moncure wrote: > no, that wasn't my intent at all, except in the sense of wondering if > a crash-safe visibility map provides a route of displacing a lot of > hint bit i/o and by extension, making alternative approaches of doing > that, including mine, a lot

[HACKERS] [v9.2] Leaky view and RLS

2011-05-10 Thread Kohei Kaigai
I'd like to summarize expected issues corresponding to leaky-view and RLS towards v9.2, and PGcon2011/Developer Meeting. We already made consensus the leaky-view is a problem to be fixed previous to the row-level security feature. We know several ways to leak/infer contents of tuples to be invisi

[HACKERS] ts_rank

2011-05-10 Thread Mark
Could somebody explain me on which methods is based ts_rank and how it works? I would appreciate some articles, if exist. Thanks a lot for reply. Mark -- View this message in context: http://postgresql.1045698.n5.nabble.com/ts-rank-tp4384120p4384120.html Sent from the PostgreSQL - hackers mailin

Re: [HACKERS] crash-safe visibility map, take five

2011-05-10 Thread Robert Haas
On Tue, May 10, 2011 at 9:45 AM, Merlin Moncure wrote: > I see: here's a comment that was throwing me off: > +       /* > +        * If we didn't get the lock and it turns out we need it, we'll have > to > +        * unlock and re-lock, to avoid holding the buffer lock across an I/O. > +        *

Re: [HACKERS] the big picture for index-only scans

2011-05-10 Thread Merlin Moncure
On Tue, May 10, 2011 at 8:22 AM, Robert Haas wrote: > On Mon, May 9, 2011 at 10:36 PM, Merlin Moncure wrote: >>> 1. The visibility map needs to be crash-safe.  The basic idea of >>> index-only scans is that, instead of checking the heap to find out >>> whether each tuple is visible, we first chec

Re: [HACKERS] crash-safe visibility map, take five

2011-05-10 Thread Merlin Moncure
On Tue, May 10, 2011 at 7:48 AM, Robert Haas wrote: > I thought I'd explained it fairly thoroughly in the comments, but > evidently not.  Suggestions for improvement are welcome. ok. that clears it up nicely. > Here goes in more detail: Every time we insert, update, or delete a > tuple in a par

Re: [HACKERS] the big picture for index-only scans

2011-05-10 Thread Robert Haas
On Mon, May 9, 2011 at 10:36 PM, Merlin Moncure wrote: >> 1. The visibility map needs to be crash-safe.  The basic idea of >> index-only scans is that, instead of checking the heap to find out >> whether each tuple is visible, we first check the visibility map.  If >> the visibility map bit is set

Re: [HACKERS] "stored procedures" - use cases?

2011-05-10 Thread Pavel Stehule
2011/5/10 Robert Haas : > On Mon, May 9, 2011 at 11:58 PM, Pavel Stehule > wrote: >> no - you are little bit confused :). CALL and function execution >> shares nothing. There is significant differences between function and >> procedure. Function is called only from executor - from some plan, and

  1   2   >