Re: [HACKERS] pg_migrator issue with contrib

2009-06-07 Thread Robert Haas
On Sun, Jun 7, 2009 at 11:50 PM, Bruce Momjian wrote: > Stefan Kaltenbrunner wrote: >> Josh Berkus wrote: >> > On 6/7/09 10:56 AM, Robert Haas wrote: >> >> OK, that's more or less what I thought, and what I intended to convey >> >> upthread.  As far as core Postgres is concerned this is a new featu

Re: [HACKERS] Partial vacuum versus pg_class.reltuples

2009-06-07 Thread Tom Lane
Robert Haas writes: > Basically, I'm trying to figure out what we're going to recommend to > someone who gets bitten by whatever remaining corner case still exists > after your recent patch, and I admit I'm not real clear on what that > is. If anyone actually shows up with a clear problem, we can

Re: [HACKERS] information_schema.columns changes needed for OLEDB

2009-06-07 Thread Tom Lane
Peter Eisentraut writes: > On Sunday 31 May 2009 18:41:55 Tom Lane wrote: >> AFAICS, the SQL standard demands that precision and scale fields be >> non-null all the time for those data types where they make sense >> (this is encoded in the CHECK CONSTRAINTs that are declared for the >> various inf

Re: [HACKERS] pg_migrator issue with contrib

2009-06-07 Thread Bruce Momjian
Tom Lane wrote: > Robert Haas writes: > > I did know that EDB had been using the tool for a while, but I admit > > I'm not familiar with the whole history. I had the impression that > > we'd gotten a lot more serious about really making this rock solid > > since Bruce took it in February.

Re: [HACKERS] pg_migrator issue with contrib

2009-06-07 Thread Bruce Momjian
Stefan Kaltenbrunner wrote: > Josh Berkus wrote: > > On 6/7/09 10:56 AM, Robert Haas wrote: > >> OK, that's more or less what I thought, and what I intended to convey > >> upthread. As far as core Postgres is concerned this is a new feature, > >> and we haven't worked out all the kinks yet. > > >

Re: [HACKERS] pg_migrator issue with contrib

2009-06-07 Thread Bruce Momjian
Robert Haas wrote: > OK, that's more or less what I thought, and what I intended to convey > upthread. As far as core Postgres is concerned this is a new feature, > and we haven't worked out all the kinks yet. As long as we set > expectations accordingly, I think that's OK. You mention CVE

Re: [HACKERS] pg_migrator issue with contrib

2009-06-07 Thread Bruce Momjian
Tom Lane wrote: > The underlying C-level get_raw_page function is still there, but > it now expects three arguments not two, and will crash if it's > passed an int4 where it's expecting a text argument. But the old > function definition will migrate without error --- there's no way > for pg_migrat

Re: [HACKERS] pg_migrator issue with contrib

2009-06-07 Thread Bruce Momjian
Tom Lane wrote: > Bruce Momjian writes: > > I think the cleanest solution is to document that these issues might > > happen and suggest solutions. > > No, the cleanest solution is to fix it before people ever see a > problem. This is trivial to do in the case of dblink and I don't > see why you

[HACKERS] B-Tree contention?

2009-06-07 Thread Alan Li
Hello, I have multiple files with that have very similar distributions and I'm seeing contention when concurrent COPY's are happening against a table with a b-tree index on the timestamp column. Each file look something like the following: ~4M rows with timestamp1 ~4M rows with timestamp2 ... T

Re: [HACKERS] Partial vacuum versus pg_class.reltuples

2009-06-07 Thread Robert Haas
On Sun, Jun 7, 2009 at 4:19 PM, Tom Lane wrote: > Robert Haas writes: >> On Sun, Jun 7, 2009 at 3:24 PM, Tom Lane wrote: >>> [ thinks a bit and reads the code some more ... ]  There is a >>> considerably safer alternative, which is to let ANALYZE update the >>> reltuples estimate based on the page

Re: [HACKERS] information_schema.columns changes needed for OLEDB

2009-06-07 Thread Peter Eisentraut
On Sunday 31 May 2009 18:41:55 Tom Lane wrote: > AFAICS, the SQL standard demands that precision and scale fields be > non-null all the time for those data types where they make sense > (this is encoded in the CHECK CONSTRAINTs that are declared for the > various information-schema tables, see part

Re: [HACKERS] Partial vacuum versus pg_class.reltuples

2009-06-07 Thread Tom Lane
Robert Haas writes: > On Sun, Jun 7, 2009 at 3:24 PM, Tom Lane wrote: >> [ thinks a bit and reads the code some more ... ]  There is a >> considerably safer alternative, which is to let ANALYZE update the >> reltuples estimate based on the pages it sampled; which should be a >> considerably less b

Re: [HACKERS] Partial vacuum versus pg_class.reltuples

2009-06-07 Thread Robert Haas
On Sun, Jun 7, 2009 at 3:24 PM, Tom Lane wrote: > Robert Haas writes: >> Am I wrong to be frightened by the implications of updating this value >> only once in a blue moon? > > It's not great, but I think it's probably not catastrophic either. > Keep in mind that all we need from reltuples is that

Re: [HACKERS] pg_migrator issue with contrib

2009-06-07 Thread Stefan Kaltenbrunner
Josh Berkus wrote: On 6/7/09 10:56 AM, Robert Haas wrote: OK, that's more or less what I thought, and what I intended to convey upthread. As far as core Postgres is concerned this is a new feature, and we haven't worked out all the kinks yet. Yes, I'm calling it "pg_migrator beta" in any advo

Re: [HACKERS] Partial vacuum versus pg_class.reltuples

2009-06-07 Thread Tom Lane
Robert Haas writes: > Am I wrong to be frightened by the implications of updating this value > only once in a blue moon? It's not great, but I think it's probably not catastrophic either. Keep in mind that all we need from reltuples is that the ratio reltuples/relpages be a reasonable estimate of

Re: [HACKERS] pg_migrator issue with contrib

2009-06-07 Thread Josh Berkus
On 6/7/09 10:56 AM, Robert Haas wrote: OK, that's more or less what I thought, and what I intended to convey upthread. As far as core Postgres is concerned this is a new feature, and we haven't worked out all the kinks yet. Yes, I'm calling it "pg_migrator beta" in any advocacy/PR about it. A

Re: [HACKERS] Managing multiple branches in git

2009-06-07 Thread Peter Eisentraut
On Wednesday 03 June 2009 01:55:48 Andrew Dunstan wrote: > Running recursive grep on a subversion working copy is quite nasty. I suggest export GREP_OPTIONS='-d skip -I --exclude=*.svn-base --exclude=tags --exclude=*~ --exclude-dir=CVS --exclude-dir=.git --exclude-dir=.svn --exclude=TAGS' --

Re: [HACKERS] Partial vacuum versus pg_class.reltuples

2009-06-07 Thread Greg Stark
On Sun, Jun 7, 2009 at 7:11 PM, Robert Haas wrote: > Am I wrong to be frightened by the implications of updating this value > only once in a blue moon?  Doesn't this have the potential to result > in really bad plans?  Do we have any reasonable manual way of forcing > VACUUM to scan the entire heap

Re: [HACKERS] Partial vacuum versus pg_class.reltuples

2009-06-07 Thread Robert Haas
On Sat, Jun 6, 2009 at 3:44 PM, Tom Lane wrote: > I complained a couple days ago that in HEAD, vacuum is putting > very bogus values into pg_class.reltuples for indexes: > http://archives.postgresql.org/pgsql-bugs/2009-06/msg00037.php > > After looking through the code a bit, I've confirmed my prio

Re: [HACKERS] pg_migrator issue with contrib

2009-06-07 Thread Robert Haas
On Jun 7, 2009, at 12:03 PM, Tom Lane wrote: Robert Haas writes: I did know that EDB had been using the tool for a while, but I admit I'm not familiar with the whole history. I had the impression that we'd gotten a lot more serious about really making this rock solid since Bruce took it in F

Re: [HACKERS] pg_migrator issue with contrib

2009-06-07 Thread Bruce Momjian
Grzegorz Jaskiewicz wrote: > > On 7 Jun 2009, at 03:27, Bruce Momjian wrote: > > > Grzegorz Jaskiewicz wrote: > >> > >> On 6 Jun 2009, at 19:50, Tom Lane wrote: > >>> We have five days. > >>> > >>> I don't think we need testing, per se. The first step should be to > >>> diff > >>> the 8.3 and 8.

Re: [Fwd: Re: [HACKERS] dblink patches for comment]

2009-06-07 Thread Joe Conway
Tom Lane wrote: Joe Conway writes: how to trigger the failure (and therefore how to test the solution). A naive test with two databases, one LATIN2, the other UTF8 does not produce the error with simple text literals. Any guidance on specific literals that would trigger the problem? Hmm, se

Re: [HACKERS] Partial vacuum versus pg_class.reltuples

2009-06-07 Thread Tom Lane
Simon Riggs writes: > On Sat, 2009-06-06 at 15:44 -0400, Tom Lane wrote: >> In the longer term, we need to do something else. > -1 for such radical change at this stage of release. Uh, by "longer term" I meant this is something to think about for 8.5. regards, tom lane

Re: [HACKERS] Partial vacuum versus pg_class.reltuples

2009-06-07 Thread Simon Riggs
On Sat, 2009-06-06 at 15:44 -0400, Tom Lane wrote: > I think probably the only workable solution for 8.4 is to use the prior > value of the relation's reltuples field as num_heap_tuples when we have > not scanned the whole heap. This will effectively mean that index > reltuples values don't chan

Re: [HACKERS] Revisiting default_statistics_target

2009-06-07 Thread Simon Riggs
On Sun, 2009-06-07 at 12:13 -0400, Tom Lane wrote: > In any case, what we seem to have here is evidence that there are some > cases where the new default value of default_statistics_target is too > high and you can get a benefit by lowering it. I'm not sure we should > panic about that. Default

Re: [HACKERS] pg_migrator issue with contrib

2009-06-07 Thread Tom Lane
I wrote: > * pageinspect has changed the ABI of get_raw_page() in a way that will > likely make it dump core if the function definition is migrated from > an old DB. This needs to be fixed. > [ and similarly for some other contrib modules ] After thinking about this some more, I think that there

Re: [HACKERS] Revisiting default_statistics_target

2009-06-07 Thread Greg Stark
On Sun, Jun 7, 2009 at 5:13 PM, Tom Lane wrote: > > In any case, what we seem to have here is evidence that there are some > cases where the new default value of default_statistics_target is too > high and you can get a benefit by lowering it.  I'm not sure we should > panic about that.  Default va

Re: [HACKERS] Revisiting default_statistics_target

2009-06-07 Thread Tom Lane
Simon Riggs writes: > On Sat, 2009-06-06 at 12:06 -0700, Josh Berkus wrote: >> Well, Jignesh and I identified two things which we think are "special" >> about DBT2: (1) it uses C stored procedures, and (2) we don't think it >> uses prepared plans. > If there is a performance regression it is al

Re: [HACKERS] pg_migrator issue with contrib

2009-06-07 Thread Tom Lane
Robert Haas writes: > I did know that EDB had been using the tool for a while, but I admit > I'm not familiar with the whole history. I had the impression that > we'd gotten a lot more serious about really making this rock solid > since Bruce took it in February. But maybe that's not the c

Re: [HACKERS] pg_migrator issue with contrib

2009-06-07 Thread Stefan Kaltenbrunner
Tom Lane wrote: Robert Haas writes: I think it's becoming increasingly clear that pg_migrator is not for the faint of heart; in fact, I think we should hedge references to it with words like "experimental". Probably ... I'm with Robert on that one - while pg_migrator is extremely inmportant

Re: [HACKERS] Revisiting default_statistics_target

2009-06-07 Thread Simon Riggs
On Sat, 2009-06-06 at 12:06 -0700, Josh Berkus wrote: > > On the DL380 GB system, where I'm using a lot more drives the Jignesh, > > I see a performance change of under 5%. 15651.14 notpm vs 16333.32 > > notpm. And this is after a bit of tuning, not sure how much the out > > of the box experien

Re: [HACKERS] pg_migrator issue with contrib

2009-06-07 Thread Robert Haas
On Jun 7, 2009, at 10:04 AM, Tom Lane wrote: Robert Haas writes: I think it's becoming increasingly clear that pg_migrator is not for the faint of heart; in fact, I think we should hedge references to it with words like "experimental". Probably ... Just to recall the history, the first pg

Re: [HACKERS] pg_migrator issue with contrib

2009-06-07 Thread Tom Lane
Robert Haas writes: > I think it's becoming increasingly clear that pg_migrator is not for > the faint of heart; in fact, I think we should hedge references to it > with words like "experimental". Probably ... > Just to recall the history, the first pgfoundry > commit messages for this tool were

Re: [HACKERS] pg_migrator issue with contrib

2009-06-07 Thread Robert Haas
On Sun, Jun 7, 2009 at 12:11 AM, Tom Lane wrote: > * intarray has removed its internal @> and <@ operators.  As I was > mentioning the other day, it might be the best thing to just revert > that change anyway, until we can get a better handle on the behavior > of GIN for empty arrays. +1. > So we

Re: [HACKERS] pg_migrator issue with contrib

2009-06-07 Thread Grzegorz Jaskiewicz
On 7 Jun 2009, at 03:27, Bruce Momjian wrote: Grzegorz Jaskiewicz wrote: On 6 Jun 2009, at 19:50, Tom Lane wrote: We have five days. I don't think we need testing, per se. The first step should be to diff the 8.3 and 8.4 versions of the various contrib .sql.in files and determine what chan

Re: [HACKERS] PostgreSQL Developer meeting minutes up

2009-06-07 Thread Nicolas Barbier
2009/6/7 Markus Wanner : > However, there's no special whitespace treatment. Nor anything remotely > as clever as "nearby variable renaming". There's no such magic, the > developer still needs to tell the tool what he wants. If I understand correctly, "nearby variable renaming" refers to changes