Re: [HACKERS] A Guide to Constraint Exclusion (Partitioning)

2005-07-22 Thread Luke Lonergan
Tom, On 7/22/05 3:32 PM, "Tom Lane" <[EMAIL PROTECTED]> wrote: > but the "scan all rows" will only happen if no index is provided on > DateKey in the child tables. Otherwise the planner will probably > select plans like this: > > -> Index Scan using i1 on sales_jan_dateitemoutlet > sale

Re: [HACKERS] Autovacuum loose ends

2005-07-22 Thread Alvaro Herrera
On Fri, Jul 22, 2005 at 07:37:53PM -0400, Tom Lane wrote: > Alvaro Herrera <[EMAIL PROTECTED]> writes: > > I've hacked the whole thing enough that I fixed most of the issues. > > However this one I don't know how to handle. What I need to do is > > compare each database's frozen Xid with the curre

Re: [HACKERS] Autovacuum loose ends

2005-07-22 Thread Tom Lane
Alvaro Herrera <[EMAIL PROTECTED]> writes: > I've hacked the whole thing enough that I fixed most of the issues. > However this one I don't know how to handle. What I need to do is > compare each database's frozen Xid with the current transaction Id. > I can get the frozenxid from the flatfile --

Re: [HACKERS] Checkpoint cost, looks like it is WAL/CRC

2005-07-22 Thread Josh Berkus
Tom, > There's something awfully weird going on here. I was prepared to see > no statistically-significant differences, but not multiple cases that > seem to be going the "wrong direction". There's a lot of variance in the tests. I'm currently running a variance test battery on one machine to

Re: [HACKERS] Checkpoint cost, looks like it is WAL/CRC

2005-07-22 Thread Tom Lane
Josh Berkus writes: >> Um, where are the test runs underlying this spreadsheet? I don't have a >> whole lot of confidence in looking at full-run average TPM numbers to >> discern whether transient dropoffs in TPM are significant or not. > Web in the form of: > http://khack.osdl.org/stp/#test_nu

Re: [HACKERS] Checkpoint cost, looks like it is WAL/CRC

2005-07-22 Thread Tom Lane
Greg Stark <[EMAIL PROTECTED]> writes: > For any benchmarking to be meaningful you have to set the checkpoint interval > to something more realistic. Something like 5 minutes. That way when the final > checkpoint cycle isn't completely included in the timing data you'll at least > be missing a stat

Re: [HACKERS] Autovacuum loose ends

2005-07-22 Thread Alvaro Herrera
On Thu, Jul 14, 2005 at 10:52:56AM -0400, Tom Lane wrote: Hey, > * Or actually, it would vacuum template0, except that since no regular > backend ever connects to template0, there will be no stats DB entry for > it and so the loop in AutoVacMain will ignore it. This is definitely > BAD as it mea

Re: [HACKERS] A Guide to Constraint Exclusion (Partitioning)

2005-07-22 Thread Tom Lane
Simon Riggs <[EMAIL PROTECTED]> writes: > In summary, the CE feature will be a huge performance gain for > qualifying queries against large tables in PostgreSQL databases. BTW, before we spend too much time chasing an emperor that may have no clothes, it's worth asking whether this feature is real

Re: [HACKERS] [PERFORM] Planner doesn't look at LIMIT?

2005-07-22 Thread Dawid Kuroczko
On 7/22/05, Tom Lane <[EMAIL PROTECTED]> wrote: > > This is quite strange. The nestloop plan definitely should be preferred > > in the context of the LIMIT, considering that it has far lower estimated > > cost. And it is preferred in simple tests for me. > > After a suitable period of contemplat

Re: [HACKERS] Buildfarm failure - pl/tcl on snake

2005-07-22 Thread Dave Page
> -Original Message- > From: Andrew Dunstan [mailto:[EMAIL PROTECTED] > Sent: 22 July 2005 17:56 > To: Tom Lane > Cc: Dave Page; PostgreSQL-development > Subject: Re: [HACKERS] Buildfarm failure - pl/tcl on snake > > > > Tom Lane wrote: > > >"Dave Page" writes: > > > > > >>CVS HE

Re: [HACKERS] Checkpoint cost, looks like it is WAL/CRC

2005-07-22 Thread Josh Berkus
Tom, > Um, where are the test runs underlying this spreadsheet? I don't have a > whole lot of confidence in looking at full-run average TPM numbers to > discern whether transient dropoffs in TPM are significant or not. Web in the form of: http://khack.osdl.org/stp/#test_number#/ Where #test_nu

Re: [HACKERS] Checkpoint cost, looks like it is WAL/CRC

2005-07-22 Thread Josh Berkus
Greg, > For any benchmarking to be meaningful you have to set the checkpoint > interval to something more realistic. Something like 5 minutes. That way > when the final checkpoint cycle isn't completely included in the timing > data you'll at least be missing a statistically insignificant portion

Re: [HACKERS] Checkpoint cost, looks like it is WAL/CRC

2005-07-22 Thread Tom Lane
Josh Berkus writes: > Bruce, >> Did you test with full_page_writes on and off? > I didn't use your full_page_writes version because Tom said it was > problematic. This is CVS from July 3rd. We already know the results: should be equivalent to the hack Josh tried first. So what we know at thi

Re: [HACKERS] Checkpoint cost, looks like it is WAL/CRC

2005-07-22 Thread Tom Lane
Josh Berkus writes: > Looks like the CRC calculation work isn't the issue. I did test runs of > no-CRC vs. regular DBT2 with different checkpoint timeouts, and didn't > discern any statistical difference. See attached spreadsheet chart (the > two different runs are on two different machines).

Re: [HACKERS] Checkpoint cost, looks like it is WAL/CRC

2005-07-22 Thread Greg Stark
Josh Berkus writes: > I think this test run http://khack.osdl.org/stp/302903/results/0/, with a > 30-min checkpoint shows pretty clearly that the behavior of the > performance drop is consistent with needing to "re-prime" the WAL will > full page images. Each checkpoint drops performance a

Re: [HACKERS] Imprecision of DAYS_PER_MONTH

2005-07-22 Thread Andrew Dunstan
Bruno Wolff III wrote: On Fri, Jul 22, 2005 at 12:27:50 -0700, Dann Corbit <[EMAIL PROTECTED]> wrote: Apparently, the Gregorian calendar has been fixed. From this: http://www.physics.uq.edu.au/people/ross/phys2081/time/calendar.htm We have this: "The Gregorian calendar has been modified

Re: [HACKERS] Imprecision of DAYS_PER_MONTH

2005-07-22 Thread Andrew - Supernews
On 2005-07-22, Bruce Momjian wrote: > Bruno Wolff III wrote: >> According to the current calendar (again ignoring leap seconds) there >> are exactly 365.2425 days per year on average. I think it makes sense to use >> this number when dealing with calendar years and months. > > Someone came up with

Re: [HACKERS] Imprecision of DAYS_PER_MONTH

2005-07-22 Thread Bruno Wolff III
On Fri, Jul 22, 2005 at 12:27:50 -0700, Dann Corbit <[EMAIL PROTECTED]> wrote: > Apparently, the Gregorian calendar has been fixed. From this: > http://www.physics.uq.edu.au/people/ross/phys2081/time/calendar.htm > > We have this: > "The Gregorian calendar has been modified since (before anythi

Re: [HACKERS] Imprecision of DAYS_PER_MONTH

2005-07-22 Thread Dann Corbit
365.2425 is the exact value computed by the formulas found in the Gregorian calendar (a very good approximation of reality). 365.2422 is the physical reality of how long it actually takes (but there are tiny wobbles in it). http://www.timeanddate.com/date/leapyear.html > -Original Message---

Re: [HACKERS] Checkpoint cost, looks like it is WAL/CRC

2005-07-22 Thread Bruce Momjian
Josh Berkus wrote: > Bruce, > > > I think we need those tests run. > > Sure. What CVS day should I grab? What's the option syntax? ( -c > full_page_writes=false)? Yes. You can grab any from the day Tom fixed it, which was I think two weeks ago. > I have about 20 tests in queue right no

Re: [HACKERS] Imprecision of DAYS_PER_MONTH

2005-07-22 Thread Dann Corbit
Apparently, the Gregorian calendar has been fixed. From this: http://www.physics.uq.edu.au/people/ross/phys2081/time/calendar.htm We have this: "The Gregorian calendar has been modified since (before anything could go wrong) to bring the Gregorian 365.2425 down to 365.2422 by cutting out "leap ce

Re: [HACKERS] Checkpoint cost, looks like it is WAL/CRC

2005-07-22 Thread Josh Berkus
Bruce, > I think we need those tests run. Sure. What CVS day should I grab? What's the option syntax? ( -c full_page_writes=false)? I have about 20 tests in queue right now but can stack yours up behind them. -- --Josh Josh Berkus Aglio Database Solutions San Francisco -

Re: [HACKERS] Checkpoint cost, looks like it is WAL/CRC

2005-07-22 Thread Bruce Momjian
Josh Berkus wrote: > Bruce, > > > Did you test with full_page_writes on and off? > > I didn't use your full_page_writes version because Tom said it was > problematic. This is CVS from July 3rd. I think we need those tests run. -- Bruce Momjian| http://candle.pha.p

Re: [HACKERS] Checkpoint cost, looks like it is WAL/CRC

2005-07-22 Thread Josh Berkus
Bruce, > Did you test with full_page_writes on and off? I didn't use your full_page_writes version because Tom said it was problematic. This is CVS from July 3rd. -- --Josh Josh Berkus Aglio Database Solutions San Francisco ---(end of broadcast)-

Re: [HACKERS] Imprecision of DAYS_PER_MONTH

2005-07-22 Thread Bruce Momjian
Bruno Wolff III wrote: > On Thu, Jul 21, 2005 at 13:47:29 -0700, > Dann Corbit <[EMAIL PROTECTED]> wrote: > > In round figures: > > > > Since there are 365.2422 days per tropical year, there are 31556926 > > seconds per year (give or take leap seconds). > > > > Ref: > > http://www.grc.nasa.gov/

Re: [HACKERS] Checkpoint cost, looks like it is WAL/CRC

2005-07-22 Thread Bruce Momjian
Did you test with full_page_writes on and off? --- Josh Berkus wrote: > Tom, > > > This will remove just the CRC calculation work associated with backed-up > > pages. ?Note that any attempt to recover from the WAL will fail

Re: [HACKERS] Checkpoint cost, looks like it is WAL/CRC

2005-07-22 Thread Josh Berkus
Tom, > This will remove just the CRC calculation work associated with backed-up > pages.  Note that any attempt to recover from the WAL will fail, but I > assume you don't need that for the purposes of the test run. Looks like the CRC calculation work isn't the issue. I did test runs of no-CRC

Re: [HACKERS] [PERFORM] Planner doesn't look at LIMIT?

2005-07-22 Thread Tom Lane
Simon Riggs <[EMAIL PROTECTED]> writes: > Looks good. I think it explains a few other wierd perf reports also. Could be. I went back to look at Sam Mason's report about three weeks ago, and it definitely seems to explain his issue. The "fuzzy cost comparison" logic is new in 8.0 so it hasn't had

Re: [HACKERS] [PERFORM] Planner doesn't look at LIMIT?

2005-07-22 Thread Simon Riggs
On Fri, 2005-07-22 at 12:20 -0400, Tom Lane wrote: > I think that this refutes the original scheme of using the same fuzz > factor for both startup and total cost comparisons, and therefore > propose the attached patch. > > Comments? Looks good. I think it explains a few other wierd perf reports

Re: [HACKERS] Buildfarm failure - pl/tcl on snake

2005-07-22 Thread Andrew Dunstan
Tom Lane wrote: "Dave Page" writes: CVS HEAD is consistently failing on snake in pl/tcl, Looks like a locale issue to me, ie, sort order of "KEY" relative to "key". Are you running the installcheck tests in C locale? (If the buildfarm script doesn't force C locale, maybe it should

Re: [HACKERS] Imprecision of DAYS_PER_MONTH

2005-07-22 Thread Bruno Wolff III
On Thu, Jul 21, 2005 at 13:47:29 -0700, Dann Corbit <[EMAIL PROTECTED]> wrote: > In round figures: > > Since there are 365.2422 days per tropical year, there are 31556926 > seconds per year (give or take leap seconds). > > Ref: > http://www.grc.nasa.gov/WWW/K-12/Numbers/Math/Mathematical_Thinki

Re: [HACKERS] Buildfarm failure - pl/tcl on snake

2005-07-22 Thread Tom Lane
"Dave Page" writes: > CVS HEAD is consistently failing on snake in pl/tcl, Looks like a locale issue to me, ie, sort order of "KEY" relative to "key". Are you running the installcheck tests in C locale? (If the buildfarm script doesn't force C locale, maybe it should.) r

[HACKERS] Buildfarm failure - pl/tcl on snake

2005-07-22 Thread Dave Page
CVS HEAD is consistently failing on snake in pl/tcl, whilst REL8_0_STABLE is passing just fine. select * from T_pkey1 order by key1 using @<, key2; key1 | key2 | txt --+--+-- - 1 | KEY1-3

Re: Writing Commit Status hint bits (was Re: [HACKERS] Constant

2005-07-22 Thread Simon Riggs
On Fri, 2005-07-22 at 09:40 -0400, Tom Lane wrote: > Simon Riggs <[EMAIL PROTECTED]> writes: > > In general, the hint bits are good. In *some* cases, not. I still seek > > control over that as a designer. ... > ... not be worth > solving with a knob as klugy and potentially dangerous as > suppres

Re: [HACKERS] [PERFORM] Planner doesn't look at LIMIT?

2005-07-22 Thread Tom Lane
I wrote: > Dawid Kuroczko <[EMAIL PROTECTED]> writes: >> qnex=# EXPLAIN SELECT * FROM log NATURAL JOIN useragents LIMIT 1; >> Limit (cost=15912.20..15912.31 rows=1 width=272) >> -> Hash Join (cost=15912.20..5328368.96 rows=47044336 width=272) >> If I set enable_hashjoin=false: >> qnex=# EXPLA

Re: [HACKERS] regressin failure on latest CVS

2005-07-22 Thread Rocco Altier
This patch fixes the interval regression on my AIX box (kookaburra) by only doing integer math on the interval, instead of float/double math. I think this is the correct way to handle this, since it's an integer data type. I don't know if it will fix Olivier's problem, since I wasn't able to repr

Re: [HACKERS] Constraint Exclusion on all tables

2005-07-22 Thread Germán Poó Caamaño
Le vendredi 22 juillet 2005 à 15:44 +0100, Sam Mason a écrit : > Dawid Kuroczko wrote: > >Hmm, methinks it wouldn't be a very difficult "beginner's project" in > >PostgreSQL hacking, to add "script-friendly" format for EXPLAIN > >command. I am not sure if I'll make it, but I'm willing to try... >

Re: [HACKERS] Constraint Exclusion on all tables

2005-07-22 Thread Sam Mason
Dawid Kuroczko wrote: >Hmm, methinks it wouldn't be a very difficult "beginner's project" in >PostgreSQL hacking, to add "script-friendly" format for EXPLAIN >command. I am not sure if I'll make it, but I'm willing to try... It occured to me that it may be good to create a new datatype for the re

Re: [HACKERS] regressin failure on latest CVS

2005-07-22 Thread Bruce Momjian
Michael Glaesemann wrote: > > On Jul 22, 2005, at 6:28 PM, ohp@pyrenet.fr wrote: > > > I tried the latest cvs this morning (07/22 11:00 CET) > > and interval test fails. > > Here's the regression.diffs. > > > > > *** ./expected/interval.outFri Jul 22 10:32:21 2005 > > --- ./results/interval.

Re: [HACKERS] [PATCHES] Roles - SET ROLE Updated

2005-07-22 Thread Stephen Frost
* Tom Lane ([EMAIL PROTECTED]) wrote: > Stephen Frost <[EMAIL PROTECTED]> writes: > > It sounds like this is essentially if 'SET ROLE all;' is allowed or not. > > If you disallow 'SET ROLE all;' (and therefore not do it on session > > start) then you would get this behaviour. I certainly see that

Re: [HACKERS] Timezone bugs

2005-07-22 Thread Andrew - Supernews
On 2005-07-22, Bruce Momjian wrote: >> >> select (CURRENT_DATE + '05:00'::time)::timestamp >>at time zone 'Canada/Pacific'; >> timezone >> >> 2005-07-19 22:00:00+00 >> (1 row) >> > What is happening here is that 2005-07-20 05:00:00 is being cast back

Re: [HACKERS] [PATCHES] Roles - SET ROLE Updated

2005-07-22 Thread Stephen Frost
* Stephen Frost ([EMAIL PROTECTED]) wrote: > * Tom Lane ([EMAIL PROTECTED]) wrote: > > If we don't do it that way then we have a bunch of API that breaks down: > > all of the has_foo_privilege functions stop working, because they don't > > have a signature that allows both a user and a role to be p

Re: [HACKERS] Timezone bugs

2005-07-22 Thread Bruce Momjian
Jeff Trout wrote: > > On Jul 21, 2005, at 11:57 PM, Bruce Momjian wrote: > > >works fine now. It will also obey whatever DST rules were in > effect at > >just that date, which the previous implementation did not. > > Speaking of that, would the nearly passed US bill to extend dayligh

Re: [HACKERS] [PATCHES] Roles - SET ROLE Updated

2005-07-22 Thread Tom Lane
Stephen Frost <[EMAIL PROTECTED]> writes: > It sounds like this is essentially if 'SET ROLE all;' is allowed or not. > If you disallow 'SET ROLE all;' (and therefore not do it on session > start) then you would get this behaviour. I certainly see that as a > reasonable option though I think we'd w

Re: Writing Commit Status hint bits (was Re: [HACKERS] Constant WAL replay)

2005-07-22 Thread Tom Lane
Simon Riggs <[EMAIL PROTECTED]> writes: > In general, the hint bits are good. In *some* cases, not. I still seek > control over that as a designer. > Specifically, the scenario I want to optimize is this: > - we load a table with lots of real time measurement data, as one child > out of a large nu

Re: [HACKERS] [PATCHES] Roles - SET ROLE Updated

2005-07-22 Thread Stephen Frost
* Peter Eisentraut ([EMAIL PROTECTED]) wrote: > Am Donnerstag, 21. Juli 2005 22:55 schrieb Tom Lane: > > What this says is that when a role A is a member of another role B, A > > automatically has all of B's privileges. But when a user U is a member > > of role R, U does *not* have R's privileges

Re: [HACKERS] Timezone bugs

2005-07-22 Thread Jeff Trout
On Jul 21, 2005, at 11:57 PM, Bruce Momjian wrote: >works fine now. It will also obey whatever DST rules were in effect at >just that date, which the previous implementation did not. Speaking of that, would the nearly passed US bill to extend daylight savings screw up our timezone

Re: [HACKERS] [PATCHES] Roles - SET ROLE Updated

2005-07-22 Thread Peter Eisentraut
Am Donnerstag, 21. Juli 2005 22:55 schrieb Tom Lane: > What this says is that when a role A is a member of another role B, A > automatically has all of B's privileges. But when a user U is a member > of role R, U does *not* have R's privileges automatically. What he has > is the right to do SET R

[HACKERS] interval->day patch and docs

2005-07-22 Thread Michael Glaesemann
Thanks, Bruce, for improving and applying the interval->day patch. I've been busy with work and slow in catching up on what you actually did (I still need to go back and look) and look forward to learning what remained to make the patch more complete. I'm hoping to get the docs and some add

Re: [HACKERS] regressin failure on latest CVS

2005-07-22 Thread Michael Glaesemann
On Jul 22, 2005, at 6:28 PM, ohp@pyrenet.fr wrote: I tried the latest cvs this morning (07/22 11:00 CET) and interval test fails. Here's the regression.diffs. *** ./expected/interval.outFri Jul 22 10:32:21 2005 --- ./results/interval.outFri Jul 22 11:07:54 2005 *** *** 2

[HACKERS] regressin failure on latest CVS

2005-07-22 Thread ohp
Hi, I tried the latest cvs this morning (07/22 11:00 CET) and interval test fails. Here's the regression.diffs. *** ./expected/interval.out Fri Jul 22 10:32:21 2005 --- ./results/interval.out Fri Jul 22 11:07:54 2005 *** *** 217,224 -- updating pg_aggregate.agginitval

Re: Writing Commit Status hint bits (was Re: [HACKERS] Constant

2005-07-22 Thread Simon Riggs
On Wed, 2005-07-20 at 13:20 -0400, Tom Lane wrote: > > When VACUUM freezes the xid, it *does* make sense at that point to > > update the hint bits as a performance optimization. > > The hint bits are not really relevant when xmin = FrozenTransactionId, > since any examiner of the tuple would cons

Re: [HACKERS] recover corrupted pg_controldata from WAL

2005-07-22 Thread yuanjia lee
Hi Tom I agree that it is wrong to use the information from the file name itself. I will try to read the xlp_pageaddr out from the segment header to figure out which one is the lastest one. In the mutilple time lines scenario, if the pg_control file crashed, and the current time line information