Re: [HACKERS] patch for new feature: Buffer Cache Hibernation

2011-05-05 Thread Dimitri Fontaine
Josh Berkus j...@agliodbs.com writes: I thought that Dimitri had already implemented this using Fincore. It's linux-only, but that should work well enough to test the general concept. Actually, Cédric did, and I have a clone of his repository where I did some debian packaging of it.

Re: [HACKERS] patch for new feature: Buffer Cache Hibernation

2011-05-05 Thread Cédric Villemain
2011/5/4 Josh Berkus j...@agliodbs.com: All, I thought that Dimitri had already implemented this using Fincore.  It's linux-only, but that should work well enough to test the general concept. Harald provided me some pointers at pgday in Stuttgart to make it work with windows but ... hum I

Re: [HACKERS] Pull up aggregate subquery

2011-05-05 Thread Hitoshi Harada
2011/5/5 Tom Lane t...@sss.pgh.pa.us: Hitoshi Harada umi.tan...@gmail.com writes: I sometimes wonder if we could pull up aggregate query in the optimizer. I don't have time to look at this right now, but please add to the upcoming commitfest. Done.

[HACKERS] Backpatching of Teach the regular expression functions to do case-insensitive matching

2011-05-05 Thread Andres Freund
Hi, In my opinion this is actually a bug in 9.0. As its a (imo) low impact fix thats constrained to two files it seems sensible to backpatch it now that the solution has proven itself in the field? The issue is hard to find and has come up several times in the field. And it has been

Re: [HACKERS] patch for new feature: Buffer Cache Hibernation

2011-05-05 Thread Mitsuru IWASAKI
Hi, thanks for good suggestions. Postgres usually starts with ZERO buffer cache. By saving the buffer cache data structure into hibernation files just before shutdown, and loading them at startup, postgres can start operations with the saved buffer cache as the same condition as just

Re: [HACKERS] patch for new feature: Buffer Cache Hibernation

2011-05-05 Thread Mitsuru IWASAKI
Hi, I think that PgFincore (http://pgfoundry.org/projects/pgfincore/) provides similar functionality. Are you familiar with that? If so, could you contrast your approach with that one? I'm not familiar with PgFincore at all sorry, but I got source code and documents and read through them

Re: [HACKERS] GSoC 2011: Fast GiST index build

2011-05-05 Thread Teodor Sigaev
gistFindCorrectParent function have branch with following comment: /* * awful!!, we need search tree to find parent ... , but before we * should release all old parent */ Can you provide me an example of case when this branch works? See several lines above: if (parent-blkno ==

Re: [HACKERS] patch for new feature: Buffer Cache Hibernation

2011-05-05 Thread Cédric Villemain
2011/5/5 Mitsuru IWASAKI iwas...@jp.freebsd.org: Hi, I think that PgFincore (http://pgfoundry.org/projects/pgfincore/) provides similar functionality.  Are you familiar with that?  If so, could you contrast your approach with that one? I'm not familiar with PgFincore at all sorry, but I got

Re: [HACKERS] GSoC 2011: Fast GiST index build

2011-05-05 Thread Alexander Korotkov
2011/5/5 Teodor Sigaev teo...@sigaev.ru See several lines above: if (parent-blkno == InvalidBlockNumber) /* * end of chain and still didn't found parent, It's very-very * rare situation when root splited */

Re: [HACKERS] VARIANT / ANYTYPE datatype

2011-05-05 Thread Merlin Moncure
On Wed, May 4, 2011 at 8:03 PM, Tom Lane t...@sss.pgh.pa.us wrote: Alvaro Herrera alvhe...@commandprompt.com writes: As a followup idea there exists the desire to store records as records and not text representation of same (given differing record types, of course), for which it'd be more

Re: [HACKERS] adding a new column in IDENTIFY_SYSTEM

2011-05-05 Thread Magnus Hagander
On Wed, May 4, 2011 at 22:42, Simon Riggs si...@2ndquadrant.com wrote: On Wed, May 4, 2011 at 3:47 AM, Tom Lane t...@sss.pgh.pa.us wrote: Jaime Casanova ja...@2ndquadrant.com writes: I want to propose the addition of a new field in IDENTIFY_SYSTEM: xlogversion, which will carry XLOG_PAGE_MAGIC

Re: [HACKERS] adding a new column in IDENTIFY_SYSTEM

2011-05-05 Thread Tom Lane
Magnus Hagander mag...@hagander.net writes: So even if people don't believe in the rationale behind the patch, would allowing it harm anything at this point? Adding it for the sake of upgrades seems very far fetched. Adding it for the sake of giving a better error message seems like a very

[HACKERS] Patch to improve style of generate_history.pl perl script

2011-05-05 Thread Bruce Momjian
I have applied the attached patch to improve the style of our generate_history.pl perl script. -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. + diff --git

Re: [HACKERS] alpha5

2011-05-05 Thread Bruce Momjian
Robert Haas wrote: On Mon, Mar 28, 2011 at 9:26 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Sat, Mar 26, 2011 at 9:41 PM, Robert Haas robertmh...@gmail.com wrote: Per previous discussion, I'm going to wrap alpha5 Monday morning Eastern time, barring

[HACKERS] Visibility map and hint bits

2011-05-05 Thread Bruce Momjian
There has been a lot of recent discussion about the visibility map (for index-only scans) and hint bits (trying to avoid double-writing a table). I wonder if we could fix both of these at the same time. Once the visibility map is reliable, can we use that to avoid updating the hint bits on all

Re: [HACKERS] VARIANT / ANYTYPE datatype

2011-05-05 Thread Jim Nasby
On May 4, 2011, at 6:24 PM, Andrew Dunstan wrote: I'm far from convinced that storing deltas per column rather than per record is a win anyway. I don't have hard numbers to hand, but my vague recollection is that my tests showed it to be a design that used more space. It depends on how many

Re: [HACKERS] Patch to improve style of generate_history.pl perl script

2011-05-05 Thread Andrew Dunstan
On 05/05/2011 12:49 PM, Bruce Momjian wrote: I have applied the attached patch to improve the style of our generate_history.pl perl script. Wow, really? There's nothing wrong with the changes but they seem pretty trivial and pointless to me. Maybe I've drunk the perl TIMTOWTDI koolaid too

Re: [HACKERS] FDW table hints

2011-05-05 Thread Magnus Hagander
On Tue, May 3, 2011 at 16:19, Tom Lane t...@sss.pgh.pa.us wrote: Dave Page dp...@pgadmin.org writes: On Tue, May 3, 2011 at 10:33 AM, Susanne Ebrecht susa...@2ndquadrant.com wrote: When we make such a hint for foreign tables then we should make a similar hint for views. A view really isn't

Re: [HACKERS] VARIANT / ANYTYPE datatype

2011-05-05 Thread Andrew Dunstan
On 05/05/2011 01:00 PM, Jim Nasby wrote: On May 4, 2011, at 6:24 PM, Andrew Dunstan wrote: I'm far from convinced that storing deltas per column rather than per record is a win anyway. I don't have hard numbers to hand, but my vague recollection is that my tests showed it to be a design

Re: [HACKERS] FDW table hints

2011-05-05 Thread Tom Lane
Magnus Hagander mag...@hagander.net writes: Since I brought it up - a patch along this line? Please don't capitalize foreign table there. Also, I think an else before the other ereport would make the code clearer, even though it's not strictly necessary. regards, tom

Re: [HACKERS] FDW table hints

2011-05-05 Thread Magnus Hagander
On Thu, May 5, 2011 at 19:22, Tom Lane t...@sss.pgh.pa.us wrote: Magnus Hagander mag...@hagander.net writes: Since I brought it up - a patch along this line? Please don't capitalize foreign table there. Yeah, I was a bit split about that myself. Will change. Also, I think an else before

Re: [HACKERS] Patch to improve style of generate_history.pl perl script

2011-05-05 Thread Bruce Momjian
Andrew Dunstan wrote: On 05/05/2011 12:49 PM, Bruce Momjian wrote: I have applied the attached patch to improve the style of our generate_history.pl perl script. Wow, really? There's nothing wrong with the changes but they seem pretty trivial and pointless to me. Maybe I've drunk

Re: [HACKERS] patch: fix race in SSI's CheckTargetForConflictsIn

2011-05-05 Thread Kevin Grittner
Dan Ports d...@csail.mit.edu wrote: While running some benchmarks to test SSI performance, I found a race condition that's capable of causing a segfault. A patch is attached. The bug is in CheckTargetForConflictsIn, which scans the list of SIREAD locks on a lock target when it's modified.

Re: [HACKERS] Visibility map and hint bits

2011-05-05 Thread Merlin Moncure
On Thu, May 5, 2011 at 11:59 AM, Bruce Momjian br...@momjian.us wrote: There has been a lot of recent discussion about the visibility map (for index-only scans) and hint bits (trying to avoid double-writing a table). I still think a small tqual.c maintained cache of hint bits will effectively

Re: [HACKERS] Visibility map and hint bits

2011-05-05 Thread Bruce Momjian
Merlin Moncure wrote: On Thu, May 5, 2011 at 11:59 AM, Bruce Momjian br...@momjian.us wrote: There has been a lot of recent discussion about the visibility map (for index-only scans) and hint bits (trying to avoid double-writing a table). I still think a small tqual.c maintained cache of

Re: [HACKERS] Comments on system tables and columns

2011-05-05 Thread Bruce Momjian
Alvaro Herrera wrote: Excerpts from Thom Brown's message of lun mar 28 08:14:07 -0300 2011: Hi, I notice that none of the system tables or columns thereof bear any comments. Is this intentional, or an oversight? I would have thought comments would be useful since the column names

Re: [HACKERS] Visibility map and hint bits

2011-05-05 Thread Robert Haas
On Thu, May 5, 2011 at 12:59 PM, Bruce Momjian br...@momjian.us wrote: I wonder if we could fix both of these at the same time.  Once the visibility map is reliable, can we use that to avoid updating the hint bits on all rows on a page? I don't think so. There are two problems: 1. If there

Re: [HACKERS] Visibility map and hint bits

2011-05-05 Thread Merlin Moncure
On Thu, May 5, 2011 at 1:34 PM, Bruce Momjian br...@momjian.us wrote: Merlin Moncure wrote: On Thu, May 5, 2011 at 11:59 AM, Bruce Momjian br...@momjian.us wrote: There has been a lot of recent discussion about the visibility map (for index-only scans) and hint bits (trying to avoid

Re: [HACKERS] Visibility map and hint bits

2011-05-05 Thread Kevin Grittner
Merlin Moncure mmonc...@gmail.com wrote: a small cache that remembers the commit/cancel status of recently seen transactions. How is that different from the head of the clog SLRU? -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your

Re: [HACKERS] new clang report

2011-05-05 Thread Peter Eisentraut
On mån, 2011-05-02 at 14:45 -0400, Tom Lane wrote: So issue here is actually that clang has an option -fmath-errno Indicate that math functions should be treated as updating errno. Really? It looks to me like the issue is that pow() is returning NaN instead of Inf

Re: [HACKERS] Visibility map and hint bits

2011-05-05 Thread Merlin Moncure
On Thu, May 5, 2011 at 2:00 PM, Kevin Grittner kevin.gritt...@wicourts.gov wrote: Merlin Moncure mmonc...@gmail.com wrote: a small cache that remembers the commit/cancel status of recently seen transactions. How is that different from the head of the clog SLRU? several things: *) any slru

Re: [HACKERS] FDW table hints

2011-05-05 Thread Magnus Hagander
On Thu, May 5, 2011 at 19:26, Magnus Hagander mag...@hagander.net wrote: On Thu, May 5, 2011 at 19:22, Tom Lane t...@sss.pgh.pa.us wrote: Magnus Hagander mag...@hagander.net writes: Since I brought it up - a patch along this line? Please don't capitalize foreign table there. Yeah, I was a

[HACKERS] Process wakeups when idle and power consumption

2011-05-05 Thread Peter Geoghegan
There is a general need to have Postgres consume fewer CPU cycles and less power when idle. Until something is done about this, shared hosting providers, particularly those who want to deploy many VM instances with databases, will continue to choose MySQL out of hand. I have quantified the

Re: [HACKERS] new clang report

2011-05-05 Thread Tom Lane
Peter Eisentraut pete...@gmx.net writes: Btw., when you build a simple test program in the default mode, pow() indeed returns Inf on overflow. There appear to be some code generation or optimization problems when it builds the postgres code, because the problem goes away with either -O0 or by

Re: [HACKERS] Process wakeups when idle and power consumption

2011-05-05 Thread Tom Lane
Peter Geoghegan pe...@2ndquadrant.com writes: Attached is the first in what I hope will become a series of patches for reducing power consumption when idle. Cool. This has been on my personal to-do list for awhile, but it keeps on failing to get to the top, so I'm glad to see somebody else

Re: [HACKERS] Process wakeups when idle and power consumption

2011-05-05 Thread Alvaro Herrera
Excerpts from Peter Geoghegan's message of jue may 05 16:49:25 -0300 2011: I'll need to take a look at statistics, autovacuum and Logger processes too, to see if they present more subtle opportunities for reduced idle power consumption. More subtle? Autovacuum wakes up once per second and it

Re: [HACKERS] Process wakeups when idle and power consumption

2011-05-05 Thread A.M.
On May 5, 2011, at 4:08 PM, Alvaro Herrera wrote: Excerpts from Peter Geoghegan's message of jue may 05 16:49:25 -0300 2011: I'll need to take a look at statistics, autovacuum and Logger processes too, to see if they present more subtle opportunities for reduced idle power consumption.

Re: [HACKERS] Process wakeups when idle and power consumption

2011-05-05 Thread Robert Haas
On Thu, May 5, 2011 at 4:05 PM, Tom Lane t...@sss.pgh.pa.us wrote: +              * The caveat about signals invalidating the timeout of +              * WaitLatch() on some platforms can be safely disregarded, Really? I'm a bit confused by the phrasing of this comment as well, but it does

[HACKERS] Re: [BUGS] BUG #5957: createdb with description and md5 auth forces to provide password twice

2011-05-05 Thread Bruce Momjian
Grzegorz Szpetkowski wrote: The following bug has been logged online: Bug reference: 5957 Logged by: Grzegorz Szpetkowski Email address: gszpetkow...@gmail.com PostgreSQL version: 9.0.3 Operating system: Ubuntu 10.10 Description:createdb with description and

Re: [HACKERS] Process wakeups when idle and power consumption

2011-05-05 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Thu, May 5, 2011 at 4:05 PM, Tom Lane t...@sss.pgh.pa.us wrote: + * The caveat about signals invalidating the timeout of + * WaitLatch() on some platforms can be safely disregarded, Really? I'm a bit confused by the

Re: [HACKERS] Re: [BUGS] BUG #5957: createdb with description and md5 auth forces to provide password twice

2011-05-05 Thread Tom Lane
Bruce Momjian br...@momjian.us writes: Prior to PG 8.2, this was necessary to put the comment on the database, but now that we have the shared comment/description table pg_shdescription, this is not necessary. Do we need createdb to be able to create databases for pre-8.2 clusters? If not,

Re: [HACKERS] Re: [BUGS] BUG #5957: createdb with description and md5 auth forces to provide password twice

2011-05-05 Thread Bruce Momjian
Tom Lane wrote: Bruce Momjian br...@momjian.us writes: Prior to PG 8.2, this was necessary to put the comment on the database, but now that we have the shared comment/description table pg_shdescription, this is not necessary. Do we need createdb to be able to create databases for

Re: [HACKERS] Backpatching of Teach the regular expression functions to do case-insensitive matching

2011-05-05 Thread Tom Lane
Andres Freund and...@anarazel.de writes: In my opinion this is actually a bug in 9.0. As its a (imo) low impact fix thats constrained to two files it seems sensible to backpatch it now that the solution has proven itself in the field? FWIW, I still don't trust that patch a lot (and I was

Re: [HACKERS] Process wakeups when idle and power consumption

2011-05-05 Thread Peter Geoghegan
On 5 May 2011 22:22, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Thu, May 5, 2011 at 4:05 PM, Tom Lane t...@sss.pgh.pa.us wrote: +              * The caveat about signals invalidating the timeout of +              * WaitLatch() on some platforms can be

Re: [HACKERS] Some surprising precedence behavior in PG's grammar

2011-05-05 Thread Tom Lane
I wrote: Andrew Dunstan and...@dunslane.net writes: If we do need a precedence setting for NULL_P, then I think it should probably be on its own and not sharing one with IS. Yeah, I was thinking that too. If we put %prec on the IS [NOT] NULL productions then there is no need for NULL_P to

Re: [HACKERS] Process wakeups when idle and power consumption

2011-05-05 Thread Tom Lane
Peter Geoghegan pe...@2ndquadrant.com writes: On 5 May 2011 22:22, Tom Lane t...@sss.pgh.pa.us wrote: What that really means is that any WaitOnLatch call with a finite timeout ought to be viewed with a jaundiced eye.  Ideally, we want them all to be waiting for latch release and nothing else.  

Re: [HACKERS] Backpatching of Teach the regular expression functions to do case-insensitive matching

2011-05-05 Thread Robert Haas
On Thu, May 5, 2011 at 5:21 AM, Andres Freund and...@anarazel.de wrote: In my opinion this is actually a bug in 9.0. As its a (imo) low impact fix thats constrained to two files it seems sensible to backpatch it now that the solution has proven itself in the field? The issue is hard to find

[HACKERS] Why is RegisterPredicateLockingXid called while holding XidGenLock?

2011-05-05 Thread Tom Lane
Inquiring minds want to know ... This seems like a pretty lousy place to do it, first because of the contention hit from holding that high-traffic lock any longer than necessary, and second because every added chance for error between ExtendCLOG() and

Re: [HACKERS] clog_redo causing very long recovery time

2011-05-05 Thread Tom Lane
Joseph Conway m...@joeconway.com writes: I'm working with a client that uses Postgres on what amounts to an appliance. The database is therefore subject to occasional torture such as, in this particular case, running out of disk space while performing a million plus queries (of mixed

Re: [HACKERS] clog_redo causing very long recovery time

2011-05-05 Thread Alvaro Herrera
Excerpts from Tom Lane's message of vie may 06 00:22:43 -0300 2011: I think a more elegant fix might be to just swap the order of the ExtendCLOG and ExtendSUBTRANS calls in GetNewTransactionId. The reason that would help is that pg_subtrans isn't WAL-logged, so if we succeed doing

Re: [HACKERS] clog_redo causing very long recovery time

2011-05-05 Thread Joe Conway
On 05/05/2011 08:22 PM, Tom Lane wrote: Joseph Conway m...@joeconway.com writes: The attached fix-clogredo diff is my proposal for a fix for this. That seems pretty grotty :-( I think a more elegant fix might be to just swap the order of the ExtendCLOG and ExtendSUBTRANS calls in

Re: [HACKERS] clog_redo causing very long recovery time

2011-05-05 Thread Tom Lane
Joe Conway m...@joeconway.com writes: Right -- I think another similar problem exists in GetNewMultiXactId where ExtendMultiXactOffset could succeed and write an XLOG entry and then ExtendMultiXactMember could fail before advancing nextMXact. The problem in this case is that they both write

[HACKERS] Re: Why is RegisterPredicateLockingXid called while holding XidGenLock?

2011-05-05 Thread Dan Ports
On Thu, May 05, 2011 at 11:12:40PM -0400, Tom Lane wrote: Even if it's actually necessary to set up that data structure while holding XidGenLock, I would *really* like the call to not be exactly where it is. Good question. I don't believe it needs to be while XidGenLock is being held at all;

[HACKERS] Re: Why is RegisterPredicateLockingXid called while holding XidGenLock?

2011-05-05 Thread Kevin Grittner
Tom Lane wrote: This seems like a pretty lousy place to do it, first because of the contention hit from holding that high-traffic lock any longer than necessary, and second because every added chance for error between ExtendCLOG() and TransactionIdAdvance(ShmemVariableCache-nextXid) gives

Re: [HACKERS] Why is RegisterPredicateLockingXid called while holding XidGenLock?

2011-05-05 Thread Tom Lane
Dan Ports d...@csail.mit.edu writes: On Thu, May 05, 2011 at 11:12:40PM -0400, Tom Lane wrote: Even if it's actually necessary to set up that data structure while holding XidGenLock, I would *really* like the call to not be exactly where it is. Good question. I don't believe it needs to be

[HACKERS] Re: Why is RegisterPredicateLockingXid called while holding XidGenLock?

2011-05-05 Thread Kevin Grittner
Tom Lane wrote: Yeah, I was thinking that it'd be better to pull it out of GetNewTransactionId and put it in a higher level. As long as it is always called when an xid is assigned. Since this function appears to be on the only path to that, it should be fine. No strong preference about

[HACKERS] Debug contrib/cube code

2011-05-05 Thread Nick Raj
Hi, I am using postgresql-8.4.6. I want to debug the contrib/cube code. Can we able to debug that cube code? Because there is no .configure file to enable debug. Is there is any way to change make file to enable debug? Thanks