Re: [HACKERS] Group commit, revised

2012-01-30 Thread Heikki Linnakangas
On 31.01.2012 01:35, Simon Riggs wrote: The plan here is to allow WAL flush and clog updates to occur concurrently. Which allows the clog contention and update time to be completely hidden behind the wait for the WAL flush. That is only possible if we have the WALwriter involved since we need two

Re: [HACKERS] Speed dblink using alternate libpq tuple storage

2012-01-30 Thread Marko Kreen
On Tue, Jan 31, 2012 at 4:59 AM, Kyotaro HORIGUCHI wrote: > This is fixed version of dblink.c for row processor. > >> i'll re-send the properly fixed patch for dblink.c later. > > - malloc error in initStoreInfo throws ERRCODE_OUT_OF_MEMORY. (new error) > > - storeHandler() now returns FALSE on ma

Re: [HACKERS] patch for parallel pg_dump

2012-01-30 Thread Joachim Wieland
On Mon, Jan 30, 2012 at 12:20 PM, Robert Haas wrote: > But the immediate problem is that pg_dump.c is heavily reliant on > global variables, which isn't going to fly if we want this code to use > threads on Windows (or anywhere else).  It's also bad style. Technically, since most of pg_dump.c dum

Re: [HACKERS] [GENERAL] Why extract( ... from timestamp ) is not immutable?

2012-01-30 Thread Tom Lane
Josh Berkus writes: > On 1/30/12 5:41 PM, Tom Lane wrote: >> Well, the current marking is clearly incorrect. What to do about that >> is a bit less clear --- should we downgrade the marking, or change the >> function's behavior so that it really is immutable? > AFAIK, the only case which is NOT

Re: [HACKERS] [GENERAL] pg_dump -s dumps data?!

2012-01-30 Thread Tom Lane
[ Note: please follow-up to pgsql-hackers not pgsql-general; I think this discussion needs to move there ] hubert depesz lubaczewski writes: > On Mon, Jan 30, 2012 at 11:30:51AM -0500, Tom Lane wrote: >> That is way too vague for my taste, as you have not shown the pg_dump >> options you're usi

Re: [HACKERS] Confusing EXPLAIN output in case of inherited tables

2012-01-30 Thread Ashutosh Bapat
I don't believe that the problem has anything to do with the names of > other tables that might happen to exist in the database. It's a matter > of what RTE names/aliases are exposed for variable references in > different parts of the query. > > Names of other tables come into picture when we sch

Re: [HACKERS] [GENERAL] Why extract( ... from timestamp ) is not immutable?

2012-01-30 Thread Josh Berkus
On 1/30/12 5:41 PM, Tom Lane wrote: > hubert depesz lubaczewski writes: >> On Mon, Jan 30, 2012 at 10:35:21AM -0800, Josh Berkus wrote: >>> We can't have functions which are immutable or not depending on their >>> inputs. That way lies madness. > >> but this is exactly what's happening now. > >

Re: [HACKERS] Speed dblink using alternate libpq tuple storage

2012-01-30 Thread Kyotaro HORIGUCHI
This is fixed version of dblink.c for row processor. > i'll re-send the properly fixed patch for dblink.c later. - malloc error in initStoreInfo throws ERRCODE_OUT_OF_MEMORY. (new error) - storeHandler() now returns FALSE on malloc failure. Garbage cleanup is done in dblink_fetch() or dblink_r

Re: [HACKERS] [GENERAL] Why extract( ... from timestamp ) is not immutable?

2012-01-30 Thread Tom Lane
hubert depesz lubaczewski writes: > On Mon, Jan 30, 2012 at 10:35:21AM -0800, Josh Berkus wrote: >> We can't have functions which are immutable or not depending on their >> inputs. That way lies madness. > but this is exactly what's happening now. Well, the current marking is clearly incorrect.

Re: [HACKERS] patch for parallel pg_dump

2012-01-30 Thread Tom Lane
Robert Haas writes: > But the immediate problem is that pg_dump.c is heavily reliant on > global variables, which isn't going to fly if we want this code to use > threads on Windows (or anywhere else). It's also bad style. So I > suggest that we refactor pg_dump.c to get rid of g_conn and g_fout

Re: [HACKERS] foreign key locks, 2nd attempt

2012-01-30 Thread Noah Misch
On Tue, Jan 24, 2012 at 03:47:16PM -0300, Alvaro Herrera wrote: > The biggest item remaining is the point you raised about multixactid > wraparound. This is closely related to multixact truncation and the way > checkpoints are to be handled. If we think that MultiXactId wraparound > is possible,

Re: [HACKERS] Group commit, revised

2012-01-30 Thread Simon Riggs
On Mon, Jan 30, 2012 at 8:04 PM, Heikki Linnakangas wrote: > So, what's the approach you're working on? I've had a few days leave at end of last week, so no time to fully discuss the next steps with the patch. That's why you were requested not to commit anything. You've suggested there was no r

Re: [HACKERS] [COMMITTERS] pgsql: Make group commit more effective.

2012-01-30 Thread Heikki Linnakangas
On 30.01.2012 23:06, Heikki Linnakangas wrote: On 30.01.2012 22:50, Heikki Linnakangas wrote: On 30.01.2012 20:27, Robert Haas wrote: Either this patch, or something else committed this morning, is causing "make check" to hang or run extremely slowly for me. I think it's this patch, because I a

Re: [HACKERS] Patch pg_is_in_backup()

2012-01-30 Thread Euler Taveira de Oliveira
On 30-01-2012 15:33, Gilles Darold wrote: > Yesterday I was facing a little issue with a backup software (NetBackup) > that do not report error when a post backup script is run. The problem > is that this script execute pg_stop_backup() and if there's any failure > PostgreSQL keeps running in on-li

Re: [HACKERS] [COMMITTERS] pgsql: Make group commit more effective.

2012-01-30 Thread Heikki Linnakangas
On 30.01.2012 22:50, Heikki Linnakangas wrote: On 30.01.2012 20:27, Robert Haas wrote: Either this patch, or something else committed this morning, is causing "make check" to hang or run extremely slowly for me. I think it's this patch, because I attached to a backend and stopped it a few times,

Re: [HACKERS] [COMMITTERS] pgsql: Make group commit more effective.

2012-01-30 Thread Heikki Linnakangas
On 30.01.2012 20:27, Robert Haas wrote: Either this patch, or something else committed this morning, is causing "make check" to hang or run extremely slowly for me. I think it's this patch, because I attached to a backend and stopped it a few times, and all the backtraces look like this: Yeah,

Re: [HACKERS] CLOG contention, part 2

2012-01-30 Thread Robert Haas
On Fri, Jan 27, 2012 at 8:21 PM, Jeff Janes wrote: > On Fri, Jan 27, 2012 at 3:16 PM, Merlin Moncure wrote: >> On Fri, Jan 27, 2012 at 4:05 PM, Jeff Janes wrote: >>> Also, I think the general approach is wrong.  The only reason to have >>> these pages in shared memory is that we can control acce

Re: [HACKERS] Group commit, revised

2012-01-30 Thread Heikki Linnakangas
On 30.01.2012 21:55, Simon Riggs wrote: On Mon, Jan 30, 2012 at 3:32 PM, Heikki Linnakangas wrote: On 30.01.2012 17:18, Simon Riggs wrote: Peter and I have been working on a new version that seems likely to improve performance over your suggestions. We should be showing something soon. Pl

Re: [HACKERS] Group commit, revised

2012-01-30 Thread Simon Riggs
On Mon, Jan 30, 2012 at 3:32 PM, Heikki Linnakangas wrote: > On 30.01.2012 17:18, Simon Riggs wrote: >> I asked clearly and specifically for you to hold back committing >> anything. Not sure why you would ignore that and commit without >> actually asking myself or Peter. On a point of principle a

Re: [HACKERS] [GENERAL] Why extract( ... from timestamp ) is not immutable?

2012-01-30 Thread hubert depesz lubaczewski
On Mon, Jan 30, 2012 at 10:35:21AM -0800, Josh Berkus wrote: > > > preferably I would see extract( epoch from timestamp ) to be really > > immutable, i.e. (in my opinion) it should treat incoming data as UTC > > - for epoch calculation. > > Alternatively - perhaps epoch extraction should be moved

Re: [HACKERS] [GENERAL] Why extract( ... from timestamp ) is not immutable?

2012-01-30 Thread Josh Berkus
> preferably I would see extract( epoch from timestamp ) to be really > immutable, i.e. (in my opinion) it should treat incoming data as UTC > - for epoch calculation. > Alternatively - perhaps epoch extraction should be moved to specialized > function, which would have swapped mutability: We can

[HACKERS] Patch pg_is_in_backup()

2012-01-30 Thread Gilles Darold
Hello, Yesterday I was facing a little issue with a backup software (NetBackup) that do not report error when a post backup script is run. The problem is that this script execute pg_stop_backup() and if there's any failure PostgreSQL keeps running in on-line backup mode. So the backup is not compl

Re: [HACKERS] [COMMITTERS] pgsql: Make group commit more effective.

2012-01-30 Thread Robert Haas
On Mon, Jan 30, 2012 at 9:55 AM, Heikki Linnakangas wrote: > Make group commit more effective. > > When a backend needs to flush the WAL, and someone else is already flushing > the WAL, wait until it releases the WALInsertLock and check if we still need > to do the flush or if the other backend al

Re: [HACKERS] [v9.2] Add GUC sepgsql.client_label

2012-01-30 Thread Robert Haas
On Sun, Jan 29, 2012 at 7:27 AM, Kohei KaiGai wrote: > 2012/1/28 Kohei KaiGai : >> 2012/1/26 Robert Haas : >>> On Thu, Jan 26, 2012 at 2:07 PM, Kohei KaiGai wrote: 2012/1/26 Robert Haas : > I'm wondering if a function would be a better fit than a GUC.  I don't > think you can really

Re: [HACKERS] Speed dblink using alternate libpq tuple storage

2012-01-30 Thread Marko Kreen
On Mon, Jan 30, 2012 at 06:06:57PM +0900, Kyotaro HORIGUCHI wrote: > The gain of performance is more than expected. Measure script now > does query via dblink ten times for stability of measuring, so > the figures become about ten times longer than the previous ones. > >sec

Re: [HACKERS] Hot standby off of hot standby?

2012-01-30 Thread Josh Berkus
On 1/29/12 8:36 PM, Igor Schtein wrote: > Hi, > > Is it possible to use a standby instance as a master/primary for another > standby in Postgres 9.0? In other words, does PG 9.0 supports cascading > standby configuration? No, that's a 9.2 feature in development. If you can build PostgreSQL from

Re: [HACKERS] [GENERAL] Why extract( ... from timestamp ) is not immutable?

2012-01-30 Thread hubert depesz lubaczewski
On Wed, Jan 25, 2012 at 11:30:49AM -0500, Tom Lane wrote: > hubert depesz lubaczewski writes: > > anyway - the point is that in \df date_part(, timestamp) says it's > > immutable, while it is not. > > Hmm, you're right. I thought we'd fixed that way back when, but > obviously not. Or maybe the

Re: [HACKERS] Simulating Clog Contention

2012-01-30 Thread Robert Haas
On Mon, Jan 30, 2012 at 10:48 AM, Jeff Janes wrote: > On Mon, Jan 30, 2012 at 7:24 AM, Robert Haas wrote: >> On Sat, Jan 28, 2012 at 3:32 PM, Jeff Janes wrote: I think that even in normal (non-initialization) usage, this message should be suppressed when the provided scale factor

Re: [HACKERS] Simulating Clog Contention

2012-01-30 Thread Robert Haas
On Mon, Jan 30, 2012 at 10:53 AM, Jeff Janes wrote: > On Thu, Jan 26, 2012 at 6:18 AM, Abhijit Menon-Sen wrote: >> At 2012-01-12 12:31:20 +, si...@2ndquadrant.com wrote: >>> >>> The following patch adds a pgbench option -I to load data using >>> INSERTs >> >> This is just to confirm that the

Re: [HACKERS] patch for parallel pg_dump

2012-01-30 Thread Robert Haas
On Sun, Jan 29, 2012 at 12:00 PM, Tom Lane wrote: > Joachim Wieland writes: >> I know that you took back some of your comments, but I'm with you >> here. Archive is allocated as an ArchiveHandle and then casted back to >> Archive*, so you always know that an Archive is an ArchiveHandle. I'm >> al

Re: [HACKERS] Confusing EXPLAIN output in case of inherited tables

2012-01-30 Thread Tom Lane
Ashutosh Bapat writes: > So, as I understand we have two problems here > 1. Prefixing schemaname to the fake alises if there is another RTE with > same name. There may not be a relation with that name (fake alias name > given) in the schema chosen as prefix. > 2. Fake aliases themselves can be con

Re: [HACKERS] Group commit, revised

2012-01-30 Thread Jeff Janes
On Sun, Jan 29, 2012 at 1:20 PM, Greg Smith wrote: > On 01/28/2012 07:48 PM, Jeff Janes wrote: >> > >> I haven't inspected that deep fall off at 30 clients for the patch. >> By way of reference, if I turn off synchronous commit, I get >> tps=1245.8 which is 100% CPU limited.  This sets an theoreti

Re: [HACKERS] Simulating Clog Contention

2012-01-30 Thread Jeff Janes
On Thu, Jan 26, 2012 at 6:18 AM, Abhijit Menon-Sen wrote: > At 2012-01-12 12:31:20 +, si...@2ndquadrant.com wrote: >> >> The following patch adds a pgbench option -I to load data using >> INSERTs > > This is just to confirm that the patch applies and builds and works > fine (though of course i

Re: [HACKERS] Simulating Clog Contention

2012-01-30 Thread Jeff Janes
On Mon, Jan 30, 2012 at 7:24 AM, Robert Haas wrote: > On Sat, Jan 28, 2012 at 3:32 PM, Jeff Janes wrote: >>> I think that even in normal (non-initialization) usage, this message >>> should be suppressed when the provided scale factor >>> is equal to the pgbench_branches table count. >> >> The att

Re: [HACKERS] JSON for PG 9.2

2012-01-30 Thread Andrew Dunstan
On 01/30/2012 09:54 AM, Abhijit Menon-Sen wrote: At 2012-01-27 09:47:05 +0530, a...@toroid.org wrote: I've started reviewing this patch, but it'll take me a bit longer to go through json.c properly. OK, I finished reading json.c. I don't have an answer to the detoasting question in the XXX co

Re: [HACKERS] Group commit, revised

2012-01-30 Thread Heikki Linnakangas
On 30.01.2012 17:18, Simon Riggs wrote: On Mon, Jan 30, 2012 at 2:57 PM, Heikki Linnakangas wrote: I committed this. I ran pgbench test on an 8-core box and didn't see any slowdown. It would still be good if you get a chance to rerun the bigger test, but I feel confident that there's no measu

Re: [HACKERS] Group commit, revised

2012-01-30 Thread Simon Riggs
On Mon, Jan 30, 2012 at 2:57 PM, Heikki Linnakangas wrote: > I committed this. I ran pgbench test on an 8-core box and didn't see any > slowdown. It would still be good if you get a chance to rerun the bigger > test, but I feel confident that there's no measurable slowdown. I asked clearly and s

Re: [HACKERS] 16-bit page checksums for 9.2

2012-01-30 Thread Robert Haas
On Fri, Jan 27, 2012 at 4:07 PM, Dan Scales wrote: > The advantage of putting the checksum calculation in smgrwrite() (or > mdwrite()) is that it catches a bunch of page writes that don't go through > the buffer pool (see calls to smgrwrite() in nbtree.c, nbtsort.c, spginsert.c) Maybe we should

Re: [HACKERS] Simulating Clog Contention

2012-01-30 Thread Robert Haas
On Sat, Jan 28, 2012 at 3:32 PM, Jeff Janes wrote: >> I think that even in normal (non-initialization) usage, this message >> should be suppressed when the provided scale factor >> is equal to the pgbench_branches table count. > > The attached patch does just that.  There is probably no reason to

Re: [HACKERS] Configuring Postgres to Add A New Source File

2012-01-30 Thread Robert Haas
On Fri, Jan 27, 2012 at 6:37 PM, Tareq Aljabban wrote: > Indeed, I'm a beginner in "Make", but I read few tutorials and was able to > do what I wanted outside of PG using a simple make file. > Now, when moving to PG, I found the Make structure much more complicated and > didn't know where to add m

Re: [HACKERS] Multithread Query Planner

2012-01-30 Thread Robert Haas
On Fri, Jan 27, 2012 at 2:56 PM, Pierre C wrote: >> Right.  I think it makes more sense to try to get parallelism working >> first with the infrastructure we have.  Converting to use threading, >> if we ever do it at all, should be something we view as a later >> performance optimization.  But I s

Re: [HACKERS] Group commit, revised

2012-01-30 Thread Heikki Linnakangas
On 27.01.2012 15:38, Robert Haas wrote: On Fri, Jan 27, 2012 at 8:35 AM, Heikki Linnakangas wrote: Yeah, we have to be careful with any overhead in there, it can be a hot spot. I wouldn't expect any measurable difference from the above, though. Could I ask you to rerun the pgbench tests you di

Re: [HACKERS] JSON for PG 9.2

2012-01-30 Thread Abhijit Menon-Sen
At 2012-01-27 09:47:05 +0530, a...@toroid.org wrote: > > I've started reviewing this patch, but it'll take me a bit longer to go > through json.c properly. OK, I finished reading json.c. I don't have an answer to the detoasting question in the XXX comment, but the code looks fine. Aside: is query

[HACKERS] Hot standby off of hot standby?

2012-01-30 Thread Igor Schtein
Hi, Is it possible to use a standby instance as a master/primary for another standby in Postgres 9.0? In other words, does PG 9.0 supports cascading standby configuration? Thanks, Igor

Re: [HACKERS] Speed dblink using alternate libpq tuple storage

2012-01-30 Thread horiguchi . kyotaro
I'm sorry. > Thank you for comments, this is revised version of the patch. The malloc error handling in dblink.c of the patch is broken. It is leaking memory and breaking control. i'll re-send the properly fixed patch for dblink.c later. # This severe back pain should have made me stupid :-p r

[HACKERS]

2012-01-30 Thread horiguchi . kyotaro
I'm sorry. > Thank you for comments, this is revised version of the patch. The malloc error handling in dblink.c of the patch is broken. It is leaking memory and breaking control. i'll re-send the properly fixed patch for dblink.c later. # This severe back pain should have made me stupid :-p r

Re: [HACKERS] Confusing EXPLAIN output in case of inherited tables

2012-01-30 Thread Ashutosh Bapat
Thanks Tom for giving a stronger case. I found the problem whille looking at inherited tables, and didn't think beyond inherited tables. Since inherited tables are expanded when subquery planner is invoked, I thought the problem will occur only in Explain output as we won't generate queries, that c

Re: [HACKERS] Remembering bug #6123

2012-01-30 Thread Simon Riggs
On Thu, Jan 12, 2012 at 4:30 PM, Tom Lane wrote: > "Kevin Grittner" writes: >> Tom Lane wrote: >>> Also, what's the point of testing update_ctid?  I don't see that >>> it matters whether the outdate was a delete or an update. > >> The update_ctid code was a carry-over from my old, slightly >> di

Re: [HACKERS] Hot standby off of hot standby?

2012-01-30 Thread Simon Riggs
On Mon, Jan 30, 2012 at 4:36 AM, Igor Schtein wrote: > Is it possible to use a standby instance as a master/primary for another > standby in Postgres 9.0?  In other words, does PG 9.0 supports cascading > standby configuration? No, but 9.2 will support that feature, known as cascading replicatio

Re: [HACKERS] Speed dblink using alternate libpq tuple storage

2012-01-30 Thread Kyotaro HORIGUCHI
Thank you for comments, this is revised version of the patch. The gain of performance is more than expected. Measure script now does query via dblink ten times for stability of measuring, so the figures become about ten times longer than the previous ones. sec% to Origi

Re: [HACKERS] Patch: Allow SQL-language functions to reference parameters by parameter name

2012-01-30 Thread Hitoshi Harada
On Sun, Jan 29, 2012 at 1:08 AM, Matthew Draper wrote: > On 25/01/12 18:37, Hitoshi Harada wrote: >>> I'm still not sure whether to just revise (almost) all the SQL function >>> examples to use parameter names, and declare them the "right" choice; as >>> it's currently written, named parameters st