[HACKERS] CF3+4 (was Re: Parallel query execution)

2013-01-16 Thread Abhijit Menon-Sen
At 2013-01-16 02:07:29 -0500, t...@sss.pgh.pa.us wrote: In case you hadn't noticed, we've totally lost control of the CF process. What can we do to get it back on track? I know various people (myself included) have been trying to keep CF3 moving, e.g. sending followup mail, adjusting patch

Re: [HACKERS] passing diff options to pg_regress

2013-01-16 Thread Jeevan Chalke
Hi Peter, Idea is really very good. About the patch: Patch looks good to me. Applied cleanly on latest sources. make / make install / make check / initdb everything works well. Tested with few options and it is working well. However, I think you need to add this in docs. Letting people know

Re: [HACKERS] Parallel query execution

2013-01-16 Thread Daniel Farina
On Tue, Jan 15, 2013 at 11:07 PM, Tom Lane t...@sss.pgh.pa.us wrote: Alvaro Herrera alvhe...@2ndquadrant.com writes: There are still 34 items needing attention in CF3. I suggest that, if you have some spare time, your help would be very much appreciated there. The commitfest that started on

Re: [HACKERS] system administration functions with hardcoded superuser checks

2013-01-16 Thread Kohei KaiGai
2013/1/15 Peter Eisentraut pete...@gmx.net: On 12/18/12 12:09 PM, Peter Eisentraut wrote: There are some system administration functions that have hardcoded superuser checks, specifically: pg_reload_conf pg_rotate_logfile pg_read_file pg_read_file_all pg_read_binary_file

Re: Review of pg_basebackup and pg_receivexlog to use non-blocking socket communication, was: Re: [HACKERS] Re: [BUGS] BUG #7534: walreceiver takes long time to detect n/w breakdown

2013-01-16 Thread Heikki Linnakangas
On 07.01.2013 16:23, Boszormenyi Zoltan wrote: Since my other patch against pg_basebackup is now committed, this patch doesn't apply cleanly, patch rejects 2 hunks. The fixed up patch is attached. Now that I look at this a high-level perspective, why are we only worried about timeouts in the

Re: [HACKERS] Parallel query execution

2013-01-16 Thread Magnus Hagander
On Wed, Jan 16, 2013 at 12:03 AM, Bruce Momjian br...@momjian.us wrote: On Tue, Jan 15, 2013 at 11:01:04PM +, Simon Riggs wrote: On 15 January 2013 22:55, Bruce Momjian br...@momjian.us wrote: Why is this being discussed now? It is for 9.4 and will take months. I didn't think there

[HACKERS] CF Progress or the lack thereof

2013-01-16 Thread Andres Freund
On 2013-01-16 02:07:29 -0500, Tom Lane wrote: Alvaro Herrera alvhe...@2ndquadrant.com writes: There are still 34 items needing attention in CF3. I suggest that, if you have some spare time, your help would be very much appreciated there. The commitfest that started on Jan 15th has 65

Re: [HACKERS] CF3+4 (was Re: Parallel query execution)

2013-01-16 Thread Magnus Hagander
On Wed, Jan 16, 2013 at 9:21 AM, Abhijit Menon-Sen a...@2ndquadrant.com wrote: At 2013-01-16 02:07:29 -0500, t...@sss.pgh.pa.us wrote: In case you hadn't noticed, we've totally lost control of the CF process. What can we do to get it back on track? Not sure. One start might be to actually

Re: [HACKERS] CF3+4 (was Re: Parallel query execution)

2013-01-16 Thread Simon Riggs
On 16 January 2013 08:21, Abhijit Menon-Sen a...@2ndquadrant.com wrote: At 2013-01-16 02:07:29 -0500, t...@sss.pgh.pa.us wrote: In case you hadn't noticed, we've totally lost control of the CF process. What can we do to get it back on track? Totally lost control is an overstatement. The

Re: [HACKERS] Curious buildfarm failures (fwd)

2013-01-16 Thread Andres Freund
Hi, On 2013-01-16 01:28:09 -0500, Tom Lane wrote: It's a compiler bug. Gah. Not again. Not that I am surprised, but still. icc 11.1 apparently thinks that this loop in doPickSplit: (Why does it think it needs to prefetch an array it's only going to write into? Is IA64's cache hardware

Re: [HACKERS] CF3+4 (was Re: Parallel query execution)

2013-01-16 Thread Magnus Hagander
On Wed, Jan 16, 2013 at 1:12 PM, Simon Riggs si...@2ndquadrant.com wrote: On 16 January 2013 08:21, Abhijit Menon-Sen a...@2ndquadrant.com wrote: At 2013-01-16 02:07:29 -0500, t...@sss.pgh.pa.us wrote: In case you hadn't noticed, we've totally lost control of the CF process. What can we do

[HACKERS] I broke pg_upgrade for GiST

2013-01-16 Thread Heikki Linnakangas
I just realized that my patch that turned XLogRecPtr into a uint64 changed the on-disk format of GiST indexes, because the NSN field in the page header is an XLogRecPtr. Oops. Fortunately that's easy to fix. I avoided the same issue with LSNs by continuing to use the old two-field struct in

Re: [HACKERS] Parallel query execution

2013-01-16 Thread Robert Haas
On Wed, Jan 16, 2013 at 2:07 AM, Tom Lane t...@sss.pgh.pa.us wrote: Alvaro Herrera alvhe...@2ndquadrant.com writes: There are still 34 items needing attention in CF3. I suggest that, if you have some spare time, your help would be very much appreciated there. The commitfest that started on

Re: [HACKERS] Parallel query execution

2013-01-16 Thread Robert Haas
On Wed, Jan 16, 2013 at 6:52 AM, Magnus Hagander mag...@hagander.net wrote: On Wed, Jan 16, 2013 at 12:03 AM, Bruce Momjian br...@momjian.us wrote: On Tue, Jan 15, 2013 at 11:01:04PM +, Simon Riggs wrote: On 15 January 2013 22:55, Bruce Momjian br...@momjian.us wrote: Why is this being

Re: [HACKERS] CF3+4 (was Re: Parallel query execution)

2013-01-16 Thread Abhijit Menon-Sen
At 2013-01-16 13:08:27 +0100, mag...@hagander.net wrote: One start might be to actually start having commitfest managers. (I'm skipping over this point, since Craig's nomination as CF manager is being discussed elsewhere in this thread.) As in it technical works, but it's better to do it in a

Re: [HACKERS] Parallel query execution

2013-01-16 Thread Stephen Frost
* Claudio Freire (klaussfre...@gmail.com) wrote: Well, there's the fault in your logic. It won't be as linear. I really don't see how this has become so difficult to communicate. It doesn't have to be linear. We're currently doing massive amounts of parallel processing by hand using

Re: [HACKERS] Parallel query execution

2013-01-16 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: In case you hadn't noticed, we've totally lost control of the CF process. I concur. Quite aside from the lack of progress on closing CF3, major hackers who should know better are submitting significant new feature patches now, despite our agreement

Re: [HACKERS] Parallel query execution

2013-01-16 Thread Stephen Frost
* Daniel Farina (dan...@heroku.com) wrote: I have been skimming the commitfest application, and unlike some of the previous commitfests a huge number of patches have had review at some point in time, but probably need more...so looking for the red Nobody in the 'reviewers' column probably

Re: [HACKERS] CF3+4 (was Re: Parallel query execution)

2013-01-16 Thread Boszormenyi Zoltan
2013-01-16 14:18 keltezéssel, Abhijit Menon-Sen írta: At 2013-01-16 13:08:27 +0100, mag...@hagander.net wrote: One start might be to actually start having commitfest managers. (I'm skipping over this point, since Craig's nomination as CF manager is being discussed elsewhere in this thread.)

Re: [HACKERS] Event Triggers: adding information

2013-01-16 Thread Robert Haas
On Wed, Jan 9, 2013 at 11:58 AM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: I made some changes to this, and I think the result (attached) is cleaner overall. Now, this review is pretty much unfinished as far as I am concerned; mainly I've been trying to figure out how it all works and

Re: [HACKERS] CF3+4 (was Re: Parallel query execution)

2013-01-16 Thread Alvaro Herrera
Simon Riggs escribió: On 16 January 2013 08:21, Abhijit Menon-Sen a...@2ndquadrant.com wrote: At 2013-01-16 02:07:29 -0500, t...@sss.pgh.pa.us wrote: In case you hadn't noticed, we've totally lost control of the CF process. What can we do to get it back on track? Totally lost

Re: [HACKERS] CF3+4 (was Re: Parallel query execution)

2013-01-16 Thread Stephen Frost
* Alvaro Herrera (alvhe...@2ndquadrant.com) wrote: I would like to nominate Craig Ringer to be independent CF mgr for Jan2013 CF. Seconded. I particularly like the fact that Craig is not already a PG developer, so he's not going to be working on his own patches. So when can he start? :D

Re: [HACKERS] pg_dump transaction's read-only mode

2013-01-16 Thread Gurjeet Singh
On Wed, Jan 16, 2013 at 1:21 AM, Pavan Deolasee pavan.deola...@gmail.comwrote: On Wed, Jan 9, 2013 at 6:42 AM, Gurjeet Singh singh.gurj...@gmail.comwrote: I have updated the commitfest submission to link to the correct patch email. Thanks Gurjeet. I initially thought that this

Re: [HACKERS] Parallel query execution

2013-01-16 Thread Andrew Dunstan
On 01/15/2013 11:32 PM, Bruce Momjian wrote: On Wed, Jan 16, 2013 at 01:28:18PM +0900, Michael Paquier wrote: On Wed, Jan 16, 2013 at 1:22 PM, Josh Berkus j...@agliodbs.com wrote: Claudio, Stephen, It really seems like the areas where we could get the most bang for the buck

Re: [HACKERS] CF3+4

2013-01-16 Thread Abhijit Menon-Sen
At 2013-01-16 09:02:45 -0500, sfr...@snowman.net wrote: So when can he start? :D Also, what should he start with? CF3 as it stands today, or CF4 with all of the pending patches moved from CF3, immense though the result may be? I slightly prefer the latter, so that we're all on the same page

Re: [HACKERS] CF3+4 (was Re: Parallel query execution)

2013-01-16 Thread Craig Ringer
On 01/16/2013 10:02 PM, Stephen Frost wrote: * Alvaro Herrera (alvhe...@2ndquadrant.com) wrote: I would like to nominate Craig Ringer to be independent CF mgr for Jan2013 CF. Seconded. I particularly like the fact that Craig is not already a PG developer, so he's not going to be working on

Re: [HACKERS] CF3+4 (was Re: Parallel query execution)

2013-01-16 Thread Craig Ringer
On 01/16/2013 08:12 PM, Simon Riggs wrote: I would like to nominate Craig Ringer to be independent CF mgr for Jan2013 CF. I'm happy to step up and help out. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via

Re: [HACKERS] pkg-config files for libpq and ecpg

2013-01-16 Thread Michael Meskes
On Tue, Jan 15, 2013 at 02:16:01PM -0500, Peter Eisentraut wrote: A side issue that arose: libecpg_compat is linked with libpq, but doesn't seem to use it. This was added many years ago in cd75f94dafd43358305811b7576ad75d889097e3, but it doesn't appear to be required anymore. Needs some

Re: [HACKERS] CF3+4

2013-01-16 Thread Stephen Frost
* Abhijit Menon-Sen (a...@2ndquadrant.com) wrote: Also, what should he start with? CF3 as it stands today, or CF4 with all of the pending patches moved from CF3, immense though the result may be? I slightly prefer the latter, so that we're all on the same page when it comes to seeing what

Re: [HACKERS] Curious buildfarm failures (fwd)

2013-01-16 Thread Sergey Koposov
Hi, On Wed, 16 Jan 2013, Andres Freund wrote: On 2013-01-16 01:28:09 -0500, Tom Lane wrote: It's a compiler bug. Thanks for investigating. But I'm not sure there is any way other way for me other than switching to gcc, because intel stopped providing their IA64 version of compilers free of

Re: [HACKERS] Curious buildfarm failures (fwd)

2013-01-16 Thread Andres Freund
On 2013-01-16 14:41:47 +, Sergey Koposov wrote: Hi, On Wed, 16 Jan 2013, Andres Freund wrote: On 2013-01-16 01:28:09 -0500, Tom Lane wrote: It's a compiler bug. Thanks for investigating. But I'm not sure there is any way other way for me other than switching to gcc, because intel

Re: [HACKERS] Curious buildfarm failures (fwd)

2013-01-16 Thread Andrew Dunstan
On 01/16/2013 09:41 AM, Sergey Koposov wrote: So unless somebody suggest otherwise, i'm going to switch to gcc on this buildfarm. If you switch compiler it should be a new buildfarm animal. (That just means re-registering so you get a new name/secret pair.) We have provision for upgrading

Re: [HACKERS] Curious buildfarm failures (fwd)

2013-01-16 Thread Sergey Koposov
On Wed, 16 Jan 2013, Andres Freund wrote: So unless somebody suggest otherwise, i'm going to switch to gcc on this buildfarm. What about switching to -O1 of 11.1? I don't know. It is up to -hackers to decide. I think that icc on ia64 have shown bugginess time after time. But if you think

Re: [HACKERS] log_lock_waits to identify transaction's relation

2013-01-16 Thread Simon Riggs
On 16 January 2013 03:47, Stephen Frost sfr...@snowman.net wrote: Simon, * Simon Riggs (si...@2ndquadrant.com) wrote: Attached patch passes through further information about the lock wait, so we can display the following message instead LOG: process %d acquired %s on transaction %u on

Re: [HACKERS] Re: [PATCH] Compile without warning with gcc's -Wtype-limits, -Wempty-body

2013-01-16 Thread Peter Eisentraut
On 1/15/13 6:36 AM, Andres Freund wrote: I just think that the price of fixing a single Assert() that hasn't changed in years where the variable isn't likely to ever get signed is acceptable. Well, once you get past that one change you proposed, you will also find pg_standby.c: In function

Re: [HACKERS] Parallel query execution

2013-01-16 Thread Claudio Freire
On Wed, Jan 16, 2013 at 10:33 AM, Stephen Frost sfr...@snowman.net wrote: * Claudio Freire (klaussfre...@gmail.com) wrote: Well, there's the fault in your logic. It won't be as linear. I really don't see how this has become so difficult to communicate. It doesn't have to be linear. We're

Re: [HACKERS] [sepgsql 1/3] add name qualified creation label

2013-01-16 Thread Robert Haas
On Tue, Jan 15, 2013 at 3:02 PM, Kohei KaiGai kai...@kaigai.gr.jp wrote: This patch adds sepgsql the feature of name qualified creation label. Background, on creation of a certain database object, sepgsql assigns a default security label according to the security policy that has a set of

Re: [HACKERS] json api WIP patch

2013-01-16 Thread Robert Haas
On Mon, Jan 14, 2013 at 11:02 PM, Andrew Dunstan and...@dunslane.net wrote: After a couple of iterations, some performance enhancements to the json parser and lexer have ended up with a net performance improvement over git tip. On our test rig, the json parse test runs at just over 13s per

Re: [HACKERS] Curious buildfarm failures (fwd)

2013-01-16 Thread Tom Lane
Sergey Koposov kopo...@ast.cam.ac.uk writes: On Wed, 16 Jan 2013, Andres Freund wrote: What about switching to -O1 of 11.1? I don't know. It is up to -hackers to decide. I think that icc on ia64 have shown bugginess time after time. But if you think that buildfarm with icc 11.1 -O1 carry

Re: [HACKERS] CF3+4 (was Re: Parallel query execution)

2013-01-16 Thread Noah Misch
On Wed, Jan 16, 2013 at 01:08:27PM +0100, Magnus Hagander wrote: On Wed, Jan 16, 2013 at 9:21 AM, Abhijit Menon-Sen a...@2ndquadrant.com wrote: At 2013-01-16 02:07:29 -0500, t...@sss.pgh.pa.us wrote: In case you hadn't noticed, we've totally lost control of the CF process. What can

Re: [HACKERS] Parallel query execution

2013-01-16 Thread Noah Misch
On Wed, Jan 16, 2013 at 08:42:29AM -0500, Stephen Frost wrote: * Daniel Farina (dan...@heroku.com) wrote: I have been skimming the commitfest application, and unlike some of the previous commitfests a huge number of patches have had review at some point in time, but probably need more...so

Re: [HACKERS] Teaching pg_receivexlog to follow timeline switches

2013-01-16 Thread Heikki Linnakangas
On 15.01.2013 20:22, Fujii Masao wrote: On Tue, Jan 15, 2013 at 11:05 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: Now that a standby server can follow timeline switches through streaming replication, we should do teach pg_receivexlog to do the same. Patch attached. I made one change

Re: [HACKERS] log_lock_waits to identify transaction's relation

2013-01-16 Thread Stephen Frost
* Simon Riggs (si...@2ndquadrant.com) wrote: A couple quick notes regarding the patch- what does GetXactLockTableRelid really provide..? The ability to access a static variable in a different module. It doesn't provide anything other than that, It isn't actually necessary for that

Re: [HACKERS] pg_trgm partial-match

2013-01-16 Thread Fujii Masao
On Fri, Nov 23, 2012 at 2:11 AM, Fujii Masao masao.fu...@gmail.com wrote: On Mon, Nov 19, 2012 at 10:56 AM, Tomas Vondra t...@fuzzy.cz wrote: I've done a quick review of the current patch: Thanks for the commit! As Alexander pointed out upthread, another infrastructure patch is required

Re: [HACKERS] Materialized views WIP patch

2013-01-16 Thread Tom Lane
Kevin Grittner kgri...@mail.com writes: I've been struggling with two areas: - pg_dump sorting for MVs which depend on other MVs Surely that should fall out automatically given that the dependency is properly expressed in pg_depend? If you mean you're trying to get it to cope with circular

Re: [HACKERS] Materialized views WIP patch

2013-01-16 Thread Kevin Grittner
Tom Lane wrote: Kevin Grittner kgri...@mail.com writes: I've been struggling with two areas: - pg_dump sorting for MVs which depend on other MVs Surely that should fall out automatically given that the dependency is properly expressed in pg_depend? If you mean you're trying to get it to

Re: [HACKERS] Teaching pg_receivexlog to follow timeline switches

2013-01-16 Thread Fujii Masao
On Thu, Jan 17, 2013 at 1:08 AM, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 15.01.2013 20:22, Fujii Masao wrote: On Tue, Jan 15, 2013 at 11:05 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: Now that a standby server can follow timeline switches through streaming

Re: [HACKERS] Materialized views WIP patch

2013-01-16 Thread Thom Brown
On 16 January 2013 05:40, Kevin Grittner kgri...@mail.com wrote: Here is a new version of the patch, with most issues discussed in previous posts fixed. I've been struggling with two areas: - pg_dump sorting for MVs which depend on other MVs - proper handling of the relisvalid flag for

Re: [HACKERS] Parallel query execution

2013-01-16 Thread Bruce Momjian
On Wed, Jan 16, 2013 at 01:37:28PM +0900, Michael Paquier wrote: On Wed, Jan 16, 2013 at 1:32 PM, Bruce Momjian br...@momjian.us wrote: On Wed, Jan 16, 2013 at 01:28:18PM +0900, Michael Paquier wrote: On Wed, Jan 16, 2013 at 1:22 PM, Josh Berkus j...@agliodbs.com

Re: [HACKERS] Parallel query execution

2013-01-16 Thread Bruce Momjian
On Wed, Jan 16, 2013 at 01:48:29AM -0300, Alvaro Herrera wrote: Bruce Momjian escribió: On Wed, Jan 16, 2013 at 01:28:18PM +0900, Michael Paquier wrote: On Wed, Jan 16, 2013 at 1:22 PM, Josh Berkus j...@agliodbs.com wrote: Claudio, Stephen, It really seems like

Re: [HACKERS] Parallel query execution

2013-01-16 Thread Bruce Momjian
On Wed, Jan 16, 2013 at 08:11:06AM -0500, Robert Haas wrote: We kind of do - when in a CF we should do reviewing of existing patches, when outside a CF we should do discussions and work on new features. It's on http://wiki.postgresql.org/wiki/CommitFest. It doesn't specifically say do this

Re: [HACKERS] Parallel query execution

2013-01-16 Thread Bruce Momjian
On Wed, Jan 16, 2013 at 09:05:39AM -0500, Andrew Dunstan wrote: On 01/15/2013 11:32 PM, Bruce Momjian wrote: On Wed, Jan 16, 2013 at 01:28:18PM +0900, Michael Paquier wrote: On Wed, Jan 16, 2013 at 1:22 PM, Josh Berkus j...@agliodbs.com wrote: Claudio, Stephen, It really

Re: [HACKERS] Materialized views WIP patch

2013-01-16 Thread Kevin Grittner
Thom Brown wrote: Some weirdness: postgres=# CREATE VIEW v_test2 AS SELECT 1 moo; CREATE VIEW postgres=# CREATE MATERIALIZED VIEW mv_test2 AS SELECT moo, 2*moo FROM v_test2 UNION ALL SELECT moo, 3*moo FROM v_test2; SELECT 2 postgres=# \d+ mv_test2  Materialized view public.mv_test2  

Re: [HACKERS] Materialized views WIP patch

2013-01-16 Thread Thom Brown
On 16 January 2013 17:20, Kevin Grittner kgri...@mail.com wrote: Thom Brown wrote: Some weirdness: postgres=# CREATE VIEW v_test2 AS SELECT 1 moo; CREATE VIEW postgres=# CREATE MATERIALIZED VIEW mv_test2 AS SELECT moo, 2*moo FROM v_test2 UNION ALL SELECT moo, 3*moo FROM v_test2;

Re: [HACKERS] Parallel query execution

2013-01-16 Thread Andrew Dunstan
On 01/16/2013 12:20 PM, Bruce Momjian wrote: On Wed, Jan 16, 2013 at 09:05:39AM -0500, Andrew Dunstan wrote: On 01/15/2013 11:32 PM, Bruce Momjian wrote: On Wed, Jan 16, 2013 at 01:28:18PM +0900, Michael Paquier wrote: On Wed, Jan 16, 2013 at 1:22 PM, Josh Berkus j...@agliodbs.com wrote:

Re: [HACKERS] CF3+4 (was Re: Parallel query execution)

2013-01-16 Thread Josh Berkus
I would like to nominate Craig Ringer to be independent CF mgr for Jan2013 CF. +1, although I'll suggest that we should have *two* CF managers for this one to keep the workload manageable. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list

Re: [HACKERS] CF3+CF4 WAS: Parallel query execution

2013-01-16 Thread Josh Berkus
I assume you know I was the most frequent committer of other people's patches for years before the commit-fests started, so I thought I would move on to other things. Why would you think that? Given the volume of incoming patches, we need more committers than ever. -- Josh Berkus

Re: [HACKERS] CF3+CF4 WAS: Parallel query execution

2013-01-16 Thread Bruce Momjian
On Wed, Jan 16, 2013 at 09:50:07AM -0800, Josh Berkus wrote: I assume you know I was the most frequent committer of other people's patches for years before the commit-fests started, so I thought I would move on to other things. Why would you think that? Given the volume of incoming

Re: [HACKERS] CF3+CF4 WAS: Parallel query execution

2013-01-16 Thread Josh Berkus
Well, I usually do stuff no one wants to do, and it seems we have people doing this. Also, I had my hand in deciding lots of things when I was committing all those patches in the past, so I thought others should get the chance. Well, we clearly don't have *enough* people committing patches.

Re: [HACKERS] Materialized views WIP patch

2013-01-16 Thread Tom Lane
Kevin Grittner kgri...@mail.com writes: Tom Lane wrote: Surely that should fall out automatically given that the dependency is properly expressed in pg_depend? The *definitions* sort properly, but what I'm trying to do is define them WITH NO DATA and load data after all the COPY statements

Re: [HACKERS] Materialized views WIP patch

2013-01-16 Thread Josh Berkus
Do we really need unlogged MVs in the first iteration? Seems like that's adding a whole bunch of new issues, when you have quite enough already without that. While I think there is strong user demand for unlogged MVs, if we can get MVs without unlogged ones for 9.3, I say go for that. We'll

Re: [HACKERS] [PATCH 3/5] Split out xlog reading into its own module called xlogreader

2013-01-16 Thread Alvaro Herrera
Andres Freund wrote: The way xlog reading was done up to now made it impossible to use that nontrivial code outside of xlog.c although it is useful for different purposes like debugging wal (xlogdump) and decoding wal back into logical changes. I have pushed this part after some more

Re: [HACKERS] CF3+4

2013-01-16 Thread Tom Lane
Stephen Frost sfr...@snowman.net writes: * Abhijit Menon-Sen (a...@2ndquadrant.com) wrote: Also, what should he start with? CF3 as it stands today, or CF4 with all of the pending patches moved from CF3, immense though the result may be? I slightly prefer the latter, so that we're all on the

Re: [HACKERS] CF3+4

2013-01-16 Thread Josh Berkus
If we decide to fold CF3 and CF4 together, either we lose that step (which would make me sad, it seems like a good idea) or we need to figure another way to work it into the process. Well, we should have the triage discussion ASAP then. We were really supposed to have it a week ago. --

Re: [HACKERS] [PATCH] COPY .. COMPRESSED

2013-01-16 Thread Peter Eisentraut
On 1/15/13 2:53 PM, Stephen Frost wrote: You're right, to clarify, for *file_fdw*, which is a backend-only operation, the popen patch is great (thought I made that clear before). I would think that if we get writable FDWs, you would want file_fdw to go through zlib so that it can write directly

Re: [HACKERS] Parallel query execution

2013-01-16 Thread Stephen Frost
* Bruce Momjian (br...@momjian.us) wrote: I am not sure how a COPY could be easily parallelized, but I supposed it could be done as part of the 1GB segment feature. People have complained that COPY is CPU-bound, so it might be very interesting to see if we could offload some of that parsing

Re: [HACKERS] [PATCH] COPY .. COMPRESSED

2013-01-16 Thread Stephen Frost
* Peter Eisentraut (pete...@gmx.net) wrote: On 1/15/13 2:53 PM, Stephen Frost wrote: You're right, to clarify, for *file_fdw*, which is a backend-only operation, the popen patch is great (thought I made that clear before). I would think that if we get writable FDWs, you would want file_fdw

Re: [HACKERS] CF3+4

2013-01-16 Thread Tom Lane
Josh Berkus j...@agliodbs.com writes: If we decide to fold CF3 and CF4 together, either we lose that step (which would make me sad, it seems like a good idea) or we need to figure another way to work it into the process. Well, we should have the triage discussion ASAP then. We were really

Re: [HACKERS] CF3+4

2013-01-16 Thread Bruce Momjian
On Wed, Jan 16, 2013 at 03:13:50PM -0500, Tom Lane wrote: I think a realistic answer might be to admit that we've slipped quite a bit. Set the end date of CF3 to perhaps end of January, do triage the first week of February, and then start CF4 after that, about three or four weeks later than

Re: [HACKERS] Parallel query execution

2013-01-16 Thread Pavel Stehule
2013/1/16 Stephen Frost sfr...@snowman.net: * Bruce Momjian (br...@momjian.us) wrote: I am not sure how a COPY could be easily parallelized, but I supposed it could be done as part of the 1GB segment feature. People have complained that COPY is CPU-bound, so it might be very interesting to

Re: [HACKERS] Event Triggers: adding information

2013-01-16 Thread Dimitri Fontaine
Robert Haas robertmh...@gmail.com writes: I think this points to a couple of problems: this patch isn't well-enough thought out, and it's got several features jammed into a single patch. This should really be split up into several patches and each one submitted separately. Ok. Now I want to

Re: [HACKERS] Parallel query execution

2013-01-16 Thread Bruce Momjian
On Wed, Jan 16, 2013 at 10:06:51PM +0100, Pavel Stehule wrote: 2013/1/16 Stephen Frost sfr...@snowman.net: * Bruce Momjian (br...@momjian.us) wrote: I am not sure how a COPY could be easily parallelized, but I supposed it could be done as part of the 1GB segment feature. People have

Re: [HACKERS] Parallel query execution

2013-01-16 Thread Dickson S. Guedes
2013/1/16 Bruce Momjian br...@momjian.us: Wiki updated: https://wiki.postgresql.org/wiki/Parallel_Query_Execution Could we add CTE to that opportunities list? I think that some kind of queries in CTE queries could be easilly parallelized. []s -- Dickson S. Guedes mail/xmpp:

Re: [HACKERS] Event Triggers: adding information

2013-01-16 Thread Robert Haas
On Wed, Jan 16, 2013 at 4:16 PM, Dimitri Fontaine dimi...@2ndquadrant.fr wrote: Ok. Now I want to talk about our process a little. That's a 2 paragraphs diversion, after that it's getting back to technical matters. There's a difference between it's not the way I would have done it and the

Re: [HACKERS] Parallel query execution

2013-01-16 Thread Bruce Momjian
On Wed, Jan 16, 2013 at 07:57:01PM -0200, Dickson S. Guedes wrote: 2013/1/16 Bruce Momjian br...@momjian.us: Wiki updated: https://wiki.postgresql.org/wiki/Parallel_Query_Execution Could we add CTE to that opportunities list? I think that some kind of queries in CTE queries

Re: [HACKERS] CF3+4 (was Re: Parallel query execution)

2013-01-16 Thread Robert Haas
On Wed, Jan 16, 2013 at 12:36 PM, Josh Berkus j...@agliodbs.com wrote: I would like to nominate Craig Ringer to be independent CF mgr for Jan2013 CF. +1, although I'll suggest that we should have *two* CF managers for this one to keep the workload manageable. That has never worked before,

Re: [HACKERS] [PATCH] COPY .. COMPRESSED

2013-01-16 Thread Robert Haas
On Mon, Jan 14, 2013 at 11:28 AM, Stephen Frost sfr...@snowman.net wrote: * Tom Lane (t...@sss.pgh.pa.us) wrote: Stephen Frost sfr...@snowman.net writes: I do like the idea of a generalized answer which just runs a user-provided command on the server but that's always going to require

Re: [HACKERS] [PATCH] COPY .. COMPRESSED

2013-01-16 Thread Robert Haas
On Tue, Jan 15, 2013 at 3:37 PM, Christopher Browne cbbro...@gmail.com wrote: That points towards a fix that involves having a set of non-arbitrary commands that we allow plain users to use. Hmm. There's an interesting thought... How about having a pg_filters table in pg_catalog which

Re: [HACKERS] Teaching pg_receivexlog to follow timeline switches

2013-01-16 Thread Dimitri Fontaine
Fujii Masao masao.fu...@gmail.com writes: Thanks for elaborating the reason why .partial suffix should be kept. I agree that keeping the .partial suffix would be safer. +1 to both points. So +2 I guess :) Regards, -- Dimitri Fontaine http://2ndQuadrant.fr PostgreSQL : Expertise,

Re: [HACKERS] [PATCH] COPY .. COMPRESSED

2013-01-16 Thread Christopher Browne
On Wed, Jan 16, 2013 at 5:15 PM, Robert Haas robertmh...@gmail.com wrote: On Tue, Jan 15, 2013 at 3:37 PM, Christopher Browne cbbro...@gmail.com wrote: That points towards a fix that involves having a set of non-arbitrary commands that we allow plain users to use. Hmm. There's an

Re: [HACKERS] review: pgbench - aggregation of info written into log

2013-01-16 Thread Tatsuo Ishii
Hi, I'm looking into this as a committer. It seems that this is the newest patch and the reviewer(Pavel) stated that it is ready for commit. However, I noticed that this patch adds a Linux/UNIX only feature(not available on Windows). So I would like to ask cores if this is ok or not. -- Tatsuo

Re: [HACKERS] Event Triggers: adding information

2013-01-16 Thread Tom Lane
Dimitri Fontaine dimi...@2ndquadrant.fr writes: There's a difference between it's not the way I would have done it and the author didn't think about what he's doing. That's also the reason why it's very hard to justify sending a polished enough patch as a non commiter. And then this patch is

Re: [HACKERS] [PATCH] COPY .. COMPRESSED

2013-01-16 Thread Robert Haas
On Tue, Jan 15, 2013 at 4:50 PM, Tom Lane t...@sss.pgh.pa.us wrote: I find the argument that this supports compression-over-the-wire to be quite weak, because COPY is only one form of bulk data transfer, and one that a lot of applications don't ever use. If we think we need to support

Re: [HACKERS] [PATCH] COPY .. COMPRESSED

2013-01-16 Thread Stephen Frost
* Bruce Momjian (br...@momjian.us) wrote: On Wed, Jan 16, 2013 at 06:19:09PM -0500, Robert Haas wrote: But presumably this would transparently compress at one end and decompress at the other end, which is again a somewhat different use case. To get compressed output on the client side, you

Re: [HACKERS] Re: Privileges for INFORMATION_SCHEMA.SCHEMATA (was Re: [DOCS] Small clarification in 34.41. schemata)

2013-01-16 Thread Ian Lawrence Barwick
2013/1/15 Tom Lane t...@sss.pgh.pa.us: Casey Allen Shobe ca...@shobe.info writes: On Wed, Jan 9, 2013 at 8:56 PM, Tom Lane t...@sss.pgh.pa.us wrote: However, it seems to me that this behavior is actually wrong for our purposes, as it represents a too-literal reading of the spec. The SQL

Re: [HACKERS] [PATCH] COPY .. COMPRESSED

2013-01-16 Thread Bruce Momjian
On Wed, Jan 16, 2013 at 06:19:09PM -0500, Robert Haas wrote: On Tue, Jan 15, 2013 at 4:50 PM, Tom Lane t...@sss.pgh.pa.us wrote: I find the argument that this supports compression-over-the-wire to be quite weak, because COPY is only one form of bulk data transfer, and one that a lot of

Re: [HACKERS] review: pgbench - aggregation of info written into log

2013-01-16 Thread Andrew Dunstan
On 01/16/2013 05:59 PM, Tatsuo Ishii wrote: Hi, I'm looking into this as a committer. It seems that this is the newest patch and the reviewer(Pavel) stated that it is ready for commit. However, I noticed that this patch adds a Linux/UNIX only feature(not available on Windows). So I would like

Re: [HACKERS] Event Triggers: adding information

2013-01-16 Thread Robert Haas
On Wed, Jan 16, 2013 at 6:14 PM, Tom Lane t...@sss.pgh.pa.us wrote: What was discussed at the last dev meeting was assigning a committer to each large patch to start with, which would reduce the risk of the goalposts moving that way. It seems to me that Robert's at least unofficially taken

Re: [HACKERS] review: pgbench - aggregation of info written into log

2013-01-16 Thread Tatsuo Ishii
I'm looking into this as a committer. It seems that this is the newest patch and the reviewer(Pavel) stated that it is ready for commit. However, I noticed that this patch adds a Linux/UNIX only feature(not available on Windows). So I would like to ask cores if this is ok or not. I

Re: [HACKERS] review: pgbench - aggregation of info written into log

2013-01-16 Thread Andrew Dunstan
On 01/16/2013 06:48 PM, Tatsuo Ishii wrote: I'm looking into this as a committer. It seems that this is the newest patch and the reviewer(Pavel) stated that it is ready for commit. However, I noticed that this patch adds a Linux/UNIX only feature(not available on Windows). So I would like to

Re: [HACKERS] Parallel query execution

2013-01-16 Thread Jeff Janes
On Tuesday, January 15, 2013, Stephen Frost wrote: * Gavin Flower (gavinflo...@archidevsys.co.nz javascript:;) wrote: How about being aware of multiple spindles - so if the requested data covers multiple spindles, then data could be extracted in parallel. This may, or may not, involve

Re: [HACKERS] Parallel query execution

2013-01-16 Thread Jeff Janes
On Tuesday, January 15, 2013, Gavin Flower wrote: On 16/01/13 11:14, Bruce Momjian wrote: I mentioned last year that I wanted to start working on parallelism: https://wiki.postgresql.org/wiki/Parallel_Query_Execution Years ago I added thread-safety to libpq. Recently I added two

Re: [HACKERS] review: pgbench - aggregation of info written into log

2013-01-16 Thread Tatsuo Ishii
It seems instr_time.h on Windows simply does not provide current timestamp. From pgbench.c: /* * if transaction finished, record the time it took in the log */ if (logfile commands[st-state + 1] == NULL) {

Re: [HACKERS] string escaping in tutorial/syscat.source

2013-01-16 Thread Jeff Janes
On Tuesday, January 15, 2013, Josh Kupershmidt wrote: On Tue, Jan 15, 2013 at 6:35 PM, Jeff Janes jeff.ja...@gmail.comjavascript:; wrote: Do you propose back-patching this? You could argue that this is a bug in 9.1 and 9.2. Before that, they generate deprecation warnings, but do not

Re: [HACKERS] review: pgbench - aggregation of info written into log

2013-01-16 Thread Andrew Dunstan
On 01/16/2013 08:05 PM, Tatsuo Ishii wrote: It seems instr_time.h on Windows simply does not provide current timestamp. From pgbench.c: /* * if transaction finished, record the time it took in the log */ if (logfile

Re: [HACKERS] CF3+4

2013-01-16 Thread Simon Riggs
On 16 January 2013 19:28, Tom Lane t...@sss.pgh.pa.us wrote: Stephen Frost sfr...@snowman.net writes: * Abhijit Menon-Sen (a...@2ndquadrant.com) wrote: Also, what should he start with? CF3 as it stands today, or CF4 with all of the pending patches moved from CF3, immense though the result may

Re: [HACKERS] review: pgbench - aggregation of info written into log

2013-01-16 Thread Tatsuo Ishii
This might be way more than we want to do, but there is an article that describes some techniques for doing what seems to be missing (AIUI): http://msdn.microsoft.com/en-us/magazine/cc163996.aspx Even this would be doable, I'm afraid it may not fit in 9.3 if we think about the current

Re: [HACKERS] Enabling Checksums

2013-01-16 Thread Jeff Davis
if the PD_ALL_VISIBLE patch is committed first then it will make reviewing this patch easier. Regardless, the second patch to be committed will need to be rebased on top of the first. Regards, Jeff Davis replace-tli-with-checksums-20130116.patch.gz Description: GNU Zip compressed data

Re: [HACKERS] log_lock_waits to identify transaction's relation

2013-01-16 Thread Simon Riggs
On 16 January 2013 16:12, Stephen Frost sfr...@snowman.net wrote: * Simon Riggs (si...@2ndquadrant.com) wrote: A couple quick notes regarding the patch- what does GetXactLockTableRelid really provide..? The ability to access a static variable in a different module. It doesn't provide

Re: [HACKERS] Parallel query execution

2013-01-16 Thread Claudio Freire
On Wed, Jan 16, 2013 at 10:04 PM, Jeff Janes jeff.ja...@gmail.com wrote: Hmm... How about being aware of multiple spindles - so if the requested data covers multiple spindles, then data could be extracted in parallel. This may, or may not, involve multiple I/O channels?

  1   2   >