Re: Review of "pg_basebackup and pg_receivexlog to use non-blocking socket communication", was: Re: [HACKERS] Re: [BUGS] BUG #7534: walreceiver takes long time to detect n/w breakdown

2013-01-17 Thread Amit Kapila
On Wednesday, January 16, 2013 4:02 PM Heikki Linnakangas wrote: > On 07.01.2013 16:23, Boszormenyi Zoltan wrote: > > Since my other patch against pg_basebackup is now committed, > > this patch doesn't apply cleanly, patch rejects 2 hunks. > > The fixed up patch is attached. > > Now that I look at

Re: [HACKERS] pg_dump --pretty-print-views

2013-01-17 Thread Jeevan Chalke
On Thu, Jan 10, 2013 at 11:07 PM, Andrew Dunstan wrote: > > On 01/10/2013 12:35 PM, Tom Lane wrote: > >> Andrew Dunstan writes: >> >>> I think there's a very good case for breaking the nexus between >>> PRETTYFLAG_PAREN and PRETTYFLAG_INDENT+line wrapping for views. Only >>> PRETTYFLAG_PAREN affe

Re: [HACKERS] Teaching pg_receivexlog to follow timeline switches

2013-01-17 Thread Phil Sorber
On Tue, Jan 15, 2013 at 9:05 AM, Heikki Linnakangas wrote: > Now that a standby server can follow timeline switches through streaming > replication, we should do teach pg_receivexlog to do the same. Patch > attached. Is it possible to re-use walreceiver code from the backend? I was thinking that

[HACKERS] FOR KEY SHARE patch ready to commit?

2013-01-17 Thread Craig Ringer
Alvaro, You said that you intended to commit your FOR KEY SHARE patch, but I don't see it in the commit logs and it's flagged as ready for committer in the CF app. The last discussion I see was between you and Andres, saying that it looks good to commit and that the concurrency tests you wrote sh

[HACKERS] Variadic "any" fix

2013-01-17 Thread Craig Ringer
Committers, The variadic "any" fix has been lurking in the queue for a while. I personally confirmed the bug, and it's one I'd love to see fixed. I just tested the patch out quickly and did a build to verify that it doesn't break Windows. Is there any chance someone can pick it up and commit it f

Re: [HACKERS] Event Triggers: adding information

2013-01-17 Thread Robert Haas
On Thu, Jan 17, 2013 at 8:33 PM, Andres Freund wrote: > I have no problem requiring C code to use the even data, be it via hooks > or via C functions called from event triggers. The problem I have with > putting in some hooks is that I doubt that you can find sensible spots > with enough informati

Re: [HACKERS] Event Triggers: adding information

2013-01-17 Thread Tom Lane
Andres Freund writes: > On 2013-01-17 21:48:01 -0500, Tom Lane wrote: >> If we're only interested in replication, let's put in some hooks whose >> contract does not allow for side-effects on the local catalogs, and be >> done. Otherwise we'll be putting in man-years of unnecessary (or at >> least

Re: [HACKERS] Move postgresql_fdw_validator into dblink

2013-01-17 Thread Craig Ringer
On 11/16/2012 08:08 AM, Noah Misch wrote: > On Thu, Nov 15, 2012 at 02:33:21PM +0900, Shigeru Hanada wrote: >> On Sat, Oct 20, 2012 at 4:24 PM, Kohei KaiGai wrote: >>> IIRC, the reason why postgresql_fdw instead of pgsql_fdw was >>> no other fdw module has shorten naming such as ora_fdw for >>> Or

Re: [HACKERS] Writable Foreign Tables

2013-01-17 Thread Tom Lane
Craig Ringer writes: > The writable foreign tables patch is flagged ready for committer. While > its last activity was in late 2012, I haven't noticed much else changing > in the area that'd be likely to break it, and FDWs are a somewhat > immature feature anyway. It's been revised based on initia

Re: [HACKERS] Event Triggers: adding information

2013-01-17 Thread Andres Freund
On 2013-01-17 21:48:01 -0500, Tom Lane wrote: > Andres Freund writes: > > I have no problem requiring C code to use the even data, be it via hooks > > or via C functions called from event triggers. The problem I have with > > putting in some hooks is that I doubt that you can find sensible spots >

[HACKERS] Writable Foreign Tables

2013-01-17 Thread Craig Ringer
Committers, The writable foreign tables patch is flagged ready for committer. While its last activity was in late 2012, I haven't noticed much else changing in the area that'd be likely to break it, and FDWs are a somewhat immature feature anyway. It's been revised based on initial reviews. Is an

Re: [HACKERS] Event Triggers: adding information

2013-01-17 Thread Tom Lane
Andres Freund writes: > I have no problem requiring C code to use the even data, be it via hooks > or via C functions called from event triggers. The problem I have with > putting in some hooks is that I doubt that you can find sensible spots > with enough information to actually recreate the DDL

Re: [HACKERS] Event Triggers: adding information

2013-01-17 Thread Craig Ringer
On 01/18/2013 10:34 AM, Andres Freund wrote: > On 2013-01-18 10:31:28 +0800, Craig Ringer wrote: >> On 01/18/2013 09:33 AM, Andres Freund wrote: >>> I have no problem requiring C code to use the even data, be it via >>> hooks or via C functions called from event triggers. The problem I >>> have wit

Re: [HACKERS] Event Triggers: adding information

2013-01-17 Thread Tom Lane
Robert Haas writes: > On Thu, Jan 17, 2013 at 4:43 PM, Dimitri Fontaine > wrote: >> because that >> sounded logical if you believe in CommandCounterIncrement. > I'm somewhat bemused by this comment. I don't think > CommandCounterIncrement() is an article of faith. If you execute a > command to

Re: [HACKERS] Event Triggers: adding information

2013-01-17 Thread Andres Freund
On 2013-01-18 10:31:28 +0800, Craig Ringer wrote: > On 01/18/2013 09:33 AM, Andres Freund wrote: > > I have no problem requiring C code to use the even data, be it via > > hooks or via C functions called from event triggers. The problem I > > have with putting in some hooks is that I doubt that you

Re: [HACKERS] Event Triggers: adding information

2013-01-17 Thread Craig Ringer
On 01/18/2013 09:33 AM, Andres Freund wrote: > I have no problem requiring C code to use the even data, be it via > hooks or via C functions called from event triggers. The problem I > have with putting in some hooks is that I doubt that you can find > sensible spots with enough information to actu

Re: [HACKERS] Event Triggers: adding information

2013-01-17 Thread Craig Ringer
On 01/18/2013 04:24 AM, Tom Lane wrote: > Simon Riggs writes: >> On 17 January 2013 16:15, Robert Haas wrote: >>> It pains me that I've evidently failed to communicate this concept >>> clearly despite a year or more of trying. Does that make sense? Is >>> there some way I can make this more cle

[HACKERS] Status of patches

2013-01-17 Thread Craig Ringer
Hi all Given the size of the 2013-01 CF and the fact that quite a bit of work dragged over from the 2012-11 CF, it'd be a huge help if anyone with stalled or abandoned work could update the CF (or let me know so I can update it) and flag the post returned with comment or rejected, as appropriate.

Re: [HACKERS] HS locking broken in HEAD

2013-01-17 Thread Andres Freund
On 2013-01-17 20:36:43 -0500, Robert Haas wrote: > On Thu, Jan 17, 2013 at 6:22 PM, Andres Freund wrote: > > That only made the bug more visible - the real bug is somewhere > > else. Which makes it even scarrier, the bug was in in the fast path > > locking patch from the start... > > > > It assume

Re: [HACKERS] HS locking broken in HEAD

2013-01-17 Thread Robert Haas
On Thu, Jan 17, 2013 at 6:22 PM, Andres Freund wrote: > That only made the bug more visible - the real bug is somewhere > else. Which makes it even scarrier, the bug was in in the fast path > locking patch from the start... > > It assumes conflicting fast path locks can only ever be in the same >

Re: [HACKERS] Event Triggers: adding information

2013-01-17 Thread Andres Freund
On 2013-01-17 23:48:23 +0100, Dimitri Fontaine wrote: > Tom Lane writes: > > Alternatively, if you want to get something into 9.3 that has not > > necessarily got a long-term-stable API, I'd be inclined to suggest that > > we forget about a SQL-level event trigger facility for now, and just put >

Re: [HACKERS] PATCH: optimized DROP of multiple tables within a transaction

2013-01-17 Thread Craig Ringer
On 01/18/2013 03:19 AM, Alvaro Herrera wrote: > Tomas Vondra wrote: >> Hi, >> >> attached is a patch that improves performance when dropping multiple >> tables within a transaction. Instead of scanning the shared buffers for >> each table separately, the patch removes this and evicts all the tables

Re: [HACKERS] patch to add \watch to psql

2013-01-17 Thread Daniel Farina
On Thu, Jan 17, 2013 at 5:07 PM, Daniel Farina wrote: > I have adjusted this patch a little bit to take care of the review > issues, along with just doing a bit of review myself. I realized while making my adjustments that I pointlessly grew some input checking in the inner loop. I just hoisted

Re: [HACKERS] Event Triggers: adding information

2013-01-17 Thread Robert Haas
On Thu, Jan 17, 2013 at 5:09 PM, Tom Lane wrote: > Perhaps it would improve matters if we refactored DDL processing so that > there were separate "parse analysis" and "execution" phases, where parse > analysis is (perhaps among other responsibilities) responsible for > identifying and locking all

Re: [HACKERS] Event Triggers: adding information

2013-01-17 Thread Robert Haas
On Thu, Jan 17, 2013 at 4:43 PM, Dimitri Fontaine wrote: > Robert Haas writes: >> Goal: Every time an ALTER command is used on object *that actually >> exists*, we will call some user-defined function and pass the object >> type, the OID of the object, and some details about what sort of >> alter

Re: [HACKERS] patch to add \watch to psql

2013-01-17 Thread Daniel Farina
I have adjusted this patch a little bit to take care of the review issues, along with just doing a bit of review myself. On Thu, Oct 25, 2012 at 2:25 AM, Will Leinweber wrote: > Thanks for the reviews and comments. Responses inline: > . > On Sat, Oct 20, 2012 at 9:19 AM, Abhijit Menon-Sen > wro

Re: [HACKERS] review: pgbench - aggregation of info written into log

2013-01-17 Thread Tatsuo Ishii
> On Thu, Jan 17, 2013 at 2:29 PM, Andrew Dunstan wrote: >> >> On 01/17/2013 06:04 AM, Tomas Vondra wrote: >>> >>> The problem is I have access to absolutely no Windows machines, >>> not mentioning the development tools (and that I have no clue about it). >>> >>> I vaguely remember there were peop

[HACKERS] Re: Slave enters in recovery and promotes when WAL stream with master is cut + delay master/slave

2013-01-17 Thread Andres Freund
On 2013-01-18 08:24:31 +0900, Michael Paquier wrote: > On Fri, Jan 18, 2013 at 3:05 AM, Fujii Masao wrote: > > > I encountered the problem that the timeline switch is not performed > > expectedly. > > I set up one master, one standby and one cascade standby. All the servers > > share the archive

[HACKERS] Re: Slave enters in recovery and promotes when WAL stream with master is cut + delay master/slave

2013-01-17 Thread Michael Paquier
On Fri, Jan 18, 2013 at 8:48 AM, Andres Freund wrote: > Can you reproduce that one with 7fcbf6a^ (i.e before xlogreader got > split off?). > Yes, it is reproducible before the xlog reader split. Just an additional report, the master jumps correctly to the new timeline. > > The replication delays

Re: [HACKERS] could not create directory "...": File exists

2013-01-17 Thread Tom Lane
Stephen Frost writes: > * Tom Lane (t...@sss.pgh.pa.us) wrote: >> This seems to provide a reasonably principled >> argument why we might want to fix this case with a localized use of an >> MVCC scan before we have such a fix globally. > I had discussed that idea a bit with Andres on IRC and my on

Re: [HACKERS] PATCH: optimized DROP of multiple tables within a transaction

2013-01-17 Thread Tomas Vondra
On 17.1.2013 20:19, Alvaro Herrera wrote: > > I'm curious -- why would you drop tables in groups of 100 instead of > just doing the 100,000 in a single transaction? Maybe that's faster > now, because you'd do a single scan of the buffer pool instead of 1000? > (I'm assuming that "in groups of" me

[HACKERS] Re: Slave enters in recovery and promotes when WAL stream with master is cut + delay master/slave

2013-01-17 Thread Andres Freund
On 2013-01-18 08:24:31 +0900, Michael Paquier wrote: > On Fri, Jan 18, 2013 at 3:05 AM, Fujii Masao wrote: > > > I encountered the problem that the timeline switch is not performed > > expectedly. > > I set up one master, one standby and one cascade standby. All the servers > > share the archive

Re: [HACKERS] could not create directory "...": File exists

2013-01-17 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: > This seems to provide a reasonably principled > argument why we might want to fix this case with a localized use of an > MVCC scan before we have such a fix globally. I had discussed that idea a bit with Andres on IRC and my only concern was if there's some

Re: [HACKERS] Re: Slave enters in recovery and promotes when WAL stream with master is cut + delay master/slave

2013-01-17 Thread Michael Paquier
On Fri, Jan 18, 2013 at 3:05 AM, Fujii Masao wrote: > I encountered the problem that the timeline switch is not performed > expectedly. > I set up one master, one standby and one cascade standby. All the servers > share the archive directory. restore_command is specified in the > recovery.conf >

Re: [HACKERS] HS locking broken in HEAD

2013-01-17 Thread Andres Freund
On 2013-01-17 23:56:16 +0100, Andres Freund wrote: > On 2013-01-17 22:46:21 +0100, Andres Freund wrote: ^ > > Note the conflicting locks held on relation foo by 28048 and 28068. > > > > I don't immediately know which patch to blame here? Looks a bit like > > broken fastpath locking,

Re: [HACKERS] Event Triggers: adding information

2013-01-17 Thread Tom Lane
Dimitri Fontaine writes: > Tom Lane writes: >> Well, that's already a problem, because as Robert keeps saying, what >> goes through utility.c and what doesn't is pretty random right at the >> moment, and we shouldn't expose that behavior to users for fear of not >> being able to change it later.

Re: [HACKERS] HS locking broken in HEAD

2013-01-17 Thread Andres Freund
On 2013-01-17 22:46:21 +0100, Andres Freund wrote: > Hi, > > > While checking whether I could reproduce the replication delay reported > by Michael Paquier I found this very nice tidbit: > > In a pretty trivial replication setup of only streaming replication I > can currently easily reproduce th

Re: [HACKERS] Removing PD_ALL_VISIBLE

2013-01-17 Thread Jeff Davis
On Thu, 2013-01-17 at 09:53 -0500, Robert Haas wrote: > The main question in my mind is whether > there are any negative consequences to holding a VM buffer pin for > that long without interruption. The usual consideration - namely, > blocking vacuum - doesn't apply here because vacuum does not ta

Re: [HACKERS] Removing PD_ALL_VISIBLE

2013-01-17 Thread Jeff Davis
On Thu, 2013-01-17 at 19:58 +, Simon Riggs wrote: > Presumably we remember the state of the VM so we can skip the re-visit > after every write? That was not a part of my patch, although I remember that you mentioned that previously and I thought it could be a good way to mitigate a problem if

Re: [HACKERS] Event Triggers: adding information

2013-01-17 Thread Dimitri Fontaine
Tom Lane writes: > Well, that's already a problem, because as Robert keeps saying, what > goes through utility.c and what doesn't is pretty random right at the > moment, and we shouldn't expose that behavior to users for fear of not > being able to change it later. It didn't feel that random to m

Re: [HACKERS] Event Triggers: adding information

2013-01-17 Thread Tom Lane
Simon Riggs writes: > On 17 January 2013 20:24, Tom Lane wrote: > My comments were in response to this >> I don't really agree with that. I think the point is to expose what >> the system is doing to the DBA. I'm OK with exposing the fact that >> creating a table with a serial column also crea

Re: [HACKERS] Event Triggers: adding information

2013-01-17 Thread Tom Lane
Dimitri Fontaine writes: > Tom Lane writes: >> I think that we're not realistically going to be able to introduce >> event triggers in very many of the places we'd like to have them >> without first doing a lot of fundamental refactoring. > We're only talking about ddl_command_start and ddl_comm

Re: [HACKERS] Event Triggers: adding information

2013-01-17 Thread Tom Lane
Robert Haas writes: > Let me try to give a concrete example of how I think another firing > point could be made to work along the lines I'm suggesting. > [ snip description of how an event trigger might safely be fired just > after identification and locking of the target object for an ALTER ] Re

Re: [HACKERS] Event Triggers: adding information

2013-01-17 Thread Simon Riggs
On 17 January 2013 20:24, Tom Lane wrote: > Simon Riggs writes: >> On 17 January 2013 16:15, Robert Haas wrote: >>> It pains me that I've evidently failed to communicate this concept >>> clearly despite a year or more of trying. Does that make sense? Is >>> there some way I can make this more

Re: [HACKERS] Re: Slave enters in recovery and promotes when WAL stream with master is cut + delay master/slave

2013-01-17 Thread Andres Freund
On 2013-01-17 23:49:22 +0200, Heikki Linnakangas wrote: > On 17.01.2013 21:57, Heikki Linnakangas wrote: > >On 17.01.2013 20:08, Andres Freund wrote: > >>On 2013-01-18 03:05:47 +0900, Fujii Masao wrote: > >>>I encountered the problem that the timeline switch is not performed > >>>expectedly. > >>>I

Re: [HACKERS] ALTER command reworks

2013-01-17 Thread Alvaro Herrera
Kohei KaiGai escribió: > This attached patch is the rebased one towards the latest master branch. Great, thanks. I played with it a bit and it looks almost done to me. The only issue I can find is that it lets you rename an aggregate by using ALTER FUNCTION, which is supposed to be forbidden. (

Re: [HACKERS] Event Triggers: adding information

2013-01-17 Thread Dimitri Fontaine
Tom Lane writes: > I have to completely disagree with that. If we don't want PostgreSQL > to soon subside into an unfixable morass, as I think Brooks puts it, > we must *not* simply patch things in a way that expediently provides > an approximation of some desired feature. We have to consider, a

Re: [HACKERS] Re: Slave enters in recovery and promotes when WAL stream with master is cut + delay master/slave

2013-01-17 Thread Heikki Linnakangas
On 17.01.2013 21:57, Heikki Linnakangas wrote: On 17.01.2013 20:08, Andres Freund wrote: On 2013-01-18 03:05:47 +0900, Fujii Masao wrote: I encountered the problem that the timeline switch is not performed expectedly. I set up one master, one standby and one cascade standby. All the servers sha

[HACKERS] HS locking broken in HEAD

2013-01-17 Thread Andres Freund
Hi, While checking whether I could reproduce the replication delay reported by Michael Paquier I found this very nice tidbit: In a pretty trivial replication setup of only streaming replication I can currently easily reproduce this: standby# BEGIN;SELECT * FROM foo; BEGIN id | data +-

Re: [HACKERS] Event Triggers: adding information

2013-01-17 Thread Dimitri Fontaine
Robert Haas writes: > Goal: Every time an ALTER command is used on object *that actually > exists*, we will call some user-defined function and pass the object > type, the OID of the object, and some details about what sort of > alteration the user has requested. Ok, in current terms of the propo

Re: [HACKERS] [sepgsql 1/3] add name qualified creation label

2013-01-17 Thread Kohei KaiGai
2013/1/16 Robert Haas : > On Tue, Jan 15, 2013 at 3:02 PM, Kohei KaiGai wrote: >> This patch adds sepgsql the feature of name qualified creation label. >> >> Background, on creation of a certain database object, sepgsql assigns >> a default security label according to the security policy that has

Re: [HACKERS] Event Triggers: adding information

2013-01-17 Thread Robert Haas
On Thu, Jan 17, 2013 at 12:06 PM, Dimitri Fontaine wrote: > Ok. Will prepare a non controversial patch for ddl_command_end. Thanks. I will make a forceful effort to review that in a timely fashion when it's posted. >> I think this is a bad idea, not only because, as I said before, it >> exposes

Re: [HACKERS] Event Triggers: adding information

2013-01-17 Thread Tom Lane
Simon Riggs writes: > On 17 January 2013 16:15, Robert Haas wrote: >> It pains me that I've evidently failed to communicate this concept >> clearly despite a year or more of trying. Does that make sense? Is >> there some way I can make this more clear? The difference seems very >> clear-cut to

Re: [HACKERS] 9.3 Pre-proposal: Range Merge Join

2013-01-17 Thread Stefan Keller
Hi Jeff 2012/4/19 Jeff Davis : > On Wed, 2012-04-18 at 01:21 -0400, Tom Lane wrote: (...) >> This is just handwaving of course. I think some digging in the >> spatial-join literature would likely find ideas better than any of >> these. > > I will look in some more detail. The merge-like approach

Re: [HACKERS] [PATCH]Tablesample Submission

2013-01-17 Thread Josh Berkus
> So I can't see this going anywhere for 9.3. I've moved it to CF1 of > 9.4 marked Waiting on Author Agreed. I wish I'd noticed that it got lost earlier. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make

Re: [HACKERS] Removing PD_ALL_VISIBLE

2013-01-17 Thread Jeff Davis
On Thu, 2013-01-17 at 17:14 +0200, Heikki Linnakangas wrote: > I don't remember if I ever actually tested that > though. Maybe I was worrying about nothing and hitting the VM page on > every update is ok. I tried, but was unable to show really anything at all, even without keeping the VM page pi

Re: [HACKERS] Removing PD_ALL_VISIBLE

2013-01-17 Thread Simon Riggs
On 17 January 2013 15:14, Heikki Linnakangas wrote: > On 17.01.2013 16:53, Robert Haas wrote: >> >> On Thu, Jan 17, 2013 at 3:49 AM, Pavan Deolasee >> wrote: >>> >>> May be you've already addressed that concern with the proven >>> performance numbers, but I'm not sure. >> >> >> It would be nice

Re: [HACKERS] Re: Slave enters in recovery and promotes when WAL stream with master is cut + delay master/slave

2013-01-17 Thread Heikki Linnakangas
On 17.01.2013 20:08, Andres Freund wrote: On 2013-01-18 03:05:47 +0900, Fujii Masao wrote: I encountered the problem that the timeline switch is not performed expectedly. I set up one master, one standby and one cascade standby. All the servers share the archive directory. restore_command is spe

Re: [HACKERS] could not create directory "...": File exists

2013-01-17 Thread Tom Lane
I wrote: > Stephen Frost writes: >> I feel like we should be able to do better than what we have now, at >> least. Using ShareLock prevented the specific case that we were >> experiencing and is therefore MUCH better (for us, anyway) than >> released versions where we ran into the error on a regu

Re: [HACKERS] Teaching pg_receivexlog to follow timeline switches

2013-01-17 Thread Alvaro Herrera
Robert Haas escribió: > Actually, I'm really glad to see all the work you've done to improve > the way that some of these scenarios work and eliminate various bugs > and other surprising failure modes over the last couple of months. > It's great stuff. +1 -- Álvaro Herrerahttp:/

Re: [HACKERS] Event Triggers: adding information

2013-01-17 Thread Simon Riggs
On 17 January 2013 16:15, Robert Haas wrote: > As a further example, suppose that in 9.4 (or 9.17) we add a command > "DROP TABLES IN SCHEMA fred WHERE name LIKE 'bob%'". Well, the > logging trigger still just works (because it's only writing the > statement, without caring about its contents).

Re: [HACKERS] PATCH: optimized DROP of multiple tables within a transaction

2013-01-17 Thread Tom Lane
Alvaro Herrera writes: > Made some tweaks and pushed (added comments to new functions, ensure > that we never try to palloc(0), renamed DropRelFileNodeAllBuffers to > plural, made the "use bsearch" logic a bit simpler). FWIW, there's nothing particularly wrong with palloc(0) ...

Re: [HACKERS] could not create directory "...": File exists

2013-01-17 Thread Tom Lane
Stephen Frost writes: > * Tom Lane (t...@sss.pgh.pa.us) wrote: >> Or maybe we should just write this off as a case we can't realistically >> fix before we have MVCC catalog scans. It seems that any other fix is >> going to be hopelessly ugly. > I feel like we should be able to do better than wha

Re: [HACKERS] Removing PD_ALL_VISIBLE

2013-01-17 Thread Jeff Davis
tion, this can happen: > > > Thank you. New patch attached with simple WAL logging. Regards, Jeff Davis rm-pd-all-visible-20130117.patch.gz Description: GNU Zip compressed data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make chan

Re: [HACKERS] PATCH: optimized DROP of multiple tables within a transaction

2013-01-17 Thread Alvaro Herrera
Tomas Vondra wrote: > Hi, > > attached is a patch that improves performance when dropping multiple > tables within a transaction. Instead of scanning the shared buffers for > each table separately, the patch removes this and evicts all the tables > in a single pass through shared buffers. Made so

Re: [HACKERS] [PATCH]Tablesample Submission

2013-01-17 Thread Simon Riggs
On 17 January 2013 18:32, Josh Berkus wrote: > On 11/04/2012 07:22 PM, Qi Huang wrote: >> Dear hackers Sorry for not replying the patch review. I didn't see the >> review until recently as my mail box is full of Postgres mails and I didn't >> notice the one for me, my mail box configuration

Re: [HACKERS] could not create directory "...": File exists

2013-01-17 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: > The case where you see a tuple twice is if an update drops a new version > of a row beyond your seqscan, and then commits before you get to the new > version. But if it drops the new version of the row *behind* your > seqscan, and then commits before you ge

Re: [HACKERS] could not create directory "...": File exists

2013-01-17 Thread Andres Freund
On 2013-01-17 13:46:44 -0500, Tom Lane wrote: > Or maybe we should just write this off as a case we can't realistically > fix before we have MVCC catalog scans. It seems that any other fix is > going to be hopelessly ugly. ISTM we could just use a MVCC snapshot in this specific case? Andres --

Re: [HACKERS] tuplesort memory usage: grow_memtuples

2013-01-17 Thread Peter Geoghegan
On 17 January 2013 18:22, Tom Lane wrote: > Applied with some changes: Thank you. That feedback is useful. -- Peter Geoghegan http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training and Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To

Re: [HACKERS] could not create directory "...": File exists

2013-01-17 Thread Tom Lane
Stephen Frost writes: > * Tom Lane (t...@sss.pgh.pa.us) wrote: >> [ thinks for a bit... ] Ugh, no, because the *other* risk you've got >> here is not seeing a row at all, which would be really bad. > I'm not sure that I see how that could happen..? I agree that it'd be > really bad if it did th

Re: [HACKERS] Removing PD_ALL_VISIBLE

2013-01-17 Thread Jeff Davis
On Thu, 2013-01-17 at 15:25 +0530, Pavan Deolasee wrote: > Now that I look at the patch, I wonder if there is another fundamental > issue with the patch. Since the patch removes WAL logging for the VM > set operation, this can happen: > Thank you. I think I was confused by this comment here: "Whe

Re: [HACKERS] [PATCH]Tablesample Submission

2013-01-17 Thread Josh Berkus
On 11/04/2012 07:22 PM, Qi Huang wrote: > Dear hackers Sorry for not replying the patch review. I didn't see the > review until recently as my mail box is full of Postgres mails and I didn't > notice the one for me, my mail box configuration problem. I am still kind > of busy with my uni

Re: [HACKERS] tuplesort memory usage: grow_memtuples

2013-01-17 Thread Tom Lane
Peter Geoghegan writes: > On 8 December 2012 14:41, Andres Freund wrote: >> Is anybody planning to work on this? There hasn't been any activity >> since the beginning of the CF and it doesn't look like there is much >> work left? > I took another look at this. Applied with some changes: * Use

Re: [HACKERS] Re: Slave enters in recovery and promotes when WAL stream with master is cut + delay master/slave

2013-01-17 Thread Andres Freund
On 2013-01-18 03:05:47 +0900, Fujii Masao wrote: > On Fri, Jan 18, 2013 at 2:35 AM, Andres Freund wrote: > > On 2013-01-17 13:47:41 +0900, Michael Paquier wrote: > >> Slave does not try anymore to reconnect to master with messages of the > >> type: > >> FATAL: could not connect to the primary se

Re: [HACKERS] Re: Slave enters in recovery and promotes when WAL stream with master is cut + delay master/slave

2013-01-17 Thread Fujii Masao
On Fri, Jan 18, 2013 at 2:35 AM, Andres Freund wrote: > On 2013-01-17 13:47:41 +0900, Michael Paquier wrote: >> Slave does not try anymore to reconnect to master with messages of the type: >> FATAL: could not connect to the primary server >> >> I also noticed that there is some delay until modifi

Re: [HACKERS] could not create directory "...": File exists

2013-01-17 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: > Ugh. Still another problem with non-MVCC catalog scans. Indeed. > It seems that the only thing we actually use from each tuple is the OID. Yes, that's true. > So there are other ways to fix it, of which probably the minimum-change > one is to keep a lis

[HACKERS] Re: Slave enters in recovery and promotes when WAL stream with master is cut + delay master/slave

2013-01-17 Thread Andres Freund
On 2013-01-17 13:47:41 +0900, Michael Paquier wrote: > Slave does not try anymore to reconnect to master with messages of the type: > FATAL: could not connect to the primary server > > I also noticed that there is some delay until modifications on master are > visible on slave. > I think that bug

[HACKERS] Re: Slave enters in recovery and promotes when WAL stream with master is cut + delay master/slave

2013-01-17 Thread Heikki Linnakangas
On 17.01.2013 18:55, Andres Freund wrote: On 2013-01-17 18:50:35 +0200, Heikki Linnakangas wrote: I was thinking of the attached. As long as we check for CheckForStandbyTrigger() after the "record == NULL" check, we won't perform extra stat() calls on successful reads, only when we're polling af

Re: [HACKERS] Event Triggers: adding information

2013-01-17 Thread Dimitri Fontaine
Robert Haas writes: > - adds ddl_command_trace and ddl_command_end events > > I think ddl_command_end is OK and I'm willing to commit that if > extracted as its own patch. I think ddl_command_trace is unnecessary > syntactic sugar. Ok. Will prepare a non controversial patch for ddl_command_end.

[HACKERS] Re: Slave enters in recovery and promotes when WAL stream with master is cut + delay master/slave

2013-01-17 Thread Andres Freund
On 2013-01-17 18:50:35 +0200, Heikki Linnakangas wrote: > On 17.01.2013 18:42, Andres Freund wrote: > >On 2013-01-17 18:33:42 +0200, Heikki Linnakangas wrote: > >>On 17.01.2013 17:42, Andres Freund wrote: > >>>Ok, the attached patch seems to fix a) and b). c) above is bogus, as > >>>explained in a

Re: [HACKERS] Re: Slave enters in recovery and promotes when WAL stream with master is cut + delay master/slave

2013-01-17 Thread Heikki Linnakangas
On 17.01.2013 18:42, Andres Freund wrote: On 2013-01-17 18:33:42 +0200, Heikki Linnakangas wrote: On 17.01.2013 17:42, Andres Freund wrote: Ok, the attached patch seems to fix a) and b). c) above is bogus, as explained in a comment in the patch. I also noticed that the TLI check didn't mark th

Re: [HACKERS] Materialized views WIP patch

2013-01-17 Thread Simon Riggs
On 16 January 2013 05:40, Kevin Grittner wrote: > Here is a new version of the patch, with most issues discussed in > previous posts fixed. Looks good. The patch implements one kind of MV. In the future, we hope to have other features or other kinds of MV alongside this: * Snapshot MV - built o

Re: [HACKERS] Latex longtable format

2013-01-17 Thread Bruce Momjian
On Sat, Jan 12, 2013 at 07:09:31PM -0500, Bruce Momjian wrote: > I have received several earnest requests over the years for LaTeX > 'longtable' output, and I have just implemented it based on a sample > LaTeX longtable output file. > > I have called it 'latex-longtable' and implemented all the be

Re: [HACKERS] Re: Slave enters in recovery and promotes when WAL stream with master is cut + delay master/slave

2013-01-17 Thread Andres Freund
On 2013-01-17 18:33:42 +0200, Heikki Linnakangas wrote: > On 17.01.2013 17:42, Andres Freund wrote: > >Ok, the attached patch seems to fix a) and b). c) above is bogus, as > >explained in a comment in the patch. I also noticed that the TLI check > >didn't mark the last source as failed. > > This

Re: [HACKERS] Re: Slave enters in recovery and promotes when WAL stream with master is cut + delay master/slave

2013-01-17 Thread Heikki Linnakangas
On 17.01.2013 17:42, Andres Freund wrote: Ok, the attached patch seems to fix a) and b). c) above is bogus, as explained in a comment in the patch. I also noticed that the TLI check didn't mark the last source as failed. This looks fragile: /*

Re: [HACKERS] Event Triggers: adding information

2013-01-17 Thread Andres Freund
On 2013-01-17 11:15:24 -0500, Robert Haas wrote: > As a further example, suppose that in 9.4 (or 9.17) we add a command > "DROP TABLES IN SCHEMA fred WHERE name LIKE 'bob%'". Well, the > logging trigger still just works (because it's only writing the > statement, without caring about its contents)

Re: [HACKERS] Event Triggers: adding information

2013-01-17 Thread Robert Haas
On Thu, Jan 17, 2013 at 5:18 AM, Dimitri Fontaine wrote: > Now, if that's what it takes, I'll spend time on it. In which exact > order do you want to be reviewing and applying that series of patches? Let's agree on which things we even want to do first. Here's my take: - adds ddl_command_trace

Re: [HACKERS] Materialized views WIP patch

2013-01-17 Thread Thom Brown
On 16 January 2013 17:25, Thom Brown wrote: > On 16 January 2013 17:20, Kevin Grittner wrote: > >> Thom Brown wrote: >> >> > Some weirdness: >> > >> > postgres=# CREATE VIEW v_test2 AS SELECT 1 moo; >> > CREATE VIEW >> > postgres=# CREATE MATERIALIZED VIEW mv_test2 AS SELECT moo, 2*moo FROM >> >

Re: [HACKERS] Hot Standby conflict resolution handling

2013-01-17 Thread Andres Freund
On 2013-01-17 10:19:23 -0500, Tom Lane wrote: > Pavan Deolasee writes: > > On Thu, Jan 17, 2013 at 12:08 PM, Tom Lane wrote: > >> ISTM that if we dare not interrupt for fear of confusing OpenSSL, we > >> cannot safely attempt to send an error message to the client either; > >> but ereport(FATAL)

Re: [HACKERS] could not create directory "...": File exists

2013-01-17 Thread Tom Lane
Stephen Frost writes: > It turns out that createdb() currently only takes an AccessShareLock > on pg_tablespace when scanning it with SnapshotNow, making it possible > for a concurrent process to make some uninteresting modification to a > tablespace (such as an ACL change) which will caus

Re: [HACKERS] Re: Slave enters in recovery and promotes when WAL stream with master is cut + delay master/slave

2013-01-17 Thread Andres Freund
On 2013-01-17 16:23:44 +0100, Andres Freund wrote: > On 2013-01-17 17:18:14 +0200, Heikki Linnakangas wrote: > > On 17.01.2013 15:05, Andres Freund wrote: > > >On 2013-01-17 13:47:41 +0900, Michael Paquier wrote: > > >>I think that bug has been introduced by commit 7fcbf6a. > > >>Before splitting x

[HACKERS] Re: Slave enters in recovery and promotes when WAL stream with master is cut + delay master/slave

2013-01-17 Thread Andres Freund
On 2013-01-17 17:18:14 +0200, Heikki Linnakangas wrote: > On 17.01.2013 15:05, Andres Freund wrote: > >On 2013-01-17 13:47:41 +0900, Michael Paquier wrote: > >>I think that bug has been introduced by commit 7fcbf6a. > >>Before splitting xlog reading as a separate facility things worked > >>correctl

Re: [HACKERS] Hot Standby conflict resolution handling

2013-01-17 Thread Tom Lane
Pavan Deolasee writes: > On Thu, Jan 17, 2013 at 12:08 PM, Tom Lane wrote: >> ISTM that if we dare not interrupt for fear of confusing OpenSSL, we >> cannot safely attempt to send an error message to the client either; >> but ereport(FATAL) will try exactly that. > I thought since FATAL will for

Re: [HACKERS] Re: Slave enters in recovery and promotes when WAL stream with master is cut + delay master/slave

2013-01-17 Thread Heikki Linnakangas
On 17.01.2013 15:05, Andres Freund wrote: On 2013-01-17 13:47:41 +0900, Michael Paquier wrote: I think that bug has been introduced by commit 7fcbf6a. Before splitting xlog reading as a separate facility things worked correctly. There are also no delay problems before this commit. Ok, my inkli

Re: [HACKERS] Teaching pg_receivexlog to follow timeline switches

2013-01-17 Thread Robert Haas
On Thu, Jan 17, 2013 at 9:59 AM, Heikki Linnakangas wrote: > The scenario I described is that you screwed up your failover environment, > and end up with a split-brain situation by accident. The DBA certainly needs > to be involved to recover from that. OK, I agree, but I still think a lot of DBA

Re: [HACKERS] Removing PD_ALL_VISIBLE

2013-01-17 Thread Heikki Linnakangas
On 17.01.2013 16:53, Robert Haas wrote: On Thu, Jan 17, 2013 at 3:49 AM, Pavan Deolasee wrote: May be you've already addressed that concern with the proven performance numbers, but I'm not sure. It would be nice to hear what Heikki's reasons were for adding PD_ALL_VISIBLE in the first place.

Re: [HACKERS] Teaching pg_receivexlog to follow timeline switches

2013-01-17 Thread Heikki Linnakangas
On 17.01.2013 16:56, Robert Haas wrote: On Wed, Jan 16, 2013 at 11:08 AM, Heikki Linnakangas wrote: I'd prefer to leave the .partial suffix in place, as the segment really isn't complete. It doesn't make a difference when you recover to the latest timeline, but if you have a more complicated s

Re: [HACKERS] Teaching pg_receivexlog to follow timeline switches

2013-01-17 Thread Robert Haas
On Wed, Jan 16, 2013 at 11:08 AM, Heikki Linnakangas wrote: > I'd prefer to leave the .partial suffix in place, as the segment really > isn't complete. It doesn't make a difference when you recover to the latest > timeline, but if you have a more complicated scenario with multiple > timelines that

Re: [HACKERS] CF3+4

2013-01-17 Thread Robert Haas
On Thu, Jan 17, 2013 at 8:17 AM, Craig Ringer wrote: > I've moved all pending patches from 2012-11 to 2013-01. I'll go through and > poke them for aliveness and start chasing things up; in the mean time, any > chance of closing 2012-11? Done. -- Robert Haas EnterpriseDB: http://www.enterprisedb

Re: [HACKERS] Removing PD_ALL_VISIBLE

2013-01-17 Thread Robert Haas
On Thu, Jan 17, 2013 at 3:49 AM, Pavan Deolasee wrote: > May be you've already addressed that concern with the proven > performance numbers, but I'm not sure. It would be nice to hear what Heikki's reasons were for adding PD_ALL_VISIBLE in the first place. Jeff's approach of holding the VM pins

  1   2   >