Re: [HACKERS] Removing PD_ALL_VISIBLE

2013-01-17 Thread Simon Riggs
On 17 January 2013 03:02, Jeff Davis pg...@j-davis.com wrote: Rebased patch attached. No significant changes. Jeff, can you summarise/collate why we're doing this, what concerns it raises and how you've dealt with them? That will help decide whether to commit. Thanks -- Simon Riggs

Re: [HACKERS] CF3+4

2013-01-17 Thread Magnus Hagander
On Jan 17, 2013 8:15 AM, Abhijit Menon-Sen a...@2ndquadrant.com wrote: At 2013-01-17 16:05:05 +0900, michael.paqu...@gmail.com wrote: Is it really necessary to create a new commit fest just to move the items? Marking the patches that are considered as being too late for 9.3 should be

Re: [HACKERS] Removing PD_ALL_VISIBLE

2013-01-17 Thread Pavan Deolasee
On Thu, Jan 17, 2013 at 2:11 PM, Simon Riggs si...@2ndquadrant.com wrote: On 17 January 2013 03:02, Jeff Davis pg...@j-davis.com wrote: Rebased patch attached. No significant changes. Jeff, can you summarise/collate why we're doing this, what concerns it raises and how you've dealt with

Re: [HACKERS] CF3+4 (was Re: Parallel query execution)

2013-01-17 Thread Magnus Hagander
On Thu, Jan 17, 2013 at 8:19 AM, Pavan Deolasee pavan.deola...@gmail.com wrote: On Wed, Jan 16, 2013 at 1:51 PM, Abhijit Menon-Sen a...@2ndquadrant.com wrote: At 2013-01-16 02:07:29 -0500, t...@sss.pgh.pa.us wrote: In case you hadn't noticed, we've totally lost control of the CF

Re: [HACKERS] Removing PD_ALL_VISIBLE

2013-01-17 Thread Abhijit Menon-Sen
At 2013-01-17 08:41:37 +, si...@2ndquadrant.com wrote: Jeff, can you summarise/collate why we're doing this, what concerns it raises and how you've dealt with them? Since I was just looking at the original patch and discussion, and since Pavan has posted an excerpt from one objection to

Re: [HACKERS] review: pgbench - aggregation of info written into log

2013-01-17 Thread Magnus Hagander
On Thu, Jan 17, 2013 at 2:35 AM, Tatsuo Ishii is...@postgresql.org wrote: This might be way more than we want to do, but there is an article that describes some techniques for doing what seems to be missing (AIUI): http://msdn.microsoft.com/en-us/magazine/cc163996.aspx Even this would be

Re: [HACKERS] Removing PD_ALL_VISIBLE

2013-01-17 Thread Pavan Deolasee
On Thu, Jan 17, 2013 at 2:57 PM, Abhijit Menon-Sen a...@2ndquadrant.comwrote: There was considerable discussion after this (accessible through the archives link above), which I won't attempt to summarise. I thought Robert made those comments after considerable discussions on Jeff's

Re: [HACKERS] review: pgbench - aggregation of info written into log

2013-01-17 Thread Dave Page
On Thu, Jan 17, 2013 at 9:36 AM, Magnus Hagander mag...@hagander.net wrote: On Thu, Jan 17, 2013 at 2:35 AM, Tatsuo Ishii is...@postgresql.org wrote: This might be way more than we want to do, but there is an article that describes some techniques for doing what seems to be missing (AIUI):

Re: [HACKERS] review: pgbench - aggregation of info written into log

2013-01-17 Thread Magnus Hagander
On Thu, Jan 17, 2013 at 11:09 AM, Dave Page dp...@pgadmin.org wrote: On Thu, Jan 17, 2013 at 9:36 AM, Magnus Hagander mag...@hagander.net wrote: On Thu, Jan 17, 2013 at 2:35 AM, Tatsuo Ishii is...@postgresql.org wrote: This might be way more than we want to do, but there is an article that

Re: [HACKERS] Event Triggers: adding information

2013-01-17 Thread Dimitri Fontaine
Robert Haas robertmh...@gmail.com writes: - adds ddl_command_trace and ddl_command_end events - causes those events to be called not only for actual SQL commands but also for things that happen, at present, to go through the same code path - adds additional magic variables to PL/pgsql to

Re: [HACKERS] Multiple --table options for other commands

2013-01-17 Thread Magnus Hagander
On Fri, Dec 14, 2012 at 4:14 PM, Karl O. Pinc k...@meme.com wrote: On 12/13/2012 11:02:56 PM, Josh Kupershmidt wrote: On Thu, Dec 13, 2012 at 9:03 PM, Karl O. Pinc k...@meme.com wrote: Sorry to be so persnickety, and unhelpful until now. It seemed like it should be doable, but something

Re: [HACKERS] Hot Standby conflict resolution handling

2013-01-17 Thread Andres Freund
On 2013-01-17 01:38:31 -0500, Tom Lane wrote: But having said that ... are we sure this code is not actually broken? ISTM that if we dare not interrupt for fear of confusing OpenSSL, we cannot safely attempt to send an error message to the client either; but ereport(FATAL) will try exactly

[HACKERS] Re: Slave enters in recovery and promotes when WAL stream with master is cut + delay master/slave

2013-01-17 Thread Andres Freund
On 2013-01-17 13:47:41 +0900, Michael Paquier wrote: Hi all, There is a strange bug with the latest master head (commit 7fcbf6a). When the WAL stream with a master is cut on a slave, slave returns a FATAL (well normal...), but then enters in recovery process and automatically promotes.

Re: [HACKERS] review: pgbench - aggregation of info written into log

2013-01-17 Thread Tomas Vondra
Dne 17.01.2013 10:36, Magnus Hagander napsal: On Thu, Jan 17, 2013 at 2:35 AM, Tatsuo Ishii is...@postgresql.org wrote: This might be way more than we want to do, but there is an article that describes some techniques for doing what seems to be missing (AIUI):

Re: [HACKERS] review: pgbench - aggregation of info written into log

2013-01-17 Thread Tomas Vondra
Dne 17.01.2013 11:16, Magnus Hagander napsal: On Thu, Jan 17, 2013 at 11:09 AM, Dave Page dp...@pgadmin.org wrote: On Thu, Jan 17, 2013 at 9:36 AM, Magnus Hagander mag...@hagander.net wrote: We have committed platform-specific features before, but generally only when it's not *possible* to do

Re: [HACKERS] Materialized views WIP patch

2013-01-17 Thread Robert Haas
On Wed, Jan 16, 2013 at 1:38 PM, Tom Lane t...@sss.pgh.pa.us wrote: Where I really need someone to hit me upside the head with a clue-stick is the code I added to the bottom of RelationBuildDesc() in relcache.c. The idea is that on first access to an unlogged MV, to detect that the heap has

Re: [HACKERS] review: pgbench - aggregation of info written into log

2013-01-17 Thread Andrew Dunstan
On 01/17/2013 06:11 AM, Tomas Vondra wrote: Dne 17.01.2013 11:16, Magnus Hagander napsal: On Thu, Jan 17, 2013 at 11:09 AM, Dave Page dp...@pgadmin.org wrote: On Thu, Jan 17, 2013 at 9:36 AM, Magnus Hagander mag...@hagander.net wrote: We have committed platform-specific features before, but

[HACKERS] Re: Slave enters in recovery and promotes when WAL stream with master is cut + delay master/slave

2013-01-17 Thread Andres Freund
On 2013-01-17 13:47:41 +0900, Michael Paquier wrote: Hi all, There is a strange bug with the latest master head (commit 7fcbf6a). When the WAL stream with a master is cut on a slave, slave returns a FATAL (well normal...), but then enters in recovery process and automatically promotes.

Re: [HACKERS] CF3+4

2013-01-17 Thread Craig Ringer
On 01/17/2013 04:43 PM, Magnus Hagander wrote: On Jan 17, 2013 8:15 AM, Abhijit Menon-Sen a...@2ndquadrant.com mailto:a...@2ndquadrant.com wrote: At 2013-01-17 16:05:05 +0900, michael.paqu...@gmail.com mailto:michael.paqu...@gmail.com wrote: Is it really necessary to create a new

Re: [HACKERS] review: pgbench - aggregation of info written into log

2013-01-17 Thread Andrew Dunstan
On 01/17/2013 06:04 AM, Tomas Vondra wrote: The problem is I have access to absolutely no Windows machines, not mentioning the development tools (and that I have no clue about it). I vaguely remember there were people on this list doing Windows development on a virtual machine or something.

Re: [HACKERS] review: pgbench - aggregation of info written into log

2013-01-17 Thread Dave Page
On Thu, Jan 17, 2013 at 1:29 PM, Andrew Dunstan and...@dunslane.net wrote: On 01/17/2013 06:04 AM, Tomas Vondra wrote: The problem is I have access to absolutely no Windows machines, not mentioning the development tools (and that I have no clue about it). I vaguely remember there were

Re: [HACKERS] review: pgbench - aggregation of info written into log

2013-01-17 Thread Magnus Hagander
On Thu, Jan 17, 2013 at 2:29 PM, Andrew Dunstan and...@dunslane.net wrote: On 01/17/2013 06:04 AM, Tomas Vondra wrote: The problem is I have access to absolutely no Windows machines, not mentioning the development tools (and that I have no clue about it). I vaguely remember there were

Re: [HACKERS] small pg_basebackup display bug

2013-01-17 Thread Magnus Hagander
On Sun, Dec 16, 2012 at 7:20 PM, Tom Lane t...@sss.pgh.pa.us wrote: Magnus Hagander mag...@hagander.net writes: On Sat, Dec 15, 2012 at 2:24 PM, Erik Rijkers e...@xs4all.nl wrote: That would make such a truncation less frequent, and after all a truncated display is not particular useful.

Re: [HACKERS] default SSL compression (was: libpq compression)

2013-01-17 Thread Magnus Hagander
On Wed, Jan 2, 2013 at 3:17 PM, Magnus Hagander mag...@hagander.net wrote: On Wed, Jan 2, 2013 at 3:15 PM, Noah Misch n...@leadboat.com wrote: On Wed, Jan 02, 2013 at 02:03:20PM +0100, Magnus Hagander wrote: On Wed, Jan 2, 2013 at 1:15 AM, Tom Lane t...@sss.pgh.pa.us wrote: So +1 for changing

[HACKERS] could not create directory ...: File exists

2013-01-17 Thread Stephen Frost
Greetings, We've come across this rather annoying error happening during our builds: ERROR: could not create directory pg_tblspc/25120/PG_9.3_201212081/231253: File exists It turns out that this is coming from copydir() when called by createdb() during a CREATE DATABASE .. FROM

Re: [HACKERS] Removing PD_ALL_VISIBLE

2013-01-17 Thread Robert Haas
On Thu, Jan 17, 2013 at 3:49 AM, Pavan Deolasee pavan.deola...@gmail.com wrote: May be you've already addressed that concern with the proven performance numbers, but I'm not sure. It would be nice to hear what Heikki's reasons were for adding PD_ALL_VISIBLE in the first place. Jeff's approach

Re: [HACKERS] CF3+4

2013-01-17 Thread Robert Haas
On Thu, Jan 17, 2013 at 8:17 AM, Craig Ringer cr...@2ndquadrant.com wrote: I've moved all pending patches from 2012-11 to 2013-01. I'll go through and poke them for aliveness and start chasing things up; in the mean time, any chance of closing 2012-11? Done. -- Robert Haas EnterpriseDB:

Re: [HACKERS] Teaching pg_receivexlog to follow timeline switches

2013-01-17 Thread Robert Haas
On Wed, Jan 16, 2013 at 11:08 AM, Heikki Linnakangas hlinnakan...@vmware.com wrote: I'd prefer to leave the .partial suffix in place, as the segment really isn't complete. It doesn't make a difference when you recover to the latest timeline, but if you have a more complicated scenario with

Re: [HACKERS] Teaching pg_receivexlog to follow timeline switches

2013-01-17 Thread Heikki Linnakangas
On 17.01.2013 16:56, Robert Haas wrote: On Wed, Jan 16, 2013 at 11:08 AM, Heikki Linnakangas hlinnakan...@vmware.com wrote: I'd prefer to leave the .partial suffix in place, as the segment really isn't complete. It doesn't make a difference when you recover to the latest timeline, but if you

Re: [HACKERS] Removing PD_ALL_VISIBLE

2013-01-17 Thread Heikki Linnakangas
On 17.01.2013 16:53, Robert Haas wrote: On Thu, Jan 17, 2013 at 3:49 AM, Pavan Deolasee pavan.deola...@gmail.com wrote: May be you've already addressed that concern with the proven performance numbers, but I'm not sure. It would be nice to hear what Heikki's reasons were for adding

Re: [HACKERS] Teaching pg_receivexlog to follow timeline switches

2013-01-17 Thread Robert Haas
On Thu, Jan 17, 2013 at 9:59 AM, Heikki Linnakangas hlinnakan...@vmware.com wrote: The scenario I described is that you screwed up your failover environment, and end up with a split-brain situation by accident. The DBA certainly needs to be involved to recover from that. OK, I agree, but I

Re: [HACKERS] Re: Slave enters in recovery and promotes when WAL stream with master is cut + delay master/slave

2013-01-17 Thread Heikki Linnakangas
On 17.01.2013 15:05, Andres Freund wrote: On 2013-01-17 13:47:41 +0900, Michael Paquier wrote: I think that bug has been introduced by commit 7fcbf6a. Before splitting xlog reading as a separate facility things worked correctly. There are also no delay problems before this commit. Ok, my

Re: [HACKERS] Hot Standby conflict resolution handling

2013-01-17 Thread Tom Lane
Pavan Deolasee pavan.deola...@gmail.com writes: On Thu, Jan 17, 2013 at 12:08 PM, Tom Lane t...@sss.pgh.pa.us wrote: ISTM that if we dare not interrupt for fear of confusing OpenSSL, we cannot safely attempt to send an error message to the client either; but ereport(FATAL) will try exactly

[HACKERS] Re: Slave enters in recovery and promotes when WAL stream with master is cut + delay master/slave

2013-01-17 Thread Andres Freund
On 2013-01-17 17:18:14 +0200, Heikki Linnakangas wrote: On 17.01.2013 15:05, Andres Freund wrote: On 2013-01-17 13:47:41 +0900, Michael Paquier wrote: I think that bug has been introduced by commit 7fcbf6a. Before splitting xlog reading as a separate facility things worked correctly. There

Re: [HACKERS] Re: Slave enters in recovery and promotes when WAL stream with master is cut + delay master/slave

2013-01-17 Thread Andres Freund
On 2013-01-17 16:23:44 +0100, Andres Freund wrote: On 2013-01-17 17:18:14 +0200, Heikki Linnakangas wrote: On 17.01.2013 15:05, Andres Freund wrote: On 2013-01-17 13:47:41 +0900, Michael Paquier wrote: I think that bug has been introduced by commit 7fcbf6a. Before splitting xlog reading

Re: [HACKERS] could not create directory ...: File exists

2013-01-17 Thread Tom Lane
Stephen Frost sfr...@snowman.net writes: It turns out that createdb() currently only takes an AccessShareLock on pg_tablespace when scanning it with SnapshotNow, making it possible for a concurrent process to make some uninteresting modification to a tablespace (such as an ACL change)

Re: [HACKERS] Hot Standby conflict resolution handling

2013-01-17 Thread Andres Freund
On 2013-01-17 10:19:23 -0500, Tom Lane wrote: Pavan Deolasee pavan.deola...@gmail.com writes: On Thu, Jan 17, 2013 at 12:08 PM, Tom Lane t...@sss.pgh.pa.us wrote: ISTM that if we dare not interrupt for fear of confusing OpenSSL, we cannot safely attempt to send an error message to the

Re: [HACKERS] Materialized views WIP patch

2013-01-17 Thread Thom Brown
On 16 January 2013 17:25, Thom Brown t...@linux.com wrote: On 16 January 2013 17:20, Kevin Grittner kgri...@mail.com wrote: Thom Brown wrote: Some weirdness: postgres=# CREATE VIEW v_test2 AS SELECT 1 moo; CREATE VIEW postgres=# CREATE MATERIALIZED VIEW mv_test2 AS SELECT moo, 2*moo

Re: [HACKERS] Event Triggers: adding information

2013-01-17 Thread Robert Haas
On Thu, Jan 17, 2013 at 5:18 AM, Dimitri Fontaine dimi...@2ndquadrant.fr wrote: Now, if that's what it takes, I'll spend time on it. In which exact order do you want to be reviewing and applying that series of patches? Let's agree on which things we even want to do first. Here's my take: -

Re: [HACKERS] Event Triggers: adding information

2013-01-17 Thread Andres Freund
On 2013-01-17 11:15:24 -0500, Robert Haas wrote: As a further example, suppose that in 9.4 (or 9.17) we add a command DROP TABLES IN SCHEMA fred WHERE name LIKE 'bob%'. Well, the logging trigger still just works (because it's only writing the statement, without caring about its contents).

Re: [HACKERS] Re: Slave enters in recovery and promotes when WAL stream with master is cut + delay master/slave

2013-01-17 Thread Heikki Linnakangas
On 17.01.2013 17:42, Andres Freund wrote: Ok, the attached patch seems to fix a) and b). c) above is bogus, as explained in a comment in the patch. I also noticed that the TLI check didn't mark the last source as failed. This looks fragile: /*

Re: [HACKERS] Re: Slave enters in recovery and promotes when WAL stream with master is cut + delay master/slave

2013-01-17 Thread Andres Freund
On 2013-01-17 18:33:42 +0200, Heikki Linnakangas wrote: On 17.01.2013 17:42, Andres Freund wrote: Ok, the attached patch seems to fix a) and b). c) above is bogus, as explained in a comment in the patch. I also noticed that the TLI check didn't mark the last source as failed. This looks

Re: [HACKERS] Latex longtable format

2013-01-17 Thread Bruce Momjian
On Sat, Jan 12, 2013 at 07:09:31PM -0500, Bruce Momjian wrote: I have received several earnest requests over the years for LaTeX 'longtable' output, and I have just implemented it based on a sample LaTeX longtable output file. I have called it 'latex-longtable' and implemented all the

Re: [HACKERS] Re: Slave enters in recovery and promotes when WAL stream with master is cut + delay master/slave

2013-01-17 Thread Heikki Linnakangas
On 17.01.2013 18:42, Andres Freund wrote: On 2013-01-17 18:33:42 +0200, Heikki Linnakangas wrote: On 17.01.2013 17:42, Andres Freund wrote: Ok, the attached patch seems to fix a) and b). c) above is bogus, as explained in a comment in the patch. I also noticed that the TLI check didn't mark

[HACKERS] Re: Slave enters in recovery and promotes when WAL stream with master is cut + delay master/slave

2013-01-17 Thread Andres Freund
On 2013-01-17 18:50:35 +0200, Heikki Linnakangas wrote: On 17.01.2013 18:42, Andres Freund wrote: On 2013-01-17 18:33:42 +0200, Heikki Linnakangas wrote: On 17.01.2013 17:42, Andres Freund wrote: Ok, the attached patch seems to fix a) and b). c) above is bogus, as explained in a comment in

Re: [HACKERS] Event Triggers: adding information

2013-01-17 Thread Dimitri Fontaine
Robert Haas robertmh...@gmail.com writes: - adds ddl_command_trace and ddl_command_end events I think ddl_command_end is OK and I'm willing to commit that if extracted as its own patch. I think ddl_command_trace is unnecessary syntactic sugar. Ok. Will prepare a non controversial patch for

[HACKERS] Re: Slave enters in recovery and promotes when WAL stream with master is cut + delay master/slave

2013-01-17 Thread Heikki Linnakangas
On 17.01.2013 18:55, Andres Freund wrote: On 2013-01-17 18:50:35 +0200, Heikki Linnakangas wrote: I was thinking of the attached. As long as we check for CheckForStandbyTrigger() after the record == NULL check, we won't perform extra stat() calls on successful reads, only when we're polling

[HACKERS] Re: Slave enters in recovery and promotes when WAL stream with master is cut + delay master/slave

2013-01-17 Thread Andres Freund
On 2013-01-17 13:47:41 +0900, Michael Paquier wrote: Slave does not try anymore to reconnect to master with messages of the type: FATAL: could not connect to the primary server I also noticed that there is some delay until modifications on master are visible on slave. I think that bug has

Re: [HACKERS] could not create directory ...: File exists

2013-01-17 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: Ugh. Still another problem with non-MVCC catalog scans. Indeed. It seems that the only thing we actually use from each tuple is the OID. Yes, that's true. So there are other ways to fix it, of which probably the minimum-change one is to keep a list

Re: [HACKERS] Re: Slave enters in recovery and promotes when WAL stream with master is cut + delay master/slave

2013-01-17 Thread Fujii Masao
On Fri, Jan 18, 2013 at 2:35 AM, Andres Freund and...@2ndquadrant.com wrote: On 2013-01-17 13:47:41 +0900, Michael Paquier wrote: Slave does not try anymore to reconnect to master with messages of the type: FATAL: could not connect to the primary server I also noticed that there is some

Re: [HACKERS] Re: Slave enters in recovery and promotes when WAL stream with master is cut + delay master/slave

2013-01-17 Thread Andres Freund
On 2013-01-18 03:05:47 +0900, Fujii Masao wrote: On Fri, Jan 18, 2013 at 2:35 AM, Andres Freund and...@2ndquadrant.com wrote: On 2013-01-17 13:47:41 +0900, Michael Paquier wrote: Slave does not try anymore to reconnect to master with messages of the type: FATAL: could not connect to the

Re: [HACKERS] tuplesort memory usage: grow_memtuples

2013-01-17 Thread Tom Lane
Peter Geoghegan pe...@2ndquadrant.com writes: On 8 December 2012 14:41, Andres Freund and...@2ndquadrant.com wrote: Is anybody planning to work on this? There hasn't been any activity since the beginning of the CF and it doesn't look like there is much work left? I took another look at this.

Re: [HACKERS] [PATCH]Tablesample Submission

2013-01-17 Thread Josh Berkus
On 11/04/2012 07:22 PM, Qi Huang wrote: Dear hackers Sorry for not replying the patch review. I didn't see the review until recently as my mail box is full of Postgres mails and I didn't notice the one for me, my mail box configuration problem. I am still kind of busy with my

Re: [HACKERS] Removing PD_ALL_VISIBLE

2013-01-17 Thread Jeff Davis
On Thu, 2013-01-17 at 15:25 +0530, Pavan Deolasee wrote: Now that I look at the patch, I wonder if there is another fundamental issue with the patch. Since the patch removes WAL logging for the VM set operation, this can happen: Thank you. I think I was confused by this comment here: When we

Re: [HACKERS] could not create directory ...: File exists

2013-01-17 Thread Tom Lane
Stephen Frost sfr...@snowman.net writes: * Tom Lane (t...@sss.pgh.pa.us) wrote: [ thinks for a bit... ] Ugh, no, because the *other* risk you've got here is not seeing a row at all, which would be really bad. I'm not sure that I see how that could happen..? I agree that it'd be really bad

Re: [HACKERS] tuplesort memory usage: grow_memtuples

2013-01-17 Thread Peter Geoghegan
On 17 January 2013 18:22, Tom Lane t...@sss.pgh.pa.us wrote: Applied with some changes: Thank you. That feedback is useful. -- Peter Geoghegan http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training and Services -- Sent via pgsql-hackers mailing list

Re: [HACKERS] could not create directory ...: File exists

2013-01-17 Thread Andres Freund
On 2013-01-17 13:46:44 -0500, Tom Lane wrote: Or maybe we should just write this off as a case we can't realistically fix before we have MVCC catalog scans. It seems that any other fix is going to be hopelessly ugly. ISTM we could just use a MVCC snapshot in this specific case? Andres --

Re: [HACKERS] could not create directory ...: File exists

2013-01-17 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: The case where you see a tuple twice is if an update drops a new version of a row beyond your seqscan, and then commits before you get to the new version. But if it drops the new version of the row *behind* your seqscan, and then commits before you get

Re: [HACKERS] [PATCH]Tablesample Submission

2013-01-17 Thread Simon Riggs
On 17 January 2013 18:32, Josh Berkus j...@agliodbs.com wrote: On 11/04/2012 07:22 PM, Qi Huang wrote: Dear hackers Sorry for not replying the patch review. I didn't see the review until recently as my mail box is full of Postgres mails and I didn't notice the one for me, my mail box

Re: [HACKERS] PATCH: optimized DROP of multiple tables within a transaction

2013-01-17 Thread Alvaro Herrera
Tomas Vondra wrote: Hi, attached is a patch that improves performance when dropping multiple tables within a transaction. Instead of scanning the shared buffers for each table separately, the patch removes this and evicts all the tables in a single pass through shared buffers. Made some

Re: [HACKERS] Removing PD_ALL_VISIBLE

2013-01-17 Thread Jeff Davis
you. New patch attached with simple WAL logging. Regards, Jeff Davis rm-pd-all-visible-20130117.patch.gz Description: GNU Zip compressed data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org

Re: [HACKERS] could not create directory ...: File exists

2013-01-17 Thread Tom Lane
Stephen Frost sfr...@snowman.net writes: * Tom Lane (t...@sss.pgh.pa.us) wrote: Or maybe we should just write this off as a case we can't realistically fix before we have MVCC catalog scans. It seems that any other fix is going to be hopelessly ugly. I feel like we should be able to do

Re: [HACKERS] PATCH: optimized DROP of multiple tables within a transaction

2013-01-17 Thread Tom Lane
Alvaro Herrera alvhe...@2ndquadrant.com writes: Made some tweaks and pushed (added comments to new functions, ensure that we never try to palloc(0), renamed DropRelFileNodeAllBuffers to plural, made the use bsearch logic a bit simpler). FWIW, there's nothing particularly wrong with palloc(0)

Re: [HACKERS] Event Triggers: adding information

2013-01-17 Thread Simon Riggs
On 17 January 2013 16:15, Robert Haas robertmh...@gmail.com wrote: As a further example, suppose that in 9.4 (or 9.17) we add a command DROP TABLES IN SCHEMA fred WHERE name LIKE 'bob%'. Well, the logging trigger still just works (because it's only writing the statement, without caring about

Re: [HACKERS] Teaching pg_receivexlog to follow timeline switches

2013-01-17 Thread Alvaro Herrera
Robert Haas escribió: Actually, I'm really glad to see all the work you've done to improve the way that some of these scenarios work and eliminate various bugs and other surprising failure modes over the last couple of months. It's great stuff. +1 -- Álvaro Herrera

Re: [HACKERS] could not create directory ...: File exists

2013-01-17 Thread Tom Lane
I wrote: Stephen Frost sfr...@snowman.net writes: I feel like we should be able to do better than what we have now, at least. Using ShareLock prevented the specific case that we were experiencing and is therefore MUCH better (for us, anyway) than released versions where we ran into the error

Re: [HACKERS] Re: Slave enters in recovery and promotes when WAL stream with master is cut + delay master/slave

2013-01-17 Thread Heikki Linnakangas
On 17.01.2013 20:08, Andres Freund wrote: On 2013-01-18 03:05:47 +0900, Fujii Masao wrote: I encountered the problem that the timeline switch is not performed expectedly. I set up one master, one standby and one cascade standby. All the servers share the archive directory. restore_command is

Re: [HACKERS] Removing PD_ALL_VISIBLE

2013-01-17 Thread Simon Riggs
On 17 January 2013 15:14, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 17.01.2013 16:53, Robert Haas wrote: On Thu, Jan 17, 2013 at 3:49 AM, Pavan Deolasee pavan.deola...@gmail.com wrote: May be you've already addressed that concern with the proven performance numbers, but I'm not

Re: [HACKERS] Removing PD_ALL_VISIBLE

2013-01-17 Thread Jeff Davis
On Thu, 2013-01-17 at 17:14 +0200, Heikki Linnakangas wrote: I don't remember if I ever actually tested that though. Maybe I was worrying about nothing and hitting the VM page on every update is ok. I tried, but was unable to show really anything at all, even without keeping the VM page

Re: [HACKERS] [PATCH]Tablesample Submission

2013-01-17 Thread Josh Berkus
So I can't see this going anywhere for 9.3. I've moved it to CF1 of 9.4 marked Waiting on Author Agreed. I wish I'd noticed that it got lost earlier. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make

Re: [HACKERS] 9.3 Pre-proposal: Range Merge Join

2013-01-17 Thread Stefan Keller
Hi Jeff 2012/4/19 Jeff Davis pg...@j-davis.com: On Wed, 2012-04-18 at 01:21 -0400, Tom Lane wrote: (...) This is just handwaving of course. I think some digging in the spatial-join literature would likely find ideas better than any of these. I will look in some more detail. The merge-like

Re: [HACKERS] Event Triggers: adding information

2013-01-17 Thread Tom Lane
Simon Riggs si...@2ndquadrant.com writes: On 17 January 2013 16:15, Robert Haas robertmh...@gmail.com wrote: It pains me that I've evidently failed to communicate this concept clearly despite a year or more of trying. Does that make sense? Is there some way I can make this more clear? The

Re: [HACKERS] Event Triggers: adding information

2013-01-17 Thread Robert Haas
On Thu, Jan 17, 2013 at 12:06 PM, Dimitri Fontaine dimi...@2ndquadrant.fr wrote: Ok. Will prepare a non controversial patch for ddl_command_end. Thanks. I will make a forceful effort to review that in a timely fashion when it's posted. I think this is a bad idea, not only because, as I said

Re: [HACKERS] [sepgsql 1/3] add name qualified creation label

2013-01-17 Thread Kohei KaiGai
2013/1/16 Robert Haas robertmh...@gmail.com: On Tue, Jan 15, 2013 at 3:02 PM, Kohei KaiGai kai...@kaigai.gr.jp wrote: This patch adds sepgsql the feature of name qualified creation label. Background, on creation of a certain database object, sepgsql assigns a default security label according

Re: [HACKERS] Event Triggers: adding information

2013-01-17 Thread Dimitri Fontaine
Robert Haas robertmh...@gmail.com writes: Goal: Every time an ALTER command is used on object *that actually exists*, we will call some user-defined function and pass the object type, the OID of the object, and some details about what sort of alteration the user has requested. Ok, in current

[HACKERS] HS locking broken in HEAD

2013-01-17 Thread Andres Freund
Hi, While checking whether I could reproduce the replication delay reported by Michael Paquier I found this very nice tidbit: In a pretty trivial replication setup of only streaming replication I can currently easily reproduce this: standby# BEGIN;SELECT * FROM foo; BEGIN id | data

Re: [HACKERS] Re: Slave enters in recovery and promotes when WAL stream with master is cut + delay master/slave

2013-01-17 Thread Heikki Linnakangas
On 17.01.2013 21:57, Heikki Linnakangas wrote: On 17.01.2013 20:08, Andres Freund wrote: On 2013-01-18 03:05:47 +0900, Fujii Masao wrote: I encountered the problem that the timeline switch is not performed expectedly. I set up one master, one standby and one cascade standby. All the servers

Re: [HACKERS] Event Triggers: adding information

2013-01-17 Thread Dimitri Fontaine
Tom Lane t...@sss.pgh.pa.us writes: I have to completely disagree with that. If we don't want PostgreSQL to soon subside into an unfixable morass, as I think Brooks puts it, we must *not* simply patch things in a way that expediently provides an approximation of some desired feature. We have

Re: [HACKERS] ALTER command reworks

2013-01-17 Thread Alvaro Herrera
Kohei KaiGai escribió: This attached patch is the rebased one towards the latest master branch. Great, thanks. I played with it a bit and it looks almost done to me. The only issue I can find is that it lets you rename an aggregate by using ALTER FUNCTION, which is supposed to be forbidden.

Re: [HACKERS] Re: Slave enters in recovery and promotes when WAL stream with master is cut + delay master/slave

2013-01-17 Thread Andres Freund
On 2013-01-17 23:49:22 +0200, Heikki Linnakangas wrote: On 17.01.2013 21:57, Heikki Linnakangas wrote: On 17.01.2013 20:08, Andres Freund wrote: On 2013-01-18 03:05:47 +0900, Fujii Masao wrote: I encountered the problem that the timeline switch is not performed expectedly. I set up one

Re: [HACKERS] Event Triggers: adding information

2013-01-17 Thread Simon Riggs
On 17 January 2013 20:24, Tom Lane t...@sss.pgh.pa.us wrote: Simon Riggs si...@2ndquadrant.com writes: On 17 January 2013 16:15, Robert Haas robertmh...@gmail.com wrote: It pains me that I've evidently failed to communicate this concept clearly despite a year or more of trying. Does that make

Re: [HACKERS] Event Triggers: adding information

2013-01-17 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: Let me try to give a concrete example of how I think another firing point could be made to work along the lines I'm suggesting. [ snip description of how an event trigger might safely be fired just after identification and locking of the target object

Re: [HACKERS] Event Triggers: adding information

2013-01-17 Thread Tom Lane
Dimitri Fontaine dimi...@2ndquadrant.fr writes: Tom Lane t...@sss.pgh.pa.us writes: I think that we're not realistically going to be able to introduce event triggers in very many of the places we'd like to have them without first doing a lot of fundamental refactoring. We're only talking

Re: [HACKERS] Event Triggers: adding information

2013-01-17 Thread Tom Lane
Simon Riggs si...@2ndquadrant.com writes: On 17 January 2013 20:24, Tom Lane t...@sss.pgh.pa.us wrote: My comments were in response to this I don't really agree with that. I think the point is to expose what the system is doing to the DBA. I'm OK with exposing the fact that creating a

Re: [HACKERS] Event Triggers: adding information

2013-01-17 Thread Dimitri Fontaine
Tom Lane t...@sss.pgh.pa.us writes: Well, that's already a problem, because as Robert keeps saying, what goes through utility.c and what doesn't is pretty random right at the moment, and we shouldn't expose that behavior to users for fear of not being able to change it later. It didn't feel

Re: [HACKERS] Removing PD_ALL_VISIBLE

2013-01-17 Thread Jeff Davis
On Thu, 2013-01-17 at 19:58 +, Simon Riggs wrote: Presumably we remember the state of the VM so we can skip the re-visit after every write? That was not a part of my patch, although I remember that you mentioned that previously and I thought it could be a good way to mitigate a problem if

Re: [HACKERS] Removing PD_ALL_VISIBLE

2013-01-17 Thread Jeff Davis
On Thu, 2013-01-17 at 09:53 -0500, Robert Haas wrote: The main question in my mind is whether there are any negative consequences to holding a VM buffer pin for that long without interruption. The usual consideration - namely, blocking vacuum - doesn't apply here because vacuum does not take

Re: [HACKERS] HS locking broken in HEAD

2013-01-17 Thread Andres Freund
On 2013-01-17 22:46:21 +0100, Andres Freund wrote: Hi, While checking whether I could reproduce the replication delay reported by Michael Paquier I found this very nice tidbit: In a pretty trivial replication setup of only streaming replication I can currently easily reproduce this:

Re: [HACKERS] Event Triggers: adding information

2013-01-17 Thread Tom Lane
Dimitri Fontaine dimi...@2ndquadrant.fr writes: Tom Lane t...@sss.pgh.pa.us writes: Well, that's already a problem, because as Robert keeps saying, what goes through utility.c and what doesn't is pretty random right at the moment, and we shouldn't expose that behavior to users for fear of not

Re: [HACKERS] HS locking broken in HEAD

2013-01-17 Thread Andres Freund
On 2013-01-17 23:56:16 +0100, Andres Freund wrote: On 2013-01-17 22:46:21 +0100, Andres Freund wrote: ^ Note the conflicting locks held on relation foo by 28048 and 28068. I don't immediately know which patch to blame here? Looks a bit like broken fastpath locking, but I

Re: [HACKERS] Re: Slave enters in recovery and promotes when WAL stream with master is cut + delay master/slave

2013-01-17 Thread Michael Paquier
On Fri, Jan 18, 2013 at 3:05 AM, Fujii Masao masao.fu...@gmail.com wrote: I encountered the problem that the timeline switch is not performed expectedly. I set up one master, one standby and one cascade standby. All the servers share the archive directory. restore_command is specified in the

Re: [HACKERS] could not create directory ...: File exists

2013-01-17 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: This seems to provide a reasonably principled argument why we might want to fix this case with a localized use of an MVCC scan before we have such a fix globally. I had discussed that idea a bit with Andres on IRC and my only concern was if there's some

[HACKERS] Re: Slave enters in recovery and promotes when WAL stream with master is cut + delay master/slave

2013-01-17 Thread Andres Freund
On 2013-01-18 08:24:31 +0900, Michael Paquier wrote: On Fri, Jan 18, 2013 at 3:05 AM, Fujii Masao masao.fu...@gmail.com wrote: I encountered the problem that the timeline switch is not performed expectedly. I set up one master, one standby and one cascade standby. All the servers share

Re: [HACKERS] PATCH: optimized DROP of multiple tables within a transaction

2013-01-17 Thread Tomas Vondra
On 17.1.2013 20:19, Alvaro Herrera wrote: I'm curious -- why would you drop tables in groups of 100 instead of just doing the 100,000 in a single transaction? Maybe that's faster now, because you'd do a single scan of the buffer pool instead of 1000? (I'm assuming that in groups of means

Re: [HACKERS] could not create directory ...: File exists

2013-01-17 Thread Tom Lane
Stephen Frost sfr...@snowman.net writes: * Tom Lane (t...@sss.pgh.pa.us) wrote: This seems to provide a reasonably principled argument why we might want to fix this case with a localized use of an MVCC scan before we have such a fix globally. I had discussed that idea a bit with Andres on

[HACKERS] Re: Slave enters in recovery and promotes when WAL stream with master is cut + delay master/slave

2013-01-17 Thread Michael Paquier
On Fri, Jan 18, 2013 at 8:48 AM, Andres Freund and...@2ndquadrant.comwrote: Can you reproduce that one with 7fcbf6a^ (i.e before xlogreader got split off?). Yes, it is reproducible before the xlog reader split. Just an additional report, the master jumps correctly to the new timeline. The

[HACKERS] Re: Slave enters in recovery and promotes when WAL stream with master is cut + delay master/slave

2013-01-17 Thread Andres Freund
On 2013-01-18 08:24:31 +0900, Michael Paquier wrote: On Fri, Jan 18, 2013 at 3:05 AM, Fujii Masao masao.fu...@gmail.com wrote: I encountered the problem that the timeline switch is not performed expectedly. I set up one master, one standby and one cascade standby. All the servers share

Re: [HACKERS] review: pgbench - aggregation of info written into log

2013-01-17 Thread Tatsuo Ishii
On Thu, Jan 17, 2013 at 2:29 PM, Andrew Dunstan and...@dunslane.net wrote: On 01/17/2013 06:04 AM, Tomas Vondra wrote: The problem is I have access to absolutely no Windows machines, not mentioning the development tools (and that I have no clue about it). I vaguely remember there were

Re: [HACKERS] patch to add \watch to psql

2013-01-17 Thread Daniel Farina
I have adjusted this patch a little bit to take care of the review issues, along with just doing a bit of review myself. On Thu, Oct 25, 2012 at 2:25 AM, Will Leinweber w...@heroku.com wrote: Thanks for the reviews and comments. Responses inline: . On Sat, Oct 20, 2012 at 9:19 AM, Abhijit

Re: [HACKERS] Event Triggers: adding information

2013-01-17 Thread Robert Haas
On Thu, Jan 17, 2013 at 4:43 PM, Dimitri Fontaine dimi...@2ndquadrant.fr wrote: Robert Haas robertmh...@gmail.com writes: Goal: Every time an ALTER command is used on object *that actually exists*, we will call some user-defined function and pass the object type, the OID of the object, and

  1   2   >