Re: [HACKERS] Removing PD_ALL_VISIBLE

2013-01-17 Thread Simon Riggs
On 17 January 2013 03:02, Jeff Davis wrote: > Rebased patch attached. No significant changes. Jeff, can you summarise/collate why we're doing this, what concerns it raises and how you've dealt with them? That will help decide whether to commit. Thanks -- Simon Riggs http://

Re: [HACKERS] CF3+4

2013-01-17 Thread Magnus Hagander
On Jan 17, 2013 8:15 AM, "Abhijit Menon-Sen" wrote: > > At 2013-01-17 16:05:05 +0900, michael.paqu...@gmail.com wrote: > > > > Is it really necessary to create a new commit fest just to move the > > items? Marking the patches that are considered as being too late for > > 9.3 should be just returne

Re: [HACKERS] Removing PD_ALL_VISIBLE

2013-01-17 Thread Pavan Deolasee
On Thu, Jan 17, 2013 at 2:11 PM, Simon Riggs wrote: > > On 17 January 2013 03:02, Jeff Davis wrote: > > > Rebased patch attached. No significant changes. > > Jeff, can you summarise/collate why we're doing this, what concerns it > raises and how you've dealt with them? That will help decide wheth

Re: [HACKERS] CF3+4 (was Re: Parallel query execution)

2013-01-17 Thread Magnus Hagander
On Thu, Jan 17, 2013 at 8:19 AM, Pavan Deolasee wrote: > > On Wed, Jan 16, 2013 at 1:51 PM, Abhijit Menon-Sen > wrote: >> >> At 2013-01-16 02:07:29 -0500, t...@sss.pgh.pa.us wrote: >> > >> > In case you hadn't noticed, we've totally lost control of >> > the CF process. >> >> What can we do to get

Re: [HACKERS] Removing PD_ALL_VISIBLE

2013-01-17 Thread Abhijit Menon-Sen
At 2013-01-17 08:41:37 +, si...@2ndquadrant.com wrote: > > Jeff, can you summarise/collate why we're doing this, what concerns it > raises and how you've dealt with them? Since I was just looking at the original patch and discussion, and since Pavan has posted an excerpt from one objection to

Re: [HACKERS] review: pgbench - aggregation of info written into log

2013-01-17 Thread Magnus Hagander
On Thu, Jan 17, 2013 at 2:35 AM, Tatsuo Ishii wrote: This might be way more than we want to do, but there is an article that describes some techniques for doing what seems to be missing (AIUI): >>> Even this would be

Re: [HACKERS] Removing PD_ALL_VISIBLE

2013-01-17 Thread Pavan Deolasee
On Thu, Jan 17, 2013 at 2:57 PM, Abhijit Menon-Sen wrote: > > > There was considerable discussion after this (accessible through the > archives link above), which I won't attempt to summarise. > > I thought Robert made those comments after considerable discussions on Jeff's approach. So he probabl

Re: [HACKERS] review: pgbench - aggregation of info written into log

2013-01-17 Thread Dave Page
On Thu, Jan 17, 2013 at 9:36 AM, Magnus Hagander wrote: > On Thu, Jan 17, 2013 at 2:35 AM, Tatsuo Ishii wrote: > This might be way more than we want to do, but there is an article > that describes some techniques for doing what seems to be missing > (AIUI): > >

Re: [HACKERS] review: pgbench - aggregation of info written into log

2013-01-17 Thread Magnus Hagander
On Thu, Jan 17, 2013 at 11:09 AM, Dave Page wrote: > On Thu, Jan 17, 2013 at 9:36 AM, Magnus Hagander wrote: >> On Thu, Jan 17, 2013 at 2:35 AM, Tatsuo Ishii wrote: >> This might be way more than we want to do, but there is an article >> that describes some techniques for doing what seem

Re: [HACKERS] Event Triggers: adding information

2013-01-17 Thread Dimitri Fontaine
Robert Haas writes: > - adds ddl_command_trace and ddl_command_end events > - causes those events to be called not only for actual SQL commands > but also for things that happen, at present, to go through the same > code path > - adds additional magic variables to PL/pgsql to expose new > informat

Re: [HACKERS] Multiple --table options for other commands

2013-01-17 Thread Magnus Hagander
On Fri, Dec 14, 2012 at 4:14 PM, Karl O. Pinc wrote: > On 12/13/2012 11:02:56 PM, Josh Kupershmidt wrote: >> On Thu, Dec 13, 2012 at 9:03 PM, Karl O. Pinc wrote: > >> > Sorry to be so persnickety, and unhelpful until now. >> > It seemed like it should be doable, but something >> > was going wrong

Re: [HACKERS] Hot Standby conflict resolution handling

2013-01-17 Thread Andres Freund
On 2013-01-17 01:38:31 -0500, Tom Lane wrote: > But having said that ... are we sure this code is not actually broken? > ISTM that if we dare not interrupt for fear of confusing OpenSSL, we > cannot safely attempt to send an error message to the client either; > but ereport(FATAL) will try exactly

[HACKERS] Re: Slave enters in recovery and promotes when WAL stream with master is cut + delay master/slave

2013-01-17 Thread Andres Freund
On 2013-01-17 13:47:41 +0900, Michael Paquier wrote: > Hi all, > > There is a strange bug with the latest master head (commit 7fcbf6a). > When the WAL stream with a master is cut on a slave, slave returns a FATAL > (well normal...), but then enters in recovery process and automatically > promotes.

Re: [HACKERS] review: pgbench - aggregation of info written into log

2013-01-17 Thread Tomas Vondra
Dne 17.01.2013 10:36, Magnus Hagander napsal: On Thu, Jan 17, 2013 at 2:35 AM, Tatsuo Ishii wrote: This might be way more than we want to do, but there is an article that describes some techniques for doing what seems to be missing (AIUI):

Re: [HACKERS] review: pgbench - aggregation of info written into log

2013-01-17 Thread Tomas Vondra
Dne 17.01.2013 11:16, Magnus Hagander napsal: On Thu, Jan 17, 2013 at 11:09 AM, Dave Page wrote: On Thu, Jan 17, 2013 at 9:36 AM, Magnus Hagander wrote: We have committed platform-specific features before, but generally only when it's not *possible* to do them for all platforms. For example t

Re: [HACKERS] Materialized views WIP patch

2013-01-17 Thread Robert Haas
On Wed, Jan 16, 2013 at 1:38 PM, Tom Lane wrote: >> Where I really need someone to hit me upside the head with a >> clue-stick is the code I added to the bottom of RelationBuildDesc() >> in relcache.c. The idea is that on first access to an unlogged MV, >> to detect that the heap has been replaced

Re: [HACKERS] review: pgbench - aggregation of info written into log

2013-01-17 Thread Andrew Dunstan
On 01/17/2013 06:11 AM, Tomas Vondra wrote: Dne 17.01.2013 11:16, Magnus Hagander napsal: On Thu, Jan 17, 2013 at 11:09 AM, Dave Page wrote: On Thu, Jan 17, 2013 at 9:36 AM, Magnus Hagander wrote: We have committed platform-specific features before, but generally only when it's not *possibl

[HACKERS] Re: Slave enters in recovery and promotes when WAL stream with master is cut + delay master/slave

2013-01-17 Thread Andres Freund
On 2013-01-17 13:47:41 +0900, Michael Paquier wrote: > Hi all, > > There is a strange bug with the latest master head (commit 7fcbf6a). > When the WAL stream with a master is cut on a slave, slave returns a FATAL > (well normal...), but then enters in recovery process and automatically > promotes.

Re: [HACKERS] CF3+4

2013-01-17 Thread Craig Ringer
On 01/17/2013 04:43 PM, Magnus Hagander wrote: > > > On Jan 17, 2013 8:15 AM, "Abhijit Menon-Sen" > wrote: > > > > At 2013-01-17 16:05:05 +0900, michael.paqu...@gmail.com > wrote: > > > > > > Is it really necessary to create a new com

Re: [HACKERS] review: pgbench - aggregation of info written into log

2013-01-17 Thread Andrew Dunstan
On 01/17/2013 06:04 AM, Tomas Vondra wrote: The problem is I have access to absolutely no Windows machines, not mentioning the development tools (and that I have no clue about it). I vaguely remember there were people on this list doing Windows development on a virtual machine or something. An

Re: [HACKERS] review: pgbench - aggregation of info written into log

2013-01-17 Thread Dave Page
On Thu, Jan 17, 2013 at 1:29 PM, Andrew Dunstan wrote: > > On 01/17/2013 06:04 AM, Tomas Vondra wrote: >> >> The problem is I have access to absolutely no Windows machines, >> not mentioning the development tools (and that I have no clue about it). >> >> I vaguely remember there were people on thi

Re: [HACKERS] review: pgbench - aggregation of info written into log

2013-01-17 Thread Magnus Hagander
On Thu, Jan 17, 2013 at 2:29 PM, Andrew Dunstan wrote: > > On 01/17/2013 06:04 AM, Tomas Vondra wrote: >> >> The problem is I have access to absolutely no Windows machines, >> not mentioning the development tools (and that I have no clue about it). >> >> I vaguely remember there were people on thi

Re: [HACKERS] small pg_basebackup display bug

2013-01-17 Thread Magnus Hagander
On Sun, Dec 16, 2012 at 7:20 PM, Tom Lane wrote: > Magnus Hagander writes: >> On Sat, Dec 15, 2012 at 2:24 PM, Erik Rijkers wrote: >>> That would make such a truncation less frequent, and after all a truncated >>> display is not >>> particular useful. > >> Agreed - it's useful during testing, b

Re: [HACKERS] default SSL compression (was: libpq compression)

2013-01-17 Thread Magnus Hagander
On Wed, Jan 2, 2013 at 3:17 PM, Magnus Hagander wrote: > On Wed, Jan 2, 2013 at 3:15 PM, Noah Misch wrote: >> On Wed, Jan 02, 2013 at 02:03:20PM +0100, Magnus Hagander wrote: >>> On Wed, Jan 2, 2013 at 1:15 AM, Tom Lane wrote: >>> > So +1 for changing it to "DEFAULT" from me, too. There's no re

[HACKERS] could not create directory "...": File exists

2013-01-17 Thread Stephen Frost
Greetings, We've come across this rather annoying error happening during our builds: ERROR: could not create directory "pg_tblspc/25120/PG_9.3_201212081/231253": File exists It turns out that this is coming from copydir() when called by createdb() during a CREATE DATABASE .. FROM TEM

Re: [HACKERS] Removing PD_ALL_VISIBLE

2013-01-17 Thread Robert Haas
On Thu, Jan 17, 2013 at 3:49 AM, Pavan Deolasee wrote: > May be you've already addressed that concern with the proven > performance numbers, but I'm not sure. It would be nice to hear what Heikki's reasons were for adding PD_ALL_VISIBLE in the first place. Jeff's approach of holding the VM pins

Re: [HACKERS] CF3+4

2013-01-17 Thread Robert Haas
On Thu, Jan 17, 2013 at 8:17 AM, Craig Ringer wrote: > I've moved all pending patches from 2012-11 to 2013-01. I'll go through and > poke them for aliveness and start chasing things up; in the mean time, any > chance of closing 2012-11? Done. -- Robert Haas EnterpriseDB: http://www.enterprisedb

Re: [HACKERS] Teaching pg_receivexlog to follow timeline switches

2013-01-17 Thread Robert Haas
On Wed, Jan 16, 2013 at 11:08 AM, Heikki Linnakangas wrote: > I'd prefer to leave the .partial suffix in place, as the segment really > isn't complete. It doesn't make a difference when you recover to the latest > timeline, but if you have a more complicated scenario with multiple > timelines that

Re: [HACKERS] Teaching pg_receivexlog to follow timeline switches

2013-01-17 Thread Heikki Linnakangas
On 17.01.2013 16:56, Robert Haas wrote: On Wed, Jan 16, 2013 at 11:08 AM, Heikki Linnakangas wrote: I'd prefer to leave the .partial suffix in place, as the segment really isn't complete. It doesn't make a difference when you recover to the latest timeline, but if you have a more complicated s

Re: [HACKERS] Removing PD_ALL_VISIBLE

2013-01-17 Thread Heikki Linnakangas
On 17.01.2013 16:53, Robert Haas wrote: On Thu, Jan 17, 2013 at 3:49 AM, Pavan Deolasee wrote: May be you've already addressed that concern with the proven performance numbers, but I'm not sure. It would be nice to hear what Heikki's reasons were for adding PD_ALL_VISIBLE in the first place.

Re: [HACKERS] Teaching pg_receivexlog to follow timeline switches

2013-01-17 Thread Robert Haas
On Thu, Jan 17, 2013 at 9:59 AM, Heikki Linnakangas wrote: > The scenario I described is that you screwed up your failover environment, > and end up with a split-brain situation by accident. The DBA certainly needs > to be involved to recover from that. OK, I agree, but I still think a lot of DBA

Re: [HACKERS] Re: Slave enters in recovery and promotes when WAL stream with master is cut + delay master/slave

2013-01-17 Thread Heikki Linnakangas
On 17.01.2013 15:05, Andres Freund wrote: On 2013-01-17 13:47:41 +0900, Michael Paquier wrote: I think that bug has been introduced by commit 7fcbf6a. Before splitting xlog reading as a separate facility things worked correctly. There are also no delay problems before this commit. Ok, my inkli

Re: [HACKERS] Hot Standby conflict resolution handling

2013-01-17 Thread Tom Lane
Pavan Deolasee writes: > On Thu, Jan 17, 2013 at 12:08 PM, Tom Lane wrote: >> ISTM that if we dare not interrupt for fear of confusing OpenSSL, we >> cannot safely attempt to send an error message to the client either; >> but ereport(FATAL) will try exactly that. > I thought since FATAL will for

[HACKERS] Re: Slave enters in recovery and promotes when WAL stream with master is cut + delay master/slave

2013-01-17 Thread Andres Freund
On 2013-01-17 17:18:14 +0200, Heikki Linnakangas wrote: > On 17.01.2013 15:05, Andres Freund wrote: > >On 2013-01-17 13:47:41 +0900, Michael Paquier wrote: > >>I think that bug has been introduced by commit 7fcbf6a. > >>Before splitting xlog reading as a separate facility things worked > >>correctl

Re: [HACKERS] Re: Slave enters in recovery and promotes when WAL stream with master is cut + delay master/slave

2013-01-17 Thread Andres Freund
On 2013-01-17 16:23:44 +0100, Andres Freund wrote: > On 2013-01-17 17:18:14 +0200, Heikki Linnakangas wrote: > > On 17.01.2013 15:05, Andres Freund wrote: > > >On 2013-01-17 13:47:41 +0900, Michael Paquier wrote: > > >>I think that bug has been introduced by commit 7fcbf6a. > > >>Before splitting x

Re: [HACKERS] could not create directory "...": File exists

2013-01-17 Thread Tom Lane
Stephen Frost writes: > It turns out that createdb() currently only takes an AccessShareLock > on pg_tablespace when scanning it with SnapshotNow, making it possible > for a concurrent process to make some uninteresting modification to a > tablespace (such as an ACL change) which will caus

Re: [HACKERS] Hot Standby conflict resolution handling

2013-01-17 Thread Andres Freund
On 2013-01-17 10:19:23 -0500, Tom Lane wrote: > Pavan Deolasee writes: > > On Thu, Jan 17, 2013 at 12:08 PM, Tom Lane wrote: > >> ISTM that if we dare not interrupt for fear of confusing OpenSSL, we > >> cannot safely attempt to send an error message to the client either; > >> but ereport(FATAL)

Re: [HACKERS] Materialized views WIP patch

2013-01-17 Thread Thom Brown
On 16 January 2013 17:25, Thom Brown wrote: > On 16 January 2013 17:20, Kevin Grittner wrote: > >> Thom Brown wrote: >> >> > Some weirdness: >> > >> > postgres=# CREATE VIEW v_test2 AS SELECT 1 moo; >> > CREATE VIEW >> > postgres=# CREATE MATERIALIZED VIEW mv_test2 AS SELECT moo, 2*moo FROM >> >

Re: [HACKERS] Event Triggers: adding information

2013-01-17 Thread Robert Haas
On Thu, Jan 17, 2013 at 5:18 AM, Dimitri Fontaine wrote: > Now, if that's what it takes, I'll spend time on it. In which exact > order do you want to be reviewing and applying that series of patches? Let's agree on which things we even want to do first. Here's my take: - adds ddl_command_trace

Re: [HACKERS] Event Triggers: adding information

2013-01-17 Thread Andres Freund
On 2013-01-17 11:15:24 -0500, Robert Haas wrote: > As a further example, suppose that in 9.4 (or 9.17) we add a command > "DROP TABLES IN SCHEMA fred WHERE name LIKE 'bob%'". Well, the > logging trigger still just works (because it's only writing the > statement, without caring about its contents)

Re: [HACKERS] Re: Slave enters in recovery and promotes when WAL stream with master is cut + delay master/slave

2013-01-17 Thread Heikki Linnakangas
On 17.01.2013 17:42, Andres Freund wrote: Ok, the attached patch seems to fix a) and b). c) above is bogus, as explained in a comment in the patch. I also noticed that the TLI check didn't mark the last source as failed. This looks fragile: /*

Re: [HACKERS] Re: Slave enters in recovery and promotes when WAL stream with master is cut + delay master/slave

2013-01-17 Thread Andres Freund
On 2013-01-17 18:33:42 +0200, Heikki Linnakangas wrote: > On 17.01.2013 17:42, Andres Freund wrote: > >Ok, the attached patch seems to fix a) and b). c) above is bogus, as > >explained in a comment in the patch. I also noticed that the TLI check > >didn't mark the last source as failed. > > This

Re: [HACKERS] Latex longtable format

2013-01-17 Thread Bruce Momjian
On Sat, Jan 12, 2013 at 07:09:31PM -0500, Bruce Momjian wrote: > I have received several earnest requests over the years for LaTeX > 'longtable' output, and I have just implemented it based on a sample > LaTeX longtable output file. > > I have called it 'latex-longtable' and implemented all the be

Re: [HACKERS] Materialized views WIP patch

2013-01-17 Thread Simon Riggs
On 16 January 2013 05:40, Kevin Grittner wrote: > Here is a new version of the patch, with most issues discussed in > previous posts fixed. Looks good. The patch implements one kind of MV. In the future, we hope to have other features or other kinds of MV alongside this: * Snapshot MV - built o

Re: [HACKERS] Re: Slave enters in recovery and promotes when WAL stream with master is cut + delay master/slave

2013-01-17 Thread Heikki Linnakangas
On 17.01.2013 18:42, Andres Freund wrote: On 2013-01-17 18:33:42 +0200, Heikki Linnakangas wrote: On 17.01.2013 17:42, Andres Freund wrote: Ok, the attached patch seems to fix a) and b). c) above is bogus, as explained in a comment in the patch. I also noticed that the TLI check didn't mark th

[HACKERS] Re: Slave enters in recovery and promotes when WAL stream with master is cut + delay master/slave

2013-01-17 Thread Andres Freund
On 2013-01-17 18:50:35 +0200, Heikki Linnakangas wrote: > On 17.01.2013 18:42, Andres Freund wrote: > >On 2013-01-17 18:33:42 +0200, Heikki Linnakangas wrote: > >>On 17.01.2013 17:42, Andres Freund wrote: > >>>Ok, the attached patch seems to fix a) and b). c) above is bogus, as > >>>explained in a

Re: [HACKERS] Event Triggers: adding information

2013-01-17 Thread Dimitri Fontaine
Robert Haas writes: > - adds ddl_command_trace and ddl_command_end events > > I think ddl_command_end is OK and I'm willing to commit that if > extracted as its own patch. I think ddl_command_trace is unnecessary > syntactic sugar. Ok. Will prepare a non controversial patch for ddl_command_end.

[HACKERS] Re: Slave enters in recovery and promotes when WAL stream with master is cut + delay master/slave

2013-01-17 Thread Heikki Linnakangas
On 17.01.2013 18:55, Andres Freund wrote: On 2013-01-17 18:50:35 +0200, Heikki Linnakangas wrote: I was thinking of the attached. As long as we check for CheckForStandbyTrigger() after the "record == NULL" check, we won't perform extra stat() calls on successful reads, only when we're polling af

[HACKERS] Re: Slave enters in recovery and promotes when WAL stream with master is cut + delay master/slave

2013-01-17 Thread Andres Freund
On 2013-01-17 13:47:41 +0900, Michael Paquier wrote: > Slave does not try anymore to reconnect to master with messages of the type: > FATAL: could not connect to the primary server > > I also noticed that there is some delay until modifications on master are > visible on slave. > I think that bug

Re: [HACKERS] could not create directory "...": File exists

2013-01-17 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: > Ugh. Still another problem with non-MVCC catalog scans. Indeed. > It seems that the only thing we actually use from each tuple is the OID. Yes, that's true. > So there are other ways to fix it, of which probably the minimum-change > one is to keep a lis

Re: [HACKERS] Re: Slave enters in recovery and promotes when WAL stream with master is cut + delay master/slave

2013-01-17 Thread Fujii Masao
On Fri, Jan 18, 2013 at 2:35 AM, Andres Freund wrote: > On 2013-01-17 13:47:41 +0900, Michael Paquier wrote: >> Slave does not try anymore to reconnect to master with messages of the type: >> FATAL: could not connect to the primary server >> >> I also noticed that there is some delay until modifi

Re: [HACKERS] Re: Slave enters in recovery and promotes when WAL stream with master is cut + delay master/slave

2013-01-17 Thread Andres Freund
On 2013-01-18 03:05:47 +0900, Fujii Masao wrote: > On Fri, Jan 18, 2013 at 2:35 AM, Andres Freund wrote: > > On 2013-01-17 13:47:41 +0900, Michael Paquier wrote: > >> Slave does not try anymore to reconnect to master with messages of the > >> type: > >> FATAL: could not connect to the primary se

Re: [HACKERS] tuplesort memory usage: grow_memtuples

2013-01-17 Thread Tom Lane
Peter Geoghegan writes: > On 8 December 2012 14:41, Andres Freund wrote: >> Is anybody planning to work on this? There hasn't been any activity >> since the beginning of the CF and it doesn't look like there is much >> work left? > I took another look at this. Applied with some changes: * Use

Re: [HACKERS] [PATCH]Tablesample Submission

2013-01-17 Thread Josh Berkus
On 11/04/2012 07:22 PM, Qi Huang wrote: > Dear hackers Sorry for not replying the patch review. I didn't see the > review until recently as my mail box is full of Postgres mails and I didn't > notice the one for me, my mail box configuration problem. I am still kind > of busy with my uni

Re: [HACKERS] Removing PD_ALL_VISIBLE

2013-01-17 Thread Jeff Davis
On Thu, 2013-01-17 at 15:25 +0530, Pavan Deolasee wrote: > Now that I look at the patch, I wonder if there is another fundamental > issue with the patch. Since the patch removes WAL logging for the VM > set operation, this can happen: > Thank you. I think I was confused by this comment here: "Whe

Re: [HACKERS] could not create directory "...": File exists

2013-01-17 Thread Tom Lane
Stephen Frost writes: > * Tom Lane (t...@sss.pgh.pa.us) wrote: >> [ thinks for a bit... ] Ugh, no, because the *other* risk you've got >> here is not seeing a row at all, which would be really bad. > I'm not sure that I see how that could happen..? I agree that it'd be > really bad if it did th

Re: [HACKERS] tuplesort memory usage: grow_memtuples

2013-01-17 Thread Peter Geoghegan
On 17 January 2013 18:22, Tom Lane wrote: > Applied with some changes: Thank you. That feedback is useful. -- Peter Geoghegan http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training and Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To

Re: [HACKERS] could not create directory "...": File exists

2013-01-17 Thread Andres Freund
On 2013-01-17 13:46:44 -0500, Tom Lane wrote: > Or maybe we should just write this off as a case we can't realistically > fix before we have MVCC catalog scans. It seems that any other fix is > going to be hopelessly ugly. ISTM we could just use a MVCC snapshot in this specific case? Andres --

Re: [HACKERS] could not create directory "...": File exists

2013-01-17 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: > The case where you see a tuple twice is if an update drops a new version > of a row beyond your seqscan, and then commits before you get to the new > version. But if it drops the new version of the row *behind* your > seqscan, and then commits before you ge

Re: [HACKERS] [PATCH]Tablesample Submission

2013-01-17 Thread Simon Riggs
On 17 January 2013 18:32, Josh Berkus wrote: > On 11/04/2012 07:22 PM, Qi Huang wrote: >> Dear hackers Sorry for not replying the patch review. I didn't see the >> review until recently as my mail box is full of Postgres mails and I didn't >> notice the one for me, my mail box configuration

Re: [HACKERS] PATCH: optimized DROP of multiple tables within a transaction

2013-01-17 Thread Alvaro Herrera
Tomas Vondra wrote: > Hi, > > attached is a patch that improves performance when dropping multiple > tables within a transaction. Instead of scanning the shared buffers for > each table separately, the patch removes this and evicts all the tables > in a single pass through shared buffers. Made so

Re: [HACKERS] Removing PD_ALL_VISIBLE

2013-01-17 Thread Jeff Davis
tion, this can happen: > > > Thank you. New patch attached with simple WAL logging. Regards, Jeff Davis rm-pd-all-visible-20130117.patch.gz Description: GNU Zip compressed data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make chan

Re: [HACKERS] could not create directory "...": File exists

2013-01-17 Thread Tom Lane
Stephen Frost writes: > * Tom Lane (t...@sss.pgh.pa.us) wrote: >> Or maybe we should just write this off as a case we can't realistically >> fix before we have MVCC catalog scans. It seems that any other fix is >> going to be hopelessly ugly. > I feel like we should be able to do better than wha

Re: [HACKERS] PATCH: optimized DROP of multiple tables within a transaction

2013-01-17 Thread Tom Lane
Alvaro Herrera writes: > Made some tweaks and pushed (added comments to new functions, ensure > that we never try to palloc(0), renamed DropRelFileNodeAllBuffers to > plural, made the "use bsearch" logic a bit simpler). FWIW, there's nothing particularly wrong with palloc(0) ...

Re: [HACKERS] Event Triggers: adding information

2013-01-17 Thread Simon Riggs
On 17 January 2013 16:15, Robert Haas wrote: > As a further example, suppose that in 9.4 (or 9.17) we add a command > "DROP TABLES IN SCHEMA fred WHERE name LIKE 'bob%'". Well, the > logging trigger still just works (because it's only writing the > statement, without caring about its contents).

Re: [HACKERS] Teaching pg_receivexlog to follow timeline switches

2013-01-17 Thread Alvaro Herrera
Robert Haas escribió: > Actually, I'm really glad to see all the work you've done to improve > the way that some of these scenarios work and eliminate various bugs > and other surprising failure modes over the last couple of months. > It's great stuff. +1 -- Álvaro Herrerahttp:/

Re: [HACKERS] could not create directory "...": File exists

2013-01-17 Thread Tom Lane
I wrote: > Stephen Frost writes: >> I feel like we should be able to do better than what we have now, at >> least. Using ShareLock prevented the specific case that we were >> experiencing and is therefore MUCH better (for us, anyway) than >> released versions where we ran into the error on a regu

Re: [HACKERS] Re: Slave enters in recovery and promotes when WAL stream with master is cut + delay master/slave

2013-01-17 Thread Heikki Linnakangas
On 17.01.2013 20:08, Andres Freund wrote: On 2013-01-18 03:05:47 +0900, Fujii Masao wrote: I encountered the problem that the timeline switch is not performed expectedly. I set up one master, one standby and one cascade standby. All the servers share the archive directory. restore_command is spe

Re: [HACKERS] Removing PD_ALL_VISIBLE

2013-01-17 Thread Simon Riggs
On 17 January 2013 15:14, Heikki Linnakangas wrote: > On 17.01.2013 16:53, Robert Haas wrote: >> >> On Thu, Jan 17, 2013 at 3:49 AM, Pavan Deolasee >> wrote: >>> >>> May be you've already addressed that concern with the proven >>> performance numbers, but I'm not sure. >> >> >> It would be nice

Re: [HACKERS] Removing PD_ALL_VISIBLE

2013-01-17 Thread Jeff Davis
On Thu, 2013-01-17 at 17:14 +0200, Heikki Linnakangas wrote: > I don't remember if I ever actually tested that > though. Maybe I was worrying about nothing and hitting the VM page on > every update is ok. I tried, but was unable to show really anything at all, even without keeping the VM page pi

Re: [HACKERS] [PATCH]Tablesample Submission

2013-01-17 Thread Josh Berkus
> So I can't see this going anywhere for 9.3. I've moved it to CF1 of > 9.4 marked Waiting on Author Agreed. I wish I'd noticed that it got lost earlier. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make

Re: [HACKERS] 9.3 Pre-proposal: Range Merge Join

2013-01-17 Thread Stefan Keller
Hi Jeff 2012/4/19 Jeff Davis : > On Wed, 2012-04-18 at 01:21 -0400, Tom Lane wrote: (...) >> This is just handwaving of course. I think some digging in the >> spatial-join literature would likely find ideas better than any of >> these. > > I will look in some more detail. The merge-like approach

Re: [HACKERS] Event Triggers: adding information

2013-01-17 Thread Tom Lane
Simon Riggs writes: > On 17 January 2013 16:15, Robert Haas wrote: >> It pains me that I've evidently failed to communicate this concept >> clearly despite a year or more of trying. Does that make sense? Is >> there some way I can make this more clear? The difference seems very >> clear-cut to

Re: [HACKERS] Event Triggers: adding information

2013-01-17 Thread Robert Haas
On Thu, Jan 17, 2013 at 12:06 PM, Dimitri Fontaine wrote: > Ok. Will prepare a non controversial patch for ddl_command_end. Thanks. I will make a forceful effort to review that in a timely fashion when it's posted. >> I think this is a bad idea, not only because, as I said before, it >> exposes

Re: [HACKERS] [sepgsql 1/3] add name qualified creation label

2013-01-17 Thread Kohei KaiGai
2013/1/16 Robert Haas : > On Tue, Jan 15, 2013 at 3:02 PM, Kohei KaiGai wrote: >> This patch adds sepgsql the feature of name qualified creation label. >> >> Background, on creation of a certain database object, sepgsql assigns >> a default security label according to the security policy that has

Re: [HACKERS] Event Triggers: adding information

2013-01-17 Thread Dimitri Fontaine
Robert Haas writes: > Goal: Every time an ALTER command is used on object *that actually > exists*, we will call some user-defined function and pass the object > type, the OID of the object, and some details about what sort of > alteration the user has requested. Ok, in current terms of the propo

[HACKERS] HS locking broken in HEAD

2013-01-17 Thread Andres Freund
Hi, While checking whether I could reproduce the replication delay reported by Michael Paquier I found this very nice tidbit: In a pretty trivial replication setup of only streaming replication I can currently easily reproduce this: standby# BEGIN;SELECT * FROM foo; BEGIN id | data +-

Re: [HACKERS] Re: Slave enters in recovery and promotes when WAL stream with master is cut + delay master/slave

2013-01-17 Thread Heikki Linnakangas
On 17.01.2013 21:57, Heikki Linnakangas wrote: On 17.01.2013 20:08, Andres Freund wrote: On 2013-01-18 03:05:47 +0900, Fujii Masao wrote: I encountered the problem that the timeline switch is not performed expectedly. I set up one master, one standby and one cascade standby. All the servers sha

Re: [HACKERS] Event Triggers: adding information

2013-01-17 Thread Dimitri Fontaine
Tom Lane writes: > I have to completely disagree with that. If we don't want PostgreSQL > to soon subside into an unfixable morass, as I think Brooks puts it, > we must *not* simply patch things in a way that expediently provides > an approximation of some desired feature. We have to consider, a

Re: [HACKERS] ALTER command reworks

2013-01-17 Thread Alvaro Herrera
Kohei KaiGai escribió: > This attached patch is the rebased one towards the latest master branch. Great, thanks. I played with it a bit and it looks almost done to me. The only issue I can find is that it lets you rename an aggregate by using ALTER FUNCTION, which is supposed to be forbidden. (

Re: [HACKERS] Re: Slave enters in recovery and promotes when WAL stream with master is cut + delay master/slave

2013-01-17 Thread Andres Freund
On 2013-01-17 23:49:22 +0200, Heikki Linnakangas wrote: > On 17.01.2013 21:57, Heikki Linnakangas wrote: > >On 17.01.2013 20:08, Andres Freund wrote: > >>On 2013-01-18 03:05:47 +0900, Fujii Masao wrote: > >>>I encountered the problem that the timeline switch is not performed > >>>expectedly. > >>>I

Re: [HACKERS] Event Triggers: adding information

2013-01-17 Thread Simon Riggs
On 17 January 2013 20:24, Tom Lane wrote: > Simon Riggs writes: >> On 17 January 2013 16:15, Robert Haas wrote: >>> It pains me that I've evidently failed to communicate this concept >>> clearly despite a year or more of trying. Does that make sense? Is >>> there some way I can make this more

Re: [HACKERS] Event Triggers: adding information

2013-01-17 Thread Tom Lane
Robert Haas writes: > Let me try to give a concrete example of how I think another firing > point could be made to work along the lines I'm suggesting. > [ snip description of how an event trigger might safely be fired just > after identification and locking of the target object for an ALTER ] Re

Re: [HACKERS] Event Triggers: adding information

2013-01-17 Thread Tom Lane
Dimitri Fontaine writes: > Tom Lane writes: >> I think that we're not realistically going to be able to introduce >> event triggers in very many of the places we'd like to have them >> without first doing a lot of fundamental refactoring. > We're only talking about ddl_command_start and ddl_comm

Re: [HACKERS] Event Triggers: adding information

2013-01-17 Thread Tom Lane
Simon Riggs writes: > On 17 January 2013 20:24, Tom Lane wrote: > My comments were in response to this >> I don't really agree with that. I think the point is to expose what >> the system is doing to the DBA. I'm OK with exposing the fact that >> creating a table with a serial column also crea

Re: [HACKERS] Event Triggers: adding information

2013-01-17 Thread Dimitri Fontaine
Tom Lane writes: > Well, that's already a problem, because as Robert keeps saying, what > goes through utility.c and what doesn't is pretty random right at the > moment, and we shouldn't expose that behavior to users for fear of not > being able to change it later. It didn't feel that random to m

Re: [HACKERS] Removing PD_ALL_VISIBLE

2013-01-17 Thread Jeff Davis
On Thu, 2013-01-17 at 19:58 +, Simon Riggs wrote: > Presumably we remember the state of the VM so we can skip the re-visit > after every write? That was not a part of my patch, although I remember that you mentioned that previously and I thought it could be a good way to mitigate a problem if

Re: [HACKERS] Removing PD_ALL_VISIBLE

2013-01-17 Thread Jeff Davis
On Thu, 2013-01-17 at 09:53 -0500, Robert Haas wrote: > The main question in my mind is whether > there are any negative consequences to holding a VM buffer pin for > that long without interruption. The usual consideration - namely, > blocking vacuum - doesn't apply here because vacuum does not ta

Re: [HACKERS] HS locking broken in HEAD

2013-01-17 Thread Andres Freund
On 2013-01-17 22:46:21 +0100, Andres Freund wrote: > Hi, > > > While checking whether I could reproduce the replication delay reported > by Michael Paquier I found this very nice tidbit: > > In a pretty trivial replication setup of only streaming replication I > can currently easily reproduce th

Re: [HACKERS] Event Triggers: adding information

2013-01-17 Thread Tom Lane
Dimitri Fontaine writes: > Tom Lane writes: >> Well, that's already a problem, because as Robert keeps saying, what >> goes through utility.c and what doesn't is pretty random right at the >> moment, and we shouldn't expose that behavior to users for fear of not >> being able to change it later.

Re: [HACKERS] HS locking broken in HEAD

2013-01-17 Thread Andres Freund
On 2013-01-17 23:56:16 +0100, Andres Freund wrote: > On 2013-01-17 22:46:21 +0100, Andres Freund wrote: ^ > > Note the conflicting locks held on relation foo by 28048 and 28068. > > > > I don't immediately know which patch to blame here? Looks a bit like > > broken fastpath locking,

Re: [HACKERS] Re: Slave enters in recovery and promotes when WAL stream with master is cut + delay master/slave

2013-01-17 Thread Michael Paquier
On Fri, Jan 18, 2013 at 3:05 AM, Fujii Masao wrote: > I encountered the problem that the timeline switch is not performed > expectedly. > I set up one master, one standby and one cascade standby. All the servers > share the archive directory. restore_command is specified in the > recovery.conf >

Re: [HACKERS] could not create directory "...": File exists

2013-01-17 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: > This seems to provide a reasonably principled > argument why we might want to fix this case with a localized use of an > MVCC scan before we have such a fix globally. I had discussed that idea a bit with Andres on IRC and my only concern was if there's some

[HACKERS] Re: Slave enters in recovery and promotes when WAL stream with master is cut + delay master/slave

2013-01-17 Thread Andres Freund
On 2013-01-18 08:24:31 +0900, Michael Paquier wrote: > On Fri, Jan 18, 2013 at 3:05 AM, Fujii Masao wrote: > > > I encountered the problem that the timeline switch is not performed > > expectedly. > > I set up one master, one standby and one cascade standby. All the servers > > share the archive

Re: [HACKERS] PATCH: optimized DROP of multiple tables within a transaction

2013-01-17 Thread Tomas Vondra
On 17.1.2013 20:19, Alvaro Herrera wrote: > > I'm curious -- why would you drop tables in groups of 100 instead of > just doing the 100,000 in a single transaction? Maybe that's faster > now, because you'd do a single scan of the buffer pool instead of 1000? > (I'm assuming that "in groups of" me

Re: [HACKERS] could not create directory "...": File exists

2013-01-17 Thread Tom Lane
Stephen Frost writes: > * Tom Lane (t...@sss.pgh.pa.us) wrote: >> This seems to provide a reasonably principled >> argument why we might want to fix this case with a localized use of an >> MVCC scan before we have such a fix globally. > I had discussed that idea a bit with Andres on IRC and my on

[HACKERS] Re: Slave enters in recovery and promotes when WAL stream with master is cut + delay master/slave

2013-01-17 Thread Michael Paquier
On Fri, Jan 18, 2013 at 8:48 AM, Andres Freund wrote: > Can you reproduce that one with 7fcbf6a^ (i.e before xlogreader got > split off?). > Yes, it is reproducible before the xlog reader split. Just an additional report, the master jumps correctly to the new timeline. > > The replication delays

[HACKERS] Re: Slave enters in recovery and promotes when WAL stream with master is cut + delay master/slave

2013-01-17 Thread Andres Freund
On 2013-01-18 08:24:31 +0900, Michael Paquier wrote: > On Fri, Jan 18, 2013 at 3:05 AM, Fujii Masao wrote: > > > I encountered the problem that the timeline switch is not performed > > expectedly. > > I set up one master, one standby and one cascade standby. All the servers > > share the archive

Re: [HACKERS] review: pgbench - aggregation of info written into log

2013-01-17 Thread Tatsuo Ishii
> On Thu, Jan 17, 2013 at 2:29 PM, Andrew Dunstan wrote: >> >> On 01/17/2013 06:04 AM, Tomas Vondra wrote: >>> >>> The problem is I have access to absolutely no Windows machines, >>> not mentioning the development tools (and that I have no clue about it). >>> >>> I vaguely remember there were peop

Re: [HACKERS] patch to add \watch to psql

2013-01-17 Thread Daniel Farina
I have adjusted this patch a little bit to take care of the review issues, along with just doing a bit of review myself. On Thu, Oct 25, 2012 at 2:25 AM, Will Leinweber wrote: > Thanks for the reviews and comments. Responses inline: > . > On Sat, Oct 20, 2012 at 9:19 AM, Abhijit Menon-Sen > wro

  1   2   >