Re: [HACKERS] DDL Damage Assessment

2014-10-03 Thread Dimitri Fontaine
Jim Nasby jim.na...@bluetreble.com writes: EXPLAIN ALTER TABLE I'm thinking it would be better to have something you could set at a session level, so you don't have to stick EXPLAIN in front of all your DDL. Yeah I'm coming into that camp too, and I think the Event Trigger idea gets us

Re: [HACKERS] Promise index tuples for UPSERT

2014-10-03 Thread Heikki Linnakangas
On 10/03/2014 11:07 AM, Simon Riggs wrote: On 1 October 2014 20:54, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 10/01/2014 02:34 PM, Simon Riggs wrote: ... When later insert scans see the promise tuple they perform XactLockTableWait() and when they get control they look again for

Re: [HACKERS] How to make ResourceOwnerForgetBuffer() O(1), instead of O(N^2) scale

2014-10-03 Thread Kouhei Kaigai
On 10/03/2014 07:08 AM, Kouhei Kaigai wrote: Hello, I recently got a trouble on development of my extension that utilizes the shared buffer when it released each buffer page. This extension transfers contents of the shared buffers to GPU device using DMA feature, then kicks a device

Re: [HACKERS] How to make ResourceOwnerForgetBuffer() O(1), instead of O(N^2) scale

2014-10-03 Thread Kouhei Kaigai
On 2014-10-03 10:35:42 +0300, Heikki Linnakangas wrote: On 10/03/2014 07:08 AM, Kouhei Kaigai wrote: Hello, I recently got a trouble on development of my extension that utilizes the shared buffer when it released each buffer page. This extension transfers contents of the

Re: [HACKERS] Promise index tuples for UPSERT

2014-10-03 Thread Peter Geoghegan
On Fri, Oct 3, 2014 at 2:03 AM, Heikki Linnakangas hlinnakan...@vmware.com wrote: That lowers the bar from what I thought everyone agreed on. Namely, if two backends run a similar UPSERT command concurrently on a table that has more than one unique constraint, they might deadlock, causing one

Re: [HACKERS] Join push-down support for foreign tables

2014-10-03 Thread Shigeru Hanada
2014-09-08 8:07 GMT+09:00 Shigeru HANADA shigeru.han...@gmail.com: (2014/09/04 21:37), Robert Haas wrote: On Wed, Sep 3, 2014 at 5:16 AM, Probably both the initial cost and final cost calculations should be delegated to the FDW, but maybe within postgres_fdw, the initial cost should do only

Re: [HACKERS] Promise index tuples for UPSERT

2014-10-03 Thread Simon Riggs
On 3 October 2014 10:03, Heikki Linnakangas hlinnakan...@vmware.com wrote: That lowers the bar from what I thought everyone agreed on. Namely, if two backends run a similar UPSERT command concurrently on a table that has more than one unique constraint, they might deadlock, causing one of them

Re: [HACKERS] Promise index tuples for UPSERT

2014-10-03 Thread Peter Geoghegan
On Fri, Oct 3, 2014 at 2:50 AM, Simon Riggs si...@2ndquadrant.com wrote: My view is that I can't see the above use case from happening in real situations, except by infrequent mistake. In most cases, unique indexes represent some form of object identity and those don't change frequently in the

Re: [HACKERS] Promise index tuples for UPSERT

2014-10-03 Thread Simon Riggs
On 3 October 2014 10:32, Peter Geoghegan p...@heroku.com wrote: On Fri, Oct 3, 2014 at 2:03 AM, Heikki Linnakangas hlinnakan...@vmware.com wrote: That lowers the bar from what I thought everyone agreed on. Namely, if two backends run a similar UPSERT command concurrently on a table that has

Re: [HACKERS] Promise index tuples for UPSERT

2014-10-03 Thread Peter Geoghegan
On Fri, Oct 3, 2014 at 3:04 AM, Simon Riggs si...@2ndquadrant.com wrote: Is there a test case that demonstrates the problem? Yes. See my e-mail to Heikki here: http://www.postgresql.org/message-id/cam3swzshbe29kpod44cvc3vpzjgmder6k_6fghiszeozgmt...@mail.gmail.com Testcase is attached. --

Re: [HACKERS] Promise index tuples for UPSERT

2014-10-03 Thread Simon Riggs
On 3 October 2014 10:57, Peter Geoghegan p...@heroku.com wrote: On Fri, Oct 3, 2014 at 2:50 AM, Simon Riggs si...@2ndquadrant.com wrote: My view is that I can't see the above use case from happening in real situations, except by infrequent mistake. In most cases, unique indexes represent some

Re: [HACKERS] Promise index tuples for UPSERT

2014-10-03 Thread Heikki Linnakangas
On 10/03/2014 01:05 PM, Peter Geoghegan wrote: On Fri, Oct 3, 2014 at 3:04 AM, Simon Riggs si...@2ndquadrant.com wrote: Is there a test case that demonstrates the problem? Yes. See my e-mail to Heikki here:

Re: [HACKERS] GiST splitting on empty pages

2014-10-03 Thread Heikki Linnakangas
On 10/03/2014 05:03 AM, Andrew Gierth wrote: This is from Bug #11555, which is still in moderation as I type this (analysis was done via IRC). The GiST insertion code appears to have no length checks at all on the inserted entry. index_form_tuple checks for length = 8191, with the default

Re: [HACKERS] TAP test breakage on MacOS X

2014-10-03 Thread Robert Haas
On Thu, Oct 2, 2014 at 10:08 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: make check-world dies ingloriously for me, like this: FWIW, it works fine for me on my Mac laptop, using the Perl 5.16.2 that comes standard with OSX 10.9.5. I did have to install

Re: [HACKERS] pg_receivexlog and replication slots

2014-10-03 Thread Andres Freund
On 2014-10-03 10:30:19 +0900, Michael Paquier wrote: On Thu, Oct 2, 2014 at 12:44 AM, Andres Freund and...@2ndquadrant.com wrote: I pushed the first part. Thanks. Attached is a rebased version of patch 2, implementing the actual feature. One thing I noticed with more testing is that if

Re: [HACKERS] Fixed xloginsert_locks for 9.4

2014-10-03 Thread Andres Freund
On 2014-10-02 20:08:33 -0400, Greg Smith wrote: I did a fair dive into double-checking the decision to just leave xloginsert_locks fixed at 8 for 9.4. My conclusion: good call, move along. Further improvements beyond what the 8-way split gives sure are possible. But my guess from chasing

Re: [HACKERS] DDL Damage Assessment

2014-10-03 Thread Robert Haas
On Thu, Oct 2, 2014 at 5:22 PM, Stephen Frost sfr...@snowman.net wrote: * Andres Freund (and...@2ndquadrant.com) wrote: On 2014-10-02 17:03:59 -0400, Stephen Frost wrote: That sounds extremely complex. You'd have to implement the fake columns, foreign keys, indexes, etc on most execution

Re: [HACKERS] DDL Damage Assessment

2014-10-03 Thread Stephen Frost
* Jim Nasby (jim.na...@bluetreble.com) wrote: I'm thinking it would be better to have something you could set at a session level, so you don't have to stick EXPLAIN in front of all your DDL. Right, I'm agreed there. As for the dry-run idea, I don't think that's really necessary. I've never

Re: [HACKERS] WAL format and API changes (9.5)

2014-10-03 Thread Andres Freund
On 2014-10-03 15:51:37 +0300, Heikki Linnakangas wrote: After a lot of experimentation, I figured out that the slowdown is already apparent with the *first* patch, the one that just refactors existing XLogInsert, without any changes to the WAL format or to the callers. I fiddled with that for

Re: [HACKERS] WAL format and API changes (9.5)

2014-10-03 Thread Heikki Linnakangas
On 10/03/2014 04:10 PM, Andres Freund wrote: On 2014-10-03 15:51:37 +0300, Heikki Linnakangas wrote: After a lot of experimentation, I figured out that the slowdown is already apparent with the *first* patch, the one that just refactors existing XLogInsert, without any changes to the WAL format

Re: [HACKERS] Fixed xloginsert_locks for 9.4

2014-10-03 Thread Gregory Smith
On 10/3/14, 8:26 AM, Andres Freund wrote: #define NUM_XLOGINSERT_LOCKS 1 tps = 52.711939 (including connections establishing) #define NUM_XLOGINSERT_LOCKS 8 tps = 286.496054 (including connections establishing) #define NUM_XLOGINSERT_LOCKS 16 tps = 346.113313 (including connections

Re: [HACKERS] Fixed xloginsert_locks for 9.4

2014-10-03 Thread Andres Freund
On 2014-10-03 10:07:39 -0400, Gregory Smith wrote: On 10/3/14, 8:26 AM, Andres Freund wrote: #define NUM_XLOGINSERT_LOCKS 1 tps = 52.711939 (including connections establishing) #define NUM_XLOGINSERT_LOCKS 8 tps = 286.496054 (including connections establishing) #define NUM_XLOGINSERT_LOCKS

Re: [HACKERS] Escaping from blocked send() reprised.

2014-10-03 Thread Heikki Linnakangas
On 09/28/2014 01:54 AM, Andres Freund wrote: I've invested some more time in this: Thanks! In 0001, the select() codepath will not return (WL_SOCKET_READABLE | WL_SOCKET_WRITEABLE) on EOF or error, like the comment says and like the poll() path does. It only sets WL_SOCKET_READABLE if

Re: [HACKERS] Escaping from blocked send() reprised.

2014-10-03 Thread Andres Freund
Hi, On 2014-10-03 17:12:18 +0300, Heikki Linnakangas wrote: On 09/28/2014 01:54 AM, Andres Freund wrote: I've invested some more time in this: Thanks! In 0001, the select() codepath will not return (WL_SOCKET_READABLE | WL_SOCKET_WRITEABLE) on EOF or error, like the comment says and like

[HACKERS] Trailing comma support in SELECT statements

2014-10-03 Thread Bogdan Pilch
Hi, I have created a small patch to postgres source (in particular the psql part of it) that accepts trailing comma at the end of list in SELECT statement. The idea is to be able to say both (with the same result): SELECT a, b, c from t; SELECT a, b, c, from t; Attached you can find a patch

Re: [HACKERS] Log notice that checkpoint is to be written on shutdown

2014-10-03 Thread Alvaro Herrera
Michael Banck wrote: diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c index 5a4dbb9..f2716ae 100644 --- a/src/backend/access/transam/xlog.c +++ b/src/backend/access/transam/xlog.c @@ -8085,10 +8085,14 @@ CreateCheckPoint(int flags) /* *

Re: [HACKERS] Escaping from blocked send() reprised.

2014-10-03 Thread Heikki Linnakangas
On 10/03/2014 05:26 PM, Andres Freund wrote: On 2014-10-03 17:12:18 +0300, Heikki Linnakangas wrote: On 09/28/2014 01:54 AM, Andres Freund wrote: 0003 Sinval/notify processing got simplified further. There really isn't any need for DisableNotifyInterrupt/DisableCatchupInterrupt

[HACKERS] [RFC] Incremental backup v2: add backup profile to base backup

2014-10-03 Thread Marco Nenciarini
Hi Hackers, I've updated the wiki page https://wiki.postgresql.org/wiki/Incremental_backup following the result of discussion on hackers. Compared to first version, we switched from a timestamp+checksum based approach to one based on LSN. This patch adds an option to pg_basebackup and to

Re: [HACKERS] Dynamic LWLock tracing via pg_stat_lwlock (proof of concept)

2014-10-03 Thread Bruce Momjian
On Thu, Oct 2, 2014 at 11:50:14AM +0200, Andres Freund wrote: The first problem that comes to my mind about collecting enough data is that we have a very large number of lwlocks (fixed_number + 2 * shared_buffers). One 'trivial' way of implementing this is to have a per backend array

Re: [HACKERS] Typo fixes in src/backend/rewrite/rewriteHandler.c

2014-10-03 Thread Robert Haas
On Thu, Oct 2, 2014 at 2:55 AM, Etsuro Fujita fujita.ets...@lab.ntt.co.jp wrote: Here is the comments in process_matched_tle() in rewriteHandler.c. 883 * such nodes; consider 884 * UPDATE tab SET col.fld1.subfld1 = x, col.fld2.subfld2 = y 885 * The two expressions produced

Re: [HACKERS] Dynamic LWLock tracing via pg_stat_lwlock (proof of concept)

2014-10-03 Thread Robert Haas
On Fri, Oct 3, 2014 at 11:33 AM, Bruce Momjian br...@momjian.us wrote: I am assuming almost no one cares about the number of locks, but rather they care about cummulative lock durations. I am having trouble seeing any other option that has such a good cost/benefit profile. I do think that

Re: [HACKERS] [RFC] Incremental backup v2: add backup profile to base backup

2014-10-03 Thread Heikki Linnakangas
On 10/03/2014 06:31 PM, Marco Nenciarini wrote: Hi Hackers, I've updated the wiki page https://wiki.postgresql.org/wiki/Incremental_backup following the result of discussion on hackers. Compared to first version, we switched from a timestamp+checksum based approach to one based on LSN. This

Re: [HACKERS] Dynamic LWLock tracing via pg_stat_lwlock (proof of concept)

2014-10-03 Thread Ilya Kosmodemiansky
On Fri, Oct 3, 2014 at 5:33 PM, Bruce Momjian br...@momjian.us wrote: As far as gathering data, I don't think we are going to do any better in terms of performance/simplicity/reliability than to have a single PGPROC entry to record when we enter/exit a lock, and having a secondary process scan

Re: [HACKERS] Dynamic LWLock tracing via pg_stat_lwlock (proof of concept)

2014-10-03 Thread Ilya Kosmodemiansky
On Fri, Oct 3, 2014 at 5:51 PM, Robert Haas robertmh...@gmail.com wrote: I do think that the instrumentation data gathered by LWLOCK_STATS is useful - very useful. Sure, quite useful. But how about this comment: /* * The LWLock stats will be updated within a critical section, which

Re: [HACKERS] Per table autovacuum vacuum cost limit behaviour strange

2014-10-03 Thread Alvaro Herrera
Stephen Frost wrote: * Alvaro Herrera (alvhe...@2ndquadrant.com) wrote: I am rather surprised that nobody has reported this problem before. I am now of the mind that this is clearly a bug that should be fixed all the way back. I'm coming around to that also, however, should we worry

Re: [HACKERS] replicating DROP commands across servers

2014-10-03 Thread Heikki Linnakangas
On 09/16/2014 09:09 PM, Brightwell, Adam wrote: I think there's been some changes to this patch since july, care to resend a new version? Sure, here it is. The only difference with the previous version is that it now also supports column defaults. This was found to be a problem when you

Re: [HACKERS] [RFC] Incremental backup v2: add backup profile to base backup

2014-10-03 Thread Marco Nenciarini
Il 03/10/14 17:53, Heikki Linnakangas ha scritto: If we're going to need a profile file - and I'm not convinced of that - is there any reason to not always include it in the backup? The main reason is to have a centralized list of files that need to be present. Without a profile, you have to

Re: [HACKERS] Last Commitfest patches waiting review

2014-10-03 Thread Heikki Linnakangas
Thanks to everyone's who's reviewed a patch so far. One last crunch, and we'll be through. We have 7 patches left in Needs Review state: pg_receivexlog: addition of --create/--drop to create/drop repslots Waiting for Magnus. Amit promised to take a look if Magnus continues to be busy.

Re: [HACKERS] Trailing comma support in SELECT statements

2014-10-03 Thread Tom Lane
Bogdan Pilch bog...@matfyz.cz writes: I have created a small patch to postgres source (in particular the psql part of it) that accepts trailing comma at the end of list in SELECT statement. This doesn't seem to me to be a remarkably good idea. What's the difference between this and accepting

Re: [HACKERS] Trailing comma support in SELECT statements

2014-10-03 Thread Bruce Momjian
On Sun, Sep 28, 2014 at 01:42:46PM +0200, Bogdan Pilch wrote: Hi, I have created a small patch to postgres source (in particular the psql part of it) that accepts trailing comma at the end of list in SELECT statement. The idea is to be able to say both (with the same result): SELECT a, b,

Re: [HACKERS] [RFC] Incremental backup v2: add backup profile to base backup

2014-10-03 Thread Claudio Freire
On Fri, Oct 3, 2014 at 1:08 PM, Marco Nenciarini marco.nenciar...@2ndquadrant.it wrote: Any comment will be appreciated. In particular I'd appreciate comments on correctness of relnode files detection and LSN extraction code. I didn't look at it in detail, but one future problem comes to mind:

Re: [HACKERS] CREATE IF NOT EXISTS INDEX

2014-10-03 Thread Fabrízio de Royes Mello
On Fri, Oct 3, 2014 at 5:26 AM, Marti Raudsepp ma...@juffo.org wrote: On Fri, Oct 3, 2014 at 6:25 AM, Fabrízio de Royes Mello fabriziome...@gmail.com wrote: Documentation: I would prefer if the explanation were consistent with Do not throw an error if the index already exists. A notice is

Re: [HACKERS] Fixed xloginsert_locks for 9.4

2014-10-03 Thread Bruce Momjian
On Fri, Oct 3, 2014 at 04:11:30PM +0200, Andres Freund wrote: On 2014-10-03 10:07:39 -0400, Gregory Smith wrote: On 10/3/14, 8:26 AM, Andres Freund wrote: #define NUM_XLOGINSERT_LOCKS 1 tps = 52.711939 (including connections establishing) #define NUM_XLOGINSERT_LOCKS 8 tps =

Re: [HACKERS] Last Commitfest patches waiting review

2014-10-03 Thread Magnus Hagander
On Fri, Oct 3, 2014 at 6:14 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: Thanks to everyone's who's reviewed a patch so far. One last crunch, and we'll be through. We have 7 patches left in Needs Review state: pg_receivexlog: addition of --create/--drop to create/drop repslots

Re: [HACKERS] replicating DROP commands across servers

2014-10-03 Thread Alvaro Herrera
Heikki Linnakangas wrote: On 09/16/2014 09:09 PM, Brightwell, Adam wrote: I have given this patch the following review: - Apply to current master (77e65bf). -- success - check-world. --success - multiple FIXME statements still exist -- are there plans to fix these items? Can the

Re: [HACKERS] replicating DROP commands across servers

2014-10-03 Thread Stephen Frost
Alvaro, * Alvaro Herrera (alvhe...@2ndquadrant.com) wrote: There are three fixmes in the code. One can be handled by just removing the line; we don't really care about duplicating 10 lines of boilerplate code. The other two mean missing support for domain constraints and for default ACLs.

Re: [HACKERS] test_shm_mq failing on anole (was: Sending out a request for more buildfarm animals?)

2014-10-03 Thread Robert Haas
On Wed, Oct 1, 2014 at 11:10 AM, Robert Haas robertmh...@gmail.com wrote: As far as I can tell, it's configured to run everything. I just checked, though, and found it wedged again. I'm not sure whether it was the same problem, though; I ended up just killing all of the postgres processes to

Re: [HACKERS] replicating DROP commands across servers

2014-10-03 Thread Heikki Linnakangas
On 10/03/2014 08:08 PM, Stephen Frost wrote: Alvaro, * Alvaro Herrera (alvhe...@2ndquadrant.com) wrote: There are three fixmes in the code. One can be handled by just removing the line; we don't really care about duplicating 10 lines of boilerplate code. The other two mean missing support

Re: [HACKERS] Proposal for updating src/timezone

2014-10-03 Thread Tom Lane
I wrote: The urgency of updating our timezone code has risen quite a bit for me, because while testing an update of the data files to tzdata2014h I became aware that the -P option is failing to print a noticeable number of zone abbreviations that clearly exist in the data files. Since the -P

Re: [HACKERS] Fixed xloginsert_locks for 9.4

2014-10-03 Thread Arthur Silva
On Fri, Oct 3, 2014 at 1:40 PM, Bruce Momjian br...@momjian.us wrote: On Fri, Oct 3, 2014 at 04:11:30PM +0200, Andres Freund wrote: On 2014-10-03 10:07:39 -0400, Gregory Smith wrote: On 10/3/14, 8:26 AM, Andres Freund wrote: #define NUM_XLOGINSERT_LOCKS 1 tps = 52.711939 (including

Re: [HACKERS] pg_receivexlog and replication slots

2014-10-03 Thread Robert Haas
On Fri, Oct 3, 2014 at 7:57 AM, Andres Freund and...@2ndquadrant.com wrote: I do wonder whether --create/--drop aren't somewhat wierd for pg_receivexlog. It's not that clear what it means. It'd be ugly, but we could rename them --create-slot/drop-slot. +1 on doing it, -1 on it being ugly. --

Re: [HACKERS] replicating DROP commands across servers

2014-10-03 Thread Alvaro Herrera
Heikki Linnakangas wrote: I had a very brief look at the docs, and these extra outputs from pg_event_trigger_dropped_objects caught my eye: +row + entryliteraladdress_names/literal/entry + entrytypetext[]/type/entry + entry + An array that,

Re: [HACKERS] Fixed xloginsert_locks for 9.4

2014-10-03 Thread Bruce Momjian
On Fri, Oct 3, 2014 at 03:00:56PM -0300, Arthur Silva wrote: I remember Informix had a setting that had no description except try different values to see if it helps performance --- we don't want to do that. What if we emit a server message if the setting is too low? That's

Re: [HACKERS] Fixed xloginsert_locks for 9.4

2014-10-03 Thread Bruce Momjian
On Fri, Oct 3, 2014 at 02:07:45PM -0400, Bruce Momjian wrote: On Fri, Oct 3, 2014 at 03:00:56PM -0300, Arthur Silva wrote: I remember Informix had a setting that had no description except try different values to see if it helps performance --- we don't want to do that.

Re: [HACKERS] [RFC] Incremental backup v2: add backup profile to base backup

2014-10-03 Thread Bruce Momjian
On Fri, Oct 3, 2014 at 06:08:47PM +0200, Marco Nenciarini wrote: Any comment will be appreciated. In particular I'd appreciate comments on correctness of relnode files detection and LSN extraction code. I didn't look at it in detail, but one future problem comes to mind: Once you

Re: [HACKERS] replicating DROP commands across servers

2014-10-03 Thread Heikki Linnakangas
On 10/03/2014 09:06 PM, Alvaro Herrera wrote: Heikki Linnakangas wrote: I had a very brief look at the docs, and these extra outputs from pg_event_trigger_dropped_objects caught my eye: +row + entryliteraladdress_names/literal/entry + entrytypetext[]/type/entry +

Re: [HACKERS] Fixed xloginsert_locks for 9.4

2014-10-03 Thread Arthur Silva
On Fri, Oct 3, 2014 at 3:10 PM, Bruce Momjian br...@momjian.us wrote: On Fri, Oct 3, 2014 at 02:07:45PM -0400, Bruce Momjian wrote: On Fri, Oct 3, 2014 at 03:00:56PM -0300, Arthur Silva wrote: I remember Informix had a setting that had no description except try different

Re: [HACKERS] replicating DROP commands across servers

2014-10-03 Thread Alvaro Herrera
Heikki Linnakangas wrote: On 10/03/2014 09:06 PM, Alvaro Herrera wrote: Well, the return value from get_object_address is an ObjectAddress. It's simple enough to create an SQL wrapper that takes the address_names/address_args arrays and return an ObjectAddress; would this be useful? An

Re: [HACKERS] test_shm_mq failing on anole (was: Sending out a request for more buildfarm animals?)

2014-10-03 Thread Robert Haas
On Fri, Oct 3, 2014 at 1:09 PM, Robert Haas robertmh...@gmail.com wrote: Further debugging reveals that sigusr1_handler() gets called repeatedly, to start autovacuum workers, and it keeps waking up and starting them. But that doesn't cause the background workers to get started either, because

Re: [HACKERS] Dynamic LWLock tracing via pg_stat_lwlock (proof of concept)

2014-10-03 Thread Bruce Momjian
On Fri, Oct 3, 2014 at 05:53:59PM +0200, Ilya Kosmodemiansky wrote: What that gives us is almost zero overhead on backends, high reliability, and the ability of the scan daemon to give higher weights to locks that are held longer. Basically, if you just stored the locks you held and

Re: [HACKERS] Fixed xloginsert_locks for 9.4

2014-10-03 Thread Tom Lane
Bruce Momjian br...@momjian.us writes: On Fri, Oct 3, 2014 at 03:00:56PM -0300, Arthur Silva wrote: Not all GUC need to be straight forward to tune. If the gains are worthy I don't see any reason not to have it. Every GUC add complexity to the system because people have to understand it to

Re: [HACKERS] Fixed xloginsert_locks for 9.4

2014-10-03 Thread Bruce Momjian
On Fri, Oct 3, 2014 at 03:30:35PM -0300, Arthur Silva wrote: Every GUC add complexity to the system because people have to understand it to know if they should tune it. No GUC is zero-cost. Please see my blog post about the cost of adding GUCs:

Re: [HACKERS] Fixed xloginsert_locks for 9.4

2014-10-03 Thread Heikki Linnakangas
On 10/03/2014 09:42 PM, Bruce Momjian wrote: On Fri, Oct 3, 2014 at 03:30:35PM -0300, Arthur Silva wrote: Every GUC add complexity to the system because people have to understand it to know if they should tune it. No GUC is zero-cost. Please see my blog post about the cost

Re: [HACKERS] Fixed xloginsert_locks for 9.4

2014-10-03 Thread Robert Haas
On Fri, Oct 3, 2014 at 2:49 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: I stand by my decision to make it a #define, at least until someone voices their objection in the form of a documentation patch. I think that's exactly right. If we knew users should tune this, then we might be

Re: [HACKERS] Fixed xloginsert_locks for 9.4

2014-10-03 Thread Merlin Moncure
On Fri, Oct 3, 2014 at 2:20 PM, Robert Haas robertmh...@gmail.com wrote: On Fri, Oct 3, 2014 at 2:49 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: I stand by my decision to make it a #define, at least until someone voices their objection in the form of a documentation patch. I think

Re: [HACKERS] replicating DROP commands across servers

2014-10-03 Thread Alvaro Herrera
Alvaro Herrera wrote: Precisely the point is not returning those values, because they are useless to identify the equivalent object in a remote database. What we need is the object names and other stuff used to uniquely identify it by user-visible name. We transmit those name arrays to a

Re: [HACKERS] replicating DROP commands across servers

2014-10-03 Thread Stephen Frost
* Alvaro Herrera (alvhe...@2ndquadrant.com) wrote: Right. In the add to objname cases, there is already some other routine that initialized it previously by filling in some stuff; in the case above, this happens in the getRelationIdentity() immediately preceding this. In the other cases we

[HACKERS] Re: Valgrind warnings in master branch (Invalid read of size 8) originating within CreatePolicy()

2014-10-03 Thread Stephen Frost
Peter, * Peter Geoghegan (p...@heroku.com) wrote: I see the following Valgrind warnings in a recent build of the master branch: Fix pushed, thanks! Stephen signature.asc Description: Digital signature

Re: [HACKERS] Missing newlines in verbose logs of pg_dump, introduced by RLS patch

2014-10-03 Thread Stephen Frost
* Stephen Frost (sfr...@snowman.net) wrote: * Michael Paquier (michael.paqu...@gmail.com) wrote: On Mon, Sep 29, 2014 at 10:07 AM, Fabrízio de Royes Mello fabriziome...@gmail.com wrote: The schema name is missing... attached patch add it. Ah, right, thanks. It didn't occur to me

Re: [HACKERS] Trailing comma support in SELECT statements

2014-10-03 Thread Andrew Dunstan
On 10/03/2014 12:20 PM, Bruce Momjian wrote: On Sun, Sep 28, 2014 at 01:42:46PM +0200, Bogdan Pilch wrote: Hi, I have created a small patch to postgres source (in particular the psql part of it) that accepts trailing comma at the end of list in SELECT statement. The idea is to be able to say

Re: [HACKERS] [RFC] Incremental backup v2: add backup profile to base backup

2014-10-03 Thread Robert Haas
On Fri, Oct 3, 2014 at 12:08 PM, Marco Nenciarini marco.nenciar...@2ndquadrant.it wrote: Il 03/10/14 17:53, Heikki Linnakangas ha scritto: If we're going to need a profile file - and I'm not convinced of that - is there any reason to not always include it in the backup? The main reason is to

Re: [HACKERS] replicating DROP commands across servers

2014-10-03 Thread Alvaro Herrera
Stephen Frost wrote: * Alvaro Herrera (alvhe...@2ndquadrant.com) wrote: Right. In the add to objname cases, there is already some other routine that initialized it previously by filling in some stuff; in the case above, this happens in the getRelationIdentity() immediately preceding

Re: [HACKERS] replicating DROP commands across servers

2014-10-03 Thread Stephen Frost
* Alvaro Herrera (alvhe...@2ndquadrant.com) wrote: Stephen Frost wrote: ahh, ok, that makes a bit more sense, sorry for missing it. Still makes me wonder why objargs gets special treatment at the top of the function and objnames doesn't- seems like both should be initialized either

Re: [HACKERS] replicating DROP commands across servers

2014-10-03 Thread Robert Haas
On Fri, Oct 3, 2014 at 2:33 PM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Heikki Linnakangas wrote: On 10/03/2014 09:06 PM, Alvaro Herrera wrote: Well, the return value from get_object_address is an ObjectAddress. It's simple enough to create an SQL wrapper that takes the

Re: [HACKERS] replicating DROP commands across servers

2014-10-03 Thread Alvaro Herrera
Robert Haas wrote: I'm not really very convinced that it's a good idea to expose this instead of just figuring out a way to parse the object identity. That's the first thing I tried. But it's not pretty: you have to extract schema names by splitting at a period (and what if a schema name

Re: [HACKERS] DDL Damage Assessment

2014-10-03 Thread José Luis Tallón
On 10/03/2014 11:02 AM, Dimitri Fontaine wrote: Jim Nasby jim.na...@bluetreble.com writes: EXPLAIN ALTER TABLE I'm thinking it would be better to have something you could set at a session level, so you don't have to stick EXPLAIN in front of all your DDL. We were considering the

Re: [HACKERS] UPSERT wiki page, and SQL MERGE syntax

2014-10-03 Thread Peter Geoghegan
On Fri, Oct 3, 2014 at 1:16 PM, Kevin Grittner kgri...@ymail.com wrote: I have two questions I hope you can clarify. I'm having trouble parsing what this statement means: ... the SQL standard does not require that MERGE be atomic in the sense of atomically providing either an INSERT or

Re: [HACKERS] Trailing comma support in SELECT statements

2014-10-03 Thread David G Johnston
Andrew Dunstan wrote On 10/03/2014 12:20 PM, Bruce Momjian wrote: On Sun, Sep 28, 2014 at 01:42:46PM +0200, Bogdan Pilch wrote: Hi, I have created a small patch to postgres source (in particular the psql part of it) that accepts trailing comma at the end of list in SELECT statement. The

Re: [HACKERS] [RFC] Incremental backup v2: add backup profile to base backup

2014-10-03 Thread Andres Freund
On 2014-10-03 17:31:45 +0200, Marco Nenciarini wrote: I've updated the wiki page https://wiki.postgresql.org/wiki/Incremental_backup following the result of discussion on hackers. Compared to first version, we switched from a timestamp+checksum based approach to one based on LSN. This

Re: [HACKERS] superuser() shortcuts

2014-10-03 Thread Brightwell, Adam
All, Thanks! Please add it to the next commitfest. Sounds good. I'll update the patch and add accordingly. Attached is an updated patch. -Adam -- Adam Brightwell - adam.brightw...@crunchydatasolutions.com Database Engineer - www.crunchydatasolutions.com diff --git

Re: [HACKERS] Trailing comma support in SELECT statements

2014-10-03 Thread Pavel Stehule
2014-09-28 13:42 GMT+02:00 Bogdan Pilch bog...@matfyz.cz: Hi, I have created a small patch to postgres source (in particular the psql part of it) that accepts trailing comma at the end of list in SELECT statement. It is ANSI/SQL ? Why we should to enable? We can be tolerant to this bug,

Re: [HACKERS] Dynamic LWLock tracing via pg_stat_lwlock (proof of concept)

2014-10-03 Thread Andres Freund
On 2014-10-03 11:33:18 -0400, Bruce Momjian wrote: On Thu, Oct 2, 2014 at 11:50:14AM +0200, Andres Freund wrote: The first problem that comes to my mind about collecting enough data is that we have a very large number of lwlocks (fixed_number + 2 * shared_buffers). One 'trivial' way of

Re: [HACKERS] Dynamic LWLock tracing via pg_stat_lwlock (proof of concept)

2014-10-03 Thread Andres Freund
On 2014-10-03 11:51:46 -0400, Robert Haas wrote: On Fri, Oct 3, 2014 at 11:33 AM, Bruce Momjian br...@momjian.us wrote: I am assuming almost no one cares about the number of locks, but rather they care about cummulative lock durations. I am having trouble seeing any other option that has

Re: [HACKERS] Promise index tuples for UPSERT

2014-10-03 Thread Peter Geoghegan
On Fri, Oct 3, 2014 at 3:54 AM, Heikki Linnakangas hlinnakan...@vmware.com wrote: Simon's approach would actually pass that test case just fine. It inserts the (promise) index tuple first, and heap tuple only after that. It will fail the test case with more than one unique index, however. Oh,

Re: [HACKERS] Last Commitfest patches waiting review

2014-10-03 Thread Andres Freund
On 2014-10-03 19:14:14 +0300, Heikki Linnakangas wrote: Thanks to everyone's who's reviewed a patch so far. One last crunch, and we'll be through. We have 7 patches left in Needs Review state: pg_receivexlog: addition of --create/--drop to create/drop repslots Waiting for Magnus.

Re: [HACKERS] replicating DROP commands across servers

2014-10-03 Thread Andres Freund
On 2014-10-03 14:02:09 -0300, Alvaro Herrera wrote: Since the patch has had good feedback and no further comments arise, I can just implement support for those two missing object types and push, and everybody will be happy. Right? I'd like to see a new version before that out here... I don't

Re: [HACKERS] Fixed xloginsert_locks for 9.4

2014-10-03 Thread Andres Freund
On 2014-10-03 12:40:21 -0400, Bruce Momjian wrote: On Fri, Oct 3, 2014 at 04:11:30PM +0200, Andres Freund wrote: On 2014-10-03 10:07:39 -0400, Gregory Smith wrote: On 10/3/14, 8:26 AM, Andres Freund wrote: #define NUM_XLOGINSERT_LOCKS 1 tps = 52.711939 (including connections

Re: [HACKERS] UPSERT wiki page, and SQL MERGE syntax

2014-10-03 Thread Kevin Grittner
Peter Geoghegan p...@heroku.com wrote: On Fri, Oct 3, 2014 at 1:16 PM, Kevin Grittner kgri...@ymail.com wrote: I'm having trouble parsing what this statement means: ... the SQL standard does not require that MERGE be atomic in the sense of atomically providing either an INSERT or UPDATE, ...

Re: [HACKERS] test_shm_mq failing on anole (was: Sending out a request for more buildfarm animals?)

2014-10-03 Thread Andres Freund
On 2014-10-03 14:38:10 -0400, Robert Haas wrote: On Fri, Oct 3, 2014 at 1:09 PM, Robert Haas robertmh...@gmail.com wrote: Further debugging reveals that sigusr1_handler() gets called repeatedly, to start autovacuum workers, and it keeps waking up and starting them. But that doesn't cause

Re: [HACKERS] pg_receivexlog and replication slots

2014-10-03 Thread Andres Freund
On 2014-10-03 14:02:08 -0400, Robert Haas wrote: On Fri, Oct 3, 2014 at 7:57 AM, Andres Freund and...@2ndquadrant.com wrote: I do wonder whether --create/--drop aren't somewhat wierd for pg_receivexlog. It's not that clear what it means. It'd be ugly, but we could rename them

Re: [HACKERS] Fixed xloginsert_locks for 9.4

2014-10-03 Thread Andres Freund
On 2014-10-03 17:55:19 -0400, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On 2014-10-03 12:40:21 -0400, Bruce Momjian wrote: Well, I think the issue is that having a GUC that can't reasonably be tuned by 95% of our users is nearly useless. Few users are going to run

Re: [HACKERS] Fixed xloginsert_locks for 9.4

2014-10-03 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On 2014-10-03 12:40:21 -0400, Bruce Momjian wrote: Well, I think the issue is that having a GUC that can't reasonably be tuned by 95% of our users is nearly useless. Few users are going to run benchmarks to see what the optimal value is. It's

Re: [HACKERS] Dynamic LWLock tracing via pg_stat_lwlock (proof of concept)

2014-10-03 Thread Bruce Momjian
On Fri, Oct 3, 2014 at 11:15:13PM +0200, Andres Freund wrote: As far as gathering data, I don't think we are going to do any better in terms of performance/simplicity/reliability than to have a single PGPROC entry to record when we enter/exit a lock, and having a secondary process scan

Re: [HACKERS] Fixed xloginsert_locks for 9.4

2014-10-03 Thread Bruce Momjian
On Fri, Oct 3, 2014 at 11:58:14PM +0200, Andres Freund wrote: On 2014-10-03 17:55:19 -0400, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On 2014-10-03 12:40:21 -0400, Bruce Momjian wrote: Well, I think the issue is that having a GUC that can't reasonably be tuned by

Re: [HACKERS] Fixed xloginsert_locks for 9.4

2014-10-03 Thread Andres Freund
On 2014-10-03 18:08:56 -0400, Bruce Momjian wrote: On Fri, Oct 3, 2014 at 11:58:14PM +0200, Andres Freund wrote: On 2014-10-03 17:55:19 -0400, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On 2014-10-03 12:40:21 -0400, Bruce Momjian wrote: Well, I think the issue is

Re: [HACKERS] Fixed xloginsert_locks for 9.4

2014-10-03 Thread Bruce Momjian
On Sat, Oct 4, 2014 at 12:13:00AM +0200, Andres Freund wrote: Do we really want to expose a setting a few of us _might_ ask customers to change? They also will try that themselves. Our customers aren't a horde of dumb people. Some of them are willing to try things if they hit scalability

Re: [HACKERS] Fixed xloginsert_locks for 9.4

2014-10-03 Thread Andres Freund
On 2014-10-03 18:16:28 -0400, Bruce Momjian wrote: On Sat, Oct 4, 2014 at 12:13:00AM +0200, Andres Freund wrote: Do we really want to expose a setting a few of us _might_ ask customers to change? They also will try that themselves. Our customers aren't a horde of dumb people. Some

Re: [HACKERS] UPSERT wiki page, and SQL MERGE syntax

2014-10-03 Thread Peter Geoghegan
On Fri, Oct 3, 2014 at 2:44 PM, Kevin Grittner kgri...@ymail.com wrote: I've never seen atomic used to mean you never get an error before. When you are saying atomic you mean something quite different. Perhaps I should have been more careful on that point. Just to be crystal clear: I don't

Re: [HACKERS] UPSERT wiki page, and SQL MERGE syntax

2014-10-03 Thread Peter Geoghegan
On Fri, Oct 3, 2014 at 3:42 PM, Peter Geoghegan p...@heroku.com wrote: Yes, it is. I am opposed to using the MERGE syntax for this *because* MERGE is useful (independently useful, when done properly), not because it is not useful. As I've mentioned, there is also the practical argument: No

Re: [HACKERS] UPSERT wiki page, and SQL MERGE syntax

2014-10-03 Thread Peter Geoghegan
On Fri, Oct 3, 2014 at 3:42 PM, Peter Geoghegan p...@heroku.com wrote: We routinely pick very different plans based on the presence or absence of an index, and we use special snapshots in the course of executing many DML statements (if FK triggers are fired) Apart from FK snapshots, we also

  1   2   >