Re: [HACKERS] CINE in CREATE TABLE AS ... and CREATE MATERIALIZED VIEW ...

2014-10-27 Thread Rushabh Lathia
Hi All, - Patch got applied cleanly. - Regression make check run fine. - Patch covered the documentation changes Here are few comments: 1) What the need of following change: diff --git a/src/backend/storage/lmgr/lwlock.c b/src/backend/storage/lmgr/lwlock.c index bcec173..9fe6855 100644 ---

Re: [HACKERS] [v9.5] Custom Plan API

2014-10-27 Thread Kouhei Kaigai
FYI, patch v12 part 2 no longer applies cleanly. Thanks. I rebased the patch set according to the latest master branch. The attached v13 can be applied to the master. -- NEC OSS Promotion Center / PG-Strom Project KaiGai Kohei kai...@ak.jp.nec.com -Original Message- From:

Re: [HACKERS] Function array_agg(array)

2014-10-27 Thread Ali Akbar
2014-10-27 9:11 GMT+07:00 Ali Akbar the.ap...@gmail.com: 2014-10-27 1:38 GMT+07:00 Pavel Stehule pavel.steh...@gmail.com: Hi My idea is using new ArrayBuilder optimized for building multidimensional arrays with own State type. I think so casting to ArrayBuildState is base of our problems,

Re: [HACKERS] DISTINCT with btree skip scan

2014-10-27 Thread David Rowley
On Sat, Jul 5, 2014 at 12:17 PM, Thomas Munro mu...@ip9.org wrote: postgres=# set enable_hashagg = false; SET Time: 0.302 ms postgres=# explain select distinct a from foo; ┌─┐ │

Re: [HACKERS] On partitioning

2014-10-27 Thread Amit Langote
Hi, On Mon, Oct 13, 2014 at 04:38:39PM -0300, Alvaro Herrera wrote: Bruce Momjian wrote: I realize there hasn't been much progress on this thread, but I wanted to chime in to say I think our current partitioning implementation is too heavy administratively, error-prone, and

Re: [HACKERS] Function array_agg(array)

2014-10-27 Thread Pavel Stehule
Hi I did some minor changes in code * move tests of old or new builder style for array sublink out of main cycles * some API simplification of new builder - we should not to create identical API, mainly it has no sense Regards Pavel Stehule 2014-10-27 8:12 GMT+01:00 Ali Akbar

Re: [HACKERS] On partitioning

2014-10-27 Thread Alvaro Herrera
Amit Langote wrote: On Mon, Oct 13, 2014 at 04:38:39PM -0300, Alvaro Herrera wrote: Bruce Momjian wrote: I realize there hasn't been much progress on this thread, but I wanted to chime in to say I think our current partitioning implementation is too heavy administratively,

Re: [HACKERS] proposal: CREATE DATABASE vs. (partial) CHECKPOINT

2014-10-27 Thread Heikki Linnakangas
On 10/26/2014 11:47 PM, Tomas Vondra wrote: After eyeballing the code for an hour or two, I think CREATE DATABASE should be fine with performing only a 'partial checkpoint' on the template database - calling FlushDatabaseBuffers and processing unlink requests, as suggested by the comment in

Re: [HACKERS] strip nulls functions for json and jsonb

2014-10-27 Thread Pavel Stehule
Hi I am sending a final review of this patch: 0. this patch implements null fields stripping. It does exactly what was proposed and we would to have this feature in core. It is requested feature for JSON types. 1. there is no problem with patch apply and with compilation - one warning is fixed

Re: [HACKERS] Function array_agg(array)

2014-10-27 Thread Ali Akbar
2014-10-27 16:15 GMT+07:00 Pavel Stehule pavel.steh...@gmail.com: Hi I did some minor changes in code * move tests of old or new builder style for array sublink out of main cycles * some API simplification of new builder - we should not to create identical API, mainly it has no sense

Re: [HACKERS] Better support of exported snapshots with pg_dump

2014-10-27 Thread Petr Jelinek
On 17/10/14 06:25, Michael Paquier wrote: Two votes in favor of that from two committers sounds like a deal. Here is an refreshed version of the patch introducing --snapshot from here, after fixing a couple of things and adding documentation:

Re: [HACKERS] END_OF_RECOVERY shutdowns and ResetUnloggedRelations()

2014-10-27 Thread Abhijit Menon-Sen
At 2014-09-25 22:41:18 +0200, and...@2ndquadrant.com wrote: On 2014-09-24 17:06:05 +0530, Abhijit Menon-Sen wrote: 1. Move the call to ResetUnloggedRelations(UNLOGGED_RELATION_INIT) to earlier in StartupXLOG. 2. Inside that function, issue fsync()s for the main forks we create by

[HACKERS] Missing FIN_CRC32 calls in logical replication code

2014-10-27 Thread Heikki Linnakangas
replication/slot.c and replication/logical/snapbuild.c use a CRC on the physical slot and snapshot files. It uses the same algorithm as used e.g. for the WAL. However, they are not doing the finalization step, FIN_CRC32() on the calculated checksums. Not that it matters much, but it's a bit

Re: [HACKERS] Function array_agg(array)

2014-10-27 Thread Pavel Stehule
2014-10-27 11:20 GMT+01:00 Ali Akbar the.ap...@gmail.com: 2014-10-27 16:15 GMT+07:00 Pavel Stehule pavel.steh...@gmail.com: Hi I did some minor changes in code * move tests of old or new builder style for array sublink out of main cycles * some API simplification of new builder - we

Re: [HACKERS] proposal: CREATE DATABASE vs. (partial) CHECKPOINT

2014-10-27 Thread Atri Sharma
To solve #1, we could redesign CREATE DATABASE so that replaying the DBASE_CREATE record doesn't zap the old directory, and also doesn't copy any files. We could instead just assume that if the transaction commits, all the files have been copied and fsync'd already, like we assume that if

Re: [HACKERS] pg_receivexlog --status-interval add fsync feedback

2014-10-27 Thread Fujii Masao
On Fri, Oct 24, 2014 at 11:21 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 10/24/2014 01:24 PM, furu...@pm.nttdata.co.jp wrote: Sorry, I'm going around in the circle. But I'd like to say again, I don't think this is good idea. It prevents asynchronous pg_receivexlog from fsyncing

Re: [HACKERS] proposal: CREATE DATABASE vs. (partial) CHECKPOINT

2014-10-27 Thread Heikki Linnakangas
On 10/27/2014 01:06 PM, Atri Sharma wrote: To solve #1, we could redesign CREATE DATABASE so that replaying the DBASE_CREATE record doesn't zap the old directory, and also doesn't copy any files. We could instead just assume that if the transaction commits, all the files have been copied

[HACKERS] Master ip from hot_standby..

2014-10-27 Thread sudalai
Hi, I need to query master ip from hot_standby. *pg_stat_replication* view only shows the slave replication status. Is there any way to get *Master IP* from standby node apart from checking *recovery.conf* file. Thanks, Sudalai - sudalai -- View this message in context:

Re: [HACKERS] proposal: CREATE DATABASE vs. (partial) CHECKPOINT

2014-10-27 Thread Atri Sharma
On Mon, Oct 27, 2014 at 4:44 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 10/27/2014 01:06 PM, Atri Sharma wrote: To solve #1, we could redesign CREATE DATABASE so that replaying the DBASE_CREATE record doesn't zap the old directory, and also doesn't copy any files. We could

Re: [HACKERS] Directory/File Access Permissions for COPY and Generic File Access Functions

2014-10-27 Thread Stephen Frost
* Peter Eisentraut (pete...@gmx.net) wrote: On 10/16/14 12:01 PM, Stephen Frost wrote: This started out as a request for a non-superuser to be able to review the log files without needing access to the server. I think that can be done with a security-definer function. Of course it can be.

Re: [HACKERS] Index-only scans for GIST

2014-10-27 Thread Thom Brown
On 18 August 2014 09:05, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 08/17/2014 07:15 PM, Anastasia Lubennikova wrote: 2014-08-07 0:30 GMT+04:00 Heikki Linnakangas hlinnakan...@vmware.com: * I'm getting two regression failures with this (opr_sanity and join). opr_sanity failure

Re: [HACKERS] Index scan optimization

2014-10-27 Thread Rajeev rastogi
On 26 October 2014 10:42, Haribabu Kommi wrote: Hi, I reviewed index scan optimization patch, the following are the observations. - Patch applies cleanly. - Compiles without warnings - All regress tests are passed. There is a good performance gain with the patch in almost all

Re: [HACKERS] KNN searches support for SP-GiST [GSOC'14]

2014-10-27 Thread Thom Brown
On 20 August 2014 08:09, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 08/20/2014 03:35 AM, Vladislav Sterzhanov wrote: Hi there, pg-Hackers! Here I go with the patch which brings up the possibility to perform nearest-neighbour searches on SP-GiSTs (as of now includes implementation

Re: [HACKERS] On partitioning

2014-10-27 Thread Andres Freund
On 2014-10-27 06:29:33 -0300, Alvaro Herrera wrote: Amit Langote wrote: On Mon, Oct 13, 2014 at 04:38:39PM -0300, Alvaro Herrera wrote: Bruce Momjian wrote: I realize there hasn't been much progress on this thread, but I wanted to chime in to say I think our current

[HACKERS] Dynamically change Master(recovery info) without restarting standby server..

2014-10-27 Thread sudalai
Hi, Is there any way to change the *master* without restarting the *standby* server. Postgresql Documentation says, --Recovery.conf file only read on the startup in standby mode. In that file we specify the Masterip. --If you want to change the master we need to change the

Re: [HACKERS] TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

2014-10-27 Thread Amit Kapila
On Sat, Oct 25, 2014 at 5:52 PM, Amit Kapila amit.kapil...@gmail.com wrote: *** *** 358,363 handle_sigint(SIGNAL_ARGS) --- 358,364 /* Send QueryCancel if we are processing a database query */ if (cancelConn != NULL) { + inAbort = true; if

Re: [HACKERS] Missing FIN_CRC32 calls in logical replication code

2014-10-27 Thread Andres Freund
On 2014-10-27 12:51:44 +0200, Heikki Linnakangas wrote: replication/slot.c and replication/logical/snapbuild.c use a CRC on the physical slot and snapshot files. It uses the same algorithm as used e.g. for the WAL. However, they are not doing the finalization step, FIN_CRC32() on the

Re: [HACKERS] proposal: CREATE DATABASE vs. (partial) CHECKPOINT

2014-10-27 Thread Tomas Vondra
Dne 27 Říjen 2014, 10:47, Heikki Linnakangas napsal(a): On 10/26/2014 11:47 PM, Tomas Vondra wrote: After eyeballing the code for an hour or two, I think CREATE DATABASE should be fine with performing only a 'partial checkpoint' on the template database - calling FlushDatabaseBuffers and

Re: [HACKERS] BUG: *FF WALs under 9.2 (WAS: .ready files appearing on slaves)

2014-10-27 Thread Fujii Masao
On Fri, Oct 24, 2014 at 10:05 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 10/23/2014 11:09 AM, Heikki Linnakangas wrote: At least for master, we should consider changing the way the archiving works so that we only archive WAL that was generated in the same server. I.e. we should

Re: [HACKERS] alter user set local_preload_libraries.

2014-10-27 Thread Fujii Masao
On Tue, Oct 21, 2014 at 3:16 PM, Kyotaro HORIGUCHI horiguchi.kyot...@lab.ntt.co.jp wrote: Wow. By the way, I became unable to login at all after wrongly setting *_preload_libraries for all available users. Is there any releaf measures for the situation? I think it's okay even if there's no

Re: [HACKERS] Reducing lock strength of adding foreign keys

2014-10-27 Thread Robert Haas
Thanks for weighing in, Noah. On Sat, Oct 25, 2014 at 2:00 PM, Noah Misch n...@leadboat.com wrote: http://www.postgresql.org/message-id/ca+tgmoy4glsxzk0tao29-ljtcuj0sl1xwcwq51xb-hfysgi...@mail.gmail.com http://www.postgresql.org/message-id/20893.1393892...@sss.pgh.pa.us

Re: [HACKERS] Reducing lock strength of adding foreign keys

2014-10-27 Thread Robert Haas
On Sun, Oct 26, 2014 at 9:48 PM, Andreas Karlsson andr...@proxel.se wrote: Agreed.. But I think reducing the lock level of the secondary table is much more important than doing the same for the primary table due to the case where the secondary table is an existing table which is hit by a

Re: [HACKERS] proposal: CREATE DATABASE vs. (partial) CHECKPOINT

2014-10-27 Thread Atri Sharma
IMHO writing all the data into a WAL would be the cleanest solution. Also, what is a small database? I don't think a static value will work, because the sweet spot between the current approach (forcing two checkpoints) and writing everything in WAL depends on the amount of dirty buffers

Re: [HACKERS] Possible problem with shm_mq spin lock

2014-10-27 Thread Robert Haas
On Sat, Oct 25, 2014 at 9:12 PM, Tom Lane t...@sss.pgh.pa.us wrote: Haribabu Kommi kommi.harib...@gmail.com writes: Thanks for the details. I am sorry It is not proc_exit. It is the exit callback functions that can cause problem. The following is the callstack where the problem can happen, if

Re: [HACKERS] proposal: CREATE DATABASE vs. (partial) CHECKPOINT

2014-10-27 Thread Tomas Vondra
Dne 27 Říjen 2014, 13:50, Atri Sharma napsal(a): IMHO writing all the data into a WAL would be the cleanest solution. Also, what is a small database? I don't think a static value will work, because the sweet spot between the current approach (forcing two checkpoints) and writing everything

Re: [HACKERS] Master ip from hot_standby..

2014-10-27 Thread Michael Paquier
On Mon, Oct 27, 2014 at 8:15 PM, sudalai sudala...@gmail.com wrote: I need to query master ip from hot_standby. *pg_stat_replication* view only shows the slave replication status. Is there any way to get *Master IP* from standby node apart from checking *recovery.conf* file. That's the

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-10-27 Thread Fujii Masao
On Fri, Oct 17, 2014 at 1:52 PM, Rahila Syed rahilasye...@gmail.com wrote: Hello, Please find the updated patch attached. Thanks for updating the patch! Here are the comments. The patch isn't applied to the master cleanly. I got the following compiler warnings. xlog.c:930: warning: ISO C90

Re: [HACKERS] Missing FIN_CRC32 calls in logical replication code

2014-10-27 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On 2014-10-27 12:51:44 +0200, Heikki Linnakangas wrote: replication/slot.c and replication/logical/snapbuild.c use a CRC on the physical slot and snapshot files. It uses the same algorithm as used e.g. for the WAL. However, they are not doing the

[HACKERS] Lockless StrategyGetBuffer() clock sweep

2014-10-27 Thread Andres Freund
Hi, I've previously posted a patch at http://archives.postgresql.org/message-id/20141010160020.GG6670%40alap3.anarazel.de that reduces contention in StrategyGetBuffer() by making the clock sweep lockless. Robert asked me to post it to a new thread; I originally wrote it to see some other

Re: [HACKERS] proposal: CREATE DATABASE vs. (partial) CHECKPOINT

2014-10-27 Thread Tom Lane
Heikki Linnakangas hlinnakan...@vmware.com writes: On 10/27/2014 03:21 PM, Tomas Vondra wrote: Thinking about this a bit more, do we really need a full checkpoint? That is a checkpoint of all the databases in the cluster? Why checkpointing the source database is not enough? A full checkpoint

Re: [HACKERS] proposal: CREATE DATABASE vs. (partial) CHECKPOINT

2014-10-27 Thread Andres Freund
On 2014-10-27 09:46:41 -0400, Tom Lane wrote: Heikki Linnakangas hlinnakan...@vmware.com writes: On 10/27/2014 03:21 PM, Tomas Vondra wrote: Thinking about this a bit more, do we really need a full checkpoint? That is a checkpoint of all the databases in the cluster? Why checkpointing the

Re: [HACKERS] jsonb generator functions

2014-10-27 Thread Andrew Dunstan
On 10/15/2014 03:54 PM, Andrew Dunstan wrote: I checked a code, and I have only two small objection - a name jsonb_object_two_arg is not good - maybe json_object_keys_values ? It's consistent with the existing json_object_two_arg. In all cases I think I kept the names the same except for

Re: [HACKERS] Review of GetUserId() Usage

2014-10-27 Thread Stephen Frost
* Stephen Frost (sfr...@snowman.net) wrote: Attached is a patch to address the pg_cancel/terminate_backend and the statistics info as discussed previously. It sounds like we're coming to And I forgot the attachment, of course. Apologies. Thanks, Stephen diff --git

Re: [HACKERS] Review of GetUserId() Usage

2014-10-27 Thread Stephen Frost
All, * Peter Eisentraut (pete...@gmx.net) wrote: It would be weird if it were inconsistent: some things require role membership, some things require SET ROLE. Try explaining that. Attached is a patch to address the pg_cancel/terminate_backend and the statistics info as discussed previously.

[HACKERS] cost_index()

2014-10-27 Thread Teodor Sigaev
Hi! Some fragment of code (src/backend/optimizer/path/costsize.c, lineno ~400): /* * Normal case: apply the Mackert and Lohman formula, and then * interpolate between that and the correlation-derived result. */ pages_fetched =

Re: [HACKERS] cost_index()

2014-10-27 Thread Tom Lane
Teodor Sigaev teo...@sigaev.ru writes: if (indexonly) pages_fetched = ceil(pages_fetched * (1.0 - baserel-allvisfrac)); As I understand the code, index_pages_fetched() returns summary of page's read for index and heap together. No. Costs for touching the index

Re: [HACKERS] superuser() shortcuts

2014-10-27 Thread Stephen Frost
* Alvaro Herrera (alvhe...@2ndquadrant.com) wrote: Brightwell, Adam wrote: If we were to make it consistent and use the old wording, what do you think about providing an errhint as well? Perhaps for example in slotfuncs.c#pg_create_physical_replication_stot: errmsg - permission

Re: [HACKERS] Typo fixes for pg_recvlogical documentation

2014-10-27 Thread Robert Haas
On Fri, Oct 24, 2014 at 5:14 PM, Michael Paquier michael.paqu...@gmail.com wrote: On Fri, Oct 24, 2014 at 7:42 PM, Robert Haas robertmh...@gmail.com wrote: [rhaas pgsql]$ patch -p1 ~/Downloads/20141023_pg_recvlogical_fixes.patch patching file doc/src/sgml/ref/pg_recvlogical.sgml Hunk #1

Re: [HACKERS] Typo fixes for pg_recvlogical documentation

2014-10-27 Thread Michael Paquier
On Tue, Oct 28, 2014 at 12:12 AM, Robert Haas robertmh...@gmail.com wrote: Committed and back-patched to 9.4. Thanks. -- Michael -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] cost_index()

2014-10-27 Thread Teodor Sigaev
No. Costs for touching the index were computed by the amcostestimate function; this code is solely about estimating costs for touching the heap. I see. Thank you. -- Teodor Sigaev E-mail: teo...@sigaev.ru WWW:

Re: [HACKERS] superuser() shortcuts

2014-10-27 Thread Stephen Frost
* Alvaro Herrera (alvhe...@2ndquadrant.com) wrote: As I started looking at this, there are multiple other places where these types of error messages occur (opclasscmds.c, user.c, postinit.c, miscinit.c are just a few), not just around the changes in this patch. If we change them in one

Re: [HACKERS] TAP test breakage on MacOS X

2014-10-27 Thread Robert Haas
On Sun, Oct 26, 2014 at 12:29 PM, Tom Lane t...@sss.pgh.pa.us wrote: The larger issue though is that even with both the above things fixed, the TAP tests would still be an expensive no-op on the majority of buildfarm members. AFAICT, I do not own a single machine on which the current TAP

Re: [HACKERS] Materialized views don't show up in information_schema

2014-10-27 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: On Fri, Oct 17, 2014 at 8:10 PM, Stephen Frost sfr...@snowman.net wrote: * Peter Eisentraut (pete...@gmx.net) wrote: On 10/16/14 9:45 AM, Stephen Frost wrote: Alright, coming back to this, I have to ask- how are matviews different from views

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-10-27 Thread Robert Haas
On Sun, Oct 26, 2014 at 4:39 PM, Peter Geoghegan p...@heroku.com wrote: I don't care whether you actually generate index-paths or not, and in fact I suspect it makes no sense to do so. But I do care that you do a cost comparison between the available indexes and pick the one that looks

Re: [HACKERS] TAP test breakage on MacOS X

2014-10-27 Thread Heikki Linnakangas
On 10/27/2014 05:41 PM, Robert Haas wrote: On Sun, Oct 26, 2014 at 12:29 PM, Tom Lane t...@sss.pgh.pa.us wrote: The larger issue though is that even with both the above things fixed, the TAP tests would still be an expensive no-op on the majority of buildfarm members. AFAICT, I do not own a

Re: [HACKERS] TAP test breakage on MacOS X

2014-10-27 Thread Andrew Dunstan
On 10/27/2014 11:41 AM, Robert Haas wrote: On Sun, Oct 26, 2014 at 12:29 PM, Tom Lane t...@sss.pgh.pa.us wrote: The larger issue though is that even with both the above things fixed, the TAP tests would still be an expensive no-op on the majority of buildfarm members. AFAICT, I do not own a

Re: [HACKERS] What exactly is our CRC algorithm?

2014-10-27 Thread Heikki Linnakangas
On 10/09/2014 12:13 AM, Andres Freund wrote: On 2014-10-08 22:13:46 +0300, Heikki Linnakangas wrote: As far as I can tell, PostgreSQL's so-called CRC algorithm doesn't correspond to any bit-by-bit CRC variant and polynomial. My math skills are not strong enough to determine what the

Re: [HACKERS] BUG: *FF WALs under 9.2 (WAS: .ready files appearing on slaves)

2014-10-27 Thread Heikki Linnakangas
On 10/27/2014 02:12 PM, Fujii Masao wrote: On Fri, Oct 24, 2014 at 10:05 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 10/23/2014 11:09 AM, Heikki Linnakangas wrote: At least for master, we should consider changing the way the archiving works so that we only archive WAL that was

Re: [HACKERS] proposal: CREATE DATABASE vs. (partial) CHECKPOINT

2014-10-27 Thread Heikki Linnakangas
On 10/27/2014 03:46 PM, Tom Lane wrote: Heikki Linnakangas hlinnakan...@vmware.com writes: On 10/27/2014 03:21 PM, Tomas Vondra wrote: Thinking about this a bit more, do we really need a full checkpoint? That is a checkpoint of all the databases in the cluster? Why checkpointing the source

Re: [HACKERS] TAP test breakage on MacOS X

2014-10-27 Thread Tom Lane
Heikki Linnakangas hlinnakan...@vmware.com writes: On 10/27/2014 05:41 PM, Robert Haas wrote: Beyond all that, I have serious doubts about whether, even if we eventually get these tests mostly working in most places, whether they will actually catch any bugs. The existing tests are not very

Re: [HACKERS] TAP test breakage on MacOS X

2014-10-27 Thread Tom Lane
Andrew Dunstan and...@dunslane.net writes: Yeah. I think at the very least they should be removed from the check-world and installcheck-world targets until this is sorted out. +1 for doing that in the 9.4 branch; I'm surprised we've not already heard bitching from packagers about how far we've

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-10-27 Thread Simon Riggs
On 27 October 2014 15:55, Robert Haas robertmh...@gmail.com wrote: Commenting on one aspect of a patch doesn't imply agreement with other aspects of the patch. Please don't put words into my mouth. I haven't reviewed this patch in detail; I've only commented on specific aspects of it as

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-10-27 Thread Peter Geoghegan
On Mon, Oct 27, 2014 at 9:43 AM, Simon Riggs si...@2ndquadrant.com wrote: On 27 October 2014 15:55, Robert Haas robertmh...@gmail.com wrote: Commenting on one aspect of a patch doesn't imply agreement with other aspects of the patch. Please don't put words into my mouth. I haven't reviewed

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-10-27 Thread Simon Riggs
On 27 October 2014 17:44, Peter Geoghegan p...@heroku.com wrote: I did a lot of copy-editing to the Wiki page yesterday. There are actually few clear open items now: https://wiki.postgresql.org/wiki/UPSERT#Open_Items I've read this page. Please do these things, both of which have been

Re: [HACKERS] jsonb generator functions

2014-10-27 Thread Pavel Stehule
Hi 2014-10-27 15:33 GMT+01:00 Andrew Dunstan and...@dunslane.net: On 10/15/2014 03:54 PM, Andrew Dunstan wrote: I checked a code, and I have only two small objection - a name jsonb_object_two_arg is not good - maybe json_object_keys_values ? It's consistent with the existing

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-10-27 Thread Peter Geoghegan
On Mon, Oct 27, 2014 at 11:12 AM, Simon Riggs si...@2ndquadrant.com wrote: 1. Take the specific docs that relate to the patch and put them in one place, so that everybody can read and understand and agree the behaviour of the patch. So that someone reading that can see *exactly* what is being

[HACKERS] TODO request: log_long_transaction

2014-10-27 Thread Josh Berkus
Hackers, I just realized that there is one thing we can't log currently: transactions which last more than #ms. This is valuable diagnostic information when looking for issues like causes of bloat and deadlocks. I'd like it to be on the TODO list because it seems like part of a good GSOC

Re: [HACKERS] TODO request: log_long_transaction

2014-10-27 Thread Thom Brown
On 27 October 2014 19:21, Josh Berkus j...@agliodbs.com wrote: Hackers, I just realized that there is one thing we can't log currently: transactions which last more than #ms. This is valuable diagnostic information when looking for issues like causes of bloat and deadlocks. I'd like it to

[HACKERS] Reducing the cost of sinval messaging

2014-10-27 Thread Tom Lane
I happened to be looking at sinvaladt.c and noticed the loop added in commit b4fbe392f8ff6ff1a66b488eb7197eef9e1770a4: /* * Now that the maxMsgNum change is globally visible, we give everyone * a swift kick to make sure they read the newly added messages. *

Re: [HACKERS] Reducing the cost of sinval messaging

2014-10-27 Thread Robert Haas
On Mon, Oct 27, 2014 at 3:31 PM, Tom Lane t...@sss.pgh.pa.us wrote: I happened to be looking at sinvaladt.c and noticed the loop added in commit b4fbe392f8ff6ff1a66b488eb7197eef9e1770a4: /* * Now that the maxMsgNum change is globally visible, we give everyone * a

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-10-27 Thread Robert Haas
On Mon, Oct 27, 2014 at 1:44 PM, Peter Geoghegan p...@heroku.com wrote: I think by far the biggest problem here is the lack of attention to the design from others. I find that attitude incredible. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company --

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-10-27 Thread Peter Geoghegan
On Mon, Oct 27, 2014 at 1:22 PM, Robert Haas robertmh...@gmail.com wrote: On Mon, Oct 27, 2014 at 1:44 PM, Peter Geoghegan p...@heroku.com wrote: I think by far the biggest problem here is the lack of attention to the design from others. I find that attitude incredible. What I mean is that

Re: [HACKERS] Reducing the cost of sinval messaging

2014-10-27 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Mon, Oct 27, 2014 at 3:31 PM, Tom Lane t...@sss.pgh.pa.us wrote: Why could we not remove the hasMessages flags again, and change the unlocked test if (!stateP-hasMessages) return 0; into if (stateP-nextMsgNum == segP-maxMsgNum

Re: [HACKERS] alter user/role CURRENT_USER

2014-10-27 Thread Adam Brightwell
All, I just ran through the patch giving it a good once over, some items to address/consider/discuss: * Trailing whitespace. * Why are you making changes in foreigncmds.c? These seem like unnecessary changes. I see that you are trying to consolidate but this refactor seems potentially out of

Re: [HACKERS] Reducing the cost of sinval messaging

2014-10-27 Thread Robert Haas
On Mon, Oct 27, 2014 at 4:27 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Mon, Oct 27, 2014 at 3:31 PM, Tom Lane t...@sss.pgh.pa.us wrote: Why could we not remove the hasMessages flags again, and change the unlocked test if (!stateP-hasMessages) return

Re: [HACKERS] proposal: CREATE DATABASE vs. (partial) CHECKPOINT

2014-10-27 Thread Tomas Vondra
On 27.10.2014 17:24, Heikki Linnakangas wrote: On 10/27/2014 03:46 PM, Tom Lane wrote: Heikki Linnakangas hlinnakan...@vmware.com writes: On 10/27/2014 03:21 PM, Tomas Vondra wrote: Thinking about this a bit more, do we really need a full checkpoint? That is a checkpoint of all the databases

Re: [HACKERS] Directory/File Access Permissions for COPY and Generic File Access Functions

2014-10-27 Thread Adam Brightwell
All, Attached is a patch with minor updates/corrections. -Adam -- Adam Brightwell - adam.brightw...@crunchydatasolutions.com Database Engineer - www.crunchydatasolutions.com diff --git a/src/backend/catalog/Makefile b/src/backend/catalog/Makefile new file mode 100644 index b257b02..8cdc5cb ***

Re: [HACKERS] proposal: CREATE DATABASE vs. (partial) CHECKPOINT

2014-10-27 Thread Tom Lane
Tomas Vondra t...@fuzzy.cz writes: That being said, our CREATE DATABASE docs currently say this Although it is possible to copy a database other than template1 by specifying its name as the template, this is not (yet) intended as a general-purpose COPY DATABASE facility. The

Re: [HACKERS] pgcrypto: PGP signatures

2014-10-27 Thread Jeff Janes
On Mon, Oct 20, 2014 at 3:32 PM, Marko Tiikkaja ma...@joh.to wrote: Hi, Here's the rebased patch -- as promised -- in a v7. Hi Marko, Using the same script as for the memory leak, I am getting seg faults using this patch. 24425 2014-10-27 15:42:11.819 PDT LOG: server process (PID

Re: [HACKERS] proposal: CREATE DATABASE vs. (partial) CHECKPOINT

2014-10-27 Thread Andrew Dunstan
On 10/27/2014 05:58 PM, Tomas Vondra wrote: On 27.10.2014 17:24, Heikki Linnakangas wrote: On 10/27/2014 03:46 PM, Tom Lane wrote: Heikki Linnakangas hlinnakan...@vmware.com writes: On 10/27/2014 03:21 PM, Tomas Vondra wrote: Thinking about this a bit more, do we really need a full

Re: [HACKERS] proposal: CREATE DATABASE vs. (partial) CHECKPOINT

2014-10-27 Thread Andres Freund
On 2014-10-27 18:57:27 -0400, Andrew Dunstan wrote: On 10/27/2014 05:58 PM, Tomas Vondra wrote: On 27.10.2014 17:24, Heikki Linnakangas wrote: I'm also thinking that for wal_level=archive and large databases, this won't really eliminate the checkpoint as it will likely generate enough WAL

Re: [HACKERS] Directory/File Access Permissions for COPY and Generic File Access Functions

2014-10-27 Thread Peter Eisentraut
I think the way this should work is that if you create a DIRALIAS, then the COPY command should refer to it by logical name, e.g., CREATE DIRALIAS dumpster AS '/tmp/trash'; COPY mytable TO dumpster; If you squint a bit, this is the same as a tablespace. Maybe those two concepts could be

Re: [HACKERS] proposal: CREATE DATABASE vs. (partial) CHECKPOINT

2014-10-27 Thread Andrew Dunstan
On 10/27/2014 07:01 PM, Andres Freund wrote: On 2014-10-27 18:57:27 -0400, Andrew Dunstan wrote: On 10/27/2014 05:58 PM, Tomas Vondra wrote: On 27.10.2014 17:24, Heikki Linnakangas wrote: I'm also thinking that for wal_level=archive and large databases, this won't really eliminate the

Re: [HACKERS] Directory/File Access Permissions for COPY and Generic File Access Functions

2014-10-27 Thread Peter Eisentraut
On 10/27/14 7:27 AM, Stephen Frost wrote: * Peter Eisentraut (pete...@gmx.net) wrote: On 10/16/14 12:01 PM, Stephen Frost wrote: This started out as a request for a non-superuser to be able to review the log files without needing access to the server. I think that can be done with a

Re: [HACKERS] proposal: CREATE DATABASE vs. (partial) CHECKPOINT

2014-10-27 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On 2014-10-27 18:57:27 -0400, Andrew Dunstan wrote: Notwithstanding what the docs say, I have seen CREATE DATABASE used plenty of times, and quite effectively, to clone databases. I don't think making it do twice the IO in the general case is going

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-10-27 Thread Simon Riggs
On 27 October 2014 20:24, Peter Geoghegan p...@heroku.com wrote: On Mon, Oct 27, 2014 at 1:22 PM, Robert Haas robertmh...@gmail.com wrote: On Mon, Oct 27, 2014 at 1:44 PM, Peter Geoghegan p...@heroku.com wrote: I think by far the biggest problem here is the lack of attention to the design from

Re: [HACKERS] Directory/File Access Permissions for COPY and Generic File Access Functions

2014-10-27 Thread Stephen Frost
* Peter Eisentraut (pete...@gmx.net) wrote: On 10/27/14 7:27 AM, Stephen Frost wrote: * Peter Eisentraut (pete...@gmx.net) wrote: On 10/16/14 12:01 PM, Stephen Frost wrote: This started out as a request for a non-superuser to be able to review the log files without needing access to the

Re: [HACKERS] proposal: CREATE DATABASE vs. (partial) CHECKPOINT

2014-10-27 Thread Tomas Vondra
On 28.10.2014 00:06, Andrew Dunstan wrote: On 10/27/2014 07:01 PM, Andres Freund wrote: On 2014-10-27 18:57:27 -0400, Andrew Dunstan wrote: On 10/27/2014 05:58 PM, Tomas Vondra wrote: On 27.10.2014 17:24, Heikki Linnakangas wrote: I'm also thinking that for wal_level=archive and large

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-10-27 Thread Peter Geoghegan
On Mon, Oct 27, 2014 at 4:34 PM, Simon Riggs si...@2ndquadrant.com wrote: Let's see if we can link these two thoughts. 1. You think the biggest problem is the lack of attention to the design. 2. I keep asking you to put the docs in a readable form. If you can't understand the link between

Re: [HACKERS] proposal: CREATE DATABASE vs. (partial) CHECKPOINT

2014-10-27 Thread Tom Lane
Tomas Vondra t...@fuzzy.cz writes: So maybe we shouldn't cling to the WAL-logging approach too much. Maybe Heikki's idea from to abandon the full checkpoint and instead assume that once the transaction commits, all the files were fsynced OK. Of couse, this will do nothing about the replay

Re: [HACKERS] Function array_agg(array)

2014-10-27 Thread Ali Akbar
super I tested last version and I have not any objections. 1. We would to have this feature - it is long time number of our ToDo List 2. Proposal and design of multidimensional aggregation is clean and nobody has objection here. 3. There is zero impact on current implementation. From

Re: [HACKERS] alter user/role CURRENT_USER

2014-10-27 Thread Marti Raudsepp
On Fri, Oct 24, 2014 at 11:29 AM, Kyotaro HORIGUCHI horiguchi.kyot...@lab.ntt.co.jp wrote: - 0001-ALTER-ROLE-CURRENT_USER_v2.patch - the patch. +RoleId:CURRENT_USER{ $$ = current_user;} + | USER { $$ =

Re: [HACKERS] Reducing lock strength of adding foreign keys

2014-10-27 Thread Noah Misch
On Mon, Oct 27, 2014 at 08:24:15AM -0400, Robert Haas wrote: On Sat, Oct 25, 2014 at 2:00 PM, Noah Misch n...@leadboat.com wrote: http://www.postgresql.org/message-id/ca+tgmoy4glsxzk0tao29-ljtcuj0sl1xwcwq51xb-hfysgi...@mail.gmail.com

Re: [HACKERS] proposal: CREATE DATABASE vs. (partial) CHECKPOINT

2014-10-27 Thread David G Johnston
Tomas Vondra wrote I mean, when we use database A as a template, why do we need to checkpoint B, C, D and F too? (Apologies if this is somehow obvious, I'm way out of my comfort zone in this part of the code.) IIUC you have to checkpoint the whole cluster because it is not possible to do

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-10-27 Thread Peter Geoghegan
--- doc/src/sgml/ddl.sgml | 23 +++ doc/src/sgml/indices.sgml | 11 +- doc/src/sgml/mvcc.sgml| 43 -- doc/src/sgml/plpgsql.sgml | 20 ++- doc/src/sgml/postgres-fdw.sgml| 8 ++ doc/src/sgml/ref/create_index.sgml|

Re: [HACKERS] Reducing the cost of sinval messaging

2014-10-27 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Mon, Oct 27, 2014 at 4:27 PM, Tom Lane t...@sss.pgh.pa.us wrote: Neither of those messages seem to me to bear on this point. AFAICS, the unlocked hasMessages test has a race condition, which the comment just above it argues isn't a problem in

Re: [HACKERS] proposal: CREATE DATABASE vs. (partial) CHECKPOINT

2014-10-27 Thread David G Johnston
Tom Lane-2 wrote Tomas Vondra lt; tv@ gt; writes: So maybe we shouldn't cling to the WAL-logging approach too much. Maybe Heikki's idea from to abandon the full checkpoint and instead assume that once the transaction commits, all the files were fsynced OK. Of couse, this will do nothing

Re: [HACKERS] Directory/File Access Permissions for COPY and Generic File Access Functions

2014-10-27 Thread Stephen Frost
* Peter Eisentraut (pete...@gmx.net) wrote: I think the way this should work is that if you create a DIRALIAS, then the COPY command should refer to it by logical name, e.g., CREATE DIRALIAS dumpster AS '/tmp/trash'; COPY mytable TO dumpster; You'd have to be able to specify the filename

Re: [HACKERS] alter user/role CURRENT_USER

2014-10-27 Thread David G Johnston
Marti Raudsepp wrote On Fri, Oct 24, 2014 at 11:29 AM, Kyotaro HORIGUCHI lt; horiguchi.kyotaro@.co gt; wrote: But should ALTER USER ALL and ALTER ROLE ALL really do the same thing? A user is a role with the LOGIN option. Every user is a role, but not every role is a user. I suspect that

Re: [HACKERS] Reducing the cost of sinval messaging

2014-10-27 Thread Robert Haas
On Mon, Oct 27, 2014 at 8:54 PM, Tom Lane t...@sss.pgh.pa.us wrote: That argument is nonsense. I complained about a lack of close analysis, but with close analysis I think this is perfectly safe; or at least no less safe than what's there now, with its not terribly bulletproof assumption that

  1   2   >