Re: [HACKERS] Final Patch for GROUPING SETS

2014-12-31 Thread Noah Misch
On Tue, Dec 23, 2014 at 02:29:58AM -0500, Noah Misch wrote: On Mon, Dec 22, 2014 at 10:46:16AM -0500, Tom Lane wrote: I still find the ChainAggregate approach too ugly at a system structural level to accept, regardless of Noah's argument about number of I/O cycles consumed. We'll be paying

Re: [HACKERS] Final Patch for GROUPING SETS

2014-12-31 Thread Atri Sharma
ChainAggregate is a bit like a node having two parents, a Sort and a GroupAggregate. However, the graph edge between ChainAggregate and its GroupAggregate is a tuplestore instead of the usual, synchronous ExecProcNode(). Well, I dont buy the two parents theory. The Sort nodes are

Re: [HACKERS] BUG: *FF WALs under 9.2 (WAS: .ready files appearing on slaves)

2014-12-31 Thread Michael Paquier
On Wed, Dec 31, 2014 at 4:44 PM, Guillaume Lelarge guilla...@lelarge.info wrote: 2014-12-12 14:58 GMT+01:00 Heikki Linnakangas hlinnakan...@vmware.com: Now, what do we do with the back-branches? I'm not sure. Changing the behaviour in back-branches could cause nasty surprises. Perhaps it's best

Re: [HACKERS] Redesigning checkpoint_segments

2014-12-31 Thread Heikki Linnakangas
(reviving an old thread) On 08/24/2013 12:53 AM, Josh Berkus wrote: On 08/23/2013 02:08 PM, Heikki Linnakangas wrote: Here's a bigger patch, which does more. It is based on the ideas in the post I started this thread with, with feedback incorporated from the long discussion. With this patch,

Re: [HACKERS] Redesigning checkpoint_segments

2014-12-31 Thread Heikki Linnakangas
On 09/01/2013 10:37 AM, Amit Kapila wrote: On Sat, Aug 24, 2013 at 2:38 AM, Heikki Linnakangas hlinnakan...@vmware.com wrote: a. In XLogFileInit(), /* ! * XXX: What should we use as max_segno? We used to use XLOGfileslop when ! * that was a constant, but that was always a bit dubious:

Re: [HACKERS] add modulo (%) operator to pgbench

2014-12-31 Thread Fabien COELHO
Here is a review updated version of the patch. Patch applies and compiles without problem on current head, and worked for the various tests I made with custom scripts. This patch is a good thing, and I recommand warmly its inclusion. It extends greatly pgbench custom capabilities by allowing

Re: [HACKERS] Initdb-cs_CZ.WIN-1250 buildfarm failures

2014-12-31 Thread Tomas Vondra
On 28.12.2014 00:46, Noah Misch wrote: On Tue, Dec 23, 2014 at 03:32:59PM +0100, Tomas Vondra wrote: On 23.12.2014 15:21, Andrew Dunstan wrote: No, config_opts is what's passed to configure. Try something like: if ($branch eq 'REL9_0_STABLE') {

Re: [HACKERS] Additional role attributes superuser review

2014-12-31 Thread José Luis Tallón
On 12/30/2014 04:16 PM, Stephen Frost wrote: [snip] The approach I was thinking was to document and implement this as impliciting granting exactly USAGE and SELECT rights, no more (not BYPASSRLS) and no less (yes, the role could execute functions). I agree that doing so would be strictly more

Re: [HACKERS] TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

2014-12-31 Thread Amit Kapila
On Mon, Dec 29, 2014 at 11:10 AM, Dilip kumar dilip.ku...@huawei.com wrote: On 29 December 2014 10:22 Amit Kapila Wrote, I think nothing more to be handled from my side, you can go ahead with review.. The patch looks good to me. I have done couple of cosmetic changes (spelling mistakes,

Re: [HACKERS] Additional role attributes superuser review

2014-12-31 Thread Magnus Hagander
On Tue, Dec 30, 2014 at 4:16 PM, Stephen Frost sfr...@snowman.net wrote: * Magnus Hagander (mag...@hagander.net) wrote: On Mon, Dec 29, 2014 at 11:01 PM, Stephen Frost sfr...@snowman.net wrote: That said, a 'DUMP' privilege which allows the user to dump the contents of the entire

[HACKERS] Performance improvement for joins where outer side is unique

2014-12-31 Thread David Rowley
Hi, I've been hacking a bit at the join code again... This time I've been putting some effort into optimising the case where the inner side of the join is known to be unique. For example, given the tables: create table t1 (id int primary key); create table t2 (id int primary key); And query

Re: [HACKERS] Using 128-bit integers for sum, avg and statistics aggregates

2014-12-31 Thread David Rowley
On 31 December 2014 at 18:20, Robert Haas robertmh...@gmail.com wrote: On Fri, Dec 26, 2014 at 7:57 AM, David Rowley dgrowle...@gmail.com wrote: 1. Do we need to keep the 128 byte aggregate state size for machines without 128 bit ints? This has been reduced to 48 bytes in the patch, which

Re: [HACKERS] Additional role attributes superuser review

2014-12-31 Thread Stephen Frost
* Magnus Hagander (mag...@hagander.net) wrote: On Tue, Dec 30, 2014 at 4:16 PM, Stephen Frost sfr...@snowman.net wrote: The approach I was thinking was to document and implement this as impliciting granting exactly USAGE and SELECT rights, no more (not BYPASSRLS) and no less (yes, the role

Re: [HACKERS] Using 128-bit integers for sum, avg and statistics aggregates

2014-12-31 Thread Andres Freund
On 2015-01-01 03:00:50 +1300, David Rowley wrote: 2. References to int16 meaning 16 bytes. I'm really in two minds about this, it's quite nice to keep the natural flow, int4, int8, int16, but I can't help think that this will confuse someone one day. I think it'll be a long time

Re: [HACKERS] Parallel Seq Scan

2014-12-31 Thread Thom Brown
On 18 December 2014 at 16:03, Amit Kapila amit.kapil...@gmail.com wrote: On Thu, Dec 18, 2014 at 9:22 PM, Amit Kapila amit.kapil...@gmail.com wrote: On Mon, Dec 8, 2014 at 10:40 AM, Amit Kapila amit.kapil...@gmail.com wrote: On Sat, Dec 6, 2014 at 5:37 PM, Stephen Frost

[HACKERS] Small doc patch about pg_service.conf

2014-12-31 Thread David Fetter
Folks, There was a slash missing, which I've added. Where is the default directory on Windows, or is there one? Cheers, David. -- David Fetter da...@fetter.org http://fetter.org/ Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter Skype: davidfetter XMPP: david.fet...@gmail.com

Re: [HACKERS] Failure on markhor with CLOBBER_CACHE_ALWAYS for test brin

2014-12-31 Thread Alvaro Herrera
Alvaro Herrera wrote: Michael Paquier wrote: HI all. markhor has run for the first time in 8 days, and there is something in range e703261..72dd233 making the regression test of brin crashing. See here:

Re: [HACKERS] Additional role attributes superuser review

2014-12-31 Thread Stephen Frost
Amit, * Amit Kapila (amit.kapil...@gmail.com) wrote: On Tue, Dec 30, 2014 at 6:35 PM, Stephen Frost sfr...@snowman.net wrote: I'm pretty sure we've agreed that having a catch-all role attribute like 'DBA' is a bad idea because we'd likely be adding privileges to it later which would expand

Re: [HACKERS] Failure on markhor with CLOBBER_CACHE_ALWAYS for test brin

2014-12-31 Thread Andres Freund
On 2014-12-31 10:02:40 -0300, Alvaro Herrera wrote: Alvaro Herrera wrote: Michael Paquier wrote: HI all. markhor has run for the first time in 8 days, and there is something in range e703261..72dd233 making the regression test of brin crashing. See here:

Re: [HACKERS] Additional role attributes superuser review

2014-12-31 Thread Magnus Hagander
On Wed, Dec 31, 2014 at 3:08 PM, Stephen Frost sfr...@snowman.net wrote: * Magnus Hagander (mag...@hagander.net) wrote: On Tue, Dec 30, 2014 at 4:16 PM, Stephen Frost sfr...@snowman.net wrote: The approach I was thinking was to document and implement this as impliciting granting exactly

Re: [HACKERS] documentation update for doc/src/sgml/func.sgml

2014-12-31 Thread Fabien COELHO
Here is a slight update so that type names are treated homogeneously between both added paragraphs. ITSM that this patch should be committed without further ado. -- Fabien.diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index 2016c5a..f91033b 100644 --- a/doc/src/sgml/func.sgml

Re: [HACKERS] Additional role attributes superuser review

2014-12-31 Thread Stephen Frost
José, * José Luis Tallón (jltal...@adv-solutions.net) wrote: On 12/30/2014 04:16 PM, Stephen Frost wrote: The approach I was thinking was to document and implement this as impliciting granting exactly USAGE and SELECT rights, no more (not BYPASSRLS) and no less (yes, the role could execute

Re: [HACKERS] Failure on markhor with CLOBBER_CACHE_ALWAYS for test brin

2014-12-31 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On 2014-12-31 10:02:40 -0300, Alvaro Herrera wrote: I can reproduce the crash in a CLOBBER_CACHE_ALWAYS build in the object_address test. The backtrace is pretty strange: Hard to say without more detail, but my guess is that the argument to

Re: [HACKERS] add modulo (%) operator to pgbench

2014-12-31 Thread Fabien COELHO
I'm not sure about what Makefile changes could be necessary for various environments, it looks ok to me as it is. Found one. EXTRA_CLEAN is needed for generated C files. Added to this v3. -- Fabien.diff --git a/contrib/pgbench/.gitignore b/contrib/pgbench/.gitignore index 489a2d6..aae819e

Re: [HACKERS] [GENERAL] ON_ERROR_ROLLBACK

2014-12-31 Thread Tom Lane
Andrew Dunstan and...@dunslane.net writes: On 12/30/2014 09:20 AM, Tom Lane wrote: In one light this is certainly a bug fix, but in another it's just definitional instability. If we'd gotten a field bug report we might well have chosen to back-patch, though, and perhaps your client's

Re: [HACKERS] Additional role attributes superuser review

2014-12-31 Thread Stephen Frost
* Magnus Hagander (mag...@hagander.net) wrote: On Wed, Dec 31, 2014 at 3:08 PM, Stephen Frost sfr...@snowman.net wrote: * Magnus Hagander (mag...@hagander.net) wrote: that doing so would be strictly more than what pg_dump actually requires but it's also what we actually have support for

Re: [HACKERS] pg_basebackup -x/X doesn't play well with archive_mode wal_keep_segments

2014-12-31 Thread Andres Freund
Hi, On 2014-12-05 16:18:02 +0900, Fujii Masao wrote: On Fri, Dec 5, 2014 at 9:28 AM, Andres Freund and...@2ndquadrant.com wrote: So I think we just need to make pg_basebackup create to .ready files. s/.ready/.done? If yes, +1. That unfortunately requires changes to both backend and

Re: [HACKERS] psql tab completion: fix COMMENT ON ... IS IS IS

2014-12-31 Thread Robert Haas
On Sun, Dec 28, 2014 at 7:44 PM, Ian Barwick i...@2ndquadrant.com wrote: Currently tab completion for 'COMMENT ON {object} foo IS' will result in the 'IS' being duplicated up to two times; not a world-shattering issue I know, but the fix is trivial and I stumble over it often enough to for it

Re: [HACKERS] The return value of allocate_recordbuf()

2014-12-31 Thread Robert Haas
On Mon, Dec 29, 2014 at 6:14 AM, Heikki Linnakangas hlinnakan...@vmware.com wrote: Hmm. There is no way to check beforehand if a palloc() will fail because of OOM. We could check for MaxAllocSize, though. I think we need a version of palloc that returns NULL instead of throwing an error. The

Re: [HACKERS] Parallel Seq Scan

2014-12-31 Thread Thom Brown
On 31 December 2014 at 14:20, Thom Brown t...@linux.com wrote: On 18 December 2014 at 16:03, Amit Kapila amit.kapil...@gmail.com wrote: On Thu, Dec 18, 2014 at 9:22 PM, Amit Kapila amit.kapil...@gmail.com wrote: On Mon, Dec 8, 2014 at 10:40 AM, Amit Kapila amit.kapil...@gmail.com

Re: [HACKERS] Publish autovacuum informations

2014-12-31 Thread Robert Haas
On Mon, Dec 29, 2014 at 11:03 AM, Tom Lane t...@sss.pgh.pa.us wrote: Either one of those approaches would cripple our freedom to change those data structures; which we've done repeatedly in the past and will surely want to do again. So I'm pretty much -1 on exposing them. We could instead add

Re: [HACKERS] Failure on markhor with CLOBBER_CACHE_ALWAYS for test brin

2014-12-31 Thread Alvaro Herrera
Tom Lane wrote: Given that CLOBBER_CACHE_ALWAYS seems to make it fail reliably, the obvious explanation is that what's being passed is a pointer into catcache or relcache storage that isn't guaranteed to be valid for long enough. The given backtrace doesn't go down far enough to show where

Re: [HACKERS] Initdb-cs_CZ.WIN-1250 buildfarm failures

2014-12-31 Thread Andrew Dunstan
On 12/31/2014 07:49 AM, Tomas Vondra wrote: On 28.12.2014 00:46, Noah Misch wrote: On Tue, Dec 23, 2014 at 03:32:59PM +0100, Tomas Vondra wrote: On 23.12.2014 15:21, Andrew Dunstan wrote: No, config_opts is what's passed to configure. Try something like: if ($branch eq 'REL9_0_STABLE')

Re: [HACKERS] Using 128-bit integers for sum, avg and statistics aggregates

2014-12-31 Thread Robert Haas
On Wed, Dec 31, 2014 at 9:00 AM, David Rowley dgrowle...@gmail.com wrote: Yeah, that's what I was talking about. I'm just looking at the code which uses this size estimate in choose_hashed_grouping(). I'd be a bit worried giving the difference between 48 and 128 that we'd under estimate the

Re: [HACKERS] Final Patch for GROUPING SETS

2014-12-31 Thread Andrew Gierth
Noah == Noah Misch n...@leadboat.com writes: Noah Suppose one node orchestrated all sorting and aggregation. Well, that has the downside of making it into an opaque blob, without actually gaining much. Noah Call it a MultiGroupAggregate for now. It wouldn't harness Noah Sort nodes, because

Re: [HACKERS] Initdb-cs_CZ.WIN-1250 buildfarm failures

2014-12-31 Thread Tomas Vondra
On 31.12.2014 17:29, Andrew Dunstan wrote: Sorry, I should have tested it. This seems to work: if ($branch eq 'REL9_0_STABLE') { $PGBuild::Options::skip_steps .= ' pl-install-check'; $main::skip_steps{'pl-install-check'} = 1; } cheers Meh, no problem. We've

Re: [HACKERS] Initdb-cs_CZ.WIN-1250 buildfarm failures

2014-12-31 Thread Andrew Dunstan
On 12/31/2014 12:26 PM, Tomas Vondra wrote: On 31.12.2014 17:29, Andrew Dunstan wrote: Sorry, I should have tested it. This seems to work: if ($branch eq 'REL9_0_STABLE') { $PGBuild::Options::skip_steps .= ' pl-install-check'; $main::skip_steps{'pl-install-check'} =

Re: [HACKERS] Publish autovacuum informations

2014-12-31 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Mon, Dec 29, 2014 at 11:03 AM, Tom Lane t...@sss.pgh.pa.us wrote: Either one of those approaches would cripple our freedom to change those data structures; which we've done repeatedly in the past and will surely want to do again. So I'm pretty much

Re: [HACKERS] orangutan seizes up during isolation-check

2014-12-31 Thread Noah Misch
On Sun, Dec 28, 2014 at 07:20:04PM -0500, Andrew Dunstan wrote: On 12/28/2014 04:58 PM, Noah Misch wrote: The gettext maintainer was open to implementing the setlocale_native_forked() technique in gettext, though the last visible progress was in October. In any event, PostgreSQL builds will

Re: [HACKERS] orangutan seizes up during isolation-check

2014-12-31 Thread Noah Misch
On Wed, Dec 31, 2014 at 12:32:37AM -0500, Robert Haas wrote: On Sun, Dec 28, 2014 at 4:58 PM, Noah Misch n...@leadboat.com wrote: I wondered whether to downgrade FATAL to LOG in back branches. Introducing a new reason to block startup is disruptive for a minor release, but having the

Re: [HACKERS] Final Patch for GROUPING SETS

2014-12-31 Thread Noah Misch
On Wed, Dec 31, 2014 at 02:45:29PM +0530, Atri Sharma wrote: Suppose one node orchestrated all sorting and aggregation. Call it a MultiGroupAggregate for now. It wouldn't harness Sort nodes, because it performs aggregation between tuplesort_puttupleslot() calls. Instead, it would

Re: [HACKERS] Final Patch for GROUPING SETS

2014-12-31 Thread Noah Misch
On Wed, Dec 31, 2014 at 05:33:43PM +, Andrew Gierth wrote: Noah == Noah Misch n...@leadboat.com writes: Noah Suppose one node orchestrated all sorting and aggregation. Well, that has the downside of making it into an opaque blob, without actually gaining much. The opaque-blob

Re: [HACKERS] Compression of full-page-writes

2014-12-31 Thread Bruce Momjian
On Tue, Dec 30, 2014 at 01:27:44PM +0100, Andres Freund wrote: On 2014-12-30 21:23:38 +0900, Michael Paquier wrote: On Tue, Dec 30, 2014 at 6:21 PM, Jeff Davis pg...@j-davis.com wrote: On Fri, 2013-08-30 at 09:57 +0300, Heikki Linnakangas wrote: Speeding up the CRC calculation obviously

Re: [HACKERS] Re: [COMMITTERS] pgsql: pg_event_trigger_dropped_objects: Add name/args output columns

2014-12-31 Thread Alvaro Herrera
Tom Lane wrote: But is there any reason to think the failure on dunlin has anything to do with default ACLs? I think you'd better work on understanding why there is a platform dependency here, before you consider either removing the regression test case or adding support for object types

Re: [HACKERS] Re: [COMMITTERS] pgsql: pg_event_trigger_dropped_objects: Add name/args output columns

2014-12-31 Thread Tom Lane
I wrote: Alvaro Herrera alvhe...@2ndquadrant.com writes: Alvaro Herrera wrote: pg_event_trigger_dropped_objects: Add name/args output columns This is causing buildfarm member dunlin to fail: ... No other member so far has reported a problem here. Not sure if that's the strangest bit, or

Re: [HACKERS] psql tab completion: fix COMMENT ON ... IS IS IS

2014-12-31 Thread Ian Barwick
On 15/01/01 1:07, Robert Haas wrote: On Sun, Dec 28, 2014 at 7:44 PM, Ian Barwick i...@2ndquadrant.com wrote: Currently tab completion for 'COMMENT ON {object} foo IS' will result in the 'IS' being duplicated up to two times; not a world-shattering issue I know, but the fix is trivial and I

Re: [HACKERS] Redesigning checkpoint_segments

2014-12-31 Thread Josh Berkus
Heikki, Thanks for getting back to this! I really look forward to simplifying WAL tuning for users. min_recycle_wal_size checkpoint_wal_size snip These settings are fairly intuitive for a DBA to tune. You begin by figuring out how much disk space you can afford to spend on WAL, and set

Re: [HACKERS] Compression of full-page-writes

2014-12-31 Thread Amit Kapila
On Thu, Jan 1, 2015 at 2:39 AM, Bruce Momjian br...@momjian.us wrote: On Tue, Dec 30, 2014 at 01:27:44PM +0100, Andres Freund wrote: On 2014-12-30 21:23:38 +0900, Michael Paquier wrote: On Tue, Dec 30, 2014 at 6:21 PM, Jeff Davis pg...@j-davis.com wrote: On Fri, 2013-08-30 at 09:57

Re: [HACKERS] Compression of full-page-writes

2014-12-31 Thread Michael Paquier
On Thu, Jan 1, 2015 at 2:10 PM, Amit Kapila amit.kapil...@gmail.com wrote: On Thu, Jan 1, 2015 at 2:39 AM, Bruce Momjian br...@momjian.us wrote: So why are you bringing it up? That's not an argument for anything, except not doing it in such a simplistic way. I still don't understand the

Re: [HACKERS] Parallel Seq Scan

2014-12-31 Thread Amit Kapila
On Wed, Dec 31, 2014 at 7:50 PM, Thom Brown t...@linux.com wrote: When attempting to recreate the plan in your example, I get an error: ➤ psql://thom@[local]:5488/pgbench # create table t1(c1 int, c2 char(500)) with (fillfactor=10); CREATE TABLE Time: 13.653 ms ➤

Re: [HACKERS] What exactly is our CRC algorithm?

2014-12-31 Thread Abhijit Menon-Sen
Hi. OK, here are the patches with the various suggestions applied. I found that the alignment didn't seem to make much difference for the CRC32* instructions, so I changed to process (len/8)*8bytes followed by (len%8)*1bytes, the way the Linux kernel does. -- Abhijit From

Re: [HACKERS] add modulo (%) operator to pgbench

2014-12-31 Thread David Rowley
On 1 January 2015 at 04:02, Fabien COELHO coe...@cri.ensmp.fr wrote: I'm not sure about what Makefile changes could be necessary for various environments, it looks ok to me as it is. Found one. EXTRA_CLEAN is needed for generated C files. Added to this v3. I was about to go and look at