Re: [HACKERS] Parallel Seq Scan

2015-01-02 Thread Thom Brown
On 1 January 2015 at 17:59, Robert Haas robertmh...@gmail.com wrote: On Thu, Jan 1, 2015 at 12:00 PM, Fabrízio de Royes Mello fabriziome...@gmail.com wrote: Can we check the number of free bgworkers slots to set the max workers? The real solution here is that this patch can't throw an error

Re: [HACKERS] Parallel Seq Scan

2015-01-02 Thread Thom Brown
On 2 January 2015 at 11:13, Amit Kapila amit.kapil...@gmail.com wrote: On Fri, Jan 2, 2015 at 4:09 PM, Thom Brown t...@linux.com wrote: On 1 January 2015 at 10:34, Amit Kapila amit.kapil...@gmail.com wrote: Running it again, I get the same issue. This is with parallel_seqscan_degree

Re: [HACKERS] Parallel Seq Scan

2015-01-02 Thread Amit Kapila
On Fri, Jan 2, 2015 at 4:09 PM, Thom Brown t...@linux.com wrote: On 1 January 2015 at 10:34, Amit Kapila amit.kapil...@gmail.com wrote: Running it again, I get the same issue. This is with parallel_seqscan_degree set to 8, and the crash occurs with 4 and 2 too. This doesn't happen if I

Re: [HACKERS] What exactly is our CRC algorithm?

2015-01-02 Thread Heikki Linnakangas
On 01/01/2015 09:17 AM, Abhijit Menon-Sen wrote: Hi. OK, here are the patches with the various suggestions applied. I found that the alignment didn't seem to make much difference for the CRC32* instructions, so I changed to process (len/8)*8bytes followed by (len%8)*1bytes, the way the Linux

Re: [HACKERS] parallel mode and parallel contexts

2015-01-02 Thread Amit Kapila
On Thu, Dec 18, 2014 at 1:23 AM, Robert Haas robertmh...@gmail.com wrote: In the meantime, I had a good chat with Heikki on IM yesterday which gave me some new ideas on how to fix up the transaction handling in here, which I am working on implementing. So hopefully I will have that by then.

Re: [HACKERS] BUG: *FF WALs under 9.2 (WAS: .ready files appearing on slaves)

2015-01-02 Thread Dennis Kögel
Am 31.12.2014 um 11:40 schrieb Michael Paquier michael.paqu...@gmail.com: As long as master is fixed, I don't actually care. But I agree with Dennis that it's hard to see what's been commited with all the different issues found, and if any commits were done, in which branch. I'd like to be able

Re: [HACKERS] TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

2015-01-02 Thread Amit Kapila
On Fri, Jan 2, 2015 at 11:47 AM, Dilip kumar dilip.ku...@huawei.com wrote: On 31 December 2014 18:36, Amit Kapila Wrote, The patch looks good to me. I have done couple of cosmetic changes (spelling mistakes, improve some comments, etc.), check the same once and if you are okay, we can

Re: [HACKERS] Compression of full-page-writes

2015-01-02 Thread Andres Freund
On 2014-12-31 16:09:31 -0500, Bruce Momjian wrote: I still don't understand the value of adding WAL compression, given the high CPU usage and minimal performance improvement. The only big advantage is WAL storage, but again, why not just compress the WAL file when archiving. before: pg_xlog

Re: [HACKERS] compress method for spgist - 2

2015-01-02 Thread Heikki Linnakangas
On 12/23/2014 03:02 PM, Teodor Sigaev wrote: I think we'll need a separate SpGistTypeDesc for the input type. Or perhaps a separate SpGistTypeDesc for the reconstructed value and an optional decompress method to turn the reconstructedValue back into an actual reconstructed input datum. Or

Re: [HACKERS] Compression of full-page-writes

2015-01-02 Thread Bruce Momjian
On Fri, Jan 2, 2015 at 10:15:57AM -0600, k...@rice.edu wrote: On Fri, Jan 02, 2015 at 01:01:06PM +0100, Andres Freund wrote: On 2014-12-31 16:09:31 -0500, Bruce Momjian wrote: I still don't understand the value of adding WAL compression, given the high CPU usage and minimal performance

Re: [HACKERS] Compression of full-page-writes

2015-01-02 Thread Andres Freund
On 2015-01-02 12:06:33 -0500, Bruce Momjian wrote: On Fri, Jan 2, 2015 at 05:55:52PM +0100, Andres Freund wrote: On 2015-01-02 11:52:42 -0500, Bruce Momjian wrote: Why are we not seeing the 33% compression and 15% performance improvement he saw? What am I missing here? To see

Re: [HACKERS] TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

2015-01-02 Thread Kevin Grittner
Amit Kapila amit.kapil...@gmail.com wrote: Notes for Committer - There is one behavioural difference in the handling of --analyze-in-stages switch, when individual tables (by using -t option) are analyzed by using this switch, patch will process (in case of concurrent jobs) all the given

Re: [HACKERS] Compression of full-page-writes

2015-01-02 Thread Claudio Freire
On Fri, Jan 2, 2015 at 2:11 PM, Andres Freund and...@2ndquadrant.com wrote: , I now see the compression patch as something that has negatives, so has to be set by the user, and only wins in certain cases. I am disappointed, and am trying to figure out how this became such a marginal win for

Re: [HACKERS] Compression of full-page-writes

2015-01-02 Thread Bruce Momjian
On Fri, Jan 2, 2015 at 05:55:52PM +0100, Andres Freund wrote: On 2015-01-02 11:52:42 -0500, Bruce Momjian wrote: Why are we not seeing the 33% compression and 15% performance improvement he saw? What am I missing here? To see performance improvements something needs to be the bottleneck.

Re: [HACKERS] Publish autovacuum informations

2015-01-02 Thread Jim Nasby
On 1/1/15, 4:17 PM, Noah Misch wrote: I'd be all right with putting the data structure declarations in a file named something like autovacuum_private.h, especially if it carried an annotation that if you depend on this, don't be surprised if we break your code in future. Such an annotation

Re: [HACKERS] Compression of full-page-writes

2015-01-02 Thread k...@rice.edu
On Fri, Jan 02, 2015 at 01:01:06PM +0100, Andres Freund wrote: On 2014-12-31 16:09:31 -0500, Bruce Momjian wrote: I still don't understand the value of adding WAL compression, given the high CPU usage and minimal performance improvement. The only big advantage is WAL storage, but again,

Re: [HACKERS] Compression of full-page-writes

2015-01-02 Thread Bruce Momjian
On Fri, Jan 2, 2015 at 06:11:29PM +0100, Andres Freund wrote: My negativity is not that I don't want it, but I want to understand why it isn't better than I remembered. You are basically telling me it was always a marginal win. :-( Boohoo! No, I didn't. I told you that *IN ONE

Re: [HACKERS] Compression of full-page-writes

2015-01-02 Thread Andres Freund
On 2015-01-02 11:52:42 -0500, Bruce Momjian wrote: Why are we not seeing the 33% compression and 15% performance improvement he saw? What am I missing here? To see performance improvements something needs to be the bottleneck. If WAL writes/flushes aren't that in the tested scenario, you won't

Re: [HACKERS] Transactions involving multiple postgres foreign servers

2015-01-02 Thread Tom Lane
Ashutosh Bapat ashutosh.ba...@enterprisedb.com writes: While looking at the patch for supporting inheritance on foreign tables, I noticed that if a transaction makes changes to more than two foreign servers the current implementation in postgres_fdw doesn't make sure that either all of them

Re: [HACKERS] Using 128-bit integers for sum, avg and statistics aggregates

2015-01-02 Thread Tom Lane
Heikki Linnakangas hlinnakan...@vmware.com writes: On 01/02/2015 11:41 PM, Tom Lane wrote: What might be worth trying is establishing a hard-and-fast boundary between C land and SQL land, with bitwise names in C and bytewise names in SQL. This would mean, for example, that int4pl() would be

Re: [HACKERS] Using 128-bit integers for sum, avg and statistics aggregates

2015-01-02 Thread Tom Lane
Jim Nasby jim.na...@bluetreble.com writes: On 12/31/14, 8:13 AM, Andres Freund wrote: Note that the C datatype has been int32/int64 for a while now, it's just the SQL datatype and the names of its support functions. Given that, afaiu, we're talking about the C datatype it seems pretty clear

Re: [HACKERS] Misaligned BufferDescriptors causing major performance problems on AMD

2015-01-02 Thread Andres Freund
On 2014-12-29 16:59:05 -0500, Bruce Momjian wrote: diff --git a/src/backend/storage/buffer/buf_init.c b/src/backend/storage/buffer/buf_init.c new file mode 100644 index ff6c713..c4dce5b *** a/src/backend/storage/buffer/buf_init.c --- b/src/backend/storage/buffer/buf_init.c ***

Re: [HACKERS] Using 128-bit integers for sum, avg and statistics aggregates

2015-01-02 Thread Jim Nasby
On 12/31/14, 8:13 AM, Andres Freund wrote: On 2015-01-01 03:00:50 +1300, David Rowley wrote: 2. References to int16 meaning 16 bytes. I'm really in two minds about this, it's quite nice to keep the natural flow, int4, int8, int16, but I can't help think that this will confuse someone one day.

Re: [HACKERS] Compression of full-page-writes

2015-01-02 Thread Bruce Momjian
On Fri, Jan 2, 2015 at 02:18:12PM -0300, Claudio Freire wrote: On Fri, Jan 2, 2015 at 2:11 PM, Andres Freund and...@2ndquadrant.com wrote: , I now see the compression patch as something that has negatives, so has to be set by the user, and only wins in certain cases. I am disappointed,

Re: [HACKERS] Final Patch for GROUPING SETS

2015-01-02 Thread Jim Nasby
On 12/31/14, 3:05 PM, Noah Misch wrote: On Wed, Dec 31, 2014 at 05:33:43PM +, Andrew Gierth wrote: Noah == Noah Mischn...@leadboat.com writes: Noah Suppose one node orchestrated all sorting and aggregation. Well, that has the downside of making it into an opaque blob, without

Re: [HACKERS] Using 128-bit integers for sum, avg and statistics aggregates

2015-01-02 Thread Heikki Linnakangas
On 01/02/2015 11:41 PM, Tom Lane wrote: What might be worth trying is establishing a hard-and-fast boundary between C land and SQL land, with bitwise names in C and bytewise names in SQL. This would mean, for example, that int4pl() would be renamed to int32pl() so far as the C function goes,

Re: [HACKERS] Compression of full-page-writes

2015-01-02 Thread Stephen Frost
* Bruce Momjian (br...@momjian.us) wrote: To be specific, desirable in streaming replication scenarios that don't use SSL compression. (What percentage is that?) It is something we should mention in the docs for this feature? Considering how painful the SSL rengeotiation problems were and

[HACKERS] Re: Problems with approach #2 to value locking (INSERT ... ON CONFLICT UPDATE/IGNORE patch)

2015-01-02 Thread Peter Geoghegan
On Thu, Jan 1, 2015 at 11:17 PM, Peter Geoghegan p...@heroku.com wrote: I've been working on fixing the bugs that Jeff Janes found [1] with approach #2 to value locking [2]. Approach #1 was unaffected. Unfortunately, exclusion constraints (which only value locking approach #2 has support for,

Re: [HACKERS] Detecting backend failures via libpq / DBD::Pg

2015-01-02 Thread Craig Ringer
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 12/30/2014 08:43 AM, Greg Sabino Mullane wrote: I am working on enhancing the ping() method of DBD::Pg. The goal of that is for a user to be able to determine if the connection to the database is still valid. This entire concept is flawed. IMO

Re: [HACKERS] Using 128-bit integers for sum, avg and statistics aggregates

2015-01-02 Thread Claudio Freire
On Fri, Jan 2, 2015 at 7:57 PM, Jim Nasby jim.na...@bluetreble.com wrote: On 1/2/15, 4:18 PM, Tom Lane wrote: Heikki Linnakangashlinnakan...@vmware.com writes: On 01/02/2015 11:41 PM, Tom Lane wrote: What might be worth trying is establishing a hard-and-fast boundary between C land and

Re: [HACKERS] Using 128-bit integers for sum, avg and statistics aggregates

2015-01-02 Thread Jim Nasby
On 1/2/15, 4:18 PM, Tom Lane wrote: Heikki Linnakangashlinnakan...@vmware.com writes: On 01/02/2015 11:41 PM, Tom Lane wrote: What might be worth trying is establishing a hard-and-fast boundary between C land and SQL land, with bitwise names in C and bytewise names in SQL. This would mean,

Re: [HACKERS] History of WAL_LEVEL (archive vs hot_standby)

2015-01-02 Thread Noah Misch
On Thu, Mar 27, 2014 at 08:16:13PM -0700, David Johnston wrote: Slightly tangential but are the locking operations associated with the recent bugfix generated in both (all?) modes or only hot_standby? All modes. I thought it strange that transient locking operations were output with WAL

Re: [HACKERS] TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

2015-01-02 Thread Amit Kapila
On Fri, Jan 2, 2015 at 8:34 PM, Kevin Grittner kgri...@ymail.com wrote: Amit Kapila amit.kapil...@gmail.com wrote: Notes for Committer - There is one behavioural difference in the handling of --analyze-in-stages switch, when individual tables (by using -t option) are analyzed by using

Re: [HACKERS] replicating DROP commands across servers

2015-01-02 Thread Jeff Janes
On Mon, Dec 29, 2014 at 2:15 PM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Here's a patch that tweaks the grammar to use TypeName in COMMENT, SECURITY LABEL, and DROP for the type and domain cases. The required changes in the code are pretty minimal, thankfully. Note the slight changes

Re: [HACKERS] Redesigning checkpoint_segments

2015-01-02 Thread Josh Berkus
On 01/02/2015 01:57 AM, Heikki Linnakangas wrote: wal_keep_segments does not affect the calculation of CheckPointSegments. If you set wal_keep_segments high enough, checkpoint_wal_size will be exceeded. The other alternative would be to force a checkpoint earlier, i.e. lower