Re: [HACKERS] Redesigning checkpoint_segments

2015-01-03 Thread Heikki Linnakangas
On 01/03/2015 12:28 AM, Josh Berkus wrote: On 01/02/2015 01:57 AM, Heikki Linnakangas wrote: wal_keep_segments does not affect the calculation of CheckPointSegments. If you set wal_keep_segments high enough, checkpoint_wal_size will be exceeded. The other alternative would be to force a checkpoi

[HACKERS] Re: Problems with approach #2 to value locking (INSERT ... ON CONFLICT UPDATE/IGNORE patch)

2015-01-03 Thread Heikki Linnakangas
On 01/03/2015 02:43 AM, Peter Geoghegan wrote: On Thu, Jan 1, 2015 at 11:17 PM, Peter Geoghegan wrote: I've been working on fixing the bugs that Jeff Janes found [1] with approach #2 to value locking [2]. Approach #1 was unaffected. Unfortunately, exclusion constraints (which only value locki

[HACKERS] Re: Problems with approach #2 to value locking (INSERT ... ON CONFLICT UPDATE/IGNORE patch)

2015-01-03 Thread Peter Geoghegan
On Sat, Jan 3, 2015 at 1:29 AM, Heikki Linnakangas wrote: > Please explain in words of one syllable how the deadlock arises. Then, > please find a way to fix it. I believe that the deadlock arises because there is no choke point. Exclusion constraint insertions always insert first, then check for

Re: [HACKERS] Logical Decoding follows timelines

2015-01-03 Thread Heikki Linnakangas
On 12/17/2014 10:35 AM, Simon Riggs wrote: On 16 December 2014 at 21:17, Simon Riggs wrote: This patch is a WIP version of doing that, but only currently attempts With the patch, XLogSendLogical uses the same logic to calculate SendRqstPtr that XLogSendPhysical does. It would be good to ref

[HACKERS] Re: Problems with approach #2 to value locking (INSERT ... ON CONFLICT UPDATE/IGNORE patch)

2015-01-03 Thread Heikki Linnakangas
On 01/03/2015 12:00 PM, Peter Geoghegan wrote: Two concurrent exclusion constraints inserters can easily insert at exactly the same time, and then wait on each other's xact, and then deadlock. That can't happen with B-Tree inserts because the checking and insertion happen at the same time, when t

Re: [HACKERS] Compression of full-page-writes

2015-01-03 Thread Michael Paquier
On Sat, Jan 3, 2015 at 1:52 AM, Bruce Momjian wrote: > I suggest we at least document that this feature as mostly useful for > I/O reduction, and maybe say CPU usage and performance might be > negatively impacted. FWIW, that's mentioned in the documentation included in the patch.. -- Michael --

Re: [HACKERS] Small doc patch about pg_service.conf

2015-01-03 Thread Magnus Hagander
On Wed, Dec 31, 2014 at 3:24 PM, David Fetter wrote: > Folks, > > There was a slash missing, which I've added. Where is the default > directory on Windows, or is there one? > Applied, thanks. And - from a quick check, it looks like it's actually using "/etc" on Windows as well. Which is somewh

[HACKERS] Updating copyright notices to 2015 for PGDG

2015-01-03 Thread Michael Paquier
Hi all, Shouldn't we update the copyright notices to 2015 for PGDG like in 7e04792? I mean those things mainly: Portions Copyright (c) 1996-2014, PostgreSQL Global Development Group Regards, -- Michael -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to you

Re: [HACKERS] recovery_min_apply_delay with a negative value

2015-01-03 Thread Michael Paquier
On Tue, Dec 30, 2014 at 4:30 AM, Fabrízio de Royes Mello wrote: > > > On Sun, Dec 28, 2014 at 12:31 PM, Michael Paquier > wrote: >> >> Hi all, >> >> While reviewing another patch, I have noticed that >> recovery_min_apply_delay can have a negative value. And the funny part is >> that we actually

Re: [HACKERS] pg_basebackup -x/X doesn't play well with archive_mode & wal_keep_segments

2015-01-03 Thread Andres Freund
On 2014-12-31 16:32:19 +0100, Andres Freund wrote: > On 2014-12-05 16:18:02 +0900, Fujii Masao wrote: > > On Fri, Dec 5, 2014 at 9:28 AM, Andres Freund > > wrote: > > > So I think we just need to make pg_basebackup create to .ready > > > files. > > > > s/.ready/.done? If yes, +1. > > That unfor

Re: [HACKERS] recovery_min_apply_delay with a negative value

2015-01-03 Thread Tom Lane
Michael Paquier writes: > On Tue, Dec 30, 2014 at 4:30 AM, Fabrízio de Royes Mello > wrote: >> Shouldn't we simply leave if recovery_min_apply_delay is lower 0, and not >> only equal to 0? > Trivial patch for master and REL9_4_STABLE attached as long as I don't > forget it.. It was originally i

Re: [HACKERS] Final Patch for GROUPING SETS

2015-01-03 Thread Noah Misch
On Fri, Jan 02, 2015 at 03:55:23PM -0600, Jim Nasby wrote: > On 12/31/14, 3:05 PM, Noah Misch wrote: > >On Wed, Dec 31, 2014 at 05:33:43PM +, Andrew Gierth wrote: > >"Noah" == Noah Misch writes: > >>> > >>> Noah> Suppose one node orchestrated all sorting and aggregation. > >>> >

Re: [HACKERS] Small doc patch about pg_service.conf

2015-01-03 Thread David Fetter
On Sat, Jan 03, 2015 at 01:22:02PM +0100, Magnus Hagander wrote: > On Wed, Dec 31, 2014 at 3:24 PM, David Fetter wrote: > > > Folks, > > > > There was a slash missing, which I've added. Where is the default > > directory on Windows, or is there one? > > > > Applied, thanks. Thank you. > And -

Re: [HACKERS] Small doc patch about pg_service.conf

2015-01-03 Thread Noah Misch
On Sat, Jan 03, 2015 at 01:22:02PM +0100, Magnus Hagander wrote: > On Wed, Dec 31, 2014 at 3:24 PM, David Fetter wrote: > > There was a slash missing, which I've added. Where is the default > > directory on Windows, or is there one? > > > > Applied, thanks. The old and the new documentation are

Re: [HACKERS] Small doc patch about pg_service.conf

2015-01-03 Thread David Fetter
On Sat, Jan 03, 2015 at 02:36:50PM -0500, Noah Misch wrote: > On Sat, Jan 03, 2015 at 01:22:02PM +0100, Magnus Hagander wrote: > > On Wed, Dec 31, 2014 at 3:24 PM, David Fetter wrote: > > > There was a slash missing, which I've added. Where is the default > > > directory on Windows, or is there o

Re: [HACKERS] Small doc patch about pg_service.conf

2015-01-03 Thread Tom Lane
David Fetter writes: > On Sat, Jan 03, 2015 at 02:36:50PM -0500, Noah Misch wrote: >> The old and the new documentation are both wrong. The directory libpq >> consults is `pg_config --sysconfdir`, which is packager-selected but rarely >> matches /etc itself. parseServiceInfo() has the code in qu

[HACKERS] Re: Problems with approach #2 to value locking (INSERT ... ON CONFLICT UPDATE/IGNORE patch)

2015-01-03 Thread Peter Geoghegan
On Sat, Jan 3, 2015 at 2:41 AM, Heikki Linnakangas wrote: > A-ha, I see. And this can happen without INSERT ON CONFLICT, too? In that > case, one of the transactions is bound to error and roll back anyway, but > you get a deadlock error instead of the constraint violation error, which is > not as

Re: [HACKERS] pg_basebackup -x/X doesn't play well with archive_mode & wal_keep_segments

2015-01-03 Thread Andres Freund
On 2015-01-03 16:03:36 +0100, Andres Freund wrote: > On 2014-12-31 16:32:19 +0100, Andres Freund wrote: > > On 2014-12-05 16:18:02 +0900, Fujii Masao wrote: > > > On Fri, Dec 5, 2014 at 9:28 AM, Andres Freund > > > wrote: > > > > So I think we just need to make pg_basebackup create to .ready > >

Re: [HACKERS] Small doc patch about pg_service.conf

2015-01-03 Thread David Fetter
On Sat, Jan 03, 2015 at 03:03:45PM -0500, Tom Lane wrote: > David Fetter writes: > > On Sat, Jan 03, 2015 at 02:36:50PM -0500, Noah Misch wrote: > >> The old and the new documentation are both wrong. The directory libpq > >> consults is `pg_config --sysconfdir`, which is packager-selected but rar

Re: [HACKERS] Small doc patch about pg_service.conf

2015-01-03 Thread Magnus Hagander
On Sat, Jan 3, 2015 at 9:03 PM, Tom Lane wrote: > David Fetter writes: > > On Sat, Jan 03, 2015 at 02:36:50PM -0500, Noah Misch wrote: > >> The old and the new documentation are both wrong. The directory libpq > >> consults is `pg_config --sysconfdir`, which is packager-selected but > rarely >

Re: [HACKERS] Small doc patch about pg_service.conf

2015-01-03 Thread Andrew Dunstan
On 01/03/2015 04:41 PM, Magnus Hagander wrote: We could do worse than to say it's in the directory printed by "pg_config --sysconfdir". Yeah. But let's make sure we include "usually /etc" or something like that. But it's not usually /etc. Certainly it's not in the PGDG RPM bui

Re: [HACKERS] Small doc patch about pg_service.conf

2015-01-03 Thread Magnus Hagander
On Sat, Jan 3, 2015 at 10:52 PM, Andrew Dunstan wrote: > > On 01/03/2015 04:41 PM, Magnus Hagander wrote: > >> >> >> We could do worse than to say it's in the directory printed by >> "pg_config --sysconfdir". >> >> >> Yeah. But let's make sure we include "usually /etc" or something like >

Re: [HACKERS] Small doc patch about pg_service.conf

2015-01-03 Thread Tom Lane
Magnus Hagander writes: > On Sat, Jan 3, 2015 at 10:52 PM, Andrew Dunstan wrote: >> On 01/03/2015 04:41 PM, Magnus Hagander wrote: >>> Yeah. But let's make sure we include "usually /etc" or something like >>> that. >> But it's not usually /etc. Certainly it's not in the PGDG RPM builds >> unless

Re: [HACKERS] logical column ordering

2015-01-03 Thread Andres Freund
On 2014-12-09 14:41:46 -0300, Alvaro Herrera wrote: > So I've been updating my very old patch to allow logical and physical > column reordering. Here's a WIP first cut for examination. Do you have a updated patch that has ripened further? > The first thing where this matters is tuple descriptor

Re: [HACKERS] Small doc patch about pg_service.conf

2015-01-03 Thread Noah Misch
On Sat, Jan 03, 2015 at 02:36:50PM -0500, Noah Misch wrote: > The directory libpq consults is `pg_config --sysconfdir` I was wrong there. `pg_config --sysconfig` uses get_etc_path(), which adjusts to post-installation moves of the installation tree. parseServiceInfo() uses the build-time SYSCONF

Re: [HACKERS] parallel mode and parallel contexts

2015-01-03 Thread Andres Freund
On 2014-12-22 14:14:31 -0500, Robert Haas wrote: > On Fri, Dec 19, 2014 at 8:59 AM, Robert Haas wrote: > > And here is a new version. > > Here is another new version, with lots of bugs fixed. A couple remarks: * Shouldn't this provide more infrastructure to deal with the fact that we might get

Re: [HACKERS] Possible micro-optimization in CacheInvalidateHeapTuple

2015-01-03 Thread Andres Freund
On 2014-10-21 19:06:41 -0500, Jim Nasby wrote: > On 10/13/14, 8:28 PM, Tom Lane wrote: > >Jim Nasby writes: > >>CacheInvalidateHeapTuple currently does the following tests first; would > >>there be a performance improvement to testing the system relation case > >>first? We're almost never in boo

Re: [HACKERS] Proposal: Log inability to lock pages during vacuum

2015-01-03 Thread Andres Freund
On 2014-12-18 16:05:23 -0600, Jim Nasby wrote: > On 12/18/14, 3:02 PM, Alvaro Herrera wrote: > >Andres Freund wrote: > >>On 2014-12-18 16:41:04 -0300, Alvaro Herrera wrote: > >>>+ if (scan_all) > >>>+ appendStringInfo(&buf, _("waited for %d buffer > >>>pin

Re: [HACKERS] Small doc patch about pg_service.conf

2015-01-03 Thread Tom Lane
Noah Misch writes: > On Sat, Jan 03, 2015 at 02:36:50PM -0500, Noah Misch wrote: >> The directory libpq consults is `pg_config --sysconfdir` > I was wrong there. `pg_config --sysconfig` uses get_etc_path(), which adjusts > to post-installation moves of the installation tree. parseServiceInfo()

Re: [HACKERS] logical column ordering

2015-01-03 Thread Alvaro Herrera
Andres Freund wrote: > On 2014-12-09 14:41:46 -0300, Alvaro Herrera wrote: > > So I've been updating my very old patch to allow logical and physical > > column reordering. Here's a WIP first cut for examination. > > Do you have a updated patch that has ripened further? Not yet. Phil was kind en

Re: [HACKERS] TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

2015-01-03 Thread Andres Freund
On 2014-12-31 18:35:38 +0530, Amit Kapila wrote: > + -j class="parameter">jobs > + --jobs= class="parameter">njobs > + > + > +Number of concurrent connections to perform the operation. > +This option will enable the vacuum operation to run on asynchronous > +

[HACKERS] Re: Problems with approach #2 to value locking (INSERT ... ON CONFLICT UPDATE/IGNORE patch)

2015-01-03 Thread Peter Geoghegan
On Sat, Jan 3, 2015 at 12:42 PM, Peter Geoghegan wrote: > There are probably additional factors that make the situation more > complicated than it is for the ON CONFLICT patch, but it's clear to me > that the mutual dependency that can be involved with two ordinary > exclusion constraint inserters

Re: [HACKERS] Small doc patch about pg_service.conf

2015-01-03 Thread Noah Misch
On Sat, Jan 03, 2015 at 07:56:06PM -0500, Tom Lane wrote: > Noah Misch writes: > > On Sat, Jan 03, 2015 at 02:36:50PM -0500, Noah Misch wrote: > >> The directory libpq consults is `pg_config --sysconfdir` > > > I was wrong there. `pg_config --sysconfig` uses get_etc_path(), which > > adjusts >