Re: [HACKERS] Bug in 9.6 tuplesort batch memory growth logic

Peter Geoghegan writes: > On Tue, Sep 6, 2016 at 6:17 AM, Tom Lane wrote: >> It doesn't seem to me that this limit has anything to do with anything, >> and the comment claiming only that it's "noncritical" isn't helping. > You've not understood the problem

Re: [HACKERS] Fun fact about autovacuum and orphan temp tables

On 9/5/16 12:14 PM, Bruce Momjian wrote: > I have certainly faced my fair share of customers with dangling temp > tables, and would like to see this changed in some way or another. I don't think we look at those temp tables frequently enough to justify keeping them around for all users. Plus,

[HACKERS] Re: [COMMITTERS] pgsql: Make initdb's suggested "pg_ctl start" command line more reliabl

On 8/20/16 3:05 PM, Tom Lane wrote: > Make initdb's suggested "pg_ctl start" command line more reliable. > > The original coding here was not nearly careful enough about quoting > special characters, and it didn't get corner cases right for constructing > the pg_ctl path either. Use

Re: [HACKERS] Obsolete TODO item "-Wcast-align" ?

On 9/4/16 5:10 PM, Christian Convey wrote: Thanks for the response. I'm unclear about how the TODO list is curated. Is there someone whose attention I should direct to this thread? There's a number of people that can edit the TODO list. I've removed that item. -- Jim Nasby, Data Architect,

Re: [HACKERS] Vacuum: allow usage of more than 1GB of work mem

On 9/4/16 1:46 AM, Simon Riggs wrote: > The patch also makes vacuum free the dead_tuples before starting > truncation. It didn't seem necessary to hold onto it beyond that > point, and it might help give the OS more cache, especially if work > mem is configured very high to avoid multiple index

Re: [HACKERS] (Comment)Bug in CteScanNext

On 9/3/16 1:30 PM, Tom Lane wrote: Or we could add something like "But first, we must deal with the special case of reversing direction after reaching EOF." I'm working on that, but one thing isn't clear to me... why do we only skip past the last tuple if (!node->leader->eof_cte)? Even if

Re: [HACKERS] Forbid use of LF and CR characters in database and role names

On 8/11/16 9:12 PM, Michael Paquier wrote: > Note that pg_dump[all] and pg_upgrade already have safeguards against > those things per the same routines putting quotes for execution as > commands into psql and shell. So attached is a patch to implement this > restriction in the backend, and I am

[HACKERS] SELECT FOR UPDATE regression in 9.5

Hello list While testing an application with PostgreSQL 9.5, we experienced an issue involving aborted subtransactions and SELECT FOR UPDATE. In certain situations, a locking query doesn't return rows that should be visible and already locked by the current transaction. I bisected this down to

Re: [HACKERS] Quorum commit for multiple synchronous replication.

On Tue, Sep 6, 2016 at 11:08 PM, Simon Riggs wrote: > On 29 August 2016 at 14:52, Fujii Masao wrote: >> On Sat, Aug 6, 2016 at 6:36 PM, Petr Jelinek wrote: >>> On 04/08/16 06:40, Masahiko Sawada wrote: On Wed, Aug 3,

Re: [HACKERS] ICU integration

On 8/31/16 1:32 PM, Peter Geoghegan wrote: > ICU is happy to have multiple versions > of a collation at a time, and you'll probably retain the old collation > version in ICU. > > Even if your old collation version isn't available in a new ICU > release (which I think is unlikely in practice) I

Re: [HACKERS] Bug in 9.6 tuplesort batch memory growth logic

On Tue, Sep 6, 2016 at 6:17 AM, Tom Lane wrote: > It doesn't seem to me that this limit has anything to do with anything, > and the comment claiming only that it's "noncritical" isn't helping. > If the point is to prevent the later LACKMEM check from failing, then > certainly

Re: [HACKERS] ICU integration

On 8/31/16 4:24 PM, Doug Doole wrote: > ICU explicitly does not provide stability in their locales and collations. We > pushed them hard to provide this, but between changes to the CLDR data and > changes to the ICU code it just wasn’t feasible for them to provide version > to version

Re: [HACKERS] ICU integration

On 8/31/16 12:32 AM, Michael Paquier wrote: > On Wed, Aug 31, 2016 at 1:12 PM, Peter Eisentraut > wrote: >> On 8/30/16 11:27 PM, Craig Ringer wrote: >>> Speaking of which, have you had a chance to try it on Windows yet? >> >> nope > > +SELECT a, b FROM

Re: [HACKERS] PATCH: Exclude additional directories in pg_basebackup

On 9/1/16 9:53 AM, Peter Eisentraut wrote: On 8/15/16 3:39 PM, David Steele wrote: That patch got me thinking about what else could be excluded and after some investigation I found the following: pg_notify, pg_serial, pg_snapshots, pg_subtrans. These directories are all cleaned, zeroed, or

Re: [HACKERS] Refactoring of heapam code.

06.09.2016 07:44, Pavan Deolasee: On Mon, Aug 8, 2016 at 3:13 PM, Anastasia Lubennikova > wrote: Thank you for the review, I'll fix these problems in final version. Posting the first message I intended to raise

Re: [HACKERS] PassDownLimitBound for ForeignScan/CustomScan

Hi, Changes look good to me. However there are couple of minor issues need to be fixed. 1. "under" repeated on second line. Please remove. +if and when CustomScanState is located under +under LimitState; which implies the underlying node is not 2. Typo: dicsussion => discussion Please

Re: [HACKERS] Bug in VACUUM_TRUNCATE_LOCK_WAIT_INTERVAL

On 6 September 2016 at 11:30, Simon Riggs wrote: > In vacuumlazy.c, VACUUM_TRUNCATE_LOCK_WAIT_INTERVAL is described as > being in ms on line 85, yet it is used on line 1759 in a call to > pg_usleep, so is treated as microseconds rather than milliseconds. > > As a result,

Re: [HACKERS] Postgres abort found in 9.3.11

Hello, I was able to find a patch file where there is a call to ExitPostmaster() in postmaster.c . @@ -3081,6 +3081,11 @@ shmem_exit(1); reset_shared(PostPortNumber); + /* recovery termination */ + ereport(FATAL, + (errmsg("recovery termination due to

Re: [HACKERS] Proposal for changes to recovery.conf API

On 9/6/16 8:07 AM, Robert Haas wrote: On Wed, Aug 31, 2016 at 9:45 PM, Simon Riggs wrote: > Related cleanup * Promotion signal file is now called "promote.trigger" rather than just "promote" * Remove user configurable "trigger_file" mechanism - use "promote.trigger"

Re: [HACKERS] Quorum commit for multiple synchronous replication.

On 29 August 2016 at 14:52, Fujii Masao wrote: > On Sat, Aug 6, 2016 at 6:36 PM, Petr Jelinek wrote: >> On 04/08/16 06:40, Masahiko Sawada wrote: >>> >>> On Wed, Aug 3, 2016 at 3:05 PM, Michael Paquier >>> wrote:

Re: [HACKERS] Proposal for changes to recovery.conf API

On Tue, Sep 6, 2016 at 10:01 PM, Petr Jelinek wrote: > That's also reasonable solution, I don't really have preference between > those. My main point was to get rid of the 5 or so variables where only one > will actually be used in the end. And no need to worry about the

Re: [HACKERS] Bug in 9.6 tuplesort batch memory growth logic

Peter Geoghegan writes: > While working on my parallel CREATE INDEX patch, I came across a > problem. I took a quick look at this. I do not follow the logic in this new bit: + if (availMemLessRefund <= + (int64) state->activeTapes *

Re: [HACKERS] condition variables

On Tue, Sep 6, 2016 at 5:29 PM, Robert Haas wrote: > On Mon, Sep 5, 2016 at 3:17 PM, Amit Kapila wrote: >> On Mon, Aug 15, 2016 at 10:35 PM, Robert Haas wrote: > Don't you need to set proc->cvSleeping = false in

Re: [HACKERS] Declarative partitioning - another take

On Tue, Sep 6, 2016 at 9:19 PM, Robert Haas wrote: > On Wed, Aug 31, 2016 at 1:05 PM, Amit Langote > wrote: >>> However, it seems a lot better to make it a property of the parent >>> from a performance point of view. Suppose there are 1000

Re: [HACKERS] Push down more full joins in postgres_fdw

On Fri, Sep 2, 2016 at 3:55 PM, Etsuro Fujita wrote: > Hi Ashutosh, > > On 2016/08/22 15:49, Ashutosh Bapat wrote: > >> 1. deparsePlaceHolderVar looks odd - each of the deparse* function is >> named as deparse + > into>. PlaceHolderVar is not a parser node, so no

Re: [HACKERS] Declarative partitioning - another take

Hi, On Tue, Sep 6, 2016 at 8:15 PM, Rajkumar Raghuwanshi wrote: > Hi, > > I have applied updated patches given by you, and observe below. > > here in the given example, t6_p3 partition is not allowed to have null, but > I am able to insert it, causing two

Re: [HACKERS] Proposal for changes to recovery.conf API

On 06/09/16 13:52, David Steele wrote: On 9/6/16 4:56 AM, Petr Jelinek wrote: On 06/09/16 07:18, Abhijit Menon-Sen wrote: Do we want something like this (easy to implement and document, perhaps not especially convenient to use): recovery_target = 'xid' # or

[HACKERS] Push down more UPDATEs/DELETEs in postgres_fdw

Hi, Attached is a WIP patch extending the postgres_fdw DML pushdown in 9.6 so that it can perform an update/delete on a join remotely. An example is shown below: * without the patch: postgres=# explain verbose delete from ft1 using ft2 where ft1.a = ft2.a; QUERY PLAN

Re: [HACKERS] pgsql: Add putenv support for msvcrt from Visual Studio 2013

(Just remembered to remove pgsql-committers here). On Tue, Sep 6, 2016 at 9:21 PM, Christian Ullrich wrote: > * Michael Paquier wrote: > >> On Tue, Sep 6, 2016 at 5:36 PM, Christian Ullrich >> wrote: >>> >>> * Michael Paquier wrote: > In order to

Re: [HACKERS] Declarative partitioning - another take

This patch uses RangeBound structure. There's also a structure defined with the same name in rangetypes.h with some slight differences. Should we rename the one in partition.c as PartRangeBound or something like that to avoid the confusion? -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation

Re: [HACKERS] pgsql: Add putenv support for msvcrt from Visual Studio 2013

* Michael Paquier wrote: On Tue, Sep 6, 2016 at 5:36 PM, Christian Ullrich wrote: * Michael Paquier wrote: In order to avoid any problems with the load and unload windows, my bet goes for 0001, 0002 and 0003, with the last two patches merged together, 0001 being only

[HACKERS] Tuplesort merge pre-reading

While reviewing Peter's latest round of sorting patches, and trying to understand the new "batch allocation" mechanism, I started to wonder how useful the pre-reading in the merge stage is in the first place. I'm talking about the code that reads a bunch of from each tape, loading them into

Re: [HACKERS] Declarative partitioning - another take

On Wed, Aug 31, 2016 at 1:05 PM, Amit Langote wrote: >> However, it seems a lot better to make it a property of the parent >> from a performance point of view. Suppose there are 1000 partitions. >> Reading one toasted value for pg_class and running stringToNode()

Re: [HACKERS] Bug in VACUUM_TRUNCATE_LOCK_WAIT_INTERVAL

On Tue, Sep 6, 2016 at 4:00 PM, Simon Riggs wrote: > I'm also wondering why we don't use lock_timeout when the user sets it? > Not a bug, but patch attached anyway. > vacuum_truncate_use_lock_timeout.v1.patch This part seems fairly random. I don't think it makes sense to

Re: [HACKERS] Proposal for changes to recovery.conf API

On Wed, Aug 31, 2016 at 9:45 PM, Simon Riggs wrote: > This is a summary of proposed changes to the recovery.conf API for > v10. These are based in part on earlier discussions, and represent a > minimal modification to current usage. This looks great. > Proposed changes

Re: [HACKERS] pgsql: Add putenv support for msvcrt from Visual Studio 2013

On Tue, Sep 6, 2016 at 5:36 PM, Christian Ullrich wrote: > * Michael Paquier wrote: >> In order to avoid any problems with the load and unload windows, my >> bet goes for 0001, 0002 and 0003, with the last two patches merged >> together, 0001 being only a set of independent

Re: [HACKERS] condition variables

On Mon, Sep 5, 2016 at 3:17 PM, Amit Kapila wrote: > On Mon, Aug 15, 2016 at 10:35 PM, Robert Haas wrote: Don't you need to set proc->cvSleeping = false in ConditionVariableSignal? >>> >>> I poked at this a bit... OK, a lot... and have some

Re: [HACKERS] Proposal for changes to recovery.conf API

On 9/6/16 4:56 AM, Petr Jelinek wrote: On 06/09/16 07:18, Abhijit Menon-Sen wrote: Do we want something like this (easy to implement and document, perhaps not especially convenient to use): recovery_target = 'xid' # or 'time'/'name'/'lsn'/'immediate' recovery_target_xid = xxx? #

Re: [HACKERS] Postgres abort found in 9.3.11

"K S, Sandhya (Nokia - IN/Bangalore)" writes: > I was able to find a patch file where there is a call to ExitPostmaster() in > postmaster.c . > @@ -3081,6 +3081,11 @@ > shmem_exit(1); > reset_shared(PostPortNumber); > + /* recovery termination */

Re: [HACKERS] Declarative partitioning - another take

Hi, I have applied updated patches given by you, and observe below. here in the given example, t6_p3 partition is not allowed to have null, but I am able to insert it, causing two nulls in the table. --create a partition table create table t6 (a int, b varchar) partition by list(a); create

Re: [HACKERS] PATCH: Batch/pipelining support for libpq

On 6 September 2016 at 16:10, Daniel Verite wrote: > Craig Ringer wrote: > >> Updated patch attached. > > Please find attached a couple fixes for typos I've came across in > the doc part. Thanks, will apply and post a rebased patch soon, or if someone picks this

[HACKERS] Bug in VACUUM_TRUNCATE_LOCK_WAIT_INTERVAL

In vacuumlazy.c, VACUUM_TRUNCATE_LOCK_WAIT_INTERVAL is described as being in ms on line 85, yet it is used on line 1759 in a call to pg_usleep, so is treated as microseconds rather than milliseconds. As a result, the timeout during lazy_truncate_heap() is actually only 5ms long, not 5s long. So

Re: [HACKERS] Override compile time log levels of specific messages/modules

On 6 Sep. 2016 17:57, "Pavan Deolasee" wrote: > > > > On Tue, Sep 6, 2016 at 3:06 PM, Craig Ringer wrote: >> >> >> I think it's worth looking at how Java handles logging. We can't achieve an exact parallel in C as we don't really have a

Re: [HACKERS] LOCK TABLE .. DEFERRABLE

On 1 September 2016 at 21:28, Simon Riggs wrote: > So the only way to handle multiple locks is to do this roughly the way > Rod suggests. I've just been reading the VACUUM code and it turns out that we already use Rod's mechanism internally. So on that basis it seems fine

Re: [HACKERS] Override compile time log levels of specific messages/modules

On Tue, Sep 6, 2016 at 3:06 PM, Craig Ringer wrote: > > I think it's worth looking at how Java handles logging. We can't achieve > an exact parallel in C as we don't really have a class hierarchy ... but we > do have subsystems roughly grouped by file and directory

Re: [HACKERS] Logical Replication WIP

On 05/09/16 23:35, Steve Singer wrote: On 09/05/2016 03:58 PM, Steve Singer wrote: On 08/31/2016 04:51 PM, Petr Jelinek wrote: Hi, and one more version with bug fixes, improved code docs and couple more tests, some general cleanup and also rebased on current master for the start of CF.

Re: [HACKERS] Speedup twophase transactions

> On 06 Sep 2016, at 12:09, Simon Riggs wrote: > > On 6 September 2016 at 09:58, Stas Kelvich wrote: >> >> I'll check it against my failure scenario with subtransactions and post >> results or updated patch here. > > Make sure tests are added

Re: [HACKERS] Logical Replication WIP

On 05/09/16 21:58, Steve Singer wrote: On 08/31/2016 04:51 PM, Petr Jelinek wrote: Hi, and one more version with bug fixes, improved code docs and couple more tests, some general cleanup and also rebased on current master for the start of CF. To get the 'subscription' TAP tests to pass I

Re: [HACKERS] Override compile time log levels of specific messages/modules

On 6 Sep. 2016 17:28, "Pavan Deolasee" wrote: > > > The patch uses some preprocessing and scripting magic to assign distinct identifiers to each module (a subdir in the source code), to each file and to each elog message. It then provides a set of functions by which an

[HACKERS] Override compile time log levels of specific messages/modules

While reviewing Jeff's notice_lock_waits patch, I came across his comment about having a general facility for promoting selected LOG messages. So I thought I should post it here, even though the patch is probably far from being accepted in Postgres. I recently wrote a patch for Postgres-XL to do

Re: [HACKERS] Proposal for changes to recovery.conf API

At 2016-09-06 10:56:53 +0200, p...@2ndquadrant.com wrote: > > So +1 on the recovery_target = 'xid:xxx' idea. OK, I'll make it work that way. New patch in a couple of days. -- Abhijit -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription:

Re: [HACKERS] Speedup twophase transactions

On 6 September 2016 at 09:58, Stas Kelvich wrote: >> On 06 Sep 2016, at 04:41, Michael Paquier wrote: >> >> On Sat, Sep 3, 2016 at 10:26 PM, Michael Paquier >> wrote: >>> On Fri, Sep 2, 2016 at 5:06 AM, Simon Riggs

Re: [HACKERS] Speedup twophase transactions

On Tue, Sep 6, 2016 at 5:58 PM, Stas Kelvich wrote: >> On 06 Sep 2016, at 04:41, Michael Paquier wrote: >> This overlaps with TwoPhaseGetGXact but I'd rather keep both things >> separated: it does not seem worth complicating the existing

[HACKERS] Re: [COMMITTERS] pgsql: Add putenv support for msvcrt from Visual Studio 2013

On 6 Sep. 2016 15:12, "Michael Paquier" wrote: > > On Thu, Sep 1, 2016 at 4:03 PM, Christian Ullrich wrote: > > > That said, introducing this requirement would be a very significant change. > > I'm not sure how many independently maintained

Re: [HACKERS] Speedup twophase transactions

> On 06 Sep 2016, at 04:41, Michael Paquier wrote: > > On Sat, Sep 3, 2016 at 10:26 PM, Michael Paquier > wrote: >> On Fri, Sep 2, 2016 at 5:06 AM, Simon Riggs wrote: >>> On 13 April 2016 at 15:31, Stas Kelvich

Re: [HACKERS] Proposal for changes to recovery.conf API

On 06/09/16 07:18, Abhijit Menon-Sen wrote: One open issue is the various assign_recovery_target_xxx functions, which Michael noted in his earlier review: +static void +assign_recovery_target_xid(const char *newval, void *extra) +{ + recovery_target_xid = *((TransactionId *) extra); + +

Re: [HACKERS] pgsql: Add putenv support for msvcrt from Visual Studio 2013

* Michael Paquier wrote: On Thu, Sep 1, 2016 at 4:03 PM, Christian Ullrich wrote: My conclusion from April stands: The fact that master looks like it does means that there have not been many (or any) complaints about missing cross-module environment variables. If

Re: [HACKERS] Suggestions for first contribution?

Hello, Christian > Can anyone suggest a project for my first PG contribution? In my experience good starting points are: * Doing code review. It's really important for PostgreSQL project. Everyone likes to write code, no one likes to read or test it :) * Finding typos, there are a lot of

Re: [HACKERS] Proposal for changes to recovery.conf API

On 6 September 2016 at 08:06, Abhijit Menon-Sen wrote: > At 2016-09-06 14:40:54 +0900, michael.paqu...@gmail.com wrote: >> >> my best advice here is to make all those recovery_target_* parameters >> PGC_POSTMASTER so as they are loaded only once when the server starts, >>

Re: [HACKERS] PATCH: Batch/pipelining support for libpq

Craig Ringer wrote: > Updated patch attached. Please find attached a couple fixes for typos I've came across in the doc part. Also it appears that PQqueriesInBatch() doesn't work as documented. It says: "Returns the number of queries still in the queue for this batch" but in fact it's

Re: [HACKERS] Notice lock waits

On Fri, Aug 5, 2016 at 10:30 PM, Jeff Janes wrote: > > > A general facility for promoting selected LOG messages to NOTICE would > be nice, but I don't know how to design or implement that. This is > much easier, and I find it quite useful. > > IMHO that's what we need and

Re: [HACKERS] make async slave to wait for lsn to be replayed

On 08/31/2016 05:54 PM, Craig Ringer wrote: How do you get the commit LSN to watch for? Grab pg_current_xlog_insert_location() just after the commit and figure that replaying to that point guarantees you get the commit? That's the point, it was created in order to provide the cosistent view

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

I'm reviewing patches 1-3 in this series, i.e. those patches that are not directly related to parallelism, but are independent improvements to merging. Let's begin with patch 1: On 08/02/2016 01:18 AM, Peter Geoghegan wrote: Cap the number of tapes used by external sorts Commit df700e6b set

Re: [HACKERS] Declarative partitioning - another take

I found a server crash while running make check in regress folder. with this set of patches. Problem is RelationBuildPartitionKey() partexprsrc may be used uninitialized. Initializing it with NIL fixes the crash. Here's patch to fix it. Came up with the fix after discussion with Amit. On Fri, Aug

Re: [HACKERS] pgsql: Add putenv support for msvcrt from Visual Studio 2013

On Thu, Sep 1, 2016 at 4:03 PM, Christian Ullrich wrote: > * Michael Paquier wrote: > >> On Wed, Apr 27, 2016 at 2:39 AM, Christian Ullrich >> wrote: > >>> * Christian Ullrich wrote: > >> And actually, by looking at those patches, isn't it a dangerous

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

On 08/16/2016 03:33 AM, Peter Geoghegan wrote: I attach a patch that changes how we maintain the heap invariant during tuplesort merging. I already mentioned this over on the "Parallel tuplesort, partitioning, merging, and the future" thread. As noted already on that thread, this patch makes

Re: [HACKERS] Proposal for changes to recovery.conf API

At 2016-09-06 14:40:54 +0900, michael.paqu...@gmail.com wrote: > > my best advice here is to make all those recovery_target_* parameters > PGC_POSTMASTER so as they are loaded only once when the server starts, > and then we define the recovery target type used in the startup > process instead of

Re: [HACKERS] Speedup twophase transactions

On 6 September 2016 at 02:41, Michael Paquier wrote: > After review the result is attached. Perhaps a committer could get a look at > it? Yes, will do, but it will be a few more days yet. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL

Re: [HACKERS] Vacuum: allow usage of more than 1GB of work mem

On 5 September 2016 at 21:58, Claudio Freire wrote: How long does that part ever take? Is there any substantial gain from this? > Btw, without a further patch to prefetch pages on the backward scan > for truncate, however, my patience ran out before it finished >

<    1   2