Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

On Sat, Nov 12, 2016 at 10:12 PM, Pavan Deolasee wrote: > > > On Tue, Nov 8, 2016 at 9:13 AM, Haribabu Kommi > wrote: > >> >> Thanks for the patch. This shows a very good performance improvement. >> >> > Thank you. Can you please share the

Re: [HACKERS] Fix checkpoint skip logic on idle systems by tracking LSN progress

Hello, At Mon, 14 Nov 2016 16:53:35 +0530, Amit Kapila wrote in > On Mon, Nov 14, 2016 at 9:33 AM, Michael Paquier > >>> Is it right to set XLOG_NO_PROGRESS flag in LogAccessExclusiveLocks? > >>> This

Re: [HACKERS] [bug fix] Cascading standby cannot catch up and get stuck emitting the same message repeatedly

From: pgsql-hackers-ow...@postgresql.org > [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Robert Haas > Let me try to be more clear. I will not commit this patch if it is not > properly commented. I doubt that anyone else will, either. > > The fact that those code changes already

Re: [HACKERS] Adding the optional clause 'AS' in CREATE TRIGGER

"Okano, Naoki" writes: > I would like to add the following support for a trigger. > This implementation enables to create a trigger efficiently > in single command. > It had been discussed before. The link is as shown below. >

Re: [HACKERS] [bug fix] Cascading standby cannot catch up and get stuck emitting the same message repeatedly

From: pgsql-hackers-ow...@postgresql.org > [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Amit Kapila > It looks like the code in 9.3 or later version uses the recptr as the target > segment location > (targetSegmentPtr) whereas 9.2 uses recptr as beginning of segment (readOff > = 0;).

Re: [HACKERS] Pinning a buffer in TupleTableSlot is unnecessary

On Mon, Nov 14, 2016 at 10:21:53AM -0800, Peter Geoghegan wrote: > BTW, I recently noticed that the latest version of Valgrind, 3.12, > added this new feature: > > * Memcheck: > > - Added meta mempool support for describing a custom allocator which: > - Auto-frees all chunks assuming that

Re: [HACKERS] Remove the comment on the countereffectiveness of large shared_buffers on Windows

From: pgsql-hackers-ow...@postgresql.org > [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Amit Kapila > Okay, not a problem. However, I am not sure the results in this thread > are sufficient proof as for read-only tests, there is no noticeable win > by increasing shared buffers and

[HACKERS] Adding the optional clause 'AS' in CREATE TRIGGER

Hi, I would like to add the following support for a trigger. This implementation enables to create a trigger efficiently in single command. It had been discussed before. The link is as shown below.

Re: [HACKERS] Patch: Implement failover on libpq connect level.

From: pgsql-hackers-ow...@postgresql.org > [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Mithun Cy > Thanks, my concern is suppose you have 3 server in cluster A(new version), > B(new version), C(old version). If we implement as above only new servers > will send ParameterStatus message

Re: [HACKERS] 9.6 TAP tests and extensions

On 14 November 2016 at 14:55, Craig Ringer wrote: > On 27 October 2016 at 00:42, Robert Haas wrote: >> On Wed, Oct 26, 2016 at 7:17 AM, Andres Freund wrote: >>> On 2016-09-23 16:04:32 -0400, Tom Lane wrote: Looking back over

Re: [HACKERS] PATCH: two slab-like memory allocators

Attached is v6 of the patch series, fixing most of the points: * common bits (valgrind/randomization/wipe) moved to memdebug.h/c Instead of introducing a new header file, I've added the prototypes to memdebug.h (which was already used for the valgrind stuff anyway), and the implementations to

[HACKERS] DECLARE STATEMENT setting up a connection in ECPG

Hi, Making developing ECPG application more efficiently and improve portability, I'd like to suggest DECLARE STATEMENT support in ECPG. This DECLARE STATEMENT is one of the statement that lets users declare an identifier pointing a connection. This identifier will be used in other embedded

Re: [HACKERS] Do we need use more meaningful variables to replace 0 in catalog head files?

Andrew Dunstan writes: > On 11/13/2016 11:11 AM, Tom Lane wrote: >> 1. Are we going to try to keep these things in the .h files, or split >> them out? I'd like to get them out, as that eliminates both the need >> to keep the things looking like macro calls, and the need for

Re: [HACKERS] [PATCH] Allow TAP tests to be run individually

On Tue, Nov 15, 2016 at 12:02 AM, Peter Eisentraut wrote: > On 11/14/16 3:52 AM, Michael Paquier wrote: >> I don't mind. This patch uses the following pattern: >> $(or $(PROVE_TESTS),t/*.pl) >> While something more spread in Postgres source would be something

Re: [HACKERS] Something is broken about connection startup

I wrote: > Robert Haas writes: >> I assume you're going to back-patch this, and the consequences of >> failing to reset it before going idle could easily be vastly worse >> than the problem you're trying to fix. So I'd rather not make >> assumptions like "the client will

Re: [HACKERS] Physical append-only tables

On Sun, Nov 13, 2016 at 3:45 PM, Magnus Hagander wrote: > For a scenario like this, would it make sense to have an option that could > be set on an individual table making it physical append only? Basically > VACUUM would run as normal and clean up the old space when rows are

Re: [HACKERS] Re: [COMMITTERS] pgsql: Change the way that LWLocks for extensions are allocated.

Andres Freund wrote: > On 2016-11-12 11:30:42 -0500, Tom Lane wrote: > > Andres Freund writes: > > > Committed after simplifying the Makefile. > > > > While I have no particular objection to adding these tests, the > > commit log's claim that this will improve buildfarm

Re: [HACKERS] Re: [COMMITTERS] pgsql: Change the way that LWLocks for extensions are allocated.

On 2016-11-14 12:14:10 -0800, Andres Freund wrote: > On 2016-11-12 11:42:12 -0500, Tom Lane wrote: > > Andres Freund writes: > > > On 2016-11-12 11:30:42 -0500, Tom Lane wrote: > > >> which is a rather blatant waste of cycles. I would suggest an explicit > > >> do-nothing

Re: [HACKERS] Re: [COMMITTERS] pgsql: Change the way that LWLocks for extensions are allocated.

On 2016-11-12 11:42:12 -0500, Tom Lane wrote: > Andres Freund writes: > > On 2016-11-12 11:30:42 -0500, Tom Lane wrote: > >> which is a rather blatant waste of cycles. I would suggest an explicit > >> do-nothing installcheck rule rather than the hack you came up with here. >

Re: [HACKERS] Do we need use more meaningful variables to replace 0 in catalog head files?

On 11/13/2016 11:11 AM, Tom Lane wrote: Andrew Dunstan writes: I'm not convinced the line prefix part is necessary, though. What I'm thinking of is something like this: PROCDATA( oid=1242 name=boolin isstrict=t volatile=i parallel=s nargs=1 rettype=bool

Re: [HACKERS] Pinning a buffer in TupleTableSlot is unnecessary

On Mon, Nov 14, 2016 at 10:17 AM, Andres Freund wrote: > I think so, yes. IIRC I discussed it with Noah and Peter G. at a > conference recently. We'd basically mark the content of shared buffers > inaccessible at backend startup, and mark it accessible whenever a > PinBuffer()

Re: [HACKERS] Pinning a buffer in TupleTableSlot is unnecessary

On 2016-11-14 13:12:28 -0500, Tom Lane wrote: > Andres Freund writes: > > On 2016-11-14 12:32:53 -0500, Tom Lane wrote: > >> Basically my concern is that this restriction isn't documented anywhere > >> and I'm not entirely certain it's been adhered to everywhere. I'd feel >

Re: [HACKERS] Pinning a buffer in TupleTableSlot is unnecessary

Andres Freund writes: > On 2016-11-14 12:32:53 -0500, Tom Lane wrote: >> Basically my concern is that this restriction isn't documented anywhere >> and I'm not entirely certain it's been adhered to everywhere. I'd feel >> much better about it if there were some way we could

Re: [HACKERS] Pinning a buffer in TupleTableSlot is unnecessary

On 2016-11-14 12:32:53 -0500, Tom Lane wrote: > Heikki Linnakangas writes: > > On 11/14/2016 06:18 PM, Tom Lane wrote: > >> You're implicitly assuming that a scan always returns its results in the > >> same slot, and that no other slot could contain a copy of that data, but > >>

Re: [HACKERS] Pinning a buffer in TupleTableSlot is unnecessary

On Mon, Nov 14, 2016 at 9:22 AM, Heikki Linnakangas wrote: > I think that difference in the API is exactly what caught Peter by surprise > and led to bug #14344. And I didn't see it either, until you two debugged > it. That is accurate, of course. -- Peter Geoghegan --

Re: [HACKERS] Pinning a buffer in TupleTableSlot is unnecessary

Heikki Linnakangas writes: > On 11/14/2016 06:18 PM, Tom Lane wrote: >> You're implicitly assuming that a scan always returns its results in the >> same slot, and that no other slot could contain a copy of that data, but >> there is no guarantee of either. See bug #14344 and

Re: [HACKERS] Pinning a buffer in TupleTableSlot is unnecessary

On 11/14/2016 06:18 PM, Tom Lane wrote: Robert Haas writes: On Mon, Nov 14, 2016 at 10:23 AM, Tom Lane wrote: I don't believe Andres' claim anyway. There are certainly cases where an allegedly-valid slot could be pointing at garbage, but table

Re: [HACKERS] Pinning a buffer in TupleTableSlot is unnecessary

On Mon, Nov 14, 2016 at 11:18 AM, Tom Lane wrote: > Robert Haas writes: >> On Mon, Nov 14, 2016 at 10:23 AM, Tom Lane wrote: >>> I don't believe Andres' claim anyway. There are certainly cases where an >>> allegedly-valid slot

Re: [HACKERS] Pinning a buffer in TupleTableSlot is unnecessary

Robert Haas writes: > On Mon, Nov 14, 2016 at 10:23 AM, Tom Lane wrote: >> I don't believe Andres' claim anyway. There are certainly cases where an >> allegedly-valid slot could be pointing at garbage, but table scans aren't >> one of them, precisely

Re: [HACKERS] Proposal: scan key push down to heap [WIP]

On Mon, Nov 14, 2016 at 9:30 PM, Robert Haas wrote: > Couldn't we just change the current memory context before calling > heap_getnext()? And then change back? Right, seems like it will not have any problem.. > > Also, what if we abandoned the idea of pushing qual

Re: [HACKERS] Proposal: scan key push down to heap [WIP]

On Sun, Nov 13, 2016 at 12:16 AM, Dilip Kumar wrote: > Problem1: As Andres has mentioned, HeapKeyTest uses heap_getattr, > whereas ExecQual use slot_getattr().So we can have worst case > performance problem when very less tuple are getting filter out and we > have table

Re: [HACKERS] Proposal for changes to recovery.conf API

On Sat, Nov 12, 2016 at 11:09 AM, Andres Freund wrote: > I'm very tempted to rename this during the move to GUCs ... > Slightly less so, but still tempted to also rename these. They're not > actually necessarily pointing towards a primary, and namespace-wise > they're not

Re: [HACKERS] Proposal for changes to recovery.conf API

On Fri, Nov 4, 2016 at 10:17 AM, Abhijit Menon-Sen wrote: > At 2016-11-04 10:35:05 +, si...@2ndquadrant.com wrote: >> Since the "lots of parameters" approach is almost exactly what we have >> now, I think we should do that first, get this patch committed and >> then sit

Re: [HACKERS] Pinning a buffer in TupleTableSlot is unnecessary

On Mon, Nov 14, 2016 at 10:23 AM, Tom Lane wrote: > Robert Haas writes: >> On Sat, Nov 12, 2016 at 10:28 AM, Andres Freund wrote: >>> On 2016-08-30 07:38:10 -0400, Tom Lane wrote: I think this is probably wrong, or at least

Re: [HACKERS] Pinning a buffer in TupleTableSlot is unnecessary

On 2016-11-14 10:09:02 -0500, Robert Haas wrote: > On Sat, Nov 12, 2016 at 10:28 AM, Andres Freund wrote: > > On 2016-08-30 07:38:10 -0400, Tom Lane wrote: > >> Heikki Linnakangas writes: > >> > While profiling some queries and looking at executor overhead, I

Re: [HACKERS] Something is broken about connection startup

Robert Haas writes: > I assume you're going to back-patch this, and the consequences of > failing to reset it before going idle could easily be vastly worse > than the problem you're trying to fix. So I'd rather not make > assumptions like "the client will probably never

Re: [HACKERS] [bug fix] Cascading standby cannot catch up and get stuck emitting the same message repeatedly

On Thu, Nov 10, 2016 at 12:13 AM, Tsunakawa, Takayuki wrote: > From: pgsql-hackers-ow...@postgresql.org >> [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Robert Haas >> OK. I agree that's a problem. However, your patch adds zero new comment >> text

Re: [HACKERS] Something is broken about connection startup

On Mon, Nov 14, 2016 at 10:17 AM, Tom Lane wrote: >> I think the really important thing is that we don't leave xmin set >> when the backend is idle. > > Agreed. I did some stats-gathering on this over the weekend, seeing how > often the various situations occur during the

Re: [HACKERS] Pinning a buffer in TupleTableSlot is unnecessary

Robert Haas writes: > On Sat, Nov 12, 2016 at 10:28 AM, Andres Freund wrote: >> On 2016-08-30 07:38:10 -0400, Tom Lane wrote: >>> I think this is probably wrong, or at least very dangerous to remove. >>> The reason for the feature is that the slot may

Re: [HACKERS] Something is broken about connection startup

Robert Haas writes: > On Fri, Nov 11, 2016 at 3:15 PM, Tom Lane wrote: >> The basic problem here, therefore, is that SnapshotResetXmin isn't aware >> that GetCatalogSnapshot is keeping a possibly-unregistered snapshot in >> its hip pocket. That has to

Re: [HACKERS] Pinning a buffer in TupleTableSlot is unnecessary

On Sat, Nov 12, 2016 at 10:28 AM, Andres Freund wrote: > On 2016-08-30 07:38:10 -0400, Tom Lane wrote: >> Heikki Linnakangas writes: >> > While profiling some queries and looking at executor overhead, I >> > realized that we're not making much use of

Re: [HACKERS] Something is broken about connection startup

On Fri, Nov 11, 2016 at 3:15 PM, Tom Lane wrote: > So this has pretty much been broken since we put in MVCC snapshots for > catalog searches. The problem would be masked when inside a transaction > that has already got a transaction snapshot, but whenever we don't have > one

Re: [HACKERS] Logical Replication WIP

On 13/11/16 10:21, Andres Freund wrote: > On 2016-11-13 00:40:12 -0500, Peter Eisentraut wrote: >> On 11/12/16 2:18 PM, Andres Freund wrote: > I also wonder if we want an easier to >>> extend form of pubinsert/update/delete (say to add pubddl, pubtruncate, >>> pub ... without changing

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

On Mon, Nov 14, 2016 at 9:57 AM, Tom Lane wrote: > Robert Haas writes: >> On Fri, Nov 4, 2016 at 6:52 AM, Ashutosh Bapat >> wrote: >>> Costing PartitionJoinPath needs more thought so that we don't end up >>> with bad

Re: [HACKERS] [PATCH] Allow TAP tests to be run individually

On 11/14/16 3:52 AM, Michael Paquier wrote: > I don't mind. This patch uses the following pattern: > $(or $(PROVE_TESTS),t/*.pl) > While something more spread in Postgres source would be something like that: > $(if $(PROVE_TESTS),$(PROVE_TESTS),t/*.pl) > It seems to me that we'd prefer that for

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

Robert Haas writes: > On Fri, Nov 4, 2016 at 6:52 AM, Ashutosh Bapat > wrote: >> Costing PartitionJoinPath needs more thought so that we don't end up >> with bad overall plans. Here's an idea. Partition-wise joins are >> better compared to

Re: [HACKERS] Minor improvement to delete.sgml

On Sun, Nov 13, 2016 at 10:55 PM, Etsuro Fujita wrote: > On 2016/10/19 2:51, Robert Haas wrote: >> >> On Fri, Oct 14, 2016 at 12:05 AM, Etsuro Fujita >> wrote: >>> >>> I think it's better to mention that an alias is needed for the target

Re: [HACKERS] Physical append-only tables

On Mon, Nov 14, 2016 at 3:39 PM, Tom Lane wrote: > Magnus Hagander writes: > > What I'm talking about is something that would be a lot simpler than > > auto-clustering. I'm not even talking about trying to detect if the row > was > > about to go into the

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

On Fri, Nov 4, 2016 at 6:52 AM, Ashutosh Bapat wrote: > Costing PartitionJoinPath needs more thought so that we don't end up > with bad overall plans. Here's an idea. Partition-wise joins are > better compared to the unpartitioned ones, because of the smaller >

Re: [HACKERS] Physical append-only tables

Magnus Hagander writes: > What I'm talking about is something that would be a lot simpler than > auto-clustering. I'm not even talking about trying to detect if the row was > about to go into the right place -- this might be expensive and certainly > more complicated. I'm

Re: [HACKERS] Physical append-only tables

On Mon, Nov 14, 2016 at 2:35 AM, Alvaro Herrera wrote: > Magnus Hagander wrote: > > > But then consider the same table. Except rows are typically updated once > or > > twice when they are new, and *then* go read only. And we also have a > > process that at some point

Re: [HACKERS] Transaction user id through logical decoding

Hi Craig, Thanks for your answer, I'll need to dig deep in the solutions you suggested although I was looking for something less convoluted. Valerio -- View this message in context: http://postgresql.nabble.com/Transaction-user-id-through-logical-decoding-tp5923261p5930219.html Sent from the

Re: [HACKERS] Danger of automatic connection reset in psql

On Fri, Nov 11, 2016 at 5:37 AM, Pavel Stehule wrote: > > 2016-11-11 5:14 GMT+01:00 Ashutosh Bapat > : > >> > >> > How about, instead of all this, adding an option to psql to suppress >> > the automatic reconnect behavior? When enabled,

Re: [HACKERS] Fix checkpoint skip logic on idle systems by tracking LSN progress

On Mon, Nov 14, 2016 at 9:33 AM, Michael Paquier wrote: > On Mon, Nov 14, 2016 at 12:49 PM, Kyotaro HORIGUCHI > wrote: >> At Sat, 12 Nov 2016 10:28:56 +0530, Amit Kapila >> wrote in

Re: [HACKERS] Patch: Implement failover on libpq connect level.

On Mon, Nov 14, 2016 at 1:37 PM, Tsunakawa, Takayuki < tsunakawa.ta...@jp.fujitsu.com> wrote: > No, there's no concern about compatibility. Please look at this: > https://www.postgresql.org/docs/devel/static/protocol- flow.html#PROTOCOL-ASYNC Thanks, my concern is suppose you have 3 server in

Re: [HACKERS] Floating point comparison inconsistencies of the geometric types

> What way to deal with it is in your mind? The problem hides > behind operators. To fix it a user should rewrite a expression > using more primitive operators. For example, (line_a # point_a) > should be rewritten as ((point_a <-> lineseg_a) < EPSILON), or in > more primitive way. I regared this

Re: [HACKERS] Remove the comment on the countereffectiveness of large shared_buffers on Windows

On Sun, Nov 13, 2016 at 1:23 AM, Jeff Janes wrote: > On Mon, Nov 7, 2016 at 7:03 PM, Amit Kapila wrote: >> >> On Tue, Nov 8, 2016 at 5:12 AM, Jeff Janes wrote: >> > On Mon, Nov 7, 2016 at 5:55 AM, Amit Kapila

Re: [HACKERS] Gather Merge

On Sat, Nov 12, 2016 at 1:56 AM, Rushabh Lathia wrote: > On Fri, Nov 4, 2016 at 8:30 AM, Thomas Munro > wrote: >> + * Portions Copyright (c) 1996-2015, PostgreSQL Global Development Group >> + * Portions Copyright (c) 1994, Regents of the

Re: [HACKERS] Remove the comment on the countereffectiveness of large shared_buffers on Windows

On Sat, Nov 12, 2016 at 11:00 PM, Magnus Hagander wrote: > On Sat, Nov 12, 2016 at 4:03 AM, Amit Kapila > wrote: >> >> On Fri, Nov 11, 2016 at 3:01 PM, Magnus Hagander >> wrote: >> >> > Based on this optimization we might want

Re: [HACKERS] [RFC] Should we fix postmaster to avoid slow shutdown?

On Mon, Nov 14, 2016 at 5:16 AM, Tsunakawa, Takayuki wrote: > From: pgsql-hackers-ow...@postgresql.org >> [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Ashutosh Bapat >> I have changed some comments around this block. See attached patch. >> Let me know

Re: [HACKERS] [PATCH] Allow TAP tests to be run individually

On 14 November 2016 at 16:52, Michael Paquier wrote: > On Mon, Nov 14, 2016 at 3:45 PM, Craig Ringer wrote: >> On 11 November 2016 at 18:13, Michael Paquier >> wrote: >>> On Fri, Nov 11, 2016 at 6:10 PM, Craig Ringer

Re: [HACKERS] Fix checkpoint skip logic on idle systems by tracking LSN progress

On Mon, Nov 14, 2016 at 6:10 PM, Kyotaro HORIGUCHI wrote: > It applies the master and compiled cleanly and no error by > regtest. (I didn't confirmed that the problem is still fixed but > seemingly no problem) Thanks for double-checking. > If I'm not missing

Re: [HACKERS] Fix checkpoint skip logic on idle systems by tracking LSN progress

Hello, It applies the master and compiled cleanly and no error by regtest. (I didn't confirmed that the problem is still fixed but seemingly no problem) At Mon, 14 Nov 2016 15:09:09 +0900, Michael Paquier wrote in

Re: [HACKERS] pg_dump, pg_dumpall and data durability

On Mon, Nov 14, 2016 at 6:04 PM, Albe Laurenz wrote: > Michael Paquier wrote: >> Meh. I forgot docs and --help output updates. > > Looks good, except that you left the "N" option in the getopt_long > call for pg_dumpall. I fixed that in the attached patch. No, v5 has

Re: [HACKERS] pg_dump, pg_dumpall and data durability

Michael Paquier wrote: > Meh. I forgot docs and --help output updates. Looks good, except that you left the "N" option in the getopt_long call for pg_dumpall. I fixed that in the attached patch. I'll mark the patch "ready for committer". Yours, Laurenz Albe pgdump-sync-v5.patch Description:

Re: [HACKERS] [PATCH] Allow TAP tests to be run individually

On Mon, Nov 14, 2016 at 3:45 PM, Craig Ringer wrote: > On 11 November 2016 at 18:13, Michael Paquier > wrote: >> On Fri, Nov 11, 2016 at 6:10 PM, Craig Ringer wrote: >>> Please backpatch to at least 9.6 since it's trivial

Re: [HACKERS] Quorum commit for multiple synchronous replication.

On Tue, Nov 8, 2016 at 10:12 PM, Michael Paquier wrote: > On Tue, Nov 8, 2016 at 12:25 AM, Masahiko Sawada > wrote: >> On Tue, Oct 25, 2016 at 10:35 PM, Michael Paquier >> wrote: >>> + if (SyncRepConfig->sync_method

Re: [HACKERS] WIP: Covering + unique indexes

Anastasia, et al, This is a review of including_columns_9.7_v5.patch. I looked through the commit fest list and this patch was interesting and I wanted to try it. I have used include columns on Microsoft SQL Server; DB2 for Linux, Unix, Windows; and DB2 for z/OS. After reading the e-mail

Re: [HACKERS] Patch: Implement failover on libpq connect level.

From: pgsql-hackers-ow...@postgresql.org > [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Mithun Cy > If you are suggesting me to change in protocol messages, I think that would > not be backward compatible to older version servers. I also think such level > of protocol changes will not

Re: [HACKERS] Floating point comparison inconsistencies of the geometric types

Hello, At Sun, 13 Nov 2016 22:41:01 +0100, Emre Hasegeli wrote in > > We can remove the fuzz factor altogether but I think we also > > should provide a means usable to do similar things. At least "is > > a