Re: [HACKERS] WAL consistency check facility

On Sun, Nov 13, 2016 at 12:06 PM, Peter Eisentraut wrote: > Could we name this something like wal_consistency_checking? > > Otherwise it sounds like you use this to select the amount of > consistency in the WAL (similar to, say, wal_level). Or wal_check? Or

Re: [HACKERS] Logical Replication WIP

On 11/12/16 2:18 PM, Andres Freund wrote: >>> I also wonder if we want an easier to >>> > > extend form of pubinsert/update/delete (say to add pubddl, pubtruncate, >>> > > pub ... without changing the schema). >>> > > >> > >> > So like, text array that's then parsed everywhere (I am not doing

Re: [HACKERS] Do we need use more meaningful variables to replace 0 in catalog head files?

On 11/11/16 11:10 AM, Tom Lane wrote: > boolin: OID=1242 proname=boolin proargtypes="cstring" prorettype=bool > boolin: prosrc=boolin provolatile=i proparallel=s Then we're not very far away from just using CREATE FUNCTION SQL commands. -- Peter Eisentraut

Re: [HACKERS] Proposal: scan key push down to heap [WIP]

I have done performance analysis for TPCH queries, I saw visible gain in 5 queries (10-25%). Performance Data: Benchmark : TPCH (S.F. 10) shared_buffer : 20GB work_mem : 50MB Machine : POWER Results are median of three run (explain analyze results for both head/patch are attached in

Re: [HACKERS] Logical Replication WIP

On 12/11/16 20:19, Andres Freund wrote: > On 2016-11-10 23:31:27 +0100, Petr Jelinek wrote: >> On 04/11/16 13:15, Andres Freund wrote: >>> >>> /* Prototypes for private functions */ >>> -static bool libpq_select(int timeout_ms); >>> +static bool libpq_select(PGconn *streamConn, >>> +

Re: [HACKERS] WAL consistency check facility

On 11/9/16 11:55 PM, Michael Paquier wrote: > + > + wal_consistency (string) > + > + wal_consistency configuration parameter > + > + > + > + > +This parameter is used to check the consistency of WAL records, i.e, > +whether the WAL

Re: [HACKERS] pg_dump, pg_dumpall and data durability

On 11/8/16 3:48 PM, Robert Haas wrote: > First question: Do we even want this? Generally, when a program > writes to a file, we rely on the operating system to decide when that > data should be written back to disk. We have to override that > distinction for things internal to PostgreSQL because

Re: [HACKERS] pg_dump, pg_dumpall and data durability

On Sun, Nov 13, 2016 at 12:17 AM, Dilip Kumar wrote: > On Sat, Nov 12, 2016 at 10:16 AM, Michael Paquier > wrote: >>> - For pg_dumpall, a short option "-N" is added for "--no-sync", but not for >>> pg_dump (because -N is already taken there).

Re: [HACKERS] More snapshot-management fun

I wrote: > * ProcArrayInstallImportedXmin and ProcArrayInstallRestoredXmin will > forcibly overwrite MyPgXact->xmin with some random xmin or other. > While they take pains to ensure that doesn't cause the global xmin > horizon to go backwards, and while we're on the subject, I doubt that those

Re: [HACKERS] PATCH: two slab-like memory allocators

Hi, Subject: [PATCH 1/2] slab allocator diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.c index 6ad7e7d..520f295 100644 --- a/src/backend/replication/logical/reorderbuffer.c +++ b/src/backend/replication/logical/reorderbuffer.c I'd

Re: [HACKERS] Logical Replication WIP

Hi, On 2016-11-11 12:04:27 +0100, Petr Jelinek wrote: > On 04/11/16 14:00, Andres Freund wrote: > > Hi, > > > > + > > + pg_publication_rel > > + > > + > > + pg_publication_rel > > + > > + > > + > > + The pg_publication_rel catalog contains > > + mapping between tables and

Re: [HACKERS] Logical Replication WIP

On 2016-11-10 23:31:27 +0100, Petr Jelinek wrote: > On 04/11/16 13:15, Andres Freund wrote: > > > > /* Prototypes for private functions */ > > -static bool libpq_select(int timeout_ms); > > +static bool libpq_select(PGconn *streamConn, > > +int

Re: [HACKERS] proposal: psql \setfileref

2016-11-10 18:56 GMT+01:00 Tom Lane : > Pavel Stehule writes: > > 2016-11-09 22:47 GMT+01:00 Tom Lane : > >> * I really dislike the notion of keying the behavior to a special type > of > >> psql variable. > > > still I am thinking

Re: [HACKERS] Remove the comment on the countereffectiveness of large shared_buffers on Windows

On Mon, Nov 7, 2016 at 7:03 PM, Amit Kapila wrote: > On Tue, Nov 8, 2016 at 5:12 AM, Jeff Janes wrote: > > On Mon, Nov 7, 2016 at 5:55 AM, Amit Kapila > wrote: > >> > >> > >> Isn't it somewhat strange that writes are

Re: [HACKERS] xlogreader.c fails with FATAL on a cluster with 4kB block size

Meh, ignore this report - I've just realized I've been running the pg_xlogdump binary built for 8kB pages, so the failures are kinda expected. Sorry about the confusion. regards On 11/12/2016 07:52 PM, Tomas Vondra wrote: Hi, I'm running some tests on a cluster with 4kB blocks, and it seems

[HACKERS] More snapshot-management fun

While pursuing the unprotected-use-of-CatalogSnapshot problem I noted earlier, I came across a couple of related issues: * ProcArrayInstallImportedXmin and ProcArrayInstallRestoredXmin will forcibly overwrite MyPgXact->xmin with some random xmin or other. While they take pains to ensure that

Re: [HACKERS] Patch to implement pg_current_logfile() function

On Mon, 7 Nov 2016 23:29:28 +0100 Gilles Darold wrote: > Here is the v13 of the patch, > - I've reverted the patch removing the call to logfile_writename in > open_csvlogfile function, it is required to write log filename at > startup when log_destination is set to

[HACKERS] xlogreader.c fails with FATAL on a cluster with 4kB block size

Hi, I'm running some tests on a cluster with 4kB blocks, and it seems there's a bug in xlogreader.c, causing FATAL errors for example when running pg_xlogdump: pg_xlogdump: FATAL: error in WAL record at 48/63970258: BKPIMAGE_HAS_HOLE not set, but hole offset 0 length 4096 at 48/63970258

Re: [HACKERS] Remove the comment on the countereffectiveness of large shared_buffers on Windows

On Sat, Nov 12, 2016 at 4:03 AM, Amit Kapila wrote: > On Fri, Nov 11, 2016 at 3:01 PM, Magnus Hagander > wrote: > >> > Based on this optimization we might want to keep the text that says > >> > large > >> > shared buffers on Windows aren't as

Re: [HACKERS] Indirect indexes

Hi, On 2016-11-01 01:43:31 -0300, Alvaro Herrera wrote: > Alvaro Herrera wrote: > > I propose we introduce the concept of "indirect indexes". > > This is a WIP non-functional patch for indirect indexes. I've been > distracted from working on it for some time already and will be off-line > for

Re: [HACKERS] Re: [COMMITTERS] pgsql: Change the way that LWLocks for extensions are allocated.

Andres Freund writes: > On 2016-11-12 11:42:12 -0500, Tom Lane wrote: >> Hm. What about inventing an additional macro NO_INSTALLCHECK that >> prevents pgxs.mk from generating an installcheck rule? > That'd work. I'd also be ok with living with the warning. I have to say > I

Re: [HACKERS] Re: [COMMITTERS] pgsql: Change the way that LWLocks for extensions are allocated.

On 2016-11-12 11:42:12 -0500, Tom Lane wrote: > Andres Freund writes: > > On 2016-11-12 11:30:42 -0500, Tom Lane wrote: > >> which is a rather blatant waste of cycles. I would suggest an explicit > >> do-nothing installcheck rule rather than the hack you came up with here. >

Re: [HACKERS] Re: [COMMITTERS] pgsql: Change the way that LWLocks for extensions are allocated.

Andres Freund writes: > On 2016-11-12 11:30:42 -0500, Tom Lane wrote: >> which is a rather blatant waste of cycles. I would suggest an explicit >> do-nothing installcheck rule rather than the hack you came up with here. > I had that at first, but that generates a warning

Re: [HACKERS] Re: [COMMITTERS] pgsql: Change the way that LWLocks for extensions are allocated.

On 2016-11-12 11:30:42 -0500, Tom Lane wrote: > Andres Freund writes: > > Committed after simplifying the Makefile. > > While I have no particular objection to adding these tests, the > commit log's claim that this will improve buildfarm testing is > quite wrong. The

Re: [HACKERS] Re: [COMMITTERS] pgsql: Change the way that LWLocks for extensions are allocated.

Andres Freund writes: > Committed after simplifying the Makefile. While I have no particular objection to adding these tests, the commit log's claim that this will improve buildfarm testing is quite wrong. The buildfarm runs "make installcheck" not "make check" in contrib.

Re: [HACKERS] Proposal for changes to recovery.conf API

Hi, On 2016-11-01 05:14:58 +0530, Abhijit Menon-Sen wrote: > Subject: Convert recovery.conf settings to GUCs I really want to get this in, we've been back and forth about this for far too long. > >- Create a recovery command file recovery.conf in the cluster >- data directory (see

Re: [HACKERS] move collation import to backend

Hi, On 2016-10-27 21:56:53 -0400, Peter Eisentraut wrote: > Currently, initdb parses locale -a output to populate pg_collation. If > additional collations are installed in the operating system, it is not > possible to repeat this process, only by doing each step manually. So I > propose to move

Re: [HACKERS] Pinning a buffer in TupleTableSlot is unnecessary

On 2016-08-30 07:38:10 -0400, Tom Lane wrote: > Heikki Linnakangas writes: > > While profiling some queries and looking at executor overhead, I > > realized that we're not making much use of TupleTableSlot's ability to > > hold a buffer pin. In a SeqScan, the buffer is held

Re: [HACKERS] pg_dump, pg_dumpall and data durability

On Sat, Nov 12, 2016 at 10:16 AM, Michael Paquier wrote: >> - For pg_dumpall, a short option "-N" is added for "--no-sync", but not for >> pg_dump (because -N is already taken there). >> I'd opt for either using the same short option for both or (IMO better) >>

Re: [HACKERS] [PATCH] Logical decoding timeline following take II

On 2016-10-24 17:49:13 +0200, Petr Jelinek wrote: > + * Determine which timeline to read an xlog page from and set the > + * XLogReaderState's state->currTLI to that timeline ID. "XLogReaderState's state->currTLI" - the state's a bit redundant. > + * The caller must also make sure it doesn't

Re: [HACKERS] Patch to implement pg_current_logfile() function

Le 09/11/2016 à 02:51, Robert Haas a écrit : > On Thu, Nov 3, 2016 at 7:34 PM, Karl O. Pinc wrote: >> To me, the CURRENT_LOG_FILENAME symbol should contain the name >> of the current log file. It does not. The CURRENT_LOG_FILENAME symbol >> holds the name of the file which itself

Re: [HACKERS] Re: [COMMITTERS] pgsql: Change the way that LWLocks for extensions are allocated.

Hi, On 2016-11-08 10:25:11 +0530, Amit Kapila wrote: > ifdef USE_PGXS > PG_CONFIG = pg_config > PGXS := $(shell $(PG_CONFIG) --pgxs) > @@ -21,3 +25,29 @@ top_builddir = ../.. > include $(top_builddir)/src/Makefile.global > include $(top_srcdir)/contrib/contrib-global.mk > endif > + > +#

Re: [HACKERS] [bug fix] Cascading standby cannot catch up and get stuck emitting the same message repeatedly

On Thu, Nov 10, 2016 at 10:43 AM, Tsunakawa, Takayuki wrote: > From: pgsql-hackers-ow...@postgresql.org >> [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Robert Haas >> OK. I agree that's a problem. However, your patch adds zero new comment >> text

Re: [HACKERS] Fix checkpoint skip logic on idle systems by tracking LSN progress

Hi, On 2016-11-11 16:42:43 +0900, Michael Paquier wrote: > + * This takes also > + * advantage to avoid 8-byte torn reads on some platforms by using the > + * fact that each insert lock is located on the same cache line. Something residing on the same cache line doens't provide that guarantee

Re: [HACKERS] pg_dump, pg_dumpall and data durability

On Sat, Nov 12, 2016 at 1:46 PM, Michael Paquier wrote: > On Fri, Nov 11, 2016 at 11:03 PM, Albe Laurenz > wrote: >> This would avoid confusion, and we expect that few people will want to use >> this option anyway, right? > > Definitely a

Re: [HACKERS] Shared memory estimation for postgres

On 2016-11-11 09:56:21 +0900, Michael Paquier wrote: > On Fri, Nov 11, 2016 at 8:40 AM, leoaaryan wrote: > > The easiest way to find the value for the shared memory computation is to > > change the logging level to DEBUG3 and start postgres DB engine and it will > > give the