Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

2017-01-31 Thread Peter Geoghegan
On Tue, Jan 31, 2017 at 11:23 PM, Thomas Munro wrote: > 2. All participants: parallel sequential scan, sort, spool to disk; > barrier; leader: merge spooled tuples and build btree. > > This patch is doing the 2nd thing. My understanding is that some > systems might choose to do that if they don'

Re: [HACKERS] [BUGS] Bug in Physical Replication Slots (at least 9.5)?

2017-01-31 Thread Kyotaro HORIGUCHI
Hello, I'll add the rebased version to the next CF. At Fri, 20 Jan 2017 11:07:29 +0900 (Tokyo Standard Time), Kyotaro HORIGUCHI wrote in <20170120.110729.107284864.horiguchi.kyot...@lab.ntt.co.jp> > > > > - Delaying recycling a segment until the last partial record on it > > > > completes. Th

Re: [HACKERS] Parallel Index Scans

2017-01-31 Thread Rahila Syed
Hello Robert, >I am a bit mystified about how this manages to work with array keys. >_bt_parallel_done() won't set btps_pageStatus to BTPARALLEL_DONE >unless so->arrayKeyCount >= btscan->btps_arrayKeyCount, but >_bt_parallel_advance_scan() won't do anything unless btps_pageStatus >is already BTPAR

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

2017-01-31 Thread Thomas Munro
On Wed, Feb 1, 2017 at 5:37 PM, Michael Paquier wrote: > On Tue, Jan 31, 2017 at 2:15 PM, Peter Geoghegan wrote: >> On Mon, Jan 30, 2017 at 8:46 PM, Thomas Munro >> wrote: >>> On Wed, Jan 4, 2017 at 12:53 PM, Peter Geoghegan wrote: Attached is V7 of the patch. >>> >>> I am doing some testi

Re: [HACKERS] [PATCH]: fix bug in SP-GiST box_ops

2017-01-31 Thread Kyotaro HORIGUCHI
At Tue, 31 Jan 2017 14:38:39 +0300, Nikita Glukhov wrote in <1622dc9f-cecf-cee3-b71e-b2bf34649...@postgrespro.ru> > On 31.01.2017 13:04, Kyotaro HORIGUCHI wrote: > > The following comment, > > > >> /* Can any range from range_box to be overlower than this argument? */ > > > > This might be better

Re: [HACKERS] WAL consistency check facility

2017-01-31 Thread Kuntal Ghosh
On Wed, Feb 1, 2017 at 8:01 AM, Michael Paquier wrote: > On Wed, Feb 1, 2017 at 1:06 AM, Robert Haas wrote: >> +/* >> + * Leave if no masking functions defined, this is possible in the case >> + * resource managers generating just full page writes, comparing an >> + * image to it

Re: [HACKERS] Logical Replication and Character encoding

2017-01-31 Thread Kyotaro HORIGUCHI
At Wed, 01 Feb 2017 12:13:04 +0900 (Tokyo Standard Time), Kyotaro HORIGUCHI wrote in <20170201.121304.267734380.horiguchi.kyot...@lab.ntt.co.jp> > > > I tried a committed Logical Replication environment. I found > > > that replication between databases of different encodings did > > > not con

Re: [HACKERS] Parallel Index-only scan

2017-01-31 Thread Michael Paquier
On Thu, Jan 19, 2017 at 9:07 PM, Rafia Sabih wrote: > Please find the attached file rebased patch of parallel index-only > scan on the latest Parallel index-scan patch [1]. Moved to CF 2017-03. -- Michael -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to

Re: [HACKERS] Protect syscache from bloating with negative cache entries

2017-01-31 Thread Kyotaro HORIGUCHI
Hello, thank you for moving this to the next CF. At Wed, 1 Feb 2017 13:09:51 +0900, Michael Paquier wrote in > On Tue, Jan 24, 2017 at 4:58 PM, Kyotaro HORIGUCHI > wrote: > > Six new syscaches in 665d1fa was conflicted and 3-way merge > > worked correctly. The new syscaches don't seem to be ta

Re: ParallelFinish-hook of FDW/CSP (Re: [HACKERS] Steps inside ExecEndGather)

2017-01-31 Thread Michael Paquier
On Fri, Dec 2, 2016 at 9:20 PM, Haribabu Kommi wrote: > Moved to next CF with "needs review" status. There has not been much interest in this patch, moved again, this time to CF 2017-03. -- Michael -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your s

Re: [HACKERS] [WIP]Vertical Clustered Index (columnar store extension)

2017-01-31 Thread Michael Paquier
On Fri, Dec 30, 2016 at 12:55 PM, Haribabu Kommi wrote: > Any Comments on the approach? I have moved this patch to CF 2017-03. -- Michael -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hacker

Re: [HACKERS] Push down more UPDATEs/DELETEs in postgres_fdw

2017-01-31 Thread Michael Paquier
On Wed, Jan 25, 2017 at 7:20 PM, Etsuro Fujita wrote: > Attached is the new version of the patch. I also addressed other comments > from you: moved rewriting the fdw_scan_tlist to postgres_fdw.c, > added/revised comments, and added regression tests for the case where a > pushed down UPDATE/DELETE

Re: [HACKERS] Logical replication existing data copy

2017-01-31 Thread Michael Paquier
On Fri, Jan 20, 2017 at 3:03 AM, Petr Jelinek wrote: > Okay, here is v3 with some small fixes and rebased on top of rename. > Also it's rebased without the > 0005-Add-separate-synchronous-commit-control-for-logical--v18.patch as I > don't expect that one to go further for now. > > Thanks for testi

Re: [HACKERS] Logical decoding on standby

2017-01-31 Thread Michael Paquier
On Tue, Jan 24, 2017 at 7:37 AM, Craig Ringer wrote: > Rebased series attached, on top of current master (which includes > logical replicaiton). > > I'm inclined to think I should split out a few of the changes from > 0005, roughly along the lines of the bullet points in its commit > message. Anyo

Re: [HACKERS] [PATCH] Transaction traceability - txid_status(bigint)

2017-01-31 Thread Michael Paquier
On Wed, Jan 25, 2017 at 4:02 PM, Craig Ringer wrote: > If we want to save the 4 bytes per xmin advance (probably not worth > caring) we can instead skip setting it on the standby, in which case > it'll be potentially wrong until the next checkpoint. I'd rather make > sure it stays correct. Those

Re: [HACKERS] Parallel Append implementation

2017-01-31 Thread Michael Paquier
On Tue, Jan 17, 2017 at 2:40 PM, Amit Langote wrote: > Hi Amit, > > On 2016/12/23 14:21, Amit Khandekar wrote: >> Currently an Append plan node does not execute its subplans in >> parallel. There is no distribution of workers across its subplans. The >> second subplan starts running only after the

Re: [HACKERS] tuplesort_gettuple_common() and *should_free argument

2017-01-31 Thread Michael Paquier
On Thu, Jan 26, 2017 at 8:16 AM, Tom Lane wrote: > [ in the service of closing out this thread... ] > > Peter Geoghegan writes: >> Finally, 0003-* is a Valgrind suppression borrowed from my parallel >> CREATE INDEX patch. It's self-explanatory. > > Um, I didn't find it all that self-explanatory.

Re: [HACKERS] asynchronous execution

2017-01-31 Thread Kyotaro HORIGUCHI
Thank you. At Wed, 1 Feb 2017 14:11:58 +0900, Michael Paquier wrote in > On Tue, Jan 31, 2017 at 12:45 PM, Kyotaro HORIGUCHI > wrote: > > I noticed that this patch is conflicting with 665d1fa (Logical > > replication) so I rebased this. Only executor/Makefile > > conflicted. > > The patches s

Re: [HACKERS] Parallel Index Scans

2017-01-31 Thread Amit Kapila
On Tue, Jan 31, 2017 at 5:53 PM, Rahila Syed wrote: > Hello, > >>Agreed, that it makes sense to consider only the number of pages to >>scan for computation of parallel workers. I think for index scan we >>should consider both index and heap pages that need to be scanned >>(costing of index scan c

Re: [HACKERS] Gather Merge

2017-01-31 Thread Michael Paquier
On Mon, Jan 23, 2017 at 6:51 PM, Kuntal Ghosh wrote: > On Wed, Jan 18, 2017 at 11:31 AM, Rushabh Lathia > wrote: >> > The patch needs a rebase after the commit 69f4b9c85f168ae006929eec4. Is an update going to be provided? I have moved this patch to next CF with "waiting on author" as status. --

Re: [HACKERS] WAL consistency check facility

2017-01-31 Thread Kuntal Ghosh
On Tue, Jan 31, 2017 at 9:36 PM, Robert Haas wrote: > On Thu, Jan 5, 2017 at 12:54 AM, Kuntal Ghosh > wrote: >> I've attached the patch with the modified changes. PFA. Thanks Robert for taking your time for the review. I'll update the patch with the changes suggested by you. -- Thanks & Regar

Re: [HACKERS] PATCH: recursive json_populate_record()

2017-01-31 Thread Michael Paquier
On Thu, Jan 26, 2017 at 6:49 AM, Tom Lane wrote: > Nikita Glukhov writes: >> On 25.01.2017 23:58, Tom Lane wrote: >>> I think you need to take a second look at the code you're producing >>> and realize that it's not so clean either. This extract from >>> populate_record_field, for example, is pr

Re: [HACKERS] proposal: session server side variables

2017-01-31 Thread Pavel Stehule
2017-02-01 6:05 GMT+01:00 Michael Paquier : > On Wed, Jan 11, 2017 at 10:42 PM, Craig Ringer > wrote: > > There is no code yet. Code review and testing is where things get firmer. > > > > My personal stance right now is that I'd like to see catalog-decared > typed > > variables. I would prefer th

Re: [HACKERS] Cache Hash Index meta page.

2017-01-31 Thread Michael Paquier
On Sun, Jan 29, 2017 at 6:13 PM, Mithun Cy wrote: >> HashMetaPage _hash_getcachedmetap(Relation rel, Buffer *metabuf, bool >> force_refresh); >> >> If the cache is initialized and force_refresh is not true, then this >> just returns the cached data, and the metabuf argument isn't used. >> Otherwis

Re: [HACKERS] Vacuum: allow usage of more than 1GB of work mem

2017-01-31 Thread Michael Paquier
On Tue, Jan 31, 2017 at 11:05 AM, Claudio Freire wrote: > Updated and rebased v7 attached. Moved to CF 2017-03. -- Michael -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] [PROPOSAL] Temporal query processing with range types

2017-01-31 Thread Michael Paquier
On Tue, Jan 24, 2017 at 6:32 PM, Peter Moser wrote: > [reviews and discussions] The patch proposed has rotten. Please provide a rebase. By the way, I am having a hard time applying your patches with patch or any other methods... I am moving it to CF 2017-03 because of the lack of reviews. -- Mic

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-01-31 Thread Pavan Deolasee
On Tue, Jan 31, 2017 at 7:21 PM, Alvaro Herrera wrote: > > > > +#define HeapTupleHeaderGetNextTid(tup, next_ctid) \ > > > +do { \ > > > + AssertMacro(!((tup)->t_infomask2 & HEAP_LATEST_TUPLE)); \ > > > + ItemPointerCopy(&(tup)->t_ctid, (next_ctid)); \ > > > +} while (0) > > > > > Actually

Re: [HACKERS] ICU integration

2017-01-31 Thread Michael Paquier
On Wed, Jan 25, 2017 at 2:44 AM, Peter Eisentraut wrote: > On 1/15/17 5:53 AM, Pavel Stehule wrote: >> the regress test fails >> >> Program received signal SIGSEGV, Segmentation fault. >> 0x007bbc2b in pattern_char_isalpha (locale_is_c=0 '\000', >> locale=0x1a73220, is_multibyte=1 '\001',

Re: [HACKERS] asynchronous execution

2017-01-31 Thread Michael Paquier
On Tue, Jan 31, 2017 at 12:45 PM, Kyotaro HORIGUCHI wrote: > I noticed that this patch is conflicting with 665d1fa (Logical > replication) so I rebased this. Only executor/Makefile > conflicted. The patches still apply, moved to CF 2017-03. Be aware of that: $ git diff HEAD~6 --check contrib/post

Re: [HACKERS] PoC: Make it possible to disallow WHERE-less UPDATE and DELETE

2017-01-31 Thread Michael Paquier
On Wed, Jan 11, 2017 at 11:55 PM, Pavel Stehule wrote: > I'll mark this patch as ready for commiter Moved to CF 2017-03. -- Michael -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] proposal: session server side variables

2017-01-31 Thread Michael Paquier
On Wed, Jan 11, 2017 at 10:42 PM, Craig Ringer wrote: > There is no code yet. Code review and testing is where things get firmer. > > My personal stance right now is that I'd like to see catalog-decared typed > variables. I would prefer them to be transactional and would at least oppose > anything

Re: [HACKERS][REVIEW] macaddr 64 bit (EUI-64) datatype support

2017-01-31 Thread Kuntal Ghosh
On Wed, Feb 1, 2017 at 12:57 AM, Vitaly Burovoy wrote: > Hello, > > I've reviewed the patch[1]. > Noting to add from my side as well. -- Thanks & Regards, Kuntal Ghosh EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make chang

Re: [HACKERS] Microvacuum support for Hash Index

2017-01-31 Thread Michael Paquier
On Sat, Jan 28, 2017 at 8:02 PM, Amit Kapila wrote: > On Fri, Jan 27, 2017 at 5:15 PM, Ashutosh Sharma > wrote: >>> >>> Don't you think we should try to identify the reason of the deadlock >>> error reported by you up thread [1]? I know that you and Ashutosh are >>> not able to reproduce it, bu

Re: [HACKERS] Write Ahead Logging for Hash Indexes

2017-01-31 Thread Michael Paquier
On Fri, Jan 13, 2017 at 12:23 PM, Amit Kapila wrote: > On Fri, Jan 13, 2017 at 1:04 AM, Jesper Pedersen > wrote: >> On 12/27/2016 01:58 AM, Amit Kapila wrote: >>> >>> After recent commit's 7819ba1e and 25216c98, this patch requires a >>> rebase. Attached is the rebased patch. >>> >> >> This need

Re: [HACKERS] Indirect indexes

2017-01-31 Thread Michael Paquier
On Sat, Dec 31, 2016 at 7:35 AM, Alvaro Herrera wrote: > Attached is v4, which fixes a couple of relatively minor bugs. There > are still things to tackle before this is committable, but coding review > of the new executor node would be welcome. Moved to CF 2017-03 because of a lack of reviews.

Re: [HACKERS] Bug in to_timestamp().

2017-01-31 Thread Michael Paquier
On Mon, Dec 5, 2016 at 11:35 AM, Haribabu Kommi wrote: > Moved to next CF with "needs review" status. Same, this time to CF 2017-03. -- Michael -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-

Re: [HACKERS] Push down more full joins in postgres_fdw

2017-01-31 Thread Michael Paquier
On Mon, Jan 30, 2017 at 8:30 PM, Etsuro Fujita wrote: > Other changes: > * I went back to make_outerrel_subquery and make_innerrel_subquery, which > are flags to indicate whether to deparse the input relations as subqueries. > is_subquery_rel would work well for handling the cases of full joins wi

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-01-31 Thread Michael Paquier
On Wed, Feb 1, 2017 at 9:36 AM, Alvaro Herrera wrote: >> I propose that we should finish the job by inventing CatalogTupleDelete(), >> which for the moment would be a trivial wrapper around >> simple_heap_delete(), maybe just a macro for it. >> >> If there's no objections I'll go make that happen

Re: [HACKERS] postgres_fdw bug in 9.6

2017-01-31 Thread Michael Paquier
On Mon, Jan 23, 2017 at 6:56 PM, Etsuro Fujita wrote: > Other changes: > > * Also modified CreateLocalJoinPath so that we pass outer/inner paths, not > outer/inner rels, because it would be more flexible for the FDW to build the > local-join path from paths it chose. > * Fixed a bug that I missed

Re: [HACKERS] WIP: [[Parallel] Shared] Hash

2017-01-31 Thread Ashutosh Bapat
> >> However, ExecHashIncreaseNumBatches() may change the >> number of buckets; the patch does not seem to account for spaceUsed changes >> because of that. > > That's what this hunk is intended to do: > > @@ -795,6 +808,12 @@ ExecHashIncreaseNumBuckets(HashJoinTable hashtable) > TRACE_POST

Re: [HACKERS] Reporting planning time with EXPLAIN

2017-01-31 Thread Michael Paquier
On Wed, Jan 4, 2017 at 7:59 PM, Ashutosh Bapat wrote: > Here are patches for following Those patches have received no code-level reviews, so moved to CF 2017-03. -- Michael -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.p

Re: [HACKERS] Pinning a buffer in TupleTableSlot is unnecessary

2017-01-31 Thread Michael Paquier
On Wed, Jan 25, 2017 at 2:53 PM, Michael Paquier wrote: > The latest patch available still applies, one person has added his > name (John G) in October though there have been no reviews. There have > been a couple of arguments against this patch, and the thread has had > no activity for the last m

Re: [HACKERS] Proposal : Parallel Merge Join

2017-01-31 Thread Michael Paquier
On Thu, Jan 5, 2017 at 12:57 AM, Dilip Kumar wrote: > On Wed, Jan 4, 2017 at 12:02 PM, Amit Kapila wrote: >> Review comments: >> 1. >> + bool is_partial); >> + >> >> Seems additional new line is not required. > Fixed This patch has a patch, no new reviews. Moved to CF 2017-03. -- Michael --

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

2017-01-31 Thread Michael Paquier
On Tue, Jan 31, 2017 at 2:15 PM, Peter Geoghegan wrote: > On Mon, Jan 30, 2017 at 8:46 PM, Thomas Munro > wrote: >> On Wed, Jan 4, 2017 at 12:53 PM, Peter Geoghegan wrote: >>> Attached is V7 of the patch. >> >> I am doing some testing. First, some superficial things from first pass: >> >> [Vari

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

2017-01-31 Thread Michael Paquier
On Tue, Jan 17, 2017 at 9:18 PM, Amit Kapila wrote: > On Tue, Jan 17, 2017 at 11:39 AM, Dilip Kumar wrote: >> On Wed, Jan 11, 2017 at 10:55 AM, Dilip Kumar wrote: >>> I have reviewed the latest patch and I don't have any more comments. >>> So if there is no objection from other reviewers I can m

Re: [HACKERS] pg_xlogdump follow into the future

2017-01-31 Thread Michael Paquier
On Fri, Dec 2, 2016 at 1:36 PM, Haribabu Kommi wrote: > Patch received feedback at the end of commitfest. > Closed in 2016-11 commitfest with "moved to next CF". > Please feel free to update the status once you submit the updated patch. And the thread has died as well weeks ago. I am marking that

Re: [HACKERS] Forbid use of LF and CR characters in database and role names

2017-01-31 Thread Michael Paquier
On Wed, Feb 1, 2017 at 1:31 PM, Michael Paquier wrote: > On Tue, Nov 29, 2016 at 1:33 PM, Michael Paquier > wrote: >> Patch moved to CF 2017-01. > > And nothing has happened since, the patch rotting a bit because of a > conflict in pg_dump's TAP tests. Attached is a rebased version. Moved to CF

Re: [HACKERS] Forbid use of LF and CR characters in database and role names

2017-01-31 Thread Michael Paquier
On Tue, Nov 29, 2016 at 1:33 PM, Michael Paquier wrote: > On Fri, Nov 25, 2016 at 4:41 PM, Michael Paquier > wrote: >> On Fri, Nov 25, 2016 at 4:02 PM, Ideriha, Takeshi >> wrote: >>> I applied your fixed patch and new one, and confirmed the applied source >>> passed the tests successfully. And

Re: [HACKERS] Measuring replay lag

2017-01-31 Thread Michael Paquier
On Sat, Jan 21, 2017 at 10:49 AM, Thomas Munro wrote: > Ok. I see that there is a new compelling reason to move the ring > buffer to the sender side: then I think lag tracking will work > automatically for the new logical replication that just landed on > master. I will try it that way. Thanks

Re: [HACKERS] PATCH: two slab-like memory allocators

2017-01-31 Thread Michael Paquier
On Tue, Dec 13, 2016 at 10:32 AM, Petr Jelinek wrote: > Okay, this version looks good to me, marked as RfC. The patches still apply, moved to CF 2017-03 with same status: RfC. -- Michael -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription

Re: [HACKERS] New SQL counter statistics view (pg_stat_sql)

2017-01-31 Thread Michael Paquier
On Fri, Jan 27, 2017 at 10:26 AM, Haribabu Kommi wrote: > Thanks for the review. > Let's wait for the committer's opinion. I have moved this patch to CF 2017-03 to wait for this to happen. -- Michael -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your

Re: [HACKERS] Supporting huge pages on Windows

2017-01-31 Thread Michael Paquier
On Mon, Jan 30, 2017 at 10:46 AM, Tsunakawa, Takayuki wrote: > From: Amit Kapila [mailto:amit.kapil...@gmail.com] >> I think it is better to document in some way if we decide to go-ahead with >> the patch. > > Sure, I added these sentences. Patch has been moved to CF 2017-03. There is a recent ne

Re: [HACKERS] Protect syscache from bloating with negative cache entries

2017-01-31 Thread Michael Paquier
On Tue, Jan 24, 2017 at 4:58 PM, Kyotaro HORIGUCHI wrote: > Six new syscaches in 665d1fa was conflicted and 3-way merge > worked correctly. The new syscaches don't seem to be targets of > this patch. To be honest, I am not completely sure what to think about this patch. Moved to next CF as there

Re: [HACKERS] pageinspect: Hash index support

2017-01-31 Thread Michael Paquier
On Sun, Jan 29, 2017 at 11:09 AM, Ashutosh Sharma wrote: > okay. Thanks. I have done changes on top of this patch. Moved to CF 2017-03 as there is a new patch, no reviews. -- Michael -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: ht

Re: [HACKERS] Transactions involving multiple postgres foreign servers

2017-01-31 Thread Michael Paquier
On Thu, Jan 26, 2017 at 6:49 PM, Masahiko Sawada wrote: > Sorry, I attached wrong version patch of pg_fdw_xact_resovler. Please > use attached patch. This patch has been moved to CF 2017-03. -- Michael -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to yo

Re: [HACKERS] Logical Replication and Character encoding

2017-01-31 Thread Kyotaro HORIGUCHI
At Wed, 01 Feb 2017 12:05:40 +0900 (Tokyo Standard Time), Kyotaro HORIGUCHI wrote in <20170201.120540.183393194.horiguchi.kyot...@lab.ntt.co.jp> > Hello, > > At Tue, 31 Jan 2017 12:46:18 +, "Shinoda, Noriyoshi" > wrote in > > > I tried a committed Logical Replication environment. I fou

Re: [HACKERS] Logical Replication and Character encoding

2017-01-31 Thread Kyotaro HORIGUCHI
Hello, At Tue, 31 Jan 2017 12:46:18 +, "Shinoda, Noriyoshi" wrote in > I tried a committed Logical Replication environment. I found > that replication between databases of different encodings did > not convert encodings in character type columns. Is this > behavior correct? The output

Re: [HACKERS] WAL consistency check facility

2017-01-31 Thread Michael Paquier
On Wed, Feb 1, 2017 at 1:06 AM, Robert Haas wrote: > On Thu, Jan 5, 2017 at 12:54 AM, Kuntal Ghosh > wrote: >> I've attached the patch with the modified changes. PFA. > > Can this patch check contrib/bloom? Only full pages are applied at redo by the generic WAL facility. So you would finish by c

Re: [HACKERS] Proposal : For Auto-Prewarm.

2017-01-31 Thread Michael Paquier
On Wed, Feb 1, 2017 at 1:17 AM, Robert Haas wrote: > Well, the question even for that case is whether it really costs > anything. My bet is that it is nearly free when it doesn't help, but > that could be wrong. My experience running pgbench tests is that > prewarming all of pgbench_accounts on

Re: [HACKERS] logical decoding of two-phase transactions

2017-01-31 Thread Michael Paquier
On Tue, Jan 31, 2017 at 6:22 PM, Craig Ringer wrote: > That's where you've misunderstood - it isn't committed yet. The point or > this change is to allow us to do logical decoding at the PREPARE TRANSACTION > point. The xact is not yet committed or rolled back. Yes, I got that. I was looking for

Re: [HACKERS] Refactoring of replication commands using printsimple

2017-01-31 Thread Michael Paquier
On Tue, Jan 31, 2017 at 11:59 PM, Robert Haas wrote: > Sorry, I have a little more nitpicking. Thanks for the input. > How about having > printsimple() use pq_sendcountedtext() instead of pq_sendint() > followed by pq_sendbytes(), as it does for TEXTOID? > > Other than that, this looks fine to m

Re: [HACKERS] sequence data type

2017-01-31 Thread Michael Paquier
On Wed, Feb 1, 2017 at 1:11 AM, Peter Eisentraut wrote: > And here is a rebased patch for the original feature. I think this > addresses all raised concerns and suggestions now. Thanks for the new version. That looks good to me after an extra lookup. -- Michael -- Sent via pgsql-hackers mail

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-01-31 Thread Alvaro Herrera
Tom Lane wrote: > BTW, the reason I think it's good cleanup is that it's something that my > colleagues at Salesforce also had to do as part of putting PG on top of a > different storage engine that had different ideas about index handling. > Essentially it's providing a bit of abstraction as to w

Re: [HACKERS] WIP: [[Parallel] Shared] Hash

2017-01-31 Thread Thomas Munro
On Wed, Feb 1, 2017 at 2:10 AM, Ashutosh Bapat wrote: >> >> 0003-hj-refactor-memory-accounting-v4.patch: >> [...] >> > I looked at this patch. I agree that it accounts the memory usage more > accurately. Here are few comments. Thanks for the review! > spaceUsed is defined with comment > Size

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-01-31 Thread Andres Freund
On 2017-01-31 17:21:28 -0500, Tom Lane wrote: > Andres Freund writes: > > Hm, sorry for missing this earlier. I think CatalogUpdateIndexes() is > > fairly widely used in extensions - it seems like a pretty harsh change > > to not leave some backward compatibility layer in place. > > If an extens

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-01-31 Thread Tom Lane
Stephen Frost writes: > * Tom Lane (t...@sss.pgh.pa.us) wrote: >> (I'm a little more concerned by Alvaro's apparent position that WARM >> is a done deal; I didn't think so. This particular change seems like >> good cleanup anyhow, however.) > Agreed. BTW, the reason I think it's good cleanup is

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-01-31 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: > Andres Freund writes: > > Hm, sorry for missing this earlier. I think CatalogUpdateIndexes() is > > fairly widely used in extensions - it seems like a pretty harsh change > > to not leave some backward compatibility layer in place. > > If an extension is

Re: [HACKERS] Should `pg_upgrade --check` check relation filenodes are present?

2017-01-31 Thread Tom Lane
Craig de Stigter writes: > We attempted to pg_upgrade a database on a replication slave, and got the > error: > error while creating link for relation "." > ("/var/lib/postgresql-ext/PG_9.2_201204301/19171/141610397" to > "/var/lib/postgresql-ext/PG_9.5_201510051/16401/9911696"): No such file or

Re: [HACKERS] Parallel Index Scans

2017-01-31 Thread Robert Haas
On Tue, Jan 24, 2017 at 4:51 PM, Robert Haas wrote: > On Mon, Jan 23, 2017 at 1:03 AM, Amit Kapila wrote: >> In spite of being careful, I missed reorganizing the functions in >> genam.h which I have done in attached patch. > > Cool. Committed parallel-generic-index-scan.2.patch. Thanks. Review

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-01-31 Thread Tom Lane
Andres Freund writes: > Hm, sorry for missing this earlier. I think CatalogUpdateIndexes() is > fairly widely used in extensions - it seems like a pretty harsh change > to not leave some backward compatibility layer in place. If an extension is doing that, it is probably constructing tuples to p

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-01-31 Thread Andres Freund
On 2017-01-31 19:10:05 -0300, Alvaro Herrera wrote: > Andres Freund wrote: > > On 2017-01-31 14:10:01 -0300, Alvaro Herrera wrote: > > > > Hmm, I was thinking we would get rid of CatalogUpdateIndexes altogether. > > > Two of the callers are in the new routines (which I propose to rename to > > > C

Re: [HACKERS] Improvements in psql hooks for variables

2017-01-31 Thread Tom Lane
BTW ... while I've been fooling with this issue, I've gotten a bit annoyed at the fact that "\set" prints the variables in, essentially, creation order. That makes the list ugly and hard to find things in, and it exposes some psql implementation details to users. I propose the attached simple pat

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-01-31 Thread Alvaro Herrera
Andres Freund wrote: > On 2017-01-31 14:10:01 -0300, Alvaro Herrera wrote: > > Hmm, I was thinking we would get rid of CatalogUpdateIndexes altogether. > > Two of the callers are in the new routines (which I propose to rename to > > CatalogTupleInsert and CatalogTupleUpdate); the only remaining on

Re: [HACKERS] Time to up bgwriter_lru_maxpages?

2017-01-31 Thread Jim Nasby
On 11/29/16 9:58 AM, Jeff Janes wrote: Considering a single SSD can do 70% of that limit, I would say yes. Next question becomes... should there even be an upper limit? Where the contortions needed to prevent calculation overflow become annoying? I'm not a big fan of nannyism in

[HACKERS] Should `pg_upgrade --check` check relation filenodes are present?

2017-01-31 Thread Craig de Stigter
Hi list We attempted to pg_upgrade a database on a replication slave, and got the error: error while creating link for relation "." >> ("/var/lib/postgresql-ext/PG_9.2_201204301/19171/141610397" to >> "/var/lib/postgresql-ext/PG_9.5_201510051/16401/9911696"): No such file or >> directory > > > Th

Re: [HACKERS] ICU integration

2017-01-31 Thread Thomas Munro
On Tue, Jan 10, 2017 at 9:45 AM, Peter Geoghegan wrote: > * I think it's worth looking into ucol_nextSortKeyPart(), and using > that as an alternative to ucol_getSortKey(). It doesn't seem any > harder, and when I tested it it was clearly faster. (I think that > ucol_nextSortKeyPart() is more or l

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-01-31 Thread Andres Freund
On 2017-01-31 14:10:01 -0300, Alvaro Herrera wrote: > Pavan Deolasee wrote: > > > Two new APIs added. > > > > - CatalogInsertHeapAndIndex which does a simple_heap_insert followed by > > catalog updates > > - CatalogUpdateHeapAndIndex which does a simple_heap_update followed by > > catalog updates

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-01-31 Thread Alvaro Herrera
Alvaro Herrera wrote: > Unless there are objections I will push this later this afternoon. Done. Let's get on with the show -- please post a rebased WARM. -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- S

Re: [HACKERS] patch: function xmltable

2017-01-31 Thread Pavel Stehule
Hi 2017-01-31 14:57 GMT+01:00 Alvaro Herrera : > Pavel Stehule wrote: > > 2017-01-24 21:38 GMT+01:00 Alvaro Herrera : > > > > I think it would be good to have a more complex test case in regress -- > > > let's say there is a table with some simple XML values, then we use > > > XMLFOREST (or maybe

Re: [HACKERS] Improvements in psql hooks for variables

2017-01-31 Thread Tom Lane
I wrote: > Attached is a draft patch for that. I chose to make a second hook rather > than complicate the assign hook API, mainly because it allows more code > sharing --- all the bool vars can share the same substitute hook, and > so can the three-way vars as long as "on" and "off" are the approp

Re: [HACKERS] logical decoding of two-phase transactions

2017-01-31 Thread Craig Ringer
On 31 Jan. 2017 22:43, "Konstantin Knizhnik" wrote: On 31.01.2017 09:29, Michael Paquier wrote: > On Fri, Jan 27, 2017 at 8:52 AM, Craig Ringer > wrote: > >> Now, if it's simpler to just xlog the gid at COMMIT PREPARED time when >> wal_level >= logical I don't think that's the end of the worl

[HACKERS][REVIEW] macaddr 64 bit (EUI-64) datatype support

2017-01-31 Thread Vitaly Burovoy
Hello, I've reviewed the patch[1]. Result of testing: make installcheck-world: tested, passed Implements feature: tested, passed Spec compliant: tested, passed Documentation:tested, passed The patch introduce a new type macaddr8 for EUI-64 addresses[2] (assuming OU

Re: [HACKERS] Floating point comparison inconsistencies of the geometric types

2017-01-31 Thread Robert Haas
On Tue, Jan 31, 2017 at 1:06 PM, Emre Hasegeli wrote: > This is what he wrote: > >> As I understand it, the key problem is that tests like "is point on line" >> would basically never succeed except in the most trivial cases, because of >> roundoff error. That's not very nice, and it might cascade

Re: \if, \elseif, \else, \endif (was Re: [HACKERS] PSQL commands: \quit_if, \quit_unless)

2017-01-31 Thread Tom Lane
Corey Huinker writes: > On Tue, Jan 31, 2017 at 1:04 AM, Fabien COELHO wrote: >> The ParseVariableBool function has been updated, and the new version is >> much cleaner, including all fixes that I suggested in your copy, so you can >> use it in your patch. > I see there's still a lot of activity

Re: [HACKERS] Improvements in psql hooks for variables

2017-01-31 Thread Tom Lane
"Daniel Verite" writes: > I notice that in the commited patch, you added the ability > for DeleteVariable() to reject the deletion if the hook > disagrees. Right. > But this can't happen in practice because as mentioned just upthread > the hook called with NULL doesn't know if the variable is g

Re: [HACKERS] Floating point comparison inconsistencies of the geometric types

2017-01-31 Thread Emre Hasegeli
> Backing up a bit here, have we lost track of the problem that we're > trying to solve? Tom gave his opinion here: > > https://www.postgresql.org/message-id/3895.1464791...@sss.pgh.pa.us > > But I don't see that the latest patch I can find does anything to fix > that. This is what he wrote: > A

Re: [HACKERS] Speedup twophase transactions

2017-01-31 Thread Robert Haas
On Mon, Jan 23, 2017 at 7:00 AM, Nikhil Sontakke wrote: > 4) Minor nit-pick on existing code. > > (errmsg_plural("%u two-phase state file was written " > "for > long-running prepared transactions", >

Re: [HACKERS] COPY as a set returning function

2017-01-31 Thread Corey Huinker
> > > Here is a 4: Refactoring BeginCopyFrom so as instead of a Relation are > used a TupleDesc, a default expression list, and a relation name. You > could as well make NextCopyFrom() smarter so as it does nothing if no > expression contexts are given by the caller, which is the case of your > fun

Re: \if, \elseif, \else, \endif (was Re: [HACKERS] PSQL commands: \quit_if, \quit_unless)

2017-01-31 Thread Corey Huinker
On Tue, Jan 31, 2017 at 1:04 AM, Fabien COELHO wrote: > > This is code lifted from variable.c's ParseVariableBool(). When the other >>> patch for "psql hooks" is committed (the one that detects when the string >>> wasn't a valid boolean), this code will go away and we'll just use >>> ParseVariab

Re: [HACKERS] Improvements in psql hooks for variables

2017-01-31 Thread Daniel Verite
I wrote: > This would allow the hook to distinguish between initialization and > unsetting, which in turn will allow it to deny the \unset in the > cases when it doesn't make any sense conceptually (like AUTOCOMMIT). I notice that in the commited patch, you added the ability for DeleteVar

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-01-31 Thread Alvaro Herrera
Pavan Deolasee wrote: > Two new APIs added. > > - CatalogInsertHeapAndIndex which does a simple_heap_insert followed by > catalog updates > - CatalogUpdateHeapAndIndex which does a simple_heap_update followed by > catalog updates > > There are only a handful callers remain for simple_heap_insert

Re: [HACKERS] Improvements in psql hooks for variables

2017-01-31 Thread Tom Lane
"Daniel Verite" writes: > Tom Lane wrote: >> One possible compromise that would address your concern about display >> is to modify the hook API some more so that variable hooks could actually >> substitute new values. Then for example the bool-variable hooks could >> effectively replace "\s

Re: [HACKERS] Proposal : For Auto-Prewarm.

2017-01-31 Thread Mithun Cy
On Tue, Jan 31, 2017 at 9:47 PM, Robert Haas wrote: > Now, I assume that this patch sorts the I/O (although I haven't > checked that) and therefore I expect that the prewarm completes really > fast. If that's not the case, then that's bad. But if it is the > case, then it's not really hurting yo

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-01-31 Thread Pavan Deolasee
On Tue, Jan 31, 2017 at 7:37 PM, Alvaro Herrera wrote: > Pavan Deolasee wrote: > > > > > Sounds good. Should I submit that as a separate patch on current master? > > Yes, please. > > Attached. Two new APIs added. - CatalogInsertHeapAndIndex which does a simple_heap_insert followed by catalog up

Re: [HACKERS] Floating point comparison inconsistencies of the geometric types

2017-01-31 Thread Robert Haas
On Thu, Jan 26, 2017 at 5:53 AM, Emre Hasegeli wrote: > Though, I know the community is against behaviour changing GUCs. I > will not spend more time on this, before I get positive feedback from > others. As if on cue, let me say that a behavior-changing GUC sounds like a terrible idea to me. I

Re: [HACKERS] Proposal : For Auto-Prewarm.

2017-01-31 Thread Robert Haas
On Tue, Jan 31, 2017 at 1:48 AM, Michael Paquier wrote: > I partially agree with this paragraph, at least there are advantages > to do so for cases where the data fits in shared buffers. Even for > data sets fitting in RAM that can be an advantage as the buffers would > get evicted from Postgres'

Re: [HACKERS] sequence data type

2017-01-31 Thread Peter Eisentraut
And here is a rebased patch for the original feature. I think this addresses all raised concerns and suggestions now. -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services >From ce2680ef072a9a4dc2cb879a70610d71ad24

Re: [HACKERS] multivariate statistics (v19)

2017-01-31 Thread Tomas Vondra
On 01/31/2017 07:52 AM, Amit Langote wrote: On 2017/01/31 6:57, Tomas Vondra wrote: On 01/30/2017 09:37 PM, Alvaro Herrera wrote: Looks good to me. I don't think we need to keep the names very short -- I would propose "standistinct", "stahistogram", "stadependencies". Yeah, I got annoyed by

Re: [HACKERS] WAL consistency check facility

2017-01-31 Thread Robert Haas
On Thu, Jan 5, 2017 at 12:54 AM, Kuntal Ghosh wrote: > I've attached the patch with the modified changes. PFA. Can this patch check contrib/bloom? +/* + * Mask some line pointer bits, particularly those marked as + * used on a master and unused on a standby. + */

Re: [HACKERS] patch: function xmltable

2017-01-31 Thread Pavel Stehule
2017-01-31 14:57 GMT+01:00 Alvaro Herrera : > Pavel Stehule wrote: > > 2017-01-24 21:38 GMT+01:00 Alvaro Herrera : > > > > I think it would be good to have a more complex test case in regress -- > > > let's say there is a table with some simple XML values, then we use > > > XMLFOREST (or maybe one

  1   2   >