Re: [HACKERS] WIP: Faster Expression Processing v4

2017-03-20 Thread Andres Freund
On 2017-03-20 16:06:27 -0400, Tom Lane wrote: > ... is there a reason why resultnum for EEOP_ASSIGN_* steps is declared > size_t and not just int? Since it's an array index, and one that > certainly can't be bigger than AttrNumber, that seems rather confusing. Not that I can see, no. I guess I m

[HACKERS] Do we create a new roadmap page for development?

2017-03-20 Thread Tsunakawa, Takayuki
Hello, I'd like to share our roadmap for PostgreSQL development, as other companies and individuals do in the following page. But this page is for PostgreSQL 10. PostgreSQL10 Roadmap https://wiki.postgresql.org/wiki/PostgreSQL10_Roadmap Should I create a page for PostgreSQL 11 likewise? Or, d

Re: [HACKERS] [COMMITTERS] pgsql: Add missing support for new node fields

2017-03-20 Thread Andres Freund
Hi, On 2017-03-21 07:22:57 +0100, Fabien COELHO wrote: > > Add missing support for new node fields > > > > Commit b6fb534f added two new node fields but neglected to add copy and > > comparison support for them, Mea culpa, should have checked for that. > > I've been annoyed by these stupid functio

Re: [HACKERS] [COMMITTERS] pgsql: Add missing support for new node fields

2017-03-20 Thread Fabien COELHO
Add missing support for new node fields Commit b6fb534f added two new node fields but neglected to add copy and comparison support for them, Mea culpa, should have checked for that. I've been annoyed by these stupid functions and forgetting to update them since I run into them while trying t

Re: [HACKERS] Adding support for Default partition in partitioning

2017-03-20 Thread Rushabh Lathia
I picked this for review and noticed that patch is not getting cleanly complied on my environment. partition.c: In function ‘RelationBuildPartitionDesc’: partition.c:269:6: error: ISO C90 forbids mixed declarations and code [-Werror=declaration-after-statement] Const*val = lfirst(c);

[HACKERS] Asymmetry between parent and child wrt "false" quals

2017-03-20 Thread Ashutosh Bapat
When I run a query like below on a child-less table, the plan comes out to be explain verbose SELECT * FROM uprt1_l WHERE a = 1 AND a = 2; QUERY PLAN -- Result (cost=0.00..11.50 rows=1 width=13)

Re: [HACKERS] exposing wait events for non-backends (was: Tracking wait event for latches)

2017-03-20 Thread Michael Paquier
On Fri, Mar 17, 2017 at 6:19 PM, Kuntal Ghosh wrote: > On Thu, Mar 16, 2017 at 1:18 PM, Michael Paquier > wrote: >> + /* We have userid for client-backends and wal-sender processes */ >> + if (beentry->st_backendType == B_BACKEND || >> beentry->st_backendType == B_WAL_SENDER) >> + beent

Re: [HACKERS] pageinspect and hash indexes

2017-03-20 Thread Ashutosh Sharma
On Mon, Mar 20, 2017 at 6:53 PM, Ashutosh Sharma wrote: > On Mon, Mar 20, 2017 at 9:31 AM, Amit Kapila wrote: >> On Sat, Mar 18, 2017 at 5:13 PM, Ashutosh Sharma >> wrote: >>> On Sat, Mar 18, 2017 at 1:34 PM, Amit Kapila >>> wrote: On Sat, Mar 18, 2017 at 12:12 AM, Ashutosh Sharma

Re: [HACKERS] ANALYZE command progress checker

2017-03-20 Thread vinayak
Hi Ashutosh, On 2017/03/19 17:56, Ashutosh Sharma wrote: Hi, I didn't find any major issues with the patch. It works as expected. However, I have few minor comments that I would like to share, + + Total number of sample rows. The sample it reads is taken randomly. + Its size d

Re: [HACKERS] ANALYZE command progress checker

2017-03-20 Thread vinayak
Hi Ashutosh, Thank you for reviewing the patch. On 2017/03/18 21:00, Ashutosh Sharma wrote: Hi Vinayak, Here are couple of review comments that may need your attention. 1. Firstly, I am seeing some trailing whitespace errors when trying to apply your v3 patch using git apply command. [ashu@l

[HACKERS] comments in hash_alloc_buckets

2017-03-20 Thread Ashutosh Sharma
Hi, While working on - [1], I realised that the following comments in _hash_alloc_buckets() needs to be corrected. /* * Initialize the freed overflow page. Just zeroing the page won't work, * See _hash_freeovflpage for similar usage. */ _hash_pageinit(page, BLCKSZ); Attac

Re: [HACKERS] Allow interrupts on waiting standby

2017-03-20 Thread Michael Paquier
On Sun, Mar 19, 2017 at 5:14 AM, Tom Lane wrote: > Couple of thoughts on this patch --- Thanks! > 1. Shouldn't WaitExceedsMaxStandbyDelay's CHECK_FOR_INTERRUPTS be moved to > after the WaitLatch call? Not much point in being woken immediately by > an interrupt if you're not going to respond. >

Re: [HACKERS] Proposal: GetOldestXminExtend for ignoring arbitrary vacuum flags

2017-03-20 Thread Seki, Eiji
On 2017-02-24 04:17:20 Haribabu Kommi wrote: >On Fri, Feb 24, 2017 at 3:17 PM, Seki, Eiji > >wrote: > >> >> Thank you for your comments. >> >> I reflected these comments to the attached patch. And I renamed IGNORE_XXX >> flags to PROCARRAY_XXX flags. > > >I checked the latest patch and I have some

Re: [HACKERS] Radix tree for character conversion

2017-03-20 Thread Kyotaro HORIGUCHI
Hello, At Fri, 17 Mar 2017 13:03:35 +0200, Heikki Linnakangas wrote in <01efd334-b839-0450-1b63-f2dea9326...@iki.fi> > On 03/17/2017 07:19 AM, Kyotaro HORIGUCHI wrote: > > I would like to use convert() function. It can be a large > > PL/PgSQL function or a series of "SELECT convert(...)"s. The >

[HACKERS] Freeze on Cygwin w/ concurrency

2017-03-20 Thread Noah Misch
"pgbench -i -s 50; pgbench -S -j2 -c16 -T900 -P5" freezes consistently on Cygwin 2.2.1 and Cygwin 2.6.0. (I suspect most other versions are affected.) I've pinged[1] the Cygwin bug thread with some additional detail. If a Cygwin buildfarm member starts using --enable-tap-tests, you may see failur

Re: [HACKERS] asynchronous execution

2017-03-20 Thread Kyotaro HORIGUCHI
Hello. This is the final report in this CF period. At Fri, 17 Mar 2017 17:35:05 +0900 (Tokyo Standard Time), Kyotaro HORIGUCHI wrote in <20170317.173505.152063931.horiguchi.kyot...@lab.ntt.co.jp> > Async-capable plan is generated in planner. An Append contains at > least one async-capable child

Re: [HACKERS] Our feature change policy

2017-03-20 Thread Robert Haas
On Mon, Mar 20, 2017 at 10:35 AM, Bruce Momjian wrote: > We keep re-litigating changes, either with pg_xlog, binaries, or > pg_stat_activity, and at some point we need to settle on a > policy. The usual "change" options are: > > 1. make the change now and mention it in the release notes > 2. #1

Re: [HACKERS] increasing the default WAL segment size

2017-03-20 Thread Robert Haas
On Mon, Mar 20, 2017 at 7:23 PM, David Steele wrote: > With 16MB WAL segments the filename neatly aligns with the LSN. For > example: > > WAL FILE 0001000100FE = LSN 1/FE00 > > This no longer holds true with this patch. It is already possible to change the WAL segment size using

Re: [HACKERS] [POC] A better way to expand hash indexes.

2017-03-20 Thread Amit Kapila
On Tue, Mar 21, 2017 at 8:16 AM, Amit Kapila wrote: > On Mon, Mar 20, 2017 at 8:58 PM, Mithun Cy wrote: >> Hi Amit, Thanks for the review, >> >> On Mon, Mar 20, 2017 at 5:17 PM, Amit Kapila wrote: >>> idea could be to make hashm_spares a two-dimensional array >>> hashm_spares[32][4] where the fi

Re: [HACKERS] [POC] A better way to expand hash indexes.

2017-03-20 Thread Amit Kapila
On Mon, Mar 20, 2017 at 8:58 PM, Mithun Cy wrote: > Hi Amit, Thanks for the review, > > On Mon, Mar 20, 2017 at 5:17 PM, Amit Kapila wrote: >> idea could be to make hashm_spares a two-dimensional array >> hashm_spares[32][4] where the first dimension will indicate the split >> point and second wi

Re: [HACKERS] Logical decoding on standby

2017-03-20 Thread Craig Ringer
On 20 March 2017 at 17:33, Andres Freund wrote: >> Subject: [PATCH 2/3] Follow timeline switches in logical decoding > > FWIW, the title doesn't really seem accurate to me. Yeah, it's not really at the logical decoding layer at all. "Teach xlogreader to follow timeline switches" ? >> Logical s

Re: [HACKERS] PATCH: Make pg_stop_backup() archive wait optional

2017-03-20 Thread Tsunakawa, Takayuki
From: David Steele [mailto:da...@pgmasters.net] > Well, that's embarrassing. When I recreated the function to add defaults > I messed up the AS clause and did not pay attention to the results of the > regression tests, apparently. > > Attached is a new version rebased on 88e66d1. Catalog version

Re: [HACKERS] [PATCH] few fts functions for jsonb

2017-03-20 Thread Andrew Dunstan
On 03/10/2017 11:13 AM, Dmitry Dolgov wrote: > > On 28 February 2017 at 19:21, Oleg Bartunov > wrote: > > 1. add json support > > I've added json support for all functions. > > > Its_headline should returns the original json with highlighting > > Yes, I see now. I d

Re: [HACKERS] Create replication slot in pg_basebackup if requested and not yet present

2017-03-20 Thread Michael Paquier
On Tue, Mar 21, 2017 at 1:32 AM, Michael Banck wrote: > On Mon, Mar 20, 2017 at 02:42:32PM +0300, Arthur Zakirov wrote: >> Also maybe it would be good if pg_basebackup had a way to drop created slot. >> Although "drop slot" is not related with concept of automatically created >> slots, it will goo

Re: [HACKERS] Logical decoding on standby

2017-03-20 Thread Craig Ringer
.On 20 March 2017 at 17:33, Andres Freund wrote: > Hi, > > Have you checked how high the overhead of XLogReadDetermineTimeline is? > A non-local function call, especially into a different translation-unit > (no partial inlining), for every single page might end up being > noticeable. That's fine

Re: [HACKERS] Logical decoding on standby

2017-03-20 Thread Craig Ringer
On 19 March 2017 at 22:12, Petr Jelinek wrote: > I am slightly worried about impact of the readTimeLineHistory() call but > I think it should be called so little that it should not matter. Pretty much my thinking too. > That brings us to the big patch 0003. > > I still don't like the "New in 10

Re: [HACKERS] Inadequate traces in TAP tests

2017-03-20 Thread Craig Ringer
On 20 March 2017 at 22:39, Alvaro Herrera wrote: > Stephen Frost wrote: > >> Is there any hope of getting a "quiet" mode, where all the "ok" lines >> aren't printed when things work..? > > Well, we currently have --verbose in PROVE_FLAGS. Maybe you can take it > out, or even add --quiet or --QUIE

Re: [HACKERS] logical replication apply to run with sync commit off by default

2017-03-20 Thread Petr Jelinek
On 18/03/17 13:31, Petr Jelinek wrote: > On 07/03/17 06:23, Petr Jelinek wrote: >> Hi, >> >> there has been discussion at the logical replication initial copy thread >> [1] about making apply work with sync commit off by default for >> performance reasons and adding option to change that per subscr

Re: [HACKERS] extended statistics: n-distinct

2017-03-20 Thread David Rowley
On 21 March 2017 at 08:02, Alvaro Herrera wrote: > Here is a closer to final version of the multivariate statistics series, > last posted at > https://www.postgresql.org/message-id/20170316222033.ncdi7nidah2gdzjx%40alvherre.pgsql I've made another pass over the patch. + A notice is issued i

Re: [HACKERS] increasing the default WAL segment size

2017-03-20 Thread David Steele
Hi Beena, On 3/20/17 2:07 PM, Beena Emerson wrote: Added check for the version, the SHOW command will be run only in v10 and above. Previous versions do not need this. I've just had the chance to have a look at this patch. This is not a complete review, just a test of something I've been cur

Re: [HACKERS] extended statistics: n-distinct

2017-03-20 Thread Alvaro Herrera
Alvaro Herrera wrote: > * I'm not terribly happy with the header organization. I think > VacAttrStats should be in its own (new) src/include/statistics/analyze.h > for example (which cleans up a bunch of existing stuff a bit) I tried this and it doesn't actually do any good. Patch attached, whi

Re: [HACKERS] Removing binaries

2017-03-20 Thread David Steele
On 3/20/17 3:40 PM, Jan de Visser wrote: On Monday, March 20, 2017 3:30:49 PM EDT Robert Haas wrote: On Sat, Mar 18, 2017 at 4:12 PM, Magnus Hagander wrote: createdb, dropdb - also not clear they're about postgres, more likely to be used by mistake but not that bad. That said, do they add any

Re: [HACKERS] Inadequate traces in TAP tests

2017-03-20 Thread Stephen Frost
Andrew, * Andrew Dunstan (andrew.duns...@2ndquadrant.com) wrote: > On 03/20/2017 10:25 AM, Craig Ringer wrote: > > I'd like to enable Carp's features to use confess for traces, and > > switch all use of die to that. We could learn a lot about > > unplanned-for test failures where a test script die

Re: [HACKERS] Inadequate traces in TAP tests

2017-03-20 Thread Andrew Dunstan
On 03/20/2017 10:25 AM, Craig Ringer wrote: > > > I'd like to enable Carp's features to use confess for traces, and > switch all use of die to that. We could learn a lot about > unplanned-for test failures where a test script dies rather than > failing a test if we used carp effectively. > > >

Re: [HACKERS] patch proposal

2017-03-20 Thread David Steele
Hi Venkata, On 2/28/17 11:59 PM, Venkata B Nagothi wrote: On Wed, Mar 1, 2017 at 1:14 AM, Venkata B Nagothi mailto:nag1...@gmail.com>> wrote: On Tue, Jan 31, 2017 at 6:49 AM, David Steele mailto:da...@pgmasters.net>> wrote: Do you know when those will be ready? Attached are bot

Re: [HACKERS] Logical replication existing data copy

2017-03-20 Thread Stephen Frost
Peter, * Peter Eisentraut (peter.eisentr...@2ndquadrant.com) wrote: > The current patch causes a failure in the pg_dump tests, because the > generated CREATE SUBSCRIPTION commands make connection attempts that > don't work. We have the pg_dump option --no-create-subscription-slots > for this, but

Re: [HACKERS] Logical replication existing data copy

2017-03-20 Thread Peter Eisentraut
The current patch causes a failure in the pg_dump tests, because the generated CREATE SUBSCRIPTION commands make connection attempts that don't work. We have the pg_dump option --no-create-subscription-slots for this, but I suppose we should expand that to --no-subscription-connect and use the new

Re: [HACKERS] WIP: Faster Expression Processing v4

2017-03-20 Thread Tom Lane
... is there a reason why resultnum for EEOP_ASSIGN_* steps is declared size_t and not just int? Since it's an array index, and one that certainly can't be bigger than AttrNumber, that seems rather confusing. regards, tom lane -- Sent via pgsql-hackers mailing list (pgs

Re: [HACKERS] Microvacuum support for Hash Index

2017-03-20 Thread Robert Haas
On Sat, Mar 18, 2017 at 4:35 AM, Amit Kapila wrote: > This version looks good to me. Committed. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscriptio

Re: [HACKERS] Removing binaries (was: createlang/droplang deprecated)

2017-03-20 Thread Jan de Visser
On Monday, March 20, 2017 3:30:49 PM EDT Robert Haas wrote: > On Sat, Mar 18, 2017 at 4:12 PM, Magnus Hagander wrote: > > createdb, dropdb - also not clear they're about postgres, more likely to > > be > > used by mistake but not that bad. That said, do they add any *value* > > beyond > > what yo

Re: [HACKERS] Removing binaries (was: createlang/droplang deprecated)

2017-03-20 Thread Robert Haas
On Sat, Mar 18, 2017 at 4:12 PM, Magnus Hagander wrote: > createdb, dropdb - also not clear they're about postgres, more likely to be > used by mistake but not that bad. That said, do they add any *value* beyond > what you can do with psql -c "CREATE DATABASE"? I don't really see one, so > I'd sug

Re: [HACKERS] logical replication access control patches

2017-03-20 Thread Petr Jelinek
On 20/03/17 13:32, Peter Eisentraut wrote: > On 3/18/17 09:31, Petr Jelinek wrote: >>> 0003 Add USAGE privilege for publications >>> >>> a way to control who can subscribe to a publication >>> >> Hmm IIUC this removes ability of REPLICATION role to subscribe to >> publications. I am not quite sure

Re: [HACKERS] increasing the default WAL segment size

2017-03-20 Thread Beena Emerson
Hello, PFA the updated patch. On Fri, Mar 17, 2017 at 6:40 AM, Robert Haas wrote: > On Tue, Mar 14, 2017 at 1:44 AM, Beena Emerson > wrote: > > Attached is the updated patch. It fixes the issues and also updates few > code > > comments. > > I did an initial readthrough of this patch tonight ju

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-03-20 Thread Robert Haas
On Mon, Mar 20, 2017 at 1:19 PM, Ashutosh Bapat wrote: >> That seems different than what I suggested and I'm not sure what the >> reason is for the difference? > > The patch adding macros IS_JOIN_REL() and IS_OTHER_REL() and changing > the code to use it will look quite odd by itself. We are not c

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-03-20 Thread Ashutosh Bapat
On Mon, Mar 20, 2017 at 10:26 PM, Robert Haas wrote: > On Mon, Mar 20, 2017 at 12:52 PM, Ashutosh Bapat > wrote: >>> Hmm. I would kind of like to move the IS_JOIN_REL() and >>> IS_OTHER_REL() stuff to the front of the series. In other words, I >>> propose that we add those macros first, each te

Re: [HACKERS] [PATCH] Removes uninitialized variable compiler warning

2017-03-20 Thread Tom Lane
Todd Sedano writes: > This patch removes a compiler warning. > warning: variable 'lenlemm' is uninitialized when used here > [-Wuninitialized] Hm, on what compiler? AFAICS, that parsetext() function hasn't changed meaningfully since 2007, and nobody complained of uninitialized-variable warnings

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-03-20 Thread Robert Haas
On Mon, Mar 20, 2017 at 12:52 PM, Ashutosh Bapat wrote: >> Hmm. I would kind of like to move the IS_JOIN_REL() and >> IS_OTHER_REL() stuff to the front of the series. In other words, I >> propose that we add those macros first, each testing for only the one >> kind of RelOptInfo that exists toda

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-03-20 Thread Ashutosh Bapat
> > On a further testing of this patch I find another case when it is > showing regression, the time taken with patch is around 160 secs and > without it is 125 secs. > Another minor thing to note that is planning time is almost twice with > this patch, though I understand that this is for scenario

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-03-20 Thread Ashutosh Bapat
> > Hmm. I would kind of like to move the IS_JOIN_REL() and > IS_OTHER_REL() stuff to the front of the series. In other words, I > propose that we add those macros first, each testing for only the one > kind of RelOptInfo that exists today, and change all the code to use > them. Then, when we ad

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-03-20 Thread Peter Geoghegan
On Sun, Mar 19, 2017 at 12:15 AM, Pavan Deolasee wrote: >> It seems like an important invariant for WARM is that any duplicate >> index values ought to have different TIDs (actually, it's a bit >> stricter than that, since btrecheck() cares about simple binary >> equality). > > Yes. I think in the

[HACKERS] [PATCH] Removes uninitialized variable compiler warning

2017-03-20 Thread Todd Sedano
This patch removes a compiler warning. warning: variable 'lenlemm' is uninitialized when used here [-Wuninitialized] This is my first commit to postgres. I've read through http://wiki.postgresql.org/wiki/Submitting_a_Patch, but I may have missed something. diff --git a/src/backend/tsearch/ts_pa

Re: [HACKERS] pg_dump emits ALTER TABLE ONLY partitioned_table

2017-03-20 Thread Stephen Frost
Amit, * Amit Langote (langote_amit...@lab.ntt.co.jp) wrote: > On 2017/02/17 22:32, Stephen Frost wrote: > > * Amit Langote (langote_amit...@lab.ntt.co.jp) wrote: > >> In certain cases, pg_dump's dumpTableSchema() emits a separate ALTER TABLE > >> command for those schema elements of a table that c

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-03-20 Thread Robert Haas
On Mon, Mar 20, 2017 at 12:07 PM, Rafia Sabih wrote: > On a further testing of this patch I find another case when it is > showing regression, the time taken with patch is around 160 secs and > without it is 125 secs. This is basically the same problem as before; the partitionwise case is doing t

Re: [HACKERS] Our feature change policy

2017-03-20 Thread David G. Johnston
On Mon, Mar 20, 2017 at 9:18 AM, Bruce Momjian wrote: > . #3 and #4 would need to be weighted depending on > whether choosing them would delay progress, e.g. it did delay progress > on standard-conforming strings, but the delay was determined to be > reasonable. > w.r.t. standard-conforming str

Re: [HACKERS] Create replication slot in pg_basebackup if requested and not yet present

2017-03-20 Thread Michael Banck
Hi, On Mon, Mar 20, 2017 at 02:42:32PM +0300, Arthur Zakirov wrote: > Also maybe it would be good if pg_basebackup had a way to drop created slot. > Although "drop slot" is not related with concept of automatically created > slots, it will good if user will have a way to drop slots. If you want t

Re: [HACKERS] WIP: Faster Expression Processing v4

2017-03-20 Thread Tom Lane
Andres Freund writes: > Additionally I added a regression test for the nearly entirely untested > nodeTidscan.c, after I'd broken it previously without noticing (thanks > Andreas). I went ahead and pushed this part, since it seemed pretty uncontroversial. I added a bit more stuff to get the LOC m

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-03-20 Thread Robert Haas
On Mon, Mar 20, 2017 at 9:44 AM, Ashutosh Bapat wrote: > Right. If we could use parent Vars to indicate parent Var or child Var > depending upon the context, a lot of memory issues would be solved; we > wouldn't need to translate a single expression. But I think that's not > straight forward. I ha

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-03-20 Thread Robert Haas
On Mon, Mar 20, 2017 at 9:44 AM, Ashutosh Bapat wrote: >> I believe it would also be best to include 0011's changes to >> adjust_appendrel_attrs_multilevel in 0001. > > The function needs to repeat the "adjustment" process for every > "other" relation (join or base) that it encounters, by testing

Re: [HACKERS] Our feature change policy

2017-03-20 Thread David G. Johnston
On Mon, Mar 20, 2017 at 8:57 AM, Stephen Frost wrote: > Tom, > > * Tom Lane (t...@sss.pgh.pa.us) wrote: > > Stephen Frost writes: > > > * Bruce Momjian (br...@momjian.us) wrote: > > >> 1. make the change now and mention it in the release notes > > >> 2. #1, but also provide backward compatibil

Re: [HACKERS] Our feature change policy

2017-03-20 Thread Bruce Momjian
On Mon, Mar 20, 2017 at 11:57:13AM -0400, Stephen Frost wrote: > Tom, > > * Tom Lane (t...@sss.pgh.pa.us) wrote: > > Stephen Frost writes: > > > * Bruce Momjian (br...@momjian.us) wrote: > > >> 1. make the change now and mention it in the release notes > > >> 2. #1, but also provide backward co

Re: [HACKERS] Partitioned tables and relfilenode

2017-03-20 Thread Robert Haas
On Fri, Mar 17, 2017 at 4:57 AM, Amit Langote wrote: >> Yes, but on the flip side, you're having to add code in a lot of >> places -- I think I counted 7 -- where you turn around and ignore >> those AppendRelInfos. > > Perhaps you were looking at the previous version with "minimal" appinfos > cont

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-03-20 Thread Alvaro Herrera
Pavan Deolasee wrote: > On Tue, Mar 14, 2017 at 7:17 AM, Alvaro Herrera > wrote: > > I didn't like this comment very much. But it's not necessary: you have > > already given relcache responsibility for setting rd_supportswarm. The > > only problem seems to be that you set it in RelationGetIndex

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-03-20 Thread Rafia Sabih
On Mon, Mar 20, 2017 at 8:21 AM, Robert Haas wrote: > On Fri, Mar 17, 2017 at 8:10 PM, Robert Haas wrote: >> While I was studying what you did with reparameterize_path_by_child(), >> I started to wonder whether reparameterize_path() doesn't need to >> start handling join paths. I think it only h

Re: [HACKERS] Our feature change policy

2017-03-20 Thread Stephen Frost
Tom, * Tom Lane (t...@sss.pgh.pa.us) wrote: > Stephen Frost writes: > > * Bruce Momjian (br...@momjian.us) wrote: > >> 1. make the change now and mention it in the release notes > >> 2. #1, but also provide backward compatibility for 5+ years > >> 3. mark the feature as deprecated and remove/c

Re: [HACKERS] Our feature change policy

2017-03-20 Thread Bruce Momjian
On Mon, Mar 20, 2017 at 11:40:34AM -0400, Tom Lane wrote: > Stephen Frost writes: > > * Bruce Momjian (br...@momjian.us) wrote: > >> 1. make the change now and mention it in the release notes > >> 2. #1, but also provide backward compatibility for 5+ years > >> 3. mark the feature as deprecated

Re: [HACKERS] Our feature change policy

2017-03-20 Thread Tom Lane
Stephen Frost writes: > * Bruce Momjian (br...@momjian.us) wrote: >> 1. make the change now and mention it in the release notes >> 2. #1, but also provide backward compatibility for 5+ years >> 3. mark the feature as deprecated and remove/change it in 5+ years >> 4. #3, but issue a warning for

Re: [HACKERS] [POC] A better way to expand hash indexes.

2017-03-20 Thread Mithun Cy
Hi Amit, Thanks for the review, On Mon, Mar 20, 2017 at 5:17 PM, Amit Kapila wrote: > idea could be to make hashm_spares a two-dimensional array > hashm_spares[32][4] where the first dimension will indicate the split > point and second will indicate the sub-split number. I am not sure > whether

Re: [HACKERS] WIP: Faster Expression Processing v4

2017-03-20 Thread Tom Lane
Andres Freund writes: > On 2017-03-15 20:09:03 -0400, Tom Lane wrote: >> I think it would be worth creating a README file giving an overview >> of how all of this patch is supposed to work. You also need to do a >> whole lot more work on the function-level comments. > I tried to improve upon bot

Re: [HACKERS] Our feature change policy

2017-03-20 Thread Stephen Frost
Bruce, * Bruce Momjian (br...@momjian.us) wrote: > On Sun, Mar 19, 2017 at 04:15:09PM -0400, Tom Lane wrote: > > Stephen Frost writes: > > > If we take your approach to its logical conclusion then we should be > > > planning to maintain all user-facing deprecated features for as long as > > > the

Re: [HACKERS] PinBuffer() no longer makes use of strategy

2017-03-20 Thread Alexander Korotkov
On Mon, Mar 20, 2017 at 6:09 PM, Teodor Sigaev wrote: > if (buf->usage_count < BM_MAX_USAGE_COUNT) >> if (BUF_STATE_GET_USAGECOUNT(buf_state) != BM_MAX_USAGE_COUNT) >> >> being prone to paranoia, I prefer the first, but I've seen both >> styles in >> the code so I don't know i

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-03-20 Thread Pavan Deolasee
On Tue, Mar 14, 2017 at 7:17 AM, Alvaro Herrera wrote: > > @@ -234,6 +236,21 @@ index_beginscan(Relation heapRelation, > > scan->heapRelation = heapRelation; > > scan->xs_snapshot = snapshot; > > > > + /* > > + * If the index supports recheck, make sure that index tuple is >

Re: [HACKERS] PinBuffer() no longer makes use of strategy

2017-03-20 Thread Teodor Sigaev
if (buf->usage_count < BM_MAX_USAGE_COUNT) if (BUF_STATE_GET_USAGECOUNT(buf_state) != BM_MAX_USAGE_COUNT) being prone to paranoia, I prefer the first, but I've seen both styles in the code so I don't know if it's worth futzing with. Ok, let's be paranoic and do this same way

Re: [HACKERS] [Proposal] Make the optimiser aware of partitions ordering

2017-03-20 Thread Ronan Dunklau
On lundi 20 mars 2017 15:52:03 CET Robert Haas wrote: > On Mon, Mar 20, 2017 at 6:31 AM, Ronan Dunklau wrote: > > With range partitioning, we guarantee that each partition contains non- > > overlapping values. Since we know the range allowed for each partition, it > > is possible to sort them acc

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-03-20 Thread Pavan Deolasee
On Wed, Mar 15, 2017 at 12:46 AM, Alvaro Herrera wrote: > Pavan Deolasee wrote: > > On Tue, Mar 14, 2017 at 7:17 AM, Alvaro Herrera < > alvhe...@2ndquadrant.com> > > wrote: > > > > I have already commented about the executor involvement in btrecheck(); > > > that doesn't seem good. I previously

Re: [HACKERS] Re: [COMMITTERS] pgsql: Improve pg_dump regression tests and code coverage

2017-03-20 Thread Stephen Frost
Tom, * Tom Lane (t...@sss.pgh.pa.us) wrote: > New tests are not zero-cost; they create a distributed burden on the > buildfarm and, by increasing the buildfarm cycle time, slow down feedback > to authors of subsequent patches. So I'm very much not on board with > any argument that "more tests are

Re: [HACKERS] Re: [COMMITTERS] pgsql: Improve pg_dump regression tests and code coverage

2017-03-20 Thread Tom Lane
Andres Freund writes: > On 2017-03-20 10:35:15 -0400, Stephen Frost wrote: >> I continue to be of the opinion that this entire discussion is quite >> flipped from how we really should be running things- adding regression >> tests to improve code coverage, particularly when they're simply adding >>

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-03-20 Thread Pavan Deolasee
On Mon, Mar 20, 2017 at 8:11 PM, Robert Haas wrote: > On Sun, Mar 19, 2017 at 3:05 AM, Pavan Deolasee > wrote: > > On Thu, Mar 16, 2017 at 12:53 PM, Robert Haas > wrote: > > >> > >> /me scratches head. > >> > >> Aren't pre-warm and post-warm just (better) names for blue and red? > >> > > > > Ye

Re: [HACKERS] [Proposal] Make the optimiser aware of partitions ordering

2017-03-20 Thread Robert Haas
On Mon, Mar 20, 2017 at 6:31 AM, Ronan Dunklau wrote: > With range partitioning, we guarantee that each partition contains non- > overlapping values. Since we know the range allowed for each partition, it is > possible to sort them according to the partition key (as is done already for > looking u

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-03-20 Thread Robert Haas
On Sun, Mar 19, 2017 at 3:05 AM, Pavan Deolasee wrote: > On Thu, Mar 16, 2017 at 12:53 PM, Robert Haas wrote: >> On Wed, Mar 15, 2017 at 3:44 PM, Pavan Deolasee >> wrote: >> > I couldn't find a better way without a lot of complex infrastructure. >> > Even >> > though we now have ability to mark

Re: [HACKERS] Inadequate traces in TAP tests

2017-03-20 Thread Stephen Frost
Alvaro, * Alvaro Herrera (alvhe...@2ndquadrant.com) wrote: > Stephen Frost wrote: > > Is there any hope of getting a "quiet" mode, where all the "ok" lines > > aren't printed when things work..? > > Well, we currently have --verbose in PROVE_FLAGS. Maybe you can take it > out, or even add --quie

Re: [HACKERS] [COMMITTERS] pgsql: Improve pg_dump regression tests and code coverage

2017-03-20 Thread Robert Haas
On Mon, Mar 20, 2017 at 10:35 AM, Stephen Frost wrote: > To be clear, I am not asking for any kind of special exception for > myself. > > I continue to be of the opinion that this entire discussion is quite > flipped from how we really should be running things- adding regression > tests to improve

Re: [HACKERS] Re: [COMMITTERS] pgsql: Improve pg_dump regression tests and code coverage

2017-03-20 Thread Andres Freund
On 2017-03-20 10:35:15 -0400, Stephen Frost wrote: > Robert, > > * Robert Haas (robertmh...@gmail.com) wrote: > > I'm glad that you are working on fixing > > pg_dump bugs and improving test coverage, but my gladness about that > > does not extend to thinking that the processes which other people >

Re: [HACKERS] Inadequate traces in TAP tests

2017-03-20 Thread Alvaro Herrera
Stephen Frost wrote: > Is there any hope of getting a "quiet" mode, where all the "ok" lines > aren't printed when things work..? Well, we currently have --verbose in PROVE_FLAGS. Maybe you can take it out, or even add --quiet or --QUIET (see the prove(1) manpage). -- Álvaro Herrera

[HACKERS] Our feature change policy

2017-03-20 Thread Bruce Momjian
On Sun, Mar 19, 2017 at 04:15:09PM -0400, Tom Lane wrote: > Stephen Frost writes: > > If we take your approach to its logical conclusion then we should be > > planning to maintain all user-facing deprecated features for as long as > > there is a version where it exists in a non-deprecated fashion,

[HACKERS] Re: [COMMITTERS] pgsql: Improve pg_dump regression tests and code coverage

2017-03-20 Thread Stephen Frost
Robert, * Robert Haas (robertmh...@gmail.com) wrote: > I'm glad that you are working on fixing > pg_dump bugs and improving test coverage, but my gladness about that > does not extend to thinking that the processes which other people > follow for their work should be waived for yours. Sorry. To

Re: [HACKERS] Inadequate traces in TAP tests

2017-03-20 Thread Stephen Frost
Alvaro, * Alvaro Herrera (alvhe...@2ndquadrant.com) wrote: > Stephen Frost wrote: > > * Andrew Dunstan (andrew.duns...@2ndquadrant.com) wrote: > > > ISTM that the test setup and breakdown code, both in individual tests > > > and in PostgresNode.pm should be liberally sprinkled with diag() calls >

Re: [HACKERS] free space map and visibility map

2017-03-20 Thread Robert Haas
On Sat, Mar 18, 2017 at 5:42 PM, Jeff Janes wrote: > Isn't HEAP2_CLEAN only issued before an intended HOT update? (Which then > can't leave the block as all visible or all frozen). I think the issue is > here is HEAP2_VISIBLE or HEAP2_FREEZE_PAGE. Am I reading this correctly, > that neither of

Re: [HACKERS] Inadequate traces in TAP tests

2017-03-20 Thread Andrew Dunstan
On 03/20/2017 10:08 AM, Tom Lane wrote: > I am *absolutely* not in favor of adding anything to the scripts' routine > output, because it will just make this problem worse by bloating the > buildfarm logs even more. What I'd like to see is for the scripts to > always report something along the li

Re: [HACKERS] Inadequate traces in TAP tests

2017-03-20 Thread Craig Ringer
On 20 Mar. 2017 22:10, "Tom Lane" wrote: FWIW, the problem I've got with the TAP tests is that when one fails in the buildfarm, you've got to dig through megabytes of all-alike-looking output just to try to determine which one failed; and once you do, you still know nothing because the script ou

Re: [HACKERS] [COMMITTERS] pgsql: Improve pg_dump regression tests and code coverage

2017-03-20 Thread Robert Haas
On Mon, Mar 20, 2017 at 9:30 AM, Stephen Frost wrote: > While I certainly agree with that when it comes to new features, changes > in work-flow, bug fixes and other things, I'm really not sure that > requiring posting to the list and waiting for responses every time > someone wants to add some reg

Re: [HACKERS] [PATCH] Incremental sort

2017-03-20 Thread Heikki Linnakangas
On 03/20/2017 11:33 AM, Alexander Korotkov wrote: Please, find rebased patch in the attachment. I had a quick look at this. * I'd love to have an explanation of what an Incremental Sort is, in the file header comment for nodeIncrementalSort.c. * I didn't understand the maxMem stuff in tuple

Re: [HACKERS] Inadequate traces in TAP tests

2017-03-20 Thread Alvaro Herrera
Stephen Frost wrote: > Andrew, > * Andrew Dunstan (andrew.duns...@2ndquadrant.com) wrote: > > ISTM that the test setup and breakdown code, both in individual tests > > and in PostgresNode.pm should be liberally sprinkled with diag() calls > > to make it easier to narrow down errors.. > > While

Re: [HACKERS] Re: [COMMITTERS] pgsql: Add TAP tests for password-based authentication methods.

2017-03-20 Thread Tom Lane
Heikki Linnakangas writes: > On 03/20/2017 02:32 AM, Peter Eisentraut wrote: >> This is missing an entry for tmp_check/ in .gitignore. But maybe we can >> do that globally instead of repeating it in every directory? > If we could also handle results and log globally, that would be nice. > But I

Re: [HACKERS] Inadequate traces in TAP tests

2017-03-20 Thread Tom Lane
Stephen Frost writes: > * Andrew Dunstan (andrew.duns...@2ndquadrant.com) wrote: >> ISTM that the test setup and breakdown code, both in individual tests >> and in PostgresNode.pm should be liberally sprinkled with diag() calls >> to make it easier to narrow down errors.. > While I'm generally i

Re: [HACKERS] logical decoding of two-phase transactions

2017-03-20 Thread Stas Kelvich
> On 20 Mar 2017, at 16:39, Craig Ringer wrote: > > On 20 March 2017 at 20:57, Stas Kelvich wrote: >> >>> On 20 Mar 2017, at 15:17, Craig Ringer wrote: >>> I thought about having special field (or reusing one of the existing fields) in snapshot struct to force filtering xmax

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-03-20 Thread Ashutosh Bapat
On Sat, Mar 18, 2017 at 5:40 AM, Robert Haas wrote: > On Fri, Mar 17, 2017 at 9:15 AM, Ashutosh Bapat > wrote: >> This set of patches fixes both of those things. > > 0001 changes the purpose of a function and then 0007 renames it. It > would be better to include the renaming in 0001 so that you'

Re: [HACKERS] logical decoding of two-phase transactions

2017-03-20 Thread Craig Ringer
On 20 March 2017 at 20:57, Stas Kelvich wrote: > >> On 20 Mar 2017, at 15:17, Craig Ringer wrote: >> >>> I thought about having special field (or reusing one of the existing fields) >>> in snapshot struct to force filtering xmax > snap->xmax or xmin = snap->xmin >>> as Petr suggested. Then this l

[HACKERS] Re: [COMMITTERS] pgsql: Improve pg_dump regression tests and code coverage

2017-03-20 Thread Stephen Frost
Robert, * Robert Haas (robertmh...@gmail.com) wrote: > On Mon, Mar 20, 2017 at 8:33 AM, Stephen Frost wrote: > > * Robert Haas (robertmh...@gmail.com) wrote: > >> So was this 3340 line patch posted or discussed anyplace before it got > >> committed? > > > > I've mentioned a few times that I'm wor

Re: [HACKERS] pageinspect and hash indexes

2017-03-20 Thread Ashutosh Sharma
On Mon, Mar 20, 2017 at 9:31 AM, Amit Kapila wrote: > On Sat, Mar 18, 2017 at 5:13 PM, Ashutosh Sharma > wrote: >> On Sat, Mar 18, 2017 at 1:34 PM, Amit Kapila wrote: >>> On Sat, Mar 18, 2017 at 12:12 AM, Ashutosh Sharma >>> wrote: On Fri, Mar 17, 2017 at 10:54 PM, Jeff Janes wrote: >>>

Re: [HACKERS] Inadequate traces in TAP tests

2017-03-20 Thread Stephen Frost
Andrew, * Andrew Dunstan (andrew.duns...@2ndquadrant.com) wrote: > If you look at this failure case > > you see: > > t/002_pg_dump.1..4449 > # Looks like your test died before it could ou

  1   2   >