Re: [HACKERS] Parallel Index Scans

2017-01-17 Thread Rahila Syed
>+ /* Check if the scan for current scan keys is finished */ >+ if (so->arrayKeyCount < btscan->btps_arrayKeyCount) >+ *status = false; >I didn't clearly understand, in which scenario the arrayKeyCount is less >than btps_arrayKeyCount? Consider following array scan keys select * from test2 where

Re: [HACKERS] RustgreSQL

2017-01-17 Thread David Fetter
On Mon, Jan 09, 2017 at 12:51:43PM -0500, Robert Haas wrote: > On Sun, Jan 8, 2017 at 4:59 AM, Gavin Flower > wrote: > >> Is this completely unrealistic or is it carved in stone PostgreSQL will > >> always be a C project forever and ever? > >> > > From my very limited understanding, PostgreSQL is

Re: [HACKERS] pg_hba_file_settings view patch

2017-01-17 Thread Haribabu Kommi
On Tue, Jan 17, 2017 at 5:24 PM, Michael Paquier wrote: > On Tue, Jan 17, 2017 at 10:19 AM, Haribabu Kommi > wrote: > > On Tue, Jan 10, 2017 at 6:35 PM, Michael Paquier < > michael.paqu...@gmail.com> > > wrote: > >> +/* LDAP supports 10 currently, keep this well above the most any > >> method ne

Re: [HACKERS] Cache Hash Index meta page.

2017-01-17 Thread Mithun Cy
On Tue, Jan 17, 2017 at 10:07 AM, Amit Kapila wrote: > 1. > (a) I think you can retain the previous comment or modify it slightly. > Just removing the whole comment and replacing it with a single line > seems like a step backward. -- Fixed, Just modified to say it > (b) Another somewhat bigger p

Re: [HACKERS] Gather Merge

2017-01-17 Thread Rushabh Lathia
On Tue, Jan 17, 2017 at 6:44 PM, Amit Kapila wrote: > On Tue, Jan 17, 2017 at 5:19 PM, Rushabh Lathia > wrote: > > > > > > > > Here is the benchmark number which I got with the latest (v6) patch: > > > > - max_worker_processes = DEFAULT (8) > > - max_parallel_workers_per_gather = 4 > > - Cold ca

Re: [HACKERS] Password identifiers, protocol aging and SCRAM protocol

2017-01-17 Thread Michael Paquier
On Tue, Dec 20, 2016 at 10:47 AM, Michael Paquier wrote: > And Heikki has mentioned me that he'd prefer not having an extra > dependency for the normalization, which is LGPL-licensed by the way. > So I have looked at the SASLprep business to see what should be done > to get a complete implementati

Re: [WIP] RE: [HACKERS] DECLARE STATEMENT setting up a connection in ECPG

2017-01-17 Thread Michael Paquier
On Fri, Jan 6, 2017 at 6:10 PM, Ideriha, Takeshi wrote: > Hi > Thank you for looking over my patch. > >> Thank you. Would it be possible for you to re-create the patch without the >> white-space changes? > > I'm sorry for adding redundant white-spaces. > Attached is a correct version. > >> I notic

Re: [HACKERS] Assignment of valid collation for SET operations on queries with UNKNOWN types.

2017-01-17 Thread Ashutosh Bapat
On Wed, Jan 18, 2017 at 10:55 AM, Michael Paquier wrote: > On Sun, Jan 8, 2017 at 10:55 AM, Tom Lane wrote: >> Ashutosh Bapat writes: >>> On Tue, Jan 3, 2017 at 5:57 PM, Rahila Syed wrote: Are you suggesting extending the patch to include coercing from unknown to text for all possible

Re: [HACKERS] Password identifiers, protocol aging and SCRAM protocol

2017-01-17 Thread Michael Paquier
On Wed, Jan 18, 2017 at 2:23 PM, Noah Misch wrote: > The latest versions document this precisely, but I agree with Peter's concern > about plain "scram". Suppose it's 2025 and PostgreSQL support SASL mechanisms > OAUTHBEARER, SCRAM-SHA-256, SCRAM-SHA-256-PLUS, and SCRAM-SHA3-512. What > should t

Re: [HACKERS] Assignment of valid collation for SET operations on queries with UNKNOWN types.

2017-01-17 Thread Michael Paquier
On Sun, Jan 8, 2017 at 10:55 AM, Tom Lane wrote: > Ashutosh Bapat writes: >> On Tue, Jan 3, 2017 at 5:57 PM, Rahila Syed wrote: >>> Are you suggesting extending the patch to include coercing from unknown to >>> text for all possible cases where a column of unknown type is being created? > >> I g

Re: [HACKERS] Password identifiers, protocol aging and SCRAM protocol

2017-01-17 Thread Noah Misch
On Tue, Nov 15, 2016 at 07:52:06AM +0900, Michael Paquier wrote: > On Sat, Nov 5, 2016 at 9:36 PM, Michael Paquier > wrote: > > On Sat, Nov 5, 2016 at 12:58 AM, Peter Eisentraut > > wrote: > > pg_hba.conf uses "scram" as keyword, but scram refers to a family of > > authentication methods. There

Re: [HACKERS] Protect syscache from bloating with negative cache entries

2017-01-17 Thread Michael Paquier
On Sat, Jan 14, 2017 at 9:36 AM, Tomas Vondra wrote: > On 01/14/2017 12:06 AM, Andres Freund wrote: >> On 2017-01-13 17:58:41 -0500, Tom Lane wrote: >>> >>> But, again, the catcache isn't the only source of per-process bloat >>> and I'm not even sure it's the main one. A more holistic approach >>

Re: [HACKERS] New SQL counter statistics view (pg_stat_sql)

2017-01-17 Thread Haribabu Kommi
On Sat, Jan 14, 2017 at 2:58 AM, Dilip Kumar wrote: > On Wed, Jan 11, 2017 at 7:47 PM, Dilip Kumar > wrote: > > +void > > +pgstat_count_sqlstmt(const char *commandTag) > > +{ > > + PgStat_SqlstmtEntry *htabent; > > + bool found; > > + > > + if (!pgstat_track_sql) > > + return > > > > Callers of

Re: [HACKERS] Parallel bitmap heap scan

2017-01-17 Thread Dilip Kumar
On Fri, Jan 13, 2017 at 6:36 PM, Dilip Kumar wrote: > > Please verify with the new patch. Patch 0001 and 0003 required to rebase on the latest head. 0002 is still the same. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com 0001-opt-parallelcost-refactoring-v11.patch Descriptio

Re: [HACKERS] PSQL commands: \quit_if, \quit_unless

2017-01-17 Thread Michael Paquier
On Wed, Jan 18, 2017 at 3:24 AM, Robert Haas wrote: > On Sat, Jan 14, 2017 at 12:22 AM, Tom Lane wrote: >> >> $ cat loop.sql >> \if :x < 1000 >> \echo :x >> \set x :x + 1 >> \include loop.sql >> \fi >> $ psql --set x=0 -f loop.sql >> >> Somebody is going to think of that workaround for not

Re: [HACKERS] DROP FUNCTION of multiple functions

2017-01-17 Thread Michael Paquier
On Wed, Jan 18, 2017 at 5:26 AM, Peter Eisentraut wrote: > On 1/10/17 1:52 AM, Michael Paquier wrote: >> I don't see any problems with 0001. > > I was wondering, should we rename funcname -> name, and funcargs -> > args, or perhaps the whole FuncWithArgs struct, so there is no confusion > when use

Re: [HACKERS] jsonb_delete with arrays

2017-01-17 Thread Michael Paquier
On Tue, Jan 17, 2017 at 8:45 PM, Magnus Hagander wrote: > On Tue, Jan 17, 2017 at 8:25 AM, Michael Paquier > wrote: >> On Sun, Dec 18, 2016 at 1:27 AM, Dmitry Dolgov <9erthali...@gmail.com> >> wrote: >> > * use variadic arguments for `jsonb_delete_array`. For rare cases, when >> > someone decides

Re: [HACKERS] PoC: Grouped base relation

2017-01-17 Thread Ashutosh Bapat
On Tue, Jan 17, 2017 at 10:07 PM, Antonin Houska wrote: > Ashutosh Bapat wrote: >> [... snip ]] >> >> This all works well, as long as the aggregate is "summing" something >> across rows. The method doesn't work when aggregation is say >> "multiplying" across the rows or "concatenating" across the

Re: [HACKERS] Patch to implement pg_current_logfile() function

2017-01-17 Thread Michael Paquier
On Wed, Jan 18, 2017 at 11:36 AM, Karl O. Pinc wrote: > On Wed, 18 Jan 2017 11:08:25 +0900 > Michael Paquier wrote: > >> Our ideas rather map here, ERRCODE_INTERNAL_ERROR would be adapted for >> this situation. Do any of you want to give it a shot or should I? > > You're welcome to it. What do y

Re: [HACKERS] Re: Clarifying "server starting" messaging in pg_ctl start without --wait

2017-01-17 Thread Peter Eisentraut
On 1/17/17 8:35 PM, Michael Paquier wrote: > Yeah, it seems to me that we are likely looking for a wait mode saying > to exit pg_ctl once Postgres is happily rejecting connections, because > that means that it is up and that it is sorting out something first > before accepting them. This would basi

Re: [HACKERS] [BUGS] Bug in Physical Replication Slots (at least 9.5)?

2017-01-17 Thread Michael Paquier
On Tue, Jan 17, 2017 at 7:36 PM, Kyotaro HORIGUCHI wrote: > I managed to reproduce this. A little tweak as the first patch > lets the standby to suicide as soon as walreceiver sees a > contrecord at the beginning of a segment. Good idea. > I believe that a continuation record cannot be span over

Re: [HACKERS] [WIP]Vertical Clustered Index (columnar store extension)

2017-01-17 Thread Peter Eisentraut
On 12/29/16 10:55 PM, Haribabu Kommi wrote: > Fujitsu was interested in developing a columnar storage extension with > minimal > changes the server backend. > > The columnar store is implemented as an extension using index access > methods. > This can be easily enhanced with pluggable storage meth

Re: [HACKERS] [PROPOSAL] Temporal query processing with range types

2017-01-17 Thread Peter Eisentraut
On 1/13/17 9:22 AM, Peter Moser wrote: > The goal of temporal aligners and normalizers is to split ranges to allow a > reduction from temporal queries to their non-temporal counterparts. > Splitting > ranges is necessary for temporal query processing. Temporal aligners and > normalizer may then be

Re: [HACKERS] postgres_fdw bug in 9.6

2017-01-17 Thread Etsuro Fujita
On 2017/01/16 11:38, Etsuro Fujita wrote: On 2017/01/14 6:39, Jeff Janes wrote: I do get a compiler warning: foreign.c: In function 'CreateLocalJoinPath': foreign.c:832: warning: implicit declaration of function 'pathkeys_contained_in' Will fix. Done. Attached is the new version. I also

Re: [HACKERS] Patch to implement pg_current_logfile() function

2017-01-17 Thread Karl O. Pinc
On Wed, 18 Jan 2017 11:08:25 +0900 Michael Paquier wrote: > Our ideas rather map here, ERRCODE_INTERNAL_ERROR would be adapted for > this situation. Do any of you want to give it a shot or should I? You're welcome to it. Karl Free Software: "You don't pay back, you pay forward."

Re: [HACKERS] Patch to implement pg_current_logfile() function

2017-01-17 Thread Michael Paquier
On Wed, Jan 18, 2017 at 7:36 AM, Karl O. Pinc wrote: > Maybe. It's not user-supplied data that's corrupted but it is > PG generated data which is generated for and supplied to the user. > I just looked at all uses of XX001 and it is true that they all > involve corruption of user-supplied data. >

Re: [HACKERS] Patch to implement pg_current_logfile() function

2017-01-17 Thread Michael Paquier
On Wed, Jan 18, 2017 at 3:06 AM, Gilles Darold wrote: > This have already been discuted previously in this thread, one of my > previous patch version has implemented this behavior but we decide that > what we really want is to be able to use the function using the > following simple query: > >

Re: [HACKERS] increasing the default WAL segment size

2017-01-17 Thread Michael Paquier
On Tue, Jan 17, 2017 at 7:19 PM, Beena Emerson wrote: > PFA the patch with the documentation included. It is usually better to keep doc lines under control of 72-80 characters if possible. + /* column 1: Wal segment size */ + len = strlen(value); + pq_sendint(&buf, len, 4); + pq_sendbyte

Re: [HACKERS] Re: Clarifying "server starting" messaging in pg_ctl start without --wait

2017-01-17 Thread Michael Paquier
On Wed, Jan 18, 2017 at 10:35 AM, Michael Paquier wrote: > On Wed, Jan 18, 2017 at 7:31 AM, Stephen Frost wrote: >> Perhaps we need a way for pg_ctl to realize a cold-standby case and >> throw an error or warning if --wait is specified then, but that hardly >> seems like the common use-case. It

Re: [HACKERS] Re: Clarifying "server starting" messaging in pg_ctl start without --wait

2017-01-17 Thread Michael Paquier
On Wed, Jan 18, 2017 at 7:31 AM, Stephen Frost wrote: > Perhaps we need a way for pg_ctl to realize a cold-standby case and > throw an error or warning if --wait is specified then, but that hardly > seems like the common use-case. It also wouldn't make any sense to have > anything in the init sys

Re: [HACKERS] Parallel Index Scans

2017-01-17 Thread Haribabu Kommi
On Mon, Jan 16, 2017 at 11:11 PM, Amit Kapila wrote: > > Changed as per suggestion. > > > I have also rebased the optimizer/executor support patch > (parallel_index_opt_exec_support_v4.patch) and added a test case in > it. Thanks for the patch. Here are comments found during review. parallel_i

Re: [HACKERS] PoC: Grouped base relation

2017-01-17 Thread Tomas Vondra
On 01/17/2017 08:05 PM, Antonin Houska wrote: [ Trying to respond to both Tomas and David. I'll check tomorrow if anything else of the thread needs my comment. ] Tomas Vondra wrote: On 01/17/2017 12:42 AM, David Rowley wrote: On 10 January 2017 at 06:56, Antonin Houska wrote: I've been t

Re: [HACKERS] [COMMITTERS] pgsql: Generate fmgr prototypes automatically

2017-01-17 Thread Tom Lane
Alvaro Herrera writes: > Alvaro Herrera wrote: >> It is possible to replace many occurrences of builtins.h with >> fmgrprotos.h. I just tried this >> git grep -l 'include.*utils/builtins.h' -- *.c | xargs perl -pi -e >> 's{utils/builtins.h}{utils/fmgrprotos.h}' >> There's a large number of chang

Re: [HACKERS] Patch to implement pg_current_logfile() function

2017-01-17 Thread Karl O. Pinc
On Tue, 17 Jan 2017 23:00:43 +0100 Gilles Darold wrote: > Le 17/01/2017 à 19:58, Karl O. Pinc a écrit : > > On Tue, 17 Jan 2017 19:06:22 +0100 > > Gilles Darold wrote: > > > >> Le 17/01/2017 à 03:22, Michael Paquier a écrit : > >>> On Tue, Jan 17, 2017 at 2:21 AM, Karl O. Pinc > >>> wrote:

Re: [HACKERS] Re: Clarifying "server starting" messaging in pg_ctl start without --wait

2017-01-17 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: > On Tue, Jan 17, 2017 at 4:46 PM, Stephen Frost wrote: > >> But what if we're restarting after, say, rebooting? Then there's > >> nobody to see the progress messages, perhaps. The system just seems > >> to take an eternity to return to the usual runl

Re: [HACKERS] Re: [COMMITTERS] pgsql: Fix an assertion failure related to an exclusive backup.

2017-01-17 Thread Michael Paquier
On Tue, Jan 17, 2017 at 11:42 PM, Fujii Masao wrote: > With the patch, what happens if pg_stop_backup exits with an error > after removing backup_label file before resetting the backup state > to none? Removing the backup_label file is the last error that can happen during the time the callback i

Re: [HACKERS] Re: Clarifying "server starting" messaging in pg_ctl start without --wait

2017-01-17 Thread Robert Haas
On Tue, Jan 17, 2017 at 4:46 PM, Stephen Frost wrote: >> But what if we're restarting after, say, rebooting? Then there's >> nobody to see the progress messages, perhaps. The system just seems >> to take an eternity to return to the usual runlevel. > > Not unlike an fsck. Right. That's why peo

Re: [HACKERS] [PATCH] Rename pg_switch_xlog to pg_switch_wal

2017-01-17 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: > I'm OK with continuing to use "xlog" as the user-facing name for the > write-ahead log, and I am OK with switching to wal. But leaving > things in the halfway in-between state where they are right now seems > like a mess. It conveniences the people w

Re: [HACKERS] [PATCH] Rename pg_switch_xlog to pg_switch_wal

2017-01-17 Thread Robert Haas
On Mon, Jan 16, 2017 at 10:48 PM, Fujii Masao wrote: > If we do that, we should vote on all the "renaming" stuff, i.e., not only > function names but also program names like pg_receivexlog, directory names > like clog, option names like xlogdir option of initdb, return value names of > the functio

Re: [HACKERS] Patch to implement pg_current_logfile() function

2017-01-17 Thread Gilles Darold
Le 17/01/2017 à 19:58, Karl O. Pinc a écrit : > On Tue, 17 Jan 2017 19:06:22 +0100 > Gilles Darold wrote: > >> Le 17/01/2017 à 03:22, Michael Paquier a écrit : >>> On Tue, Jan 17, 2017 at 2:21 AM, Karl O. Pinc >>> wrote: On January 15, 2017 11:47:51 PM CST, Michael Paquier wrote: >

Re: pg_authid.rolpassword format (was Re: [HACKERS] Password identifiers, protocol aging and SCRAM protocol)

2017-01-17 Thread Peter Eisentraut
On 1/3/17 9:09 AM, Heikki Linnakangas wrote: > Since not everyone agrees with this approach, I split this patch into > two. The first patch refactors things, replacing the isMD5() function > with get_password_type(), without changing the representation of > pg_authid.rolpassword. That is hopeful

Re: [HACKERS] Re: Clarifying "server starting" messaging in pg_ctl start without --wait

2017-01-17 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: > On Tue, Jan 17, 2017 at 11:27 AM, Peter Eisentraut > wrote: > > On 1/15/17 11:40 PM, Fujii Masao wrote: > >> This change may confuse the users who run "pg_ctl start" to perform a crash > >> recovery, archive recovery and standby server (with hot_stand

Re: [HACKERS] Logical Replication WIP

2017-01-17 Thread Robert Haas
On Tue, Jan 17, 2017 at 11:15 AM, Petr Jelinek wrote: >> Is there anything stopping anyone from implementing it? > > No, just didn't seem priority for the functionality right now. Why is it OK for this to not support rename like everything else does? It shouldn't be more than a few hours of work

Re: [HACKERS] Re: Clarifying "server starting" messaging in pg_ctl start without --wait

2017-01-17 Thread Robert Haas
On Tue, Jan 17, 2017 at 11:27 AM, Peter Eisentraut wrote: > On 1/15/17 11:40 PM, Fujii Masao wrote: >> This change may confuse the users who run "pg_ctl start" to perform a crash >> recovery, archive recovery and standby server (with hot_standby=off) because >> "pg_ctl start" would not return so l

Re: [HACKERS] [COMMITTERS] pgsql: Generate fmgr prototypes automatically

2017-01-17 Thread Alvaro Herrera
Alvaro Herrera wrote: > It is possible to replace many occurrences of builtins.h with > fmgrprotos.h. I just tried this >git grep -l 'include.*utils/builtins.h' -- *.c | xargs perl -pi -e > 's{utils/builtins.h}{utils/fmgrprotos.h}' > There's a large number of changes that the oneliner produc

Re: [HACKERS] New SQL counter statistics view (pg_stat_sql)

2017-01-17 Thread Robert Haas
On Fri, Jan 13, 2017 at 10:58 AM, Dilip Kumar wrote: > If you execute the query with EXECUTE then commandTag will be EXECUTE > that way we will not show the actual query type, I mean all the > statements will get the common tag "EXECUTE". Somebody might say that's a feature, not a bug. But then

Re: [HACKERS] Too many autovacuum workers spawned during forced auto-vacuum

2017-01-17 Thread Robert Haas
On Fri, Jan 13, 2017 at 8:45 AM, Alvaro Herrera wrote: > Amit Khandekar wrote: >> In a server where autovacuum is disabled and its databases reach >> autovacuum_freeze_max_age limit, an autovacuum is forced to prevent >> xid wraparound issues. At this stage, when the server is loaded with a >> lot

Re: [HACKERS] Gather Merge

2017-01-17 Thread Peter Geoghegan
On Tue, Jan 17, 2017 at 4:26 AM, Rushabh Lathia wrote: > Another observation is, HashAggregate (case 1) is performs better compare to > GroupAggregate (case 2), but still it doesn't justify the cost difference of > those two. It may not be the only issue, or even the main issue, but I'm fairly su

Re: [HACKERS] [COMMITTERS] pgsql: Generate fmgr prototypes automatically

2017-01-17 Thread Alvaro Herrera
Tom Lane wrote: > Alternatively ... is there a specific reason why you chose to make > builtins.h the key inclusion file for this change, rather than having > callers include fmgrprotos.h directly? It seems like the stuff remaining > in builtins.h is just a laundry list of random utility function

Re: [HACKERS] generating fmgr prototypes automatically

2017-01-17 Thread Peter Eisentraut
On 1/5/17 12:24 PM, Pavel Stehule wrote: > I checked last set of patches and I didn't find any issue. > > There are no problems with patching, compilation and all regress tests > passed. > > I'll mark this patch as ready for commiter This has been committed. -- Peter Eisentraut h

Re: [HACKERS] DROP FUNCTION of multiple functions

2017-01-17 Thread Peter Eisentraut
On 1/10/17 1:52 AM, Michael Paquier wrote: > I don't see any problems with 0001. I was wondering, should we rename funcname -> name, and funcargs -> args, or perhaps the whole FuncWithArgs struct, so there is no confusion when used with operators? In 0002, the comment of > class_args/CreateOpCla

Re: [HACKERS] Implement targetlist SRFs using ROWS FROM() (was Changed SRF in targetlist handling)

2017-01-17 Thread Tom Lane
Andres Freund writes: > On 2017-01-17 13:43:38 -0500, Tom Lane wrote: >> I'm not convinced that that optimization is worth preserving, but if we >> keep it then ProjectSet isn't le mot juste here, any more than you'd want >> to rename Result to Project without changing its existing >> functionalit

Re: [HACKERS] PoC: Grouped base relation

2017-01-17 Thread Antonin Houska
[ Trying to respond to both Tomas and David. I'll check tomorrow if anything else of the thread needs my comment. ] Tomas Vondra wrote: > On 01/17/2017 12:42 AM, David Rowley wrote: > > On 10 January 2017 at 06:56, Antonin Houska wrote: > > I've been thinking about this aggtransmultifn and I'm

Re: [HACKERS] Patch to implement pg_current_logfile() function

2017-01-17 Thread Karl O. Pinc
On Tue, 17 Jan 2017 19:06:22 +0100 Gilles Darold wrote: > Le 17/01/2017 à 03:22, Michael Paquier a écrit : > > On Tue, Jan 17, 2017 at 2:21 AM, Karl O. Pinc > > wrote: > >> On January 15, 2017 11:47:51 PM CST, Michael Paquier > >> wrote: > >>> Also, I would rather see an ERROR returned to

Re: [HACKERS] Implement targetlist SRFs using ROWS FROM() (was Changed SRF in targetlist handling)

2017-01-17 Thread Andres Freund
On 2017-01-17 13:43:38 -0500, Tom Lane wrote: > Although ... looking closer at Andres' patch, the new node type *is* > channeling Result, in the sense that it might or might not have any input > plan. This probably traces to what I wrote in September: > > + * XXX Possibly-temporary hack: if th

Re: [HACKERS] Implement targetlist SRFs using ROWS FROM() (was Changed SRF in targetlist handling)

2017-01-17 Thread Tom Lane
Robert Haas writes: > On Tue, Jan 17, 2017 at 1:18 PM, Tom Lane wrote: >> Using Result for two completely different things is a wart though. If we >> had it to do over I think we'd define Result as a scan node that produces >> rows from no input, and create a separate Project node for the case o

Re: [HACKERS] PSQL commands: \quit_if, \quit_unless

2017-01-17 Thread Robert Haas
On Sat, Jan 14, 2017 at 12:22 AM, Tom Lane wrote: > > $ cat loop.sql > \if :x < 1000 > \echo :x > \set x :x + 1 > \include loop.sql > \fi > $ psql --set x=0 -f loop.sql > > Somebody is going to think of that workaround for not having loops, and > then whine about how psql runs out of file de

Re: [HACKERS] Implement targetlist SRFs using ROWS FROM() (was Changed SRF in targetlist handling)

2017-01-17 Thread Robert Haas
On Tue, Jan 17, 2017 at 1:18 PM, Tom Lane wrote: > Andres Freund writes: >> I'd not have gone for SetResult if we didn't already have Result. I'm >> not super happy ending up having Project in ProjectSet but not in the >> Result that end up doing the majority of the projection. But eh, we can >

Re: [HACKERS] Implement targetlist SRFs using ROWS FROM() (was Changed SRF in targetlist handling)

2017-01-17 Thread Tom Lane
Andres Freund writes: > I'd not have gone for SetResult if we didn't already have Result. I'm > not super happy ending up having Project in ProjectSet but not in the > Result that end up doing the majority of the projection. But eh, we can > live with it. Using Result for two completely differe

Re: [HACKERS] patch: function xmltable

2017-01-17 Thread Pavel Stehule
2017-01-16 23:51 GMT+01:00 Alvaro Herrera : > Given > https://www.postgresql.org/message-id/20170116210019. > a3glfwspg5lnf...@alap3.anarazel.de > which is going to heavily change how the executor works in this area, I > am returning this patch to you again. I would like a few rather minor > chan

Re: [HACKERS] Patch to implement pg_current_logfile() function

2017-01-17 Thread Gilles Darold
Le 17/01/2017 à 03:22, Michael Paquier a écrit : > On Tue, Jan 17, 2017 at 2:21 AM, Karl O. Pinc wrote: >> On January 15, 2017 11:47:51 PM CST, Michael Paquier >> wrote: >>> On Mon, Jan 16, 2017 at 1:14 PM, Karl O. Pinc wrote: >>> With all those issues fixed, I finish with the attached, that I

Re: [HACKERS] Implement targetlist SRFs using ROWS FROM() (was Changed SRF in targetlist handling)

2017-01-17 Thread Andres Freund
Hi, On 2017-01-17 12:52:20 -0500, Tom Lane wrote: > Robert Haas writes: > > On Mon, Jan 16, 2017 at 2:13 PM, Tom Lane wrote: > >> "Srf" is ugly as can be, and unintelligible. SetResult might be OK. > > > The operation we're performing here, IIUC, is projection. SetResult > > lacks a verb, alth

Re: [HACKERS] Implement targetlist SRFs using ROWS FROM() (was Changed SRF in targetlist handling)

2017-01-17 Thread Robert Haas
On Tue, Jan 17, 2017 at 12:52 PM, Tom Lane wrote: > Robert Haas writes: >> On Mon, Jan 16, 2017 at 2:13 PM, Tom Lane wrote: >>> "Srf" is ugly as can be, and unintelligible. SetResult might be OK. > >> The operation we're performing here, IIUC, is projection. SetResult >> lacks a verb, although

Re: [HACKERS] Parallel Index Scans

2017-01-17 Thread Robert Haas
On Mon, Jan 16, 2017 at 7:11 AM, Amit Kapila wrote: > Fixed. Thanks for the update. Some more comments: It shouldn't be necessary for MultiExecBitmapIndexScan to modify the IndexScanDesc. That seems really broken. If a parallel scan isn't supported here (and I'm sure that's the case right now

Re: [HACKERS] Implement targetlist SRFs using ROWS FROM() (was Changed SRF in targetlist handling)

2017-01-17 Thread Tom Lane
Robert Haas writes: > On Mon, Jan 16, 2017 at 2:13 PM, Tom Lane wrote: >> "Srf" is ugly as can be, and unintelligible. SetResult might be OK. > The operation we're performing here, IIUC, is projection. SetResult > lacks a verb, although Set could be confused with one; someone might > think thi

Re: [HACKERS] Improving RLS planning

2017-01-17 Thread Stephen Frost
Tom, * Tom Lane (t...@sss.pgh.pa.us) wrote: > Stephen Frost writes: > > * Tom Lane (t...@sss.pgh.pa.us) wrote: > >> Here's an updated version of the RLS planning patch that gets rid of > >> the incorrect interaction with Query.hasRowSecurity and adjusts > >> terminology as agreed. > > > I've spe

Re: [HACKERS] Packages: Again

2017-01-17 Thread Robert Haas
On Fri, Jan 13, 2017 at 7:24 PM, Peter Geoghegan wrote: > MERGE isn't UPSERT, and isn't even in competition with UPSERT as a > feature. I've written reams of text explaining why this is so in > precise detail, ... No matter how much text you write, I doubt that I will ever believe that statement.

Re: [HACKERS] move collation import to backend

2017-01-17 Thread Peter Eisentraut
On 1/9/17 10:04 PM, Euler Taveira wrote: > On 18-12-2016 18:30, Peter Eisentraut wrote: >> Updated patch with that fix. >> > Peter, I reviewed and improved your patch. > > * I document the new function. Since collation is a database object, I > chose "Database Object Management Functions" section.

Re: [HACKERS] Implement targetlist SRFs using ROWS FROM() (was Changed SRF in targetlist handling)

2017-01-17 Thread Robert Haas
On Mon, Jan 16, 2017 at 2:13 PM, Tom Lane wrote: > Andres Freund writes: >> That worked quite well. So we have a few questions, before I clean this >> up: > >> - For now the node is named 'Srf' both internally and in explain - not >> sure if we want to make that something longer/easier to unde

Re: [HACKERS] Hash support for grouping sets

2017-01-17 Thread Robert Haas
On Mon, Jan 16, 2017 at 10:59 AM, Finnerty, Jim wrote: > The ability to exploit hashed aggregation within sorted groups, when the > order of the input stream can be exploited this way, is potentially a useful > way to improve aggregation performance more generally. This would > potentially be

Re: [HACKERS] Declarative partitioning - another take

2017-01-17 Thread Robert Haas
On Mon, Jan 16, 2017 at 4:09 AM, Amit Langote wrote: > The problem is that whereas the SlotValueDescription that we build to show > in the error message should be based on the tuple that was passed to > ExecInsert() or whatever NextCopyFrom() returned for CopyFrom() to > process, it might fail to

Re: [HACKERS] [COMMITTERS] pgsql: Fix cardinality estimates for parallel joins.

2017-01-17 Thread Robert Haas
On Mon, Jan 16, 2017 at 7:23 AM, Amit Kapila wrote: > On Sat, Jan 14, 2017 at 12:07 AM, Robert Haas wrote: >> Fix cardinality estimates for parallel joins. >> > > + /* > +* In the case of a parallel plan, the row count needs to represent > +* the number of tuples processed p

Re: [HACKERS] [PATCH] Rename pg_switch_xlog to pg_switch_wal

2017-01-17 Thread Stephen Frost
Peter, * Peter Eisentraut (peter.eisentr...@2ndquadrant.com) wrote: > On 1/13/17 10:18 AM, Stephen Frost wrote: > > Certainly, check_postgres is going to have to be changed to address this > > and, unsurprisingly, it's already had to address a variety of major > > version differences that have bee

Re: [HACKERS] Logical Replication WIP

2017-01-17 Thread Petr Jelinek
On 17/01/17 17:11, Peter Eisentraut wrote: > On 1/15/17 1:48 PM, Petr Jelinek wrote: >> It's meant to decouple the synchronous commit setting for logical >> replication workers from the one set for normal clients. Now that we >> have owners for subscription and subscription runs as that owner, mayb

Re: [HACKERS] PoC: Grouped base relation

2017-01-17 Thread Antonin Houska
Ashutosh Bapat wrote: > [... snip ]] > > This all works well, as long as the aggregate is "summing" something > across rows. The method doesn't work when aggregation is say > "multiplying" across the rows or "concatenating" across the rows like > array_agg() or string_agg(). They need a different

Re: [HACKERS] [PATCH] Rename pg_switch_xlog to pg_switch_wal

2017-01-17 Thread Peter Eisentraut
On 1/13/17 10:18 AM, Stephen Frost wrote: > The point I was making was that serious montioring systems would have to > be changed and I stand by that. I don't think my monitoring systems are any less serious than yours. > Certainly, check_postgres is going to have to be changed to address this >

Re: [HACKERS] Re: Clarifying "server starting" messaging in pg_ctl start without --wait

2017-01-17 Thread Peter Eisentraut
On 1/15/17 11:40 PM, Fujii Masao wrote: > This change may confuse the users who run "pg_ctl start" to perform a crash > recovery, archive recovery and standby server (with hot_standby=off) because > "pg_ctl start" would not return so long time. Well, this change was made because the previous behav

Re: [HACKERS] New CORRESPONDING clause design

2017-01-17 Thread Tom Lane
Surafel Temsgen writes: > My design is > *In parsing stage* > 1. Check at least one common column name appear in queries > 2. If corresponding column list is not specified, then make corresponding > list from common column name in queries target lists in the order > that those column names a

Re: [HACKERS] Logical Replication WIP

2017-01-17 Thread Petr Jelinek
On 17/01/17 17:09, Peter Eisentraut wrote: > >> Yes, that will need some discussion about corner case behaviour. For >> example, have partitioned table 'foo' which is in publication, then you >> have table 'bar' which is not in publication, you attach it to the >> partitioned table 'foo', should i

Re: [HACKERS] Logical Replication WIP

2017-01-17 Thread Peter Eisentraut
On 1/15/17 1:48 PM, Petr Jelinek wrote: > It's meant to decouple the synchronous commit setting for logical > replication workers from the one set for normal clients. Now that we > have owners for subscription and subscription runs as that owner, maybe > we could do that via ALTER USER. I was thin

Re: [HACKERS] Logical Replication WIP

2017-01-17 Thread Peter Eisentraut
On 1/15/17 5:20 PM, Petr Jelinek wrote: > Well, it's 4 because max_worker_processes is 8, I think default > max_worker_processes should be higher than > max_logical_replication_workers so that's why I picked 4. If we are okay > wit bumping the max_worker_processes a bit, I am all for increasing > m

Re: [HACKERS] Logical Replication WIP

2017-01-17 Thread Peter Eisentraut
On 1/15/17 2:28 PM, Petr Jelinek wrote: > Well the preinstalled information_schema is excluded by the > FirstNormalObjectId filter as it's created by initdb. If user drops and > recreates it that means it was created as user object. > > My opinion is that FOR ALL TABLES should replicate all user t

Re: [HACKERS] New CORRESPONDING clause design

2017-01-17 Thread David Fetter
On Tue, Jan 17, 2017 at 08:20:25AM -0600, Merlin Moncure wrote: > On Tue, Jan 17, 2017 at 12:37 AM, Surafel Temsgen > wrote: > > I am new here and I really want to contribute, I have read same resource > > that help understanding database system and postgresql. I would like to > > start implement

Re: [HACKERS] GSoC 2017

2017-01-17 Thread Anastasia Lubennikova
I'm ready to be a mentor. 10.01.2017 12:53, Alexander Korotkov: Hi all! In 2016 PostgreSQL project didn't pass to GSoC program. In my understanding the reasons for that are following. 1. We did last-minute submission of our application to GSoC. 2. In 2016 GSoC application form for mentoring

Re: [HACKERS] ISO/IEC 9075-2:2016 for postgres community

2017-01-17 Thread Oleg Bartunov
On Tue, Jan 17, 2017 at 6:26 PM, Alvaro Herrera wrote: > Oleg Bartunov wrote: > > Hi there, > > > > I just bought ISO/IEC 9075-2:2016 > > http://www.iso.org/iso/home/store/catalogue_tc/catalogue_ > detail.htm?csnumber=63556 > > to satisfy my interest on json support in SQL. I am not ready to dis

Re: [HACKERS] ISO/IEC 9075-2:2016 for postgres community

2017-01-17 Thread Alvaro Herrera
Oleg Bartunov wrote: > Hi there, > > I just bought ISO/IEC 9075-2:2016 > http://www.iso.org/iso/home/store/catalogue_tc/catalogue_detail.htm?csnumber=63556 > to satisfy my interest on json support in SQL. I am not ready to discuss > here implementation details, but there is one problem with the s

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-01-17 Thread Alvaro Herrera
Reading through the track_root_lp patch now. > + /* > + * For HOT (or WARM) updated tuples, we store the offset of the > root > + * line pointer of this chain in the ip_posid field of the new > tuple. > + * Usually this information will be avail

[HACKERS] reminder: PGCon 2017 CFP

2017-01-17 Thread Dan Langille
Hello, There are two days left in the PGCon 2017 CFP, which closes on 19 January. Please get your submissions in soon. PGCon 2017 will be on 23-26 May 2017 at University of Ottawa. * 23-24 (Tue-Wed) tutorials * 24 (Wed) The Unconference * 25-26 (Thu-Fri) talks - the main part of the conference

Re: [HACKERS] Re: [COMMITTERS] pgsql: Fix an assertion failure related to an exclusive backup.

2017-01-17 Thread Fujii Masao
On Tue, Jan 17, 2017 at 10:37 PM, Michael Paquier wrote: > On Tue, Jan 17, 2017 at 5:40 PM, Fujii Masao wrote: >> Fix an assertion failure related to an exclusive backup. >> >> Previously multiple sessions could execute pg_start_backup() and >> pg_stop_backup() to start and stop an exclusive back

Re: [HACKERS] New CORRESPONDING clause design

2017-01-17 Thread Merlin Moncure
On Tue, Jan 17, 2017 at 12:37 AM, Surafel Temsgen wrote: > I am new here and I really want to contribute, I have read same resource > that help understanding database system and postgresql. I would like to > start implementing sql syntax corresponding by clause because I believe > implementing sql

Re: [HACKERS] Patch to implement pg_current_logfile() function

2017-01-17 Thread Karl O. Pinc
On Tue, 17 Jan 2017 07:58:43 -0600 "Karl O. Pinc" wrote: > On Tue, 17 Jan 2017 11:22:45 +0900 > Michael Paquier wrote: > > > On Tue, Jan 17, 2017 at 2:21 AM, Karl O. Pinc > > wrote: > > > >>Also, I would rather see an ERROR returned to the user in case of > > >>bad data in current_logfiles,

Re: [HACKERS] Patch to implement pg_current_logfile() function

2017-01-17 Thread Karl O. Pinc
On Tue, 17 Jan 2017 11:22:45 +0900 Michael Paquier wrote: > On Tue, Jan 17, 2017 at 2:21 AM, Karl O. Pinc wrote: > >>Also, I would rather see an ERROR returned to the user in case of > >>bad data in current_logfiles, I did not change that either as > >>that's the original intention of Gilles.

[HACKERS] Re: [COMMITTERS] pgsql: Fix an assertion failure related to an exclusive backup.

2017-01-17 Thread Michael Paquier
On Tue, Jan 17, 2017 at 5:40 PM, Fujii Masao wrote: > Fix an assertion failure related to an exclusive backup. > > Previously multiple sessions could execute pg_start_backup() and > pg_stop_backup() to start and stop an exclusive backup at the same time. > This could trigger the assertion failure

Re: [HACKERS] Re: [sqlsmith] FailedAssertion("!(XLogCtl->Insert.exclusiveBackup)", File: "xlog.c", Line: 10200)

2017-01-17 Thread Michael Paquier
I sent this email one month ago but forgot to cc pgsql-hackers ;) For the record, it is the set of patches attached that have been pushed as 974ece5, and only Fujii-san has received them... Thanks for committing the fix by the way! On Thu, Dec 15, 2016 at 3:30 AM, Fujii Masao wrote: > On Wed, Dec

Re: [HACKERS] Gather Merge

2017-01-17 Thread Amit Kapila
On Tue, Jan 17, 2017 at 5:19 PM, Rushabh Lathia wrote: > > > > Here is the benchmark number which I got with the latest (v6) patch: > > - max_worker_processes = DEFAULT (8) > - max_parallel_workers_per_gather = 4 > - Cold cache environment is ensured. With every query execution - server is > sto

[HACKERS] ISO/IEC 9075-2:2016 for postgres community

2017-01-17 Thread Oleg Bartunov
Hi there, I just bought ISO/IEC 9075-2:2016 http://www.iso.org/iso/home/store/catalogue_tc/catalogue_detail.htm?csnumber=63556 to satisfy my interest on json support in SQL. I am not ready to discuss here implementation details, but there is one problem with the status of this document, which is

[HACKERS] New CORRESPONDING clause design

2017-01-17 Thread Surafel Temsgen
I am new here and I really want to contribute, I have read same resource that help understanding database system and postgresql. I would like to start implementing sql syntax corresponding by clause because I believe implementing sql syntax gives an opportunity to familiarize many part of postgres

Re: [HACKERS] pageinspect: Hash index support

2017-01-17 Thread Amit Kapila
On Fri, Jan 13, 2017 at 12:49 AM, Jesper Pedersen wrote: > Hi, > > On 01/11/2017 03:16 PM, Ashutosh Sharma wrote: >> >> >> I have rephrased it to make it more clear. >> > > Rebased, and removed the compile warn in hashfuncs.c > Review comments: 1. +static Page +verify_hash_page(bytea *raw_page,

Re: [HACKERS] Gather Merge

2017-01-17 Thread Rushabh Lathia
On Tue, Jan 17, 2017 at 5:19 PM, Rushabh Lathia wrote: > > > On Fri, Jan 13, 2017 at 10:52 AM, Rushabh Lathia > wrote: > >> >> >> On Thu, Jan 12, 2017 at 8:50 AM, Robert Haas >> wrote: >> >>> On Sun, Dec 4, 2016 at 7:36 PM, Haribabu Kommi >>> wrote: >>> > On Thu, Nov 24, 2016 at 11:12 PM, Rush

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

2017-01-17 Thread Amit Kapila
On Tue, Jan 17, 2017 at 11:39 AM, Dilip Kumar wrote: > On Wed, Jan 11, 2017 at 10:55 AM, Dilip Kumar wrote: >> I have reviewed the latest patch and I don't have any more comments. >> So if there is no objection from other reviewers I can move it to >> "Ready For Committer"? > > Seeing no objectio

  1   2   >