Re: [HACKERS] tuplesort_gettuple_common() and *should_free argument

2017-04-06 Thread Andres Freund
On 2017-04-04 13:49:11 -0700, Peter Geoghegan wrote: > On Tue, Apr 4, 2017 at 1:32 PM, Andres Freund wrote: > >> static bool > >> tuplesort_gettuple_common(Tuplesortstate *state, bool forward, > >> @@ -2091,12 +2092,15 @@ tuplesort_gettuple_common(Tuplesortst

Re: [HACKERS] parallel explain analyze support not exercised

2017-04-06 Thread Andres Freund
Hi, On 2017-04-03 17:11:33 -0400, Robert Haas wrote: > On Mon, Apr 3, 2017 at 3:31 PM, Andres Freund wrote: > >> If this is 'make check', then we should have 8 parallel workers > >> allowed, so if we only do one of these at a time, then I think we're &g

Re: [HACKERS] parallel bitmapscan isn't exercised in regression tests

2017-04-06 Thread Andres Freund
On 2017-04-06 10:00:32 +0530, Dilip Kumar wrote: > On Tue, Apr 4, 2017 at 5:51 AM, Dilip Kumar wrote: > > Sure I can do that, In attached patch, I only fixed the problem of not > > executing the bitmap test. Now, I will add few cases to cover other > > parts especially rescan and prefetching logi

Re: [HACKERS] Fast Default WIP patch for discussion

2017-04-06 Thread Andres Freund
Hi, On 2017-04-05 22:31:15 -0700, Serge Rielau wrote: > Andres, > Yes, I still want to push this in. However I have not had time to get back to > it. I’m embarrassed to say that I don’t even know where the comments that > were issued occurred. > Cheers Serge You mean https://www.postgresql.org/

Re: [HACKERS] Other formats in pset like markdown, rst, mediawiki

2017-04-06 Thread Andres Freund
On April 5, 2017 11:19:04 PM PDT, "Jan Michálek" wrote: >2017-04-06 0:55 GMT+02:00 Andres Freund : > >> Hi, >> >> On 2017-04-02 22:28:40 +0200, Jan Michálek wrote: >> > 2017-03-23 17:26 GMT+01:00 Pierre Ducroquet : >> >

Re: [HACKERS] Parallel Append implementation

2017-04-05 Thread Andres Freund
ers Does that make sense? > BTW all of the above points apply only for non-partial plans. Indeed. But I think that's going to be a pretty common type of plan, especially if we get partitionwise joins. Greetings, Andres Freund -- Sent via pgsql-hackers mailing list (pgsql-hackers

Re: [HACKERS] Re: new set of psql patches for loading (saving) data from (to) text, binary files

2017-04-05 Thread Andres Freund
Hi, On 2017-04-05 21:07:59 -0400, Stephen Frost wrote: > * Andres Freund (and...@anarazel.de) wrote: > > I don't like the API here much. Loading requires knowledge of some > > magic $1 value and allows only a single column, printing doesn't mean > > much when

Re: [HACKERS] Re: new set of psql patches for loading (saving) data from (to) text, binary files

2017-04-05 Thread Andres Freund
On 2017-03-18 17:51:48 +0100, Pavel Stehule wrote: > What is done: > > create table foo foo(a bytea); > > -- import > insert into foo values($1) > \gloadfrom ~/xxx.jpg bytea > > -- export > \pset format binary > select a from foo > \g ~/xxx2.jpg > > tested on import 55MB binary file > > Commen

Re: [HACKERS] pgbench - allow to store select results into variables

2017-04-05 Thread Andres Freund
On 2017-04-05 20:24:19 -0400, Tom Lane wrote: > Having said all that, I think we're at the point in the commitfest > where if there's any design question at all about a patch, it should > get booted to the next cycle. We are going to have more than enough > to do to stabilize what's already commit

Re: [HACKERS] Fast Default WIP patch for discussion

2017-04-05 Thread Andres Freund
Hi Serge, On 2016-10-28 08:28:11 -0700, Serge Rielau wrote: > Time for me to dig into that then. Are you planning to update your POC at some point? This'd be a very welcome improvement. Regards, Andres -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to y

Re: [HACKERS] pgbench more operators & functions

2017-04-05 Thread Andres Freund
27;ll have to punt this to v11. Greetings, Andres Freund -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] [PROPOSAL] Temporal query processing with range types

2017-04-05 Thread Andres Freund
Hi, On 2017-03-30 14:11:28 +0200, Peter Moser wrote: > 2017-03-01 10:56 GMT+01:00 Peter Moser : > > A similar walkthrough for ALIGN will follow soon. > > > > We are thankful for any suggestion or ideas, to be used to write a > > good SGML documentation. > > The attached README explains the ALIGN

Re: [HACKERS] LWLock optimization for multicore Power machines

2017-04-05 Thread Andres Freund
Hi, On 2017-04-03 11:56:13 -0700, Andres Freund wrote: > > > +/* > > + * Generic implementation of pg_atomic_fetch_mask_add_u32() via loop > > + * of compare & exchange. > > + */ > > +static inline uint32 > > +pg_atomic_fetch_mask_

Re: [HACKERS] [GENERAL] C++ port of Postgres

2017-04-05 Thread Andres Freund
Hi Peter, On 2017-02-28 22:30:16 -0800, Andres Freund wrote: > On 2017-02-28 23:42:45 -0500, Peter Eisentraut wrote: > > On 1/26/17 22:46, Andres Freund wrote: > > > On 2016-09-30 15:24:09 -0400, Peter Eisentraut wrote: > > >> Yeah, I have committed a few of the pat

Re: [HACKERS] Re: proposal - psql: possibility to specify sort for describe commands, when size is printed

2017-04-05 Thread Andres Freund
Hi, On 2017-03-11 13:06:13 +0100, Pavel Stehule wrote: > 2017-03-10 15:45 GMT+01:00 Alexander Korotkov : > > > On Fri, Mar 10, 2017 at 5:10 PM, Peter Eisentraut < > > peter.eisentr...@2ndquadrant.com> wrote: > > > >> On 2/24/17 16:32, Pavel Stehule wrote: > >> > set EXTENDED_DESCRIBE_SORT siz

Re: [HACKERS] Re: new set of psql patches for loading (saving) data from (to) text, binary files

2017-04-05 Thread Andres Freund
Hi, On 2017-03-18 17:51:48 +0100, Pavel Stehule wrote: > What is done: > > create table foo foo(a bytea); > > -- import > insert into foo values($1) > \gloadfrom ~/xxx.jpg bytea > > -- export > \pset format binary > select a from foo > \g ~/xxx2.jpg > > tested on import 55MB binary file > > C

Re: [HACKERS] Other formats in pset like markdown, rst, mediawiki

2017-04-05 Thread Andres Freund
Hi, On 2017-04-02 22:28:40 +0200, Jan Michálek wrote: > 2017-03-23 17:26 GMT+01:00 Pierre Ducroquet : > > The new status of this patch is: Waiting on Author > > > > Corrected problem with \pset linestyle when format is set to markdown, or > rst. > > Corrected tuples only for markdown and rst (no

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-04-05 Thread Andres Freund
Hi, On 2017-04-01 03:05:07 +0900, Masahiko Sawada wrote: > On Fri, Mar 31, 2017 at 11:44 PM, Robert Haas wrote: > [ lots of valuable discussion ] I think this patch clearly still is in the design stage, and has received plenty feedback this CF. I'll therefore move this to the next commitfest.

Re: [HACKERS] PATCH: Batch/pipelining support for libpq

2017-04-05 Thread Andres Freund
Hi, On 2017-04-05 17:00:42 +1000, Vaishnavi Prabakaran wrote: > Regarding test patch, I have corrected the test suite after David Steele's > comments. > Also, I would like to mention that a companion patch was submitted by David > Steele up-thread. > > Attached the latest code and test patch. My

Re: [HACKERS] Proposal : For Auto-Prewarm.

2017-04-05 Thread Andres Freund
On 2017-03-13 18:45:00 +0530, Mithun Cy wrote: > I have implemented a similar logic now. The prewarm bgworker will > launch a sub-worker per database in the dump file. And, each > sub-worker will load its database block info. The sub-workers will be > launched only after previous one is finished. A

Re: [HACKERS] PoC plpgsql - possibility to force custom or generic plan

2017-04-05 Thread Andres Freund
On 2017-04-05 17:22:34 -0400, Tom Lane wrote: > Andres Freund writes: > > I'd like some input from other committers whether we want this. I'm > > somewhat doubtful, but don't have particularly strong feelings. > > I don't really want to expose the worki

Re: [HACKERS] pg_stat_wal_write statistics view

2017-04-05 Thread Andres Freund
XLogBackgroundFlush(void) > { > XLogWrite(WriteRqst, flexible); > } > + > + /* Collect all the wal write shared counters into local, and report it > to stats collector */ > + memcpy(&LocalWalWritesStats.stats, &XLogCtl->stats, > sizeof(PgStat_WalWritesCounts)); > LWLockRelease(WALWriteLock); Hm. I think in a good number of workloads hti sis never reached, because we return early. I think this is an interesting feature, but I don't think it's ready, and it was submitted very late in the v10 release cycle. Therefore I think this should be moved to the next CF. Greetings, Andres Freund -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PoC plpgsql - possibility to force custom or generic plan

2017-04-05 Thread Andres Freund
Hi, I'd like some input from other committers whether we want this. I'm somewhat doubtful, but don't have particularly strong feelings. > + > + > + Block level PRAGMA > + > + > +PRAGMA > +in PL/pgSQL > + > + > + > +The block level PRAGMA allows to change the > +PL/

Re: [HACKERS] New SQL counter statistics view (pg_stat_sql)

2017-04-05 Thread Andres Freund
Tag); Wait, we're re-fetching the statement here? That doesn't sound alright. > +Datum > +pg_stat_sql(PG_FUNCTION_ARGS) > +{ > + TupleDesc tupdesc; > + Datum values[2]; > + boolnulls[2]; > + ReturnSetInfo *rsi; > +

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-04-05 Thread Andres Freund
On 2017-04-05 09:36:47 -0400, Robert Haas wrote: > By the way, the "Converting WARM chains back to HOT chains" section of > README.WARM seems to be out of date. Any chance you could update that > to reflect the current state and thinking of the patch? I propose we move this patch to the next CF.

Re: [HACKERS] partitioned tables and contrib/sepgsql

2017-04-05 Thread Andres Freund
On April 5, 2017 9:04:00 AM PDT, Tom Lane wrote: >Joe Conway writes: >> On 04/04/2017 09:58 PM, Tom Lane wrote: >>> Another issue is whether you won't get compiler complaints about >>> redefinition of the "true" and "false" macros. But those would >>> likely only be warnings, not flat-out erro

Re: [HACKERS] Rewriting the test of pg_upgrade as a TAP test

2017-04-05 Thread Andres Freund
On 2017-04-05 10:50:19 -0400, Stephen Frost wrote: > Andres, > > * Andres Freund (and...@anarazel.de) wrote: > > On 2017-04-05 10:40:41 -0400, Stephen Frost wrote: > > > * Tom Lane (t...@sss.pgh.pa.us) wrote: > > > > Stephen Frost writes: > > > &g

Re: [HACKERS] partitioned tables and contrib/sepgsql

2017-04-05 Thread Andres Freund
On 2017-04-05 10:48:27 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2017-03-31 20:23:39 -0400, Robert Haas wrote: > >> 0001 has the problem that we have a firm rule against putting any > >> #includes whatsoever before "postgres.h". This stdbool issue

Re: [HACKERS] partitioned tables and contrib/sepgsql

2017-04-05 Thread Andres Freund
On 2017-04-05 10:45:06 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2017-04-05 09:43:53 -0400, Tom Lane wrote: > >> Yeah, I was just thinking about that. The core problem though is that > >> we need the "bool" fields in the system catalog s

Re: [HACKERS] Rewriting the test of pg_upgrade as a TAP test

2017-04-05 Thread Andres Freund
ther (at least, not if we're going to stick > to the pg_upgrade test basically being if pg_dump returns the same > results before-and-after). I think we *should* have populated GIN indexes. Yes, the coverage isn't perfect, but the VACUUM definitely gives a decent amount of coverage

Re: [HACKERS] Statement timeout behavior in extended queries

2017-04-05 Thread Andres Freund
On 2017-04-05 09:46:31 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2017-04-05 00:39:51 -0400, Tom Lane wrote: > >> but > >> if that's what we're doing, let's make sure we do it consistently. > >> I haven't read the patch, but

Re: [HACKERS] Logical decoding on standby

2017-04-05 Thread Andres Freund
On 2017-04-05 17:18:24 +0800, Craig Ringer wrote: > On 5 April 2017 at 04:19, Andres Freund wrote: > > On 2017-04-04 22:32:40 +0800, Craig Ringer wrote: > >> I'm much happier with this. I'm still fixing some issues in the tests > >> for 03 and tidying them up

Re: [HACKERS] partitioned tables and contrib/sepgsql

2017-04-05 Thread Andres Freund
On 2017-03-31 20:23:39 -0400, Robert Haas wrote: > On Fri, Mar 31, 2017 at 2:14 PM, Mike Palmiotto > wrote: > > Attached you will find two patches, which were rebased on master as of > > 156d388 (applied with `git am --revert [patch file]`). The first gets > > rid of some pesky compiler warnings a

Re: [HACKERS] partitioned tables and contrib/sepgsql

2017-04-05 Thread Andres Freund
On 2017-04-05 09:43:53 -0400, Tom Lane wrote: > Andres Freund writes: > > I argued before that we should migrate to stdbool.h by default, because > > it's only going to get more common. We already do so in a way for c++ > > compilers... > > Yeah, I was jus

Re: [HACKERS] Re: Query fails when SRFs are part of FROM clause (Commit id: 69f4b9c85f)

2017-04-05 Thread Andres Freund
On 2017-04-05 02:47:55 -0400, Noah Misch wrote: > On Fri, Mar 10, 2017 at 11:49:46AM -0800, Andres Freund wrote: > > On 2017-03-09 13:34:22 -0500, Robert Haas wrote: > > > On Mon, Jan 30, 2017 at 6:54 PM, Tom Lane wrote: > > > > Andres Freund writes: > > >

Re: [HACKERS] Statement timeout behavior in extended queries

2017-04-05 Thread Andres Freund
On 2017-04-05 00:39:51 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2017-04-05 10:05:19 +0900, Tatsuo Ishii wrote: > >> What's your point of the question? What kind of problem do you expect > >> if the timeout starts only once at the first parse me

Re: [HACKERS] partitioned tables and contrib/sepgsql

2017-04-05 Thread Andres Freund
On 2017-04-05 00:58:07 -0400, Tom Lane wrote: > Robert Haas writes: > > On Tue, Apr 4, 2017 at 6:56 PM, Joe Conway wrote: > >> Any objections? > > > I'm guessing Tom's going to have a strong feeling about whether 0001a > > is the right way to address the stdbool issue, > > I will? [ looks ...

Re: [HACKERS] Supporting huge pages on Windows

2017-04-04 Thread Andres Freund
On 2017-04-05 04:25:41 +, Tsunakawa, Takayuki wrote: > From: Craig Ringer [mailto:craig.rin...@2ndquadrant.com] > > On 5 April 2017 at 10:37, Tsunakawa, Takayuki > > wrote: > > > > OTOH, I tried again to leave the DISABLE_MAX_PRIVILEGE as is and add Lock > > Pages in Memory, using the attache

Re: [HACKERS] Statement timeout behavior in extended queries

2017-04-04 Thread &#x27;Andres Freund'
On 2017-04-04 16:56:26 -0700, 'Andres Freund' wrote: > On 2017-04-04 23:52:28 +, Tsunakawa, Takayuki wrote: > > From: Andres Freund [mailto:and...@anarazel.de] > > > Looks to me like npgsql doesn't do that either. None of libpq, pgjdbs and > > > npg

Re: [HACKERS] Statement timeout behavior in extended queries

2017-04-04 Thread Andres Freund
messages? It's perfectly valid to send a lot of Parse messages without interspersed Sync or Bind/Execute message. There'll be one timeout covering all of those Parse messages, which can thus lead to a timeout, even though nothing actually takes long individually. Greetings, Andres

[HACKERS] Outdated comments around HandleFunctionRequest

2017-04-04 Thread Andres Freund
Hi, PostgresMain() has the following blurb for fastpath functions: /* * Note: we may at this point be inside an aborted * transaction. We can't throw error for that until we've

Re: [HACKERS] Statement timeout behavior in extended queries

2017-04-04 Thread &#x27;Andres Freund'
On 2017-04-04 23:52:28 +, Tsunakawa, Takayuki wrote: > From: Andres Freund [mailto:and...@anarazel.de] > > Looks to me like npgsql doesn't do that either. None of libpq, pgjdbs and > > npgsql doing it seems like some evidence that it's ok. > > And psqlODBC

Re: [HACKERS] Statement timeout behavior in extended queries

2017-04-04 Thread Andres Freund
On 2017-04-04 16:38:53 -0700, Andres Freund wrote: > On 2017-04-04 16:10:32 +0900, Tatsuo Ishii wrote: > > >> If what Tatsuo-san said to Tom is correct (i.e. each Parse/Bind/Execute > > >> starts and stops the timer), then it's a concern and the patch should &

Re: [HACKERS] Statement timeout behavior in extended queries

2017-04-04 Thread Andres Freund
On 2017-04-05 08:34:43 +0900, Tatsuo Ishii wrote: > Andres, > > >> I think the code needs a few clarifying comments around this, but > >> otherwise seems good. Not restarting the timeout in those cases > >> obviously isn't entirely "perfect"/"correct", but a tradeoff - the > >> comments should no

Re: [HACKERS] Statement timeout behavior in extended queries

2017-04-04 Thread Andres Freund
On 2017-04-04 16:10:32 +0900, Tatsuo Ishii wrote: > >> If what Tatsuo-san said to Tom is correct (i.e. each Parse/Bind/Execute > >> starts and stops the timer), then it's a concern and the patch should not > >> be ready for committer. However, the current patch is not like that -- it > >> seems

Re: [HACKERS] bug in SlabAlloc / VALGRIND_MAKE_MEM_DEFINED

2017-04-04 Thread Andres Freund
On 2017-04-04 23:23:30 +0200, Tomas Vondra wrote: > On 04/04/2017 10:42 PM, Tomas Vondra wrote: > > Hi, > > > > Andres nagged to me about valgrind runs taking much longer since > > 9fab40ad introduced the SlabContext into reorderbuffer.c. And by > > "longer" I mean hours instead of minutes. > > >

Re: [HACKERS] tuplesort_gettuple_common() and *should_free argument

2017-04-04 Thread Andres Freund
On 2017-03-13 18:14:07 -0700, Peter Geoghegan wrote: > From 5351b5db257cb39832647d9117465c0217e6268b Mon Sep 17 00:00:00 2001 > From: Peter Geoghegan > Date: Thu, 13 Oct 2016 10:54:31 -0700 > Subject: [PATCH 1/2] Avoid copying within tuplesort_gettupleslot(). s/Avoid/Allow to avoid/ > Add a "cop

Re: [HACKERS] Logical decoding on standby

2017-04-04 Thread Andres Freund
On 2017-04-04 22:32:40 +0800, Craig Ringer wrote: > I'm much happier with this. I'm still fixing some issues in the tests > for 03 and tidying them up, but 03 should allow 01 and 02 to be > reviewed in their proper context now. To me this very clearly is too late for v10, and now should be moved t

Re: [HACKERS] Parallel Append implementation

2017-04-04 Thread Andres Freund
On 2017-04-04 08:01:32 -0400, Robert Haas wrote: > On Tue, Apr 4, 2017 at 12:47 AM, Andres Freund wrote: > > I don't think the parallel seqscan is comparable in complexity with the > > parallel append case. Each worker there does the same kind of work, and > > if on

Re: [HACKERS] logical decoding of two-phase transactions

2017-04-04 Thread Andres Freund
On 2017-04-04 13:06:13 +0300, Stas Kelvich wrote: > That is just argument against Andres concern that prepared transaction > is able to deadlock with decoding process — at least no such cases in > regression tests. There's few longer / adverse xacts, that doesn't say much. > And that concern is

Re: [HACKERS] Statement timeout behavior in extended queries

2017-04-03 Thread &#x27;Andres Freund'
On 2017-04-04 06:35:00 +, Tsunakawa, Takayuki wrote: > From: Andres Freund [mailto:and...@anarazel.de] > Given the concern raised in > > http://archives.postgresql.org/message-id/12207.1491228316%40sss.pgh.p > > a.us > > I don't see this being ready for committer.

Re: [HACKERS] Statement timeout behavior in extended queries

2017-04-03 Thread Andres Freund
On 2017-04-04 06:18:04 +, Tsunakawa, Takayuki wrote: > From: Tatsuo Ishii [mailto:is...@sraoss.co.jp] > > It's too late. Someone has already moved the patch to the next CF (for > > PostgreSQL 11). > > Yes, but this patch will be necessary by the final release of PG 10 if the > libpq batch/pip

Re: [HACKERS] Supporting huge pages on Windows

2017-04-03 Thread Andres Freund
On 2017-01-05 03:12:09 +, Tsunakawa, Takayuki wrote: > From: pgsql-hackers-ow...@postgresql.org > > [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Magnus Hagander > > For the pg_ctl changes, we're going from removing all privilieges from the > > token, to removing none. Are there any

Re: [HACKERS] [PATCH] Incremental sort

2017-04-03 Thread Andres Freund
On 2017-04-03 22:18:21 -0400, Robert Haas wrote: > On Mon, Apr 3, 2017 at 5:09 PM, Andres Freund wrote: > > To me this hasn't gotten even remotely enough performance evaluation. > > And I don't think it's fair to characterize it as pending since 2013, > >

Re: [HACKERS] PATCH: Batch/pipelining support for libpq

2017-04-03 Thread Andres Freund
On 2017-04-04 08:57:33 +0900, Michael Paquier wrote: > On Tue, Apr 4, 2017 at 8:26 AM, Andres Freund wrote: > > On 2017-04-04 09:24:23 +1000, Vaishnavi Prabakaran wrote: > >> Just quickly, Is it not ok to consider only the code patch for this CF > >> without test patc

Re: [HACKERS] Parallel Append implementation

2017-04-03 Thread Andres Freund
On 2017-04-03 22:13:18 -0400, Robert Haas wrote: > On Mon, Apr 3, 2017 at 4:17 PM, Andres Freund wrote: > > Hm. I'm not really convinced by the logic here. Wouldn't it be better > > to try to compute the minimum total cost across all workers for > > 1..#max_worker

Re: [HACKERS] PATCH: Batch/pipelining support for libpq

2017-04-03 Thread Andres Freund
On 2017-04-04 09:24:23 +1000, Vaishnavi Prabakaran wrote: > Hi, > On Tue, Apr 4, 2017 at 7:05 AM, Andres Freund wrote: > > > On 2017-04-03 14:10:47 +1000, Vaishnavi Prabakaran wrote: > > > > The CF has been extended until April 7 but time is still growing short. > &

Re: [HACKERS] Candidate for local inline function?

2017-04-03 Thread Andres Freund
On 2017-03-17 15:29:27 -0500, Kevin Grittner wrote: > On Fri, Mar 17, 2017 at 3:23 PM, Andres Freund wrote: > > On 2017-03-17 15:17:33 -0500, Kevin Grittner wrote: > >> Why do we warn of a hazard here instead of eliminating said hazard > >> with a static inline functio

Re: [HACKERS] Supporting huge pages on Windows

2017-04-03 Thread Andres Freund
> DISABLE_MAX_PRIVILEGE, > + 0, > sizeof(dropSids) / > sizeof(dropSids[0]), > dropSids, > 0, NULL, Uh - isn't that a behavioural change? Was this discussed? Greetings, Andres Freund -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Making clausesel.c Smarter

2017-04-03 Thread Andres Freund
On 2017-04-04 09:22:07 +1200, David Rowley wrote: > On 4 April 2017 at 08:24, Andres Freund wrote: > > On 2017-04-03 20:59:42 +1200, David Rowley wrote: > >> Updated patch attached. > >> > >> Thanks for reviewing it. > > > > Given the time in the

Re: [HACKERS] PATCH: recursive json_populate_record()

2017-04-03 Thread Andres Freund
On 2017-03-21 14:31:08 -0400, Andrew Dunstan wrote: > > > On 03/21/2017 01:37 PM, David Steele wrote: > > On 3/16/17 11:54 AM, David Steele wrote: > >> On 2/1/17 12:53 AM, Michael Paquier wrote: > >>> On Thu, Jan 26, 2017 at 6:49 AM, Tom Lane wrote: > Nikita Glukhov writes: > > On 25.01.

Re: [HACKERS] WIP: [[Parallel] Shared] Hash

2017-04-03 Thread Andres Freund
Hi, On 2017-03-31 17:53:12 +1300, Thomas Munro wrote: > Thanks very much to Rafia for testing, and to Andres for his copious > review feedback. Here's a new version. Changes: I unfortunately think it's too late to get this into v10. There's still heavy development going on, several pieces chan

Re: [HACKERS] [PATCH] Incremental sort

2017-04-03 Thread Andres Freund
#x27;s fair to characterize it as pending since 2013, given it was essentially "waiting on author" for most of that. Greetings, Andres Freund -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PATCH: Batch/pipelining support for libpq

2017-04-03 Thread Andres Freund
libpq_gettext_noop("internal error, IDLE in > batch mode")); > + break; > + case PGASYNC_QUEUED: > + /* next query please */ > + break; > + } Once more, I'm very unconvinced by the switch. Unless you do anything with the errors, this seems pointless. > + if (conn->batch_status == PQBATCH_MODE_ABORTED && conn->queryclass != > PGQUERY_SYNC) > + { > + /* > + * In an aborted batch we don't get anything from the server > for each > + * result; we're just discarding input until we get to the next > sync > + * from the server. The client needs to know its queries got > aborted > + * so we create a fake PGresult to return immediately from > + * PQgetResult. > + */ > + conn->result = PQmakeEmptyPGresult(conn, > + >PGRES_BATCH_ABORTED); > + if (!conn->result) > + { > + printfPQExpBuffer(&conn->errorMessage, > + libpq_gettext("out of > memory")); > + pqSaveErrorResult(conn); > + } > + conn->asyncStatus = PGASYNC_READY; So we still return true in the OOM case? Greetings, Andres Freund -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] identity columns

2017-04-03 Thread Andres Freund
te type, but that wouldn't actually do > anything. Uh. What about 32bit systems (and 64bit systems without float8passbyval)? Greetings, Andres Freund -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Making clausesel.c Smarter

2017-04-03 Thread Andres Freund
On 2017-04-03 20:59:42 +1200, David Rowley wrote: > Updated patch attached. > > Thanks for reviewing it. Given the time in the release cycle I'm afraid that this it's too late to get this into v10. Does anybody disagree? If not, it should be moved to the next CF. Greeti

Re: [HACKERS] Parallel Append implementation

2017-04-03 Thread Andres Freund
Hi, On 2017-03-24 21:32:57 +0530, Amit Khandekar wrote: > diff --git a/src/backend/executor/nodeAppend.c > b/src/backend/executor/nodeAppend.c > index a107545..e9e8676 100644 > --- a/src/backend/executor/nodeAppend.c > +++ b/src/backend/executor/nodeAppend.c > @@ -59,9 +59,47 @@ > > #include

Re: [HACKERS] parallel explain analyze support not exercised

2017-04-03 Thread Andres Freund
On 2017-04-03 15:13:13 -0400, Robert Haas wrote: > On Mon, Apr 3, 2017 at 1:53 PM, Andres Freund wrote: > >> Please find the attached for the same. > > > >> +-- to increase the parallel query test coverage > >> +EXPLAIN (analyze, timing off, summary

Re: [HACKERS] [PATCH] Incremental sort

2017-04-03 Thread Andres Freund
On April 3, 2017 12:03:56 PM PDT, Alexander Korotkov wrote: >On Mon, Apr 3, 2017 at 9:34 PM, Andres Freund >wrote: > >> On 2017-03-29 00:17:02 +0300, Alexander Korotkov wrote: >> > On Tue, Mar 28, 2017 at 5:27 PM, David Steele >> wrote: >> > > On

Re: [HACKERS] Should we cacheline align PGXACT?

2017-04-03 Thread Andres Freund
816 bytes to 848 bytes. So, size of PGPROC remains 16-byte > aligned. So, probably effect is related to distance between PGPROC > members... > > See comparison of 16-bytes alignment of PGXACT + reduce PGXACT access vs. > padding of PGPROC. My earlier testing had showed that padding e

Re: [HACKERS] LWLock optimization for multicore Power machines

2017-04-03 Thread Andres Freund
value += add_; > + > + /* > + * Attempt to swap in the value we are expecting. If we didn't > see > + * masked bits to be clear, that's just the old value. If we > saw them > + * as clear, we'll attempt to make an increment. The r

Re: [HACKERS] Improve OR conditions on joined columns (common star schema problem)

2017-04-03 Thread Andres Freund
Hi, On 2017-02-14 14:18:54 -0500, Tom Lane wrote: > I think this might be code-complete, modulo the question of whether we > want an enabling GUC for it. I'm still concerned about the question > of whether it adds more planning time than it's worth for most users. > (Obviously it needs some regre

Re: [HACKERS] SERIALIZABLE with parallel query

2017-04-03 Thread Andres Freund
Hi, On 2017-03-11 15:19:23 +1300, Thomas Munro wrote: > Here is a rebased patch. It seems that this patch is still undergoing development, review and performance evaluation. Therefore it seems like it'd be a bad idea to try to get this into v10. Any arguments against moving this to the next CF?

Re: [HACKERS] [PATCH] Incremental sort

2017-04-03 Thread Andres Freund
On 2017-03-29 00:17:02 +0300, Alexander Korotkov wrote: > On Tue, Mar 28, 2017 at 5:27 PM, David Steele wrote: > > > Hi Alexander, > > > > On 3/20/17 10:19 AM, Heikki Linnakangas wrote: > > > >> On 03/20/2017 11:33 AM, Alexander Korotkov wrote: > >> > >>> Please, find rebased patch in the attachm

Re: [HACKERS] [PATCH v.7] Move all am-related reloption code into src/backend/access/[am-name] and get rid of relopt_kind for custom AM

2017-04-03 Thread Andres Freund
Hi Nikolay, On 2017-03-29 15:51:43 +0300, Nikolay Shaplov wrote: > Here is a new version of the patch. Thanks for the new version of the patch. As this commitfest is running out of time, and as the patch has been submitted late in the v10 development cycle, I've moved this patch to the next comm

Re: [HACKERS] Statement timeout behavior in extended queries

2017-04-03 Thread Andres Freund
On 2017-04-03 23:31:59 +0900, Tatsuo Ishii wrote: > > That seems like it could represent quite a lot of added overhead, > > on machines where gettimeofday() is slow ... which is still a lot > > of them, unfortunately. > > Maybe. I think we could eliminate restarting the timer for parse and > bind.

Re: [HACKERS] [PATCH] few fts functions for jsonb

2017-04-03 Thread Andres Freund
On 2017-04-01 16:20:46 -0400, Andrew Dunstan wrote: > > > On 03/31/2017 03:17 PM, Oleg Bartunov wrote: > > > > > > On 30 Mar 2017 23:43, "Dmitry Dolgov" <9erthali...@gmail.com > > > wrote: > > > > On 31 March 2017 at 00:01, Andrew Dunstan > > >

Re: [HACKERS] postgres_fdw: evaluate placeholdervars on remote server

2017-04-03 Thread Andres Freund
Hi, On 2017-02-28 21:45:22 +0900, Etsuro Fujita wrote: > Here is a patch for $subject. This is a nontrivial patch, submitted just before the start of the last CF for postgres 10. Therefore I think we should move this to the next CF. - Andres -- Sent via pgsql-hackers mailing list (pgsql-hac

Re: [HACKERS] identity columns

2017-04-03 Thread Andres Freund
pr(ExprState *state, ExprContext > *econtext, bool *isnull) > EEO_NEXT(); > } > > + EEO_CASE(EEOP_NEXTVALUEEXPR) > + { > + *op->resvalue = > Int64GetDatum(nextval_internal(op->d.nextvalueex

Re: [HACKERS] parallel explain analyze support not exercised

2017-04-03 Thread Andres Freund
On 2017-04-03 10:26:27 +0530, Rafia Sabih wrote: > On Sat, Apr 1, 2017 at 12:25 AM, Andres Freund wrote: > > Hi, > > > > As visible in [1], the explain analyze codepaths of parallel query isn't > > exercised in the tests. That used to be not entirely trivial if th

Re: [HACKERS] parallel bitmapscan isn't exercised in regression tests

2017-04-03 Thread Andres Freund
On 2017-04-01 17:23:04 +0530, Dilip Kumar wrote: > On Sat, Apr 1, 2017 at 12:16 AM, Andres Freund wrote: > > Hi, > > > > The parallel code-path isn't actually exercised in the tests added in > > [1], as evidenced by [2] (they just explain). That imo needs

Re: [HACKERS] Rewriting the test of pg_upgrade as a TAP test

2017-04-03 Thread Andres Freund
ere are certainly > some tests now but perhaps we need more or need to have improvmenets > made there. I don't fundamentally disagree with anything here, but I think it'd be a serious mistake to link this to the conversion of the pg_upgrade tests to tap tests. Greetings, Andres Freund -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Rewriting the test of pg_upgrade as a TAP test

2017-04-03 Thread Andres Freund
full serial tests to populate the pre-upgrade database, we > determine a useful subset of what that ends up generating and just > populate with that. That doesn't strike as particularly future proof. We intentionally leave objects behind pg_regress runs, but that only works if we actual

Re: [HACKERS] New expression evaluator and indirect jumps

2017-04-01 Thread Andres Freund
the computed goto stuff isn't that high, I'm not really concerned here. Greetings, Andres Freund -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] Re: [COMMITTERS] pgsql: Avoid GatherMerge crash when there are no workers.

2017-03-31 Thread Andres Freund
Hi, On 2017-04-01 01:22:14 +, Robert Haas wrote: > Avoid GatherMerge crash when there are no workers. I think the gather merge code needs a bit more test coverage (sorry to make this a larger theme today). As shown by https://coverage.postgresql.org/src/backend/executor/nodeGatherMerge.c.gco

Re: [HACKERS] logical replication launcher crash on buildfarm

2017-03-31 Thread Andres Freund
On 2017-03-31 21:30:17 -0400, Robert Haas wrote: > On Fri, Mar 31, 2017 at 9:26 PM, Petr Jelinek > wrote: > >> Hmm, I don't know if there's any good reason not to just use strcmp(), > >> but sure, OK. Committed and back-patched. > > > > Hmm culicidae still fails, this time only in parallel worker

Re: [HACKERS] logical replication launcher crash on buildfarm

2017-03-31 Thread Andres Freund
On 2017-04-01 03:57:05 +0200, Petr Jelinek wrote: > On 01/04/17 03:44, Andres Freund wrote: > > On 2017-04-01 03:26:01 +0200, Petr Jelinek wrote: > >> On 01/04/17 02:53, Robert Haas wrote: > >>> On Fri, Mar 31, 2017 at 8:32 PM, Petr Jelinek > >>>

Re: [HACKERS] logical replication launcher crash on buildfarm

2017-03-31 Thread Andres Freund
On 2017-04-01 03:26:01 +0200, Petr Jelinek wrote: > On 01/04/17 02:53, Robert Haas wrote: > > On Fri, Mar 31, 2017 at 8:32 PM, Petr Jelinek > > wrote: > >> Right, changed to BGW_MAXLEN. > > > > Hmm, I don't know if there's any good reason not to just use strcmp(), > > but sure, OK. Committed and

Re: [HACKERS] postgres_fdw IMPORT SCHEMA and partitioned tables

2017-03-31 Thread Andres Freund
because otherwise using postgres_fdw from an old server against a newer one will do weird stuff. I don't know what kind of policy we've committed to with postgresImportForeignSchema... Greetings, Andres Freund -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make c

[HACKERS] parallel explain analyze support not exercised

2017-03-31 Thread Andres Freund
_series < 0) │ │ Rows Removed by Filter: 1000 │ └───┘ should be reproducible. I'd suggest additionally adding one tests that throws the EXPLAIN output away, but actually enables paralellism. Greetings, Andres

[HACKERS] parallel bitmapscan isn't exercised in regression tests

2017-03-31 Thread Andres Freund
Hi, The parallel code-path isn't actually exercised in the tests added in [1], as evidenced by [2] (they just explain). That imo needs to be fixed. Greetings, Andres Freund [1] https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=f35742ccb7aa53ee3ed8416bbb378b0c3eeb6bb9 [2]

Re: [HACKERS] WIP: [[Parallel] Shared] Hash

2017-03-31 Thread Andres Freund
Hi Thomas, On 2017-03-31 17:53:12 +1300, Thomas Munro wrote: > Thanks very much to Rafia for testing, and to Andres for his copious > review feedback. Here's a new version. Changes: I've not looked at that aspect, but one thing I think would be good is to first add patch that increases coverage

Re: [HACKERS] Allow to specify #columns in heap/index_form_tuple

2017-03-31 Thread Andres Freund
On 2017-03-31 13:44:52 -0400, Tom Lane wrote: > Andres Freund writes: > > The covering indexes patch [1] really needs a version of > > heap_form_tuple/index_form_tuple that allows to specify the number of > > columns in the to-be generated tuple. > > I was thinking a

Re: [HACKERS] WIP: Covering + unique indexes.

2017-03-31 Thread Andres Freund
On 2017-03-31 20:40:59 +0300, Anastasia Lubennikova wrote: > 30.03.2017 22:11, Andres Freund > > Any objection from reviewers to push both patches? > > First of all, I want to thank you and Robert for reviewing this patch. > Your expertise in postgres subsystems is really nece

[HACKERS] Allow to specify #columns in heap/index_form_tuple

2017-03-31 Thread Andres Freund
Hi, The covering indexes patch [1] really needs a version of heap_form_tuple/index_form_tuple that allows to specify the number of columns in the to-be generated tuple. Previously the faster expression evaluation stuff could also have benefited form the same for both forming and deforming tuples.

Re: [HACKERS] Logical decoding on standby

2017-03-30 Thread Andres Freund
On 2017-03-30 19:40:08 +0100, Simon Riggs wrote: > On 30 March 2017 at 18:16, Andres Freund wrote: > > >> /* > >> * Each page of XLOG file has a header like this: > >> */ > >> -#define XLOG_PAGE_MAGIC 0xD097 /* can be used as WA

Re: [HACKERS] WIP: Covering + unique indexes.

2017-03-30 Thread Andres Freund
On 2017-03-30 18:26:05 +0300, Teodor Sigaev wrote: > Any objection from reviewers to push both patches? > diff --git a/contrib/bloom/blutils.c b/contrib/bloom/blutils.c > index f2eda67..59029b9 100644 > --- a/contrib/bloom/blutils.c > +++ b/contrib/bloom/blutils.c > @@ -120,6 +120,7 @@ blhandler(

Re: [HACKERS] Logical decoding on standby

2017-03-30 Thread Andres Freund
On 2017-03-29 08:01:34 +0800, Craig Ringer wrote: > On 28 March 2017 at 23:22, Andres Freund wrote: > > >> --- a/doc/src/sgml/protocol.sgml > >> +++ b/doc/src/sgml/protocol.sgml > >> @@ -2034,6 +2034,8 @@ The commands accepted in walsender mode are: > >&

Re: [HACKERS] Logical decoding on standby

2017-03-30 Thread Andres Freund
logXmin guarantees that no valid catalog tuples >= than it are removed. That property is used for logical decoding.". or similar? > /* > * Each page of XLOG file has a header like this: > */ > -#define XLOG_PAGE_MAGIC 0xD097 /* can be used as WAL version indicator > */ > +#define XLOG_PAGE_MAGIC 0xD100 /* can be used as WAL version indicator > */ We normally only advance this by one, it's not tied to the poistgres version. I'm sorry, but to me this patch isn't ready. I'm also doubtful that it makes a whole lot of sense on its own, without having finished the design for decoding on a standby - it seems quite likely that we'll have to redesign the mechanisms in here a bit for that. For 10 this seems to do nothing but add overhead? Greetings, Andres Freund -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] Use \if ... \elif ... \else ... \endif to simplify alternative in psql scripting.

2017-03-30 Thread Andres Freund
On 2017-03-30 16:59:39 +, Tom Lane wrote: > Support \if ... \elif ... \else ... \endif in psql scripting. Could we use this to simplify maintenance of some of the larger alternative output files? -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your s

<    4   5   6   7   8   9   10   11   12   13   >