Re: [HACKERS] [POC] Faster processing at Gather node

2017-10-17 Thread Andres Freund
Hi, On 2017-10-17 14:39:57 -0700, Andres Freund wrote: > I've spent some time looking into this, and I'm not quite convinced this > is the right approach. Let me start by describing where I see the > current performance problems around gather stemming from. One further approach to several of the

Re: [HACKERS] Partition-wise aggregation/grouping

2017-10-17 Thread Dilip Kumar
On Tue, Oct 17, 2017 at 10:44 PM, Jeevan Chalke wrote: > > I didn't get what you mean by regression here. Can you please explain? > > I see that PWA plan is selected over regular plan when enabled on the basis > of costing. > Regular planning need a Result node due to which costing increases wher

[HACKERS] A handful of typos in allpaths.c

2017-10-17 Thread David Rowley
A small patch to fix these is attached. -- David Rowley http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services allpath_typos_fix.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make chan

Re: [HACKERS] Re: Is anything preventing us from allowing write to foreign tables from standby?

2017-10-17 Thread Michael Paquier
On Wed, Oct 18, 2017 at 9:14 AM, Craig Ringer wrote: > Superficially at least, it sounds like a good idea. Indeed. > We should only need a virtual xid when we're working with foreign > tables since we don't do any local heap changes. > > How's it work with savepoints? That's one thing to worry

Re: [HACKERS] [COMMITTERS] pgsql: Implement table partitioning.

2017-10-17 Thread Amit Langote
On 2017/10/18 1:52, Alvaro Herrera wrote: > Alvaro Herrera wrote: >> Robert Haas wrote: >>> Implement table partitioning. >> >> Is it intentional that you can use ALTER TABLE OWNER TO on the parent >> table, and that this does not recurse to modify the partitions' owners? >> This doesn't seem to be

[HACKERS] alter table doc fix

2017-10-17 Thread Amit Langote
Hi. Noticed that a alter table sub-command's name in Description (where it's OWNER) differs from that in synopsis (where it's OWNER TO). Attached patch to make them match, if the difference is unintentional. Thanks, Amit diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_ta

Re: [HACKERS] Re: Is anything preventing us from allowing write to foreign tables from standby?

2017-10-17 Thread Craig Ringer
On 18 October 2017 at 02:01, Alexander Korotkov wrote: > On Wed, Sep 6, 2017 at 4:42 PM, Alexander Korotkov > wrote: >> >> We're currently blocking writing queries on standby if even they are >> modifying contents of foreign tables. But do we have serious reasons for >> that? >> Keeping in the m

Re: [HACKERS] [POC] Faster processing at Gather node

2017-10-17 Thread Andres Freund
Hi Everyone, On 2017-05-19 17:25:38 +0530, Rafia Sabih wrote: > While analysing the performance of TPC-H queries for the newly developed > parallel-operators, viz, parallel index, bitmap heap scan, etc. we noticed > that the time taken by gather node is significant. On investigation, as per > the

[HACKERS] v10 telease note for pg_basebackup refers to old --xlog-method argument

2017-10-17 Thread David G. Johnston
diff --git a/doc/src/sgml/release-10.sgml b/doc/src/sgml/release-10.sgml index 116f7224da..f1f7cfed5f 100644 --- a/doc/src/sgml/release-10.sgml +++ b/doc/src/sgml/release-10.sgml @@ -242,7 +242,7 @@ This changes pg_basebackup's - -X/--xlog-method default to stream. + -X/--w

Re: [HACKERS] Determine state of cluster (HA)

2017-10-17 Thread Vik Fearing
On 10/17/2017 08:40 PM, Joshua D. Drake wrote: > On 10/16/2017 07:31 PM, Craig Ringer wrote: >> On 17 October 2017 at 01:02, Joshua D. Drake >> wrote: >>> On 10/15/2017 07:39 PM, Craig Ringer wrote: > - Get info about master. We should finish merging recovery.conf into postgresql.conf.

Re: [HACKERS] pgbench: Skipping the creating primary keys after initialization

2017-10-17 Thread Fabien COELHO
Hello Masahiko-san, Attached the updated version patch. Applies, compiles, make check & tap test ok, doc is fine. All is well for me: the feature is useful, code is simple and clean, it is easy to invoke, easy to extend as well, which I'm planning to do once it gets in. I switched the pa

Re: [HACKERS] Windows warnings from VS 2017

2017-10-17 Thread Andres Freund
On 2017-10-11 18:20:24 -0400, Tom Lane wrote: > Well, it's not that much work to try it and see. I compared results > of this simplistic test case: > pgbench -S -c 1 -T 60 bench > (using a scale-factor-10 pgbench database) on current HEAD and HEAD > with the attached patch, which just loboto

Re: [HACKERS] [COMMITTERS] pgsql: Fix traversal of half-frozen update chains

2017-10-17 Thread Peter Geoghegan
On Tue, Oct 17, 2017 at 3:02 AM, Alvaro Herrera wrote: > Yeah, me too. If you see another way to fix the problem, let's discuss > it. I doubt that there is a better way. > I think a possible way is to avoid considering that the relfrozenxid > value computed by the caller is final. While that a

Re: [HACKERS] Determine state of cluster (HA)

2017-10-17 Thread Joshua D. Drake
On 10/16/2017 07:31 PM, Craig Ringer wrote: On 17 October 2017 at 01:02, Joshua D. Drake wrote: On 10/15/2017 07:39 PM, Craig Ringer wrote: - Get info about master. We should finish merging recovery.conf into postgresql.conf. Definitely. There's a patch from Abhijit Menon-Sen you could

[HACKERS] Re: Is anything preventing us from allowing write to foreign tables from standby?

2017-10-17 Thread Alexander Korotkov
On Wed, Sep 6, 2017 at 4:42 PM, Alexander Korotkov < a.korot...@postgrespro.ru> wrote: > We're currently blocking writing queries on standby if even they are > modifying contents of foreign tables. But do we have serious reasons for > that? > Keeping in the mind FDW-sharding, making FDW-tables wr

Re: [HACKERS] Partition-wise aggregation/grouping

2017-10-17 Thread Jeevan Chalke
On Tue, Oct 17, 2017 at 7:13 PM, Dilip Kumar wrote: > On Fri, Oct 13, 2017 at 12:06 PM, Jeevan Chalke > wrote: > > > While playing around with the patch I have noticed one regression with > the partial partition-wise aggregate. > > I am consistently able to reproduce this on my local machine. >

Re: [HACKERS] [COMMITTERS] pgsql: Implement table partitioning.

2017-10-17 Thread Alvaro Herrera
Alvaro Herrera wrote: > Robert Haas wrote: > > Implement table partitioning. > > Is it intentional that you can use ALTER TABLE OWNER TO on the parent > table, and that this does not recurse to modify the partitions' owners? > This doesn't seem to be mentioned in comments nor documentation, so it

Re: [HACKERS] SIGSEGV in BRIN autosummarize

2017-10-17 Thread Tom Lane
Alvaro Herrera writes: > cur_datname here seems corrupted -- it points halfway into cur_nspname, > which is also a corrupt value. Yeah. > And I think that's because we're not > checking that the namespace OID is a valid value before calling > get_namespace_name on it. No, because get_namespace_

Re: [HACKERS] [COMMITTERS] pgsql: Implement table partitioning.

2017-10-17 Thread Alvaro Herrera
Robert Haas wrote: > Implement table partitioning. Is it intentional that you can use ALTER TABLE OWNER TO on the parent table, and that this does not recurse to modify the partitions' owners? This doesn't seem to be mentioned in comments nor documentation, so it seems an oversight to me. Thought

Re: [HACKERS] PATCH: enabling parallel execution for cursors explicitly (experimental)

2017-10-17 Thread Tomas Vondra
On 10/17/2017 03:16 PM, Robert Haas wrote: > On Sat, Oct 14, 2017 at 2:14 PM, Tomas Vondra > wrote: >> I propose is to add a new cursor option (PARALLEL), which would allow >> parallel plans for that particular user-defined cursor. Attached is an >> experimental patch doing this (I'm sure there

Re: [HACKERS] Query got Killed with CTE.

2017-10-17 Thread Craig Ringer
On 17 October 2017 at 21:18, Prabhat Sahu wrote: > Hi, > > While quering with CTE against PG HEAD , i found that query got killed > with this below error logs > -- Machine Configuration: (d1.xlarge) CUPs : 8 , RAM : 16GB , SIze : 640GB > > postgres=# with x as (select 5 c1 from generate_series(1

Re: [HACKERS] SIGSEGV in BRIN autosummarize

2017-10-17 Thread Craig Ringer
On 17 October 2017 at 22:39, Tom Lane wrote: > Justin Pryzby writes: >> On Tue, Oct 17, 2017 at 09:34:24AM -0400, Tom Lane wrote: >>> So: where did you get the existing binaries? If it's from some vendor >>> packaging system, what you should do is fetch the package source, add >>> the patch to t

Re: [HACKERS] SIGSEGV in BRIN autosummarize

2017-10-17 Thread Alvaro Herrera
Justin Pryzby wrote: > I'm happy to try the patch, but in case it makes any difference, we have few > DBs/schemas: I don't expect that it does. -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsq

Re: [HACKERS] SIGSEGV in BRIN autosummarize

2017-10-17 Thread Tom Lane
Justin Pryzby writes: > On Tue, Oct 17, 2017 at 09:34:24AM -0400, Tom Lane wrote: >> So: where did you get the existing binaries? If it's from some vendor >> packaging system, what you should do is fetch the package source, add >> the patch to the probably-nonempty set of patches the vendor is ap

Re: [HACKERS] SIGSEGV in BRIN autosummarize

2017-10-17 Thread Justin Pryzby
On Tue, Oct 17, 2017 at 09:34:24AM -0400, Tom Lane wrote: > Justin Pryzby writes: > > On Tue, Oct 17, 2017 at 12:59:16PM +0200, Alvaro Herrera wrote: > >> Anyway, can give this patch a try? > > > I've only compiled postgres once before and this is a production environment > > (althought nothing s

Re: [HACKERS] [PATCH] Add recovery_min_apply_delay_reconnect recovery option

2017-10-17 Thread Michael Paquier
On Tue, Oct 17, 2017 at 10:40 PM, Eric Radman wrote: > On Tue, Oct 17, 2017 at 12:34:17PM +0900, Michael Paquier wrote: > I thought I had observed cases where the WalReceiver was shut down > without causing XLogCtl->recoveryWakeupLatch to return. If I'm wrong > about this then there's no reason to

Re: [HACKERS] Partition-wise aggregation/grouping

2017-10-17 Thread Dilip Kumar
On Fri, Oct 13, 2017 at 12:06 PM, Jeevan Chalke wrote: > While playing around with the patch I have noticed one regression with the partial partition-wise aggregate. I am consistently able to reproduce this on my local machine. Scenario: Group by on non-key column and only one tuple per group.

Re: [HACKERS] [PATCH] Add recovery_min_apply_delay_reconnect recovery option

2017-10-17 Thread Eric Radman
On Tue, Oct 17, 2017 at 12:34:17PM +0900, Michael Paquier wrote: > On Tue, Oct 17, 2017 at 12:51 AM, Eric Radman wrote: > > This administrative compromise is necessary because the WalReceiver is > > not resumed after a network interruption until all records are read, > > verified, and applied from

Re: [HACKERS] SIGSEGV in BRIN autosummarize

2017-10-17 Thread Tom Lane
Justin Pryzby writes: > On Tue, Oct 17, 2017 at 12:59:16PM +0200, Alvaro Herrera wrote: >> Anyway, can give this patch a try? > I've only compiled postgres once before and this is a production environment > (althought nothing so important that the crashes are a serious concern > either). > Is i

[HACKERS] Query got Killed with CTE.

2017-10-17 Thread Prabhat Sahu
Hi, While quering with CTE against PG HEAD , i found that query got killed with this below error logs -- Machine Configuration: (d1.xlarge) CUPs : 8 , RAM : 16GB , SIze : 640GB postgres=# with x as (select 5 c1 from generate_series(1,100) x) select * from x x1 join x x2 using(c1); Killed 201

Re: [HACKERS] PATCH: enabling parallel execution for cursors explicitly (experimental)

2017-10-17 Thread Robert Haas
On Sat, Oct 14, 2017 at 2:14 PM, Tomas Vondra wrote: > I propose is to add a new cursor option (PARALLEL), which would allow > parallel plans for that particular user-defined cursor. Attached is an > experimental patch doing this (I'm sure there are some loose ends). I think you would need to do

Re: [HACKERS] SIGSEGV in BRIN autosummarize

2017-10-17 Thread Tomas Vondra
On 10/17/2017 02:29 PM, Justin Pryzby wrote: > On Tue, Oct 17, 2017 at 12:59:16PM +0200, Alvaro Herrera wrote: >> Anyway, can give this patch a try? > > I've only compiled postgres once before and this is a production environment > (althought nothing so important that the crashes are a serious con

Re: [HACKERS] SIGSEGV in BRIN autosummarize

2017-10-17 Thread Justin Pryzby
On Tue, Oct 17, 2017 at 12:59:16PM +0200, Alvaro Herrera wrote: > Justin Pryzby wrote: > > > #1 0x006a52e9 in perform_work_item (workitem=0x7f8ad1f94824) at > > autovacuum.c:2676 > > cur_datname = 0x298c740 "no 1 :vartype 1184 :vartypmod -1 > > :varcollid 0 :varlevelsup 0 :varno

Re: [HACKERS] SIGSEGV in BRIN autosummarize

2017-10-17 Thread Justin Pryzby
On Tue, Oct 17, 2017 at 12:59:16PM +0200, Alvaro Herrera wrote: > Anyway, can give this patch a try? I've only compiled postgres once before and this is a production environment (althought nothing so important that the crashes are a serious concern either). Is it reasonable to wget the postgres t

[HACKERS] A GUC to prevent leader processes from running subplans?

2017-10-17 Thread Thomas Munro
Hi hackers, While testing parallelism work I've wanted to be able to prevent gather nodes from running the plan in the leader process, and I've heard others say the same. One way would be to add a GUC "multiplex_gather", like in the attached patch. If you set it to off, Gather and Gather Merge w

Re: [HACKERS] SIGSEGV in BRIN autosummarize

2017-10-17 Thread Alvaro Herrera
Justin Pryzby wrote: > #1 0x006a52e9 in perform_work_item (workitem=0x7f8ad1f94824) at > autovacuum.c:2676 > cur_datname = 0x298c740 "no 1 :vartype 1184 :vartypmod -1 :varcollid > 0 :varlevelsup 0 :varnoold 1 :varoattno 1 :location 146} {CONST :consttype > 1184 :consttypmod -1

Re: [HACKERS] [COMMITTERS] pgsql: Fix traversal of half-frozen update chains

2017-10-17 Thread Alvaro Herrera
Robert Haas wrote: > I haven't really followed this thread in depth, but I'm very nervous > about the idea that we should ever be examining the raw-xmin of a > tuple that has been marked HEAP_XMIN_FROZEN for anything other than > forensic purposes. Yeah, me too. If you see another way to fix the

Re: [HACKERS] SIGSEGV in BRIN autosummarize

2017-10-17 Thread Alvaro Herrera
Justin Pryzby wrote: > On Sun, Oct 15, 2017 at 02:44:58PM +0200, Tomas Vondra wrote: > > Thanks, but I'm not sure that'll help, at this point. We already know > > what happened (corrupted memory), we don't know "how". And core files > > are mostly just "snapshots" so are not very useful in answerin

Re: [HACKERS] [PATCH] pageinspect function to decode infomasks

2017-10-17 Thread Masahiko Sawada
On Thu, Sep 14, 2017 at 11:00 PM, Craig Ringer wrote: > On 14 September 2017 at 19:57, Ashutosh Sharma > wrote: > >> >> >> Are you planning to work on the review comments from Robert, Moon >> Insung and supply the new patch. I just had a quick glance into this >> mail thread (after a long time) a