Re: [HACKERS] Parallel Index Scans

2016-12-22 Thread tushar
On 12/22/2016 01:35 PM, tushar wrote: On 12/22/2016 09:49 AM, Amit Kapila wrote: I think you can focus on the handling of array scan keys for testing. In general, one of my colleagues has shown interest in testing this patch and I think he has tested as well but never posted his findings. I

Re: [HACKERS] multi-level partitions and partition-wise joins

2016-12-22 Thread Ashutosh Bapat
Another question: do we want to commit the code for creating unflattened hierarchy separately or along with partition-wise join? On Fri, Dec 23, 2016 at 9:58 AM, Ashutosh Bapat wrote: > On Thu, Dec 22, 2016 at 10:52 PM, Robert Haas wrote:

[HACKERS] Parallel Append implementation

2016-12-22 Thread Amit Khandekar
Currently an Append plan node does not execute its subplans in parallel. There is no distribution of workers across its subplans. The second subplan starts running only after the first subplan finishes, although the individual subplans may be running parallel scans. Secondly, we create a partial

Re: [HACKERS] Parallel Index-only scan

2016-12-22 Thread rafia.sabih
Please find the attached file for the latest version of patch. parallel_index_only_v2.patch -- View this message in context: http://postgresql.nabble.com/Parallel-Index-only-scan-tp5934352p5936010.html Sent from the

Re: [HACKERS] multi-level partitions and partition-wise joins

2016-12-22 Thread Ashutosh Bapat
On Thu, Dec 22, 2016 at 10:52 PM, Robert Haas wrote: > On Wed, Dec 21, 2016 at 11:31 PM, Ashutosh Bapat > wrote: >> Given the scenario described above, it looks like we have to retain >> partition hierarchy in the form of inheritance

Re: [HACKERS] [patch] psql tab completion for ALTER DEFAULT PRIVILEGES

2016-12-22 Thread Stephen Frost
Gilles, * Gilles Darold (gilles.dar...@dalibo.com) wrote: > Added to next commitfest. To explain more this patch, the completion of > SQL command: > > ALTER DEFAULT PRIVILEGES FOR ROLE xxx [tab] Here is a cleaned up patch for master and 9.6. Tomorrow I'll look into what we can do for 9.5

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

2016-12-22 Thread Amit Kapila
On Thu, Dec 22, 2016 at 6:59 PM, Tomas Vondra wrote: > Hi, > > But as discussed with Amit in Tokyo at pgconf.asia, I got access to a > Power8e machine (IBM 8247-22L to be precise). It's a much smaller machine > compared to the x86 one, though - it only has 24 cores

Re: [HACKERS] RLS related docs

2016-12-22 Thread Joe Conway
On 09/15/2016 02:34 PM, Joe Conway wrote: > On 09/15/2016 01:33 PM, Robert Haas wrote: >> On Sun, Aug 28, 2016 at 4:23 PM, Joe Conway wrote: > For COPY, I think perhaps it would be more logical to put the new note > immediately after the third note which describes the

Re: [HACKERS] Fix checkpoint skip logic on idle systems by tracking LSN progress

2016-12-22 Thread Michael Paquier
On Fri, Dec 23, 2016 at 8:13 AM, Robert Haas wrote: > On Thu, Dec 22, 2016 at 2:34 PM, Andres Freund wrote: >> On 2016-12-22 08:32:56 -0800, Andres Freund wrote: >>> I plan to commit this later today. Hope I got the reviewers roughly right. >> >> And

Re: [HACKERS] varlena beyond 1GB and matrix

2016-12-22 Thread Kohei KaiGai
2016-12-23 8:24 GMT+09:00 Robert Haas : > On Wed, Dec 7, 2016 at 11:01 PM, Kohei KaiGai wrote: >> Regardless of the ExpandedObject, does the flatten format need to >> contain fully flatten data chunks? > > I suspect it does, and I think that's why this

Re: [HACKERS] varlena beyond 1GB and matrix

2016-12-22 Thread Kohei KaiGai
2016-12-23 8:23 GMT+09:00 Robert Haas : > On Wed, Dec 7, 2016 at 10:44 PM, Kohei KaiGai wrote: >>> Handling objects >1GB at all seems like the harder part of the >>> problem. >>> >> I could get your point almost. Does the last line above mention about

Re: [HACKERS] Potential data loss of 2PC files

2016-12-22 Thread Michael Paquier
On Fri, Dec 23, 2016 at 6:33 AM, Jim Nasby wrote: > On 12/22/16 12:02 PM, Andres Freund wrote: >> >> >> On December 22, 2016 6:44:22 PM GMT+01:00, Robert Haas >> wrote: >>> >>> On Thu, Dec 22, 2016 at 12:39 PM, Andres Freund

Re: [HACKERS] Write Ahead Logging for Hash Indexes

2016-12-22 Thread Amit Kapila
On Thu, Dec 22, 2016 at 9:56 PM, Robert Haas wrote: > On Mon, Dec 5, 2016 at 2:46 AM, Amit Kapila wrote: >>> I'll review after that, since I have other things to review meanwhile. >> >> Attached, please find the rebased patch attached with this

Re: [HACKERS] DROP FUNCTION of multiple functions

2016-12-22 Thread Robert Haas
On Thu, Dec 1, 2016 at 9:32 PM, Peter Eisentraut wrote: > I think it would be better to get rid of objargs and have objname be a > general Node that can contain more specific node types so that there is > some amount of type tracking. FuncWithArgs would be one

Re: [HACKERS] Re: [COMMITTERS] pgsql: Build HTML documentation using XSLT stylesheets by default

2016-12-22 Thread Tom Lane
Peter Eisentraut writes: > On 11/30/16 11:37 PM, Tom Lane wrote: >> OK, I updated docbook-style-xsl to 1.79.1 from Fedora rawhide (building >> and installing that was quite painless btw, didn't need a pile of build >> dependencies like I'd feared it would take).

Re: [HACKERS] varlena beyond 1GB and matrix

2016-12-22 Thread Robert Haas
On Wed, Dec 7, 2016 at 11:01 PM, Kohei KaiGai wrote: > Regardless of the ExpandedObject, does the flatten format need to > contain fully flatten data chunks? I suspect it does, and I think that's why this isn't going to get very far without a super-varlena format. --

Re: [HACKERS] varlena beyond 1GB and matrix

2016-12-22 Thread Robert Haas
On Wed, Dec 7, 2016 at 10:44 PM, Kohei KaiGai wrote: >> Handling objects >1GB at all seems like the harder part of the >> problem. >> > I could get your point almost. Does the last line above mention about > amount of the data object >1GB? even if the "super-varlena" format >

Re: [HACKERS] Fix checkpoint skip logic on idle systems by tracking LSN progress

2016-12-22 Thread Robert Haas
On Thu, Dec 22, 2016 at 2:34 PM, Andres Freund wrote: > On 2016-12-22 08:32:56 -0800, Andres Freund wrote: >> I plan to commit this later today. Hope I got the reviewers roughly right. > > And pushed. Thanks for the work on this everyone. Cool. Also, +1 for the

Re: [HACKERS] Declarative partitioning - another take

2016-12-22 Thread Robert Haas
On Thu, Dec 22, 2016 at 3:35 AM, Amit Langote wrote: > While working on that, I discovered yet-another-bug having to do with the > tuple descriptor that's used as we route a tuple down a partition tree. If > attnums of given key attribute(s) are different on

Re: [HACKERS] Re: [COMMITTERS] pgsql: Build HTML documentation using XSLT stylesheets by default

2016-12-22 Thread Peter Eisentraut
On 11/30/16 11:37 PM, Tom Lane wrote: > Peter Eisentraut writes: >> OK, I got it. The component of concern is the DocBook XSL stylesheets, >> called docbook-style-xsl on RH-like systems (docbook-xsl on Debian). If >> it runs too slow, it's probably too old. >

Re: [HACKERS] pg_background contrib module proposal

2016-12-22 Thread Robert Haas
On Thu, Dec 22, 2016 at 4:41 PM, Jim Nasby wrote: > On 12/22/16 4:20 AM, amul sul wrote: >> • pg_background_detach : This API takes the process id and detach the >> background process. Stored worker's session is not dropped until this >> called. > > When I hear "detach"

Re: [HACKERS] [patch] psql tab completion for ALTER DEFAULT PRIVILEGES

2016-12-22 Thread Stephen Frost
Gilles, * Gilles Darold (gilles.dar...@dalibo.com) wrote: > Le 20/11/2016 à 15:46, Gilles Darold a écrit : > > When tab-completing after ALTER DEFAULT PRIVILEGES ... GRANT|REVOKE, > > currently psql injects completion from the GRANT|REVOKE order, rather > > than the one expected. > > > > A patch

Re: [HACKERS] [BUG?] pg_event_trigger_ddl_commands() error with ALTER TEXT SEARCH CONFIGURATION

2016-12-22 Thread Stephen Frost
Artur, * Artur Zakirov (a.zaki...@postgrespro.ru) wrote: > 2016-12-21 20:34 GMT+03:00 Stephen Frost : > > Did you happen to look at adding a regression test for this to > > test_ddl_deparse? > > Of course. I updated the patch. I added a few comments and back-patched the

Re: [HACKERS] Indirect indexes

2016-12-22 Thread Alvaro Herrera
Here's a still-WIP patch that's a better candidate for inclusion. In this patch, I have created an executor node for indirect index scans. This node is created whenever an indexscan path is chosen for an indirect index. The planner treats indirect indexes just like regular indexes, except that

Re: [HACKERS] pg_background contrib module proposal

2016-12-22 Thread Jim Nasby
On 12/22/16 4:20 AM, amul sul wrote: • pg_background_detach : This API takes the process id and detach the background process. Stored worker's session is not dropped until this called. When I hear "detach" I think that whatever I'm detaching from is going to stick around, which I don't think

Re: [HACKERS] Potential data loss of 2PC files

2016-12-22 Thread Jim Nasby
On 12/22/16 12:02 PM, Andres Freund wrote: On December 22, 2016 6:44:22 PM GMT+01:00, Robert Haas wrote: On Thu, Dec 22, 2016 at 12:39 PM, Andres Freund wrote: It makes more sense of you mentally separate between filename(s) and file contents.

Re: [HACKERS] Fix checkpoint skip logic on idle systems by tracking LSN progress

2016-12-22 Thread Andres Freund
Hi, On 2016-12-22 08:32:56 -0800, Andres Freund wrote: > I plan to commit this later today. Hope I got the reviewers roughly right. And pushed. Thanks for the work on this everyone. Andres -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your

Re: [HACKERS] Ilegal type cast in _hash_doinsert()

2016-12-22 Thread Robert Haas
On Thu, Dec 22, 2016 at 2:04 AM, Mithun Cy wrote: > There seems to be some base bug in _hash_doinsert(). > +* XXX this is useless code if we are only storing hash keys. > > + */ > > +if (itemsz > HashMaxItemSize((Page) metap)) > > +ereport(ERROR, > >

[HACKERS] Read/write expanded objects versus domains and CASE

2016-12-22 Thread Tom Lane
I looked into the problem reported in bug #14472, https://www.postgresql.org/message-id/20161221214744.25622.71...@wrigleys.postgresql.org Although the submitter thought it was related to bug #14414, it isn't particularly. The failure scenario is that the input value to a CoerceToDomain execution

Re: [HACKERS] Vacuum: allow usage of more than 1GB of work mem

2016-12-22 Thread Claudio Freire
On Thu, Dec 22, 2016 at 12:22 PM, Claudio Freire wrote: > On Thu, Dec 22, 2016 at 12:15 PM, Anastasia Lubennikova > wrote: >> The following review has been posted through the commitfest application: >> make installcheck-world: tested, failed >>

Re: [HACKERS] increasing the default WAL segment size

2016-12-22 Thread Tomas Vondra
On 12/20/2016 02:19 PM, Andres Freund wrote: On 2016-12-20 08:10:29 -0500, Robert Haas wrote: We could use the GUC assign hook to compute a mask and a shift, so that this could be written as (CurrPos & mask_variable) == 0. That would avoid the division instruction, though not the memory

Re: [HACKERS] Potential data loss of 2PC files

2016-12-22 Thread Andres Freund
On December 22, 2016 6:44:22 PM GMT+01:00, Robert Haas wrote: >On Thu, Dec 22, 2016 at 12:39 PM, Andres Freund >wrote: >> It makes more sense of you mentally separate between filename(s) and >file contents. Having to do filesystem metatata

Re: [HACKERS] An attempt to reduce WALWriteLock contention

2016-12-22 Thread Tomas Vondra
On 12/22/2016 04:00 PM, Kuntal Ghosh wrote: Hello all, > ... > \t select wait_event_type, wait_event from pg_stat_activity where pid != pg_backend_pid(); \watch 0.5 HEAD 48642 LWLockNamed | WALWriteLock With Patch -- 31889 LWLockNamed

Re: [HACKERS] Getting rid of "unknown error" in dblink and postgres_fdw

2016-12-22 Thread Joe Conway
On 12/22/2016 06:55 AM, Tom Lane wrote: > Joe Conway writes: >> On 12/21/2016 09:20 PM, Tom Lane wrote: >>> I see that you need to pass the PGconn into dblink_res_error() now, but >>> what's the point of the new "bool fail" parameter? > >> That part isn't new -- we added it

Re: [HACKERS] [COMMITTERS] pgsql: Implement table partitioning.

2016-12-22 Thread Robert Haas
On Wed, Dec 21, 2016 at 8:00 PM, Amit Langote wrote: > On 2016/12/22 0:31, Robert Haas wrote: >> On Tue, Dec 20, 2016 at 12:22 PM, Alvaro Herrera >> wrote: >>> Robert Haas wrote: Implement table partitioning. >>> >>> I thought it was

Re: [HACKERS] Potential data loss of 2PC files

2016-12-22 Thread Robert Haas
On Thu, Dec 22, 2016 at 12:39 PM, Andres Freund wrote: > It makes more sense of you mentally separate between filename(s) and file > contents. Having to do filesystem metatata transactions for an fsync > intended to sync contents would be annoying... I thought that's why

Re: [HACKERS] Potential data loss of 2PC files

2016-12-22 Thread Andres Freund
On December 22, 2016 5:50:38 PM GMT+01:00, Robert Haas wrote: >On Wed, Dec 21, 2016 at 8:30 PM, Michael Paquier > wrote: >> Hi all, >> >> 2PC files are created using RecreateTwoPhaseFile() in two places >currently: >> - at replay on a

Re: [HACKERS] dblink get_connect_string() passes FDW option "updatable" to the connect string, connection fails.

2016-12-22 Thread Joe Conway
On 12/18/2016 02:47 PM, Corey Huinker wrote: > On Sun, Dec 18, 2016 at 4:57 PM, Michael Paquier wrote: >> On Mon, Dec 19, 2016 at 6:48 AM, Joe Conway wrote: >>> Maybe if "CREATE FOREIGN DATA WRAPPER" had a way to specify that the FDW >>> supports a libpq connection it would make sense to allows

Re: [HACKERS] [PATCH] Transaction traceability - txid_status(bigint)

2016-12-22 Thread Robert Haas
On Thu, Dec 22, 2016 at 12:12 AM, Craig Ringer wrote: > On 22 December 2016 at 09:55, Craig Ringer wrote: > >> Updated. >> >> If you think it's better to just take XidGenLock again, let me know. > > Here's a further update that merges in Robert's

Re: [HACKERS] multi-level partitions and partition-wise joins

2016-12-22 Thread Robert Haas
On Wed, Dec 21, 2016 at 11:31 PM, Ashutosh Bapat wrote: > Given the scenario described above, it looks like we have to retain > partition hierarchy in the form of inheritance hierarchy in order to > implement partition-wise joins for multi-leveled partition

Re: [HACKERS] Potential data loss of 2PC files

2016-12-22 Thread Robert Haas
On Wed, Dec 21, 2016 at 8:30 PM, Michael Paquier wrote: > Hi all, > > 2PC files are created using RecreateTwoPhaseFile() in two places currently: > - at replay on a XLOG_XACT_PREPARE record. > - At checkpoint with CheckPointTwoPhase(). > > Now RecreateTwoPhaseFile() is

Re: [HACKERS] Fix checkpoint skip logic on idle systems by tracking LSN progress

2016-12-22 Thread Andres Freund
Hi, On 2016-12-21 13:28:54 -0800, Andres Freund wrote: > A mime-type of invalid/octet-stream? That's an, uh, odd choice. > > Working on committing this (tomorrow morning, not tonight). There's > some relatively minor things I want to change: > > - I don't like the name XLogSetFlags() - it's

Re: [HACKERS] Write Ahead Logging for Hash Indexes

2016-12-22 Thread Robert Haas
On Mon, Dec 5, 2016 at 2:46 AM, Amit Kapila wrote: >> I'll review after that, since I have other things to review meanwhile. > > Attached, please find the rebased patch attached with this e-mail. > There is no fundamental change in patch except for adapting the new >

Re: [HACKERS] patch: function xmltable

2016-12-22 Thread Alvaro Herrera
Pavel Stehule wrote: > another update - lot of cleaning Ah, the tupledesc stuff in this one appears much more reasonable to me. -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers

Re: [HACKERS] SET NOT NULL [NOT VALID / CONCURRENTLY]?

2016-12-22 Thread Robert Haas
On Wed, Dec 21, 2016 at 7:55 PM, Joel Jacobson wrote: > Attached is the function SET_NOT_NULL(_Schema name, _Table name, _Column > name) which does the following: > > 1. LOCK TABLE %I.%I IN ACCESS EXCLUSIVE MODE > just like the normal DDL commands would do > > 2. SELECT EXISTS

Re: [HACKERS] Hang in pldebugger after git commit : 98a64d0

2016-12-22 Thread Robert Haas
On Wed, Dec 21, 2016 at 9:34 PM, Amit Kapila wrote: > On Wed, Dec 21, 2016 at 9:46 PM, Robert Haas wrote: >> On Tue, Dec 20, 2016 at 11:32 PM, Amit Kapila >> wrote: >>> Ashutosh Sharma has helped to test that pldebugger

Re: [HACKERS] postgres_fdw bug in 9.6

2016-12-22 Thread Robert Haas
On Wed, Dec 21, 2016 at 11:04 AM, Ashutosh Bapat wrote: > Some review comments > > 1. postgres_fdw doesn't push down semi and anti joins so you may want to > discount these two too. > + jointype == JOIN_SEMI || > + jointype == JOIN_ANTI); But

Re: [HACKERS] Vacuum: allow usage of more than 1GB of work mem

2016-12-22 Thread Claudio Freire
On Thu, Dec 22, 2016 at 12:15 PM, Anastasia Lubennikova wrote: > The following review has been posted through the commitfest application: > make installcheck-world: tested, failed > Implements feature: not tested > Spec compliant: not tested >

Re: [HACKERS] Vacuum: allow usage of more than 1GB of work mem

2016-12-22 Thread Anastasia Lubennikova
The following review has been posted through the commitfest application: make installcheck-world: tested, failed Implements feature: not tested Spec compliant: not tested Documentation:not tested Hi, I haven't read through the thread yet, just tried to apply the patch

Re: [HACKERS] Minor correction in alter_table.sgml

2016-12-22 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: > Alvaro Herrera writes: > >> I had considered removing those but thought that some users might think > >> that they're only "altering" one table and therefore felt it made sense > >> to keep those explicitly listed. > > > I agree

[HACKERS] An attempt to reduce WALWriteLock contention

2016-12-22 Thread Kuntal Ghosh
Hello all, In a recent post[1] by Robert, wait events for different LWLOCKS have been analyzed. The results clearly indicate a significant lock contention overhead on WAL Write locks. To get an idea of this overhead, we did the following two tests. 1. Hacked the code to comment out WAL write and

Re: [HACKERS] Getting rid of "unknown error" in dblink and postgres_fdw

2016-12-22 Thread Tom Lane
Joe Conway writes: > On 12/21/2016 09:20 PM, Tom Lane wrote: >> I see that you need to pass the PGconn into dblink_res_error() now, but >> what's the point of the new "bool fail" parameter? > That part isn't new -- we added it sometime prior to 9.2: Oh! I misread the patch

Re: [HACKERS] Minor correction in alter_table.sgml

2016-12-22 Thread Tom Lane
Alvaro Herrera writes: >> I had considered removing those but thought that some users might think >> that they're only "altering" one table and therefore felt it made sense >> to keep those explicitly listed. > I agree with Stephen; it's not crystal clear from the

Re: [HACKERS] Minor correction in alter_table.sgml

2016-12-22 Thread Alvaro Herrera
Stephen Frost wrote: > Tom, > > * Tom Lane (t...@sss.pgh.pa.us) wrote: > > BTW, so far as the HEAD version of this patch goes, I notice that > > ATTACH PARTITION and DETACH PARTITION were recently added to the > > list of exceptions. But they're not exceptions according to this > > wording:

Re: [HACKERS] Minor correction in alter_table.sgml

2016-12-22 Thread Stephen Frost
Tom, * Tom Lane (t...@sss.pgh.pa.us) wrote: > Stephen Frost writes: > > * Tom Lane (t...@sss.pgh.pa.us) wrote: > >> So maybe something like > >> > >> All the forms of ALTER TABLE that act on a single table, > >> except RENAME and SET SCHEMA, can be combined into a > >> list

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

2016-12-22 Thread Tomas Vondra
Hi, The attached results show that: (a) master shows the same zig-zag behavior - No idea why this wasn't observed on the previous runs. (b) group_update actually seems to improve the situation, because the performance keeps stable up to 72 clients, while on master the fluctuation starts way

Re: [HACKERS] pg_background contrib module proposal

2016-12-22 Thread amul sul
Hi all, As we have discussed previously, we need to rework this patch as a client of Peter Eisentraut's background sessions code[1]. Attaching trial version patch to discussed possible design and api. We could have following APIs : • pg_background_launch : This function start and stores new

Re: [HACKERS] Fix checkpoint skip logic on idle systems by tracking LSN progress

2016-12-22 Thread Amit Kapila
On Thu, Dec 22, 2016 at 3:10 AM, Andres Freund wrote: > On 2016-12-21 16:35:28 -0500, Robert Haas wrote: >> On Wed, Dec 21, 2016 at 4:28 PM, Andres Freund wrote: >> > - Similarly I don't like the name "progress LSN" much. What does >> > "progress" really

Re: [HACKERS] Proposal for CSN based snapshots

2016-12-22 Thread Amit Kapila
On Wed, Aug 24, 2016 at 2:24 PM, Heikki Linnakangas wrote: > > And here are the results on the 72 core machine (thanks again, Alexander!). > The test setup was the same as on the 32-core machine, except that I ran it > with more clients since the system has more CPU cores. In

Re: [HACKERS] Declarative partitioning - another take

2016-12-22 Thread Amit Langote
On 2016/12/22 1:50, Robert Haas wrote: > On Wed, Dec 21, 2016 at 5:33 AM, Amit Langote > wrote: >> Breaking changes into multiple commits/patches does not seem to work for >> adding regression tests. So, I've combined multiple patches into a single >> patch which