Re: [HACKERS] Parallel Index Scans

On 12/22/2016 01:35 PM, tushar wrote: On 12/22/2016 09:49 AM, Amit Kapila wrote: I think you can focus on the handling of array scan keys for testing. In general, one of my colleagues has shown interest in testing this patch and I think he has tested as well but never posted his findings. I will

Re: [HACKERS] multi-level partitions and partition-wise joins

Another question: do we want to commit the code for creating unflattened hierarchy separately or along with partition-wise join? On Fri, Dec 23, 2016 at 9:58 AM, Ashutosh Bapat wrote: > On Thu, Dec 22, 2016 at 10:52 PM, Robert Haas wrote: >> On Wed, Dec 21, 2016 at 11:31 PM, Ashutosh Bapat >> w

[HACKERS] Parallel Append implementation

Currently an Append plan node does not execute its subplans in parallel. There is no distribution of workers across its subplans. The second subplan starts running only after the first subplan finishes, although the individual subplans may be running parallel scans. Secondly, we create a partial A

Re: [HACKERS] Parallel Index-only scan

Please find the attached file for the latest version of patch. parallel_index_only_v2.patch -- View this message in context: http://postgresql.nabble.com/Parallel-Index-only-scan-tp5934352p5936010.html Sent from the Po

Re: [HACKERS] multi-level partitions and partition-wise joins

On Thu, Dec 22, 2016 at 10:52 PM, Robert Haas wrote: > On Wed, Dec 21, 2016 at 11:31 PM, Ashutosh Bapat > wrote: >> Given the scenario described above, it looks like we have to retain >> partition hierarchy in the form of inheritance hierarchy in order to >> implement partition-wise joins for mul

Re: [HACKERS] [patch] psql tab completion for ALTER DEFAULT PRIVILEGES

Gilles, * Gilles Darold (gilles.dar...@dalibo.com) wrote: > Added to next commitfest. To explain more this patch, the completion of > SQL command: > > ALTER DEFAULT PRIVILEGES FOR ROLE xxx [tab] Here is a cleaned up patch for master and 9.6. Tomorrow I'll look into what we can do for 9.5 an

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

On Thu, Dec 22, 2016 at 6:59 PM, Tomas Vondra wrote: > Hi, > > But as discussed with Amit in Tokyo at pgconf.asia, I got access to a > Power8e machine (IBM 8247-22L to be precise). It's a much smaller machine > compared to the x86 one, though - it only has 24 cores in 2 sockets, 128GB > of RAM and

Re: [HACKERS] RLS related docs

On 09/15/2016 02:34 PM, Joe Conway wrote: > On 09/15/2016 01:33 PM, Robert Haas wrote: >> On Sun, Aug 28, 2016 at 4:23 PM, Joe Conway wrote: > For COPY, I think perhaps it would be more logical to put the new note > immediately after the third note which describes the privileges > requ

Re: [HACKERS] Fix checkpoint skip logic on idle systems by tracking LSN progress

On Fri, Dec 23, 2016 at 8:13 AM, Robert Haas wrote: > On Thu, Dec 22, 2016 at 2:34 PM, Andres Freund wrote: >> On 2016-12-22 08:32:56 -0800, Andres Freund wrote: >>> I plan to commit this later today. Hope I got the reviewers roughly right. >> >> And pushed. Thanks for the work on this everyone.

Re: [HACKERS] varlena beyond 1GB and matrix

2016-12-23 8:24 GMT+09:00 Robert Haas : > On Wed, Dec 7, 2016 at 11:01 PM, Kohei KaiGai wrote: >> Regardless of the ExpandedObject, does the flatten format need to >> contain fully flatten data chunks? > > I suspect it does, and I think that's why this isn't going to get very > far without a super

Re: [HACKERS] varlena beyond 1GB and matrix

2016-12-23 8:23 GMT+09:00 Robert Haas : > On Wed, Dec 7, 2016 at 10:44 PM, Kohei KaiGai wrote: >>> Handling objects >1GB at all seems like the harder part of the >>> problem. >>> >> I could get your point almost. Does the last line above mention about >> amount of the data object >1GB? even if the

Re: [HACKERS] Potential data loss of 2PC files

On Fri, Dec 23, 2016 at 6:33 AM, Jim Nasby wrote: > On 12/22/16 12:02 PM, Andres Freund wrote: >> >> >> On December 22, 2016 6:44:22 PM GMT+01:00, Robert Haas >> wrote: >>> >>> On Thu, Dec 22, 2016 at 12:39 PM, Andres Freund >>> wrote: It makes more sense of you mentally separate betwe

Re: [HACKERS] Write Ahead Logging for Hash Indexes

On Thu, Dec 22, 2016 at 9:56 PM, Robert Haas wrote: > On Mon, Dec 5, 2016 at 2:46 AM, Amit Kapila wrote: >>> I'll review after that, since I have other things to review meanwhile. >> >> Attached, please find the rebased patch attached with this e-mail. >> There is no fundamental change in patch e

Re: [HACKERS] DROP FUNCTION of multiple functions

On Thu, Dec 1, 2016 at 9:32 PM, Peter Eisentraut wrote: > I think it would be better to get rid of objargs and have objname be a > general Node that can contain more specific node types so that there is > some amount of type tracking. FuncWithArgs would be one such type, > Typename would be anoth

Re: [HACKERS] Re: [COMMITTERS] pgsql: Build HTML documentation using XSLT stylesheets by default

Peter Eisentraut writes: > On 11/30/16 11:37 PM, Tom Lane wrote: >> OK, I updated docbook-style-xsl to 1.79.1 from Fedora rawhide (building >> and installing that was quite painless btw, didn't need a pile of build >> dependencies like I'd feared it would take). The extraneous commas are >> gone,

Re: [HACKERS] varlena beyond 1GB and matrix

On Wed, Dec 7, 2016 at 11:01 PM, Kohei KaiGai wrote: > Regardless of the ExpandedObject, does the flatten format need to > contain fully flatten data chunks? I suspect it does, and I think that's why this isn't going to get very far without a super-varlena format. -- Robert Haas EnterpriseDB: h

Re: [HACKERS] varlena beyond 1GB and matrix

On Wed, Dec 7, 2016 at 10:44 PM, Kohei KaiGai wrote: >> Handling objects >1GB at all seems like the harder part of the >> problem. >> > I could get your point almost. Does the last line above mention about > amount of the data object >1GB? even if the "super-varlena" format > allows 64bit length?

Re: [HACKERS] Fix checkpoint skip logic on idle systems by tracking LSN progress

On Thu, Dec 22, 2016 at 2:34 PM, Andres Freund wrote: > On 2016-12-22 08:32:56 -0800, Andres Freund wrote: >> I plan to commit this later today. Hope I got the reviewers roughly right. > > And pushed. Thanks for the work on this everyone. Cool. Also, +1 for the important/unimportant terminology

Re: [HACKERS] Declarative partitioning - another take

On Thu, Dec 22, 2016 at 3:35 AM, Amit Langote wrote: > While working on that, I discovered yet-another-bug having to do with the > tuple descriptor that's used as we route a tuple down a partition tree. If > attnums of given key attribute(s) are different on different levels, it > would be incorre

Re: [HACKERS] Re: [COMMITTERS] pgsql: Build HTML documentation using XSLT stylesheets by default

On 11/30/16 11:37 PM, Tom Lane wrote: > Peter Eisentraut writes: >> OK, I got it. The component of concern is the DocBook XSL stylesheets, >> called docbook-style-xsl on RH-like systems (docbook-xsl on Debian). If >> it runs too slow, it's probably too old. > > OK, I updated docbook-style-xsl t

Re: [HACKERS] pg_background contrib module proposal

On Thu, Dec 22, 2016 at 4:41 PM, Jim Nasby wrote: > On 12/22/16 4:20 AM, amul sul wrote: >> • pg_background_detach : This API takes the process id and detach the >> background process. Stored worker's session is not dropped until this >> called. > > When I hear "detach" I think that whatever I'm d

Re: [HACKERS] [patch] psql tab completion for ALTER DEFAULT PRIVILEGES

Gilles, * Gilles Darold (gilles.dar...@dalibo.com) wrote: > Le 20/11/2016 à 15:46, Gilles Darold a écrit : > > When tab-completing after ALTER DEFAULT PRIVILEGES ... GRANT|REVOKE, > > currently psql injects completion from the GRANT|REVOKE order, rather > > than the one expected. > > > > A patch i

Re: [HACKERS] [BUG?] pg_event_trigger_ddl_commands() error with ALTER TEXT SEARCH CONFIGURATION

Artur, * Artur Zakirov (a.zaki...@postgrespro.ru) wrote: > 2016-12-21 20:34 GMT+03:00 Stephen Frost : > > Did you happen to look at adding a regression test for this to > > test_ddl_deparse? > > Of course. I updated the patch. I added a few comments and back-patched the relevant bits as discusse

Re: [HACKERS] Indirect indexes

Here's a still-WIP patch that's a better candidate for inclusion. In this patch, I have created an executor node for indirect index scans. This node is created whenever an indexscan path is chosen for an indirect index. The planner treats indirect indexes just like regular indexes, except that th

Re: [HACKERS] pg_background contrib module proposal

On 12/22/16 4:20 AM, amul sul wrote: • pg_background_detach : This API takes the process id and detach the background process. Stored worker's session is not dropped until this called. When I hear "detach" I think that whatever I'm detaching from is going to stick around, which I don't think i

Re: [HACKERS] Potential data loss of 2PC files

On 12/22/16 12:02 PM, Andres Freund wrote: On December 22, 2016 6:44:22 PM GMT+01:00, Robert Haas wrote: On Thu, Dec 22, 2016 at 12:39 PM, Andres Freund wrote: It makes more sense of you mentally separate between filename(s) and file contents. Having to do filesystem metatata transactions

Re: [HACKERS] Fix checkpoint skip logic on idle systems by tracking LSN progress

Hi, On 2016-12-22 08:32:56 -0800, Andres Freund wrote: > I plan to commit this later today. Hope I got the reviewers roughly right. And pushed. Thanks for the work on this everyone. Andres -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscript

Re: [HACKERS] Ilegal type cast in _hash_doinsert()

On Thu, Dec 22, 2016 at 2:04 AM, Mithun Cy wrote: > There seems to be some base bug in _hash_doinsert(). > +* XXX this is useless code if we are only storing hash keys. > > + */ > > +if (itemsz > HashMaxItemSize((Page) metap)) > > +ereport(ERROR, > > +(errcode(ERR

[HACKERS] Read/write expanded objects versus domains and CASE

I looked into the problem reported in bug #14472, https://www.postgresql.org/message-id/20161221214744.25622.71...@wrigleys.postgresql.org Although the submitter thought it was related to bug #14414, it isn't particularly. The failure scenario is that the input value to a CoerceToDomain execution

Re: [HACKERS] Vacuum: allow usage of more than 1GB of work mem

On Thu, Dec 22, 2016 at 12:22 PM, Claudio Freire wrote: > On Thu, Dec 22, 2016 at 12:15 PM, Anastasia Lubennikova > wrote: >> The following review has been posted through the commitfest application: >> make installcheck-world: tested, failed >> Implements feature: not tested >> Spec compli

Re: [HACKERS] increasing the default WAL segment size

On 12/20/2016 02:19 PM, Andres Freund wrote: On 2016-12-20 08:10:29 -0500, Robert Haas wrote: We could use the GUC assign hook to compute a mask and a shift, so that this could be written as (CurrPos & mask_variable) == 0. That would avoid the division instruction, though not the memory access.

Re: [HACKERS] Potential data loss of 2PC files

On December 22, 2016 6:44:22 PM GMT+01:00, Robert Haas wrote: >On Thu, Dec 22, 2016 at 12:39 PM, Andres Freund >wrote: >> It makes more sense of you mentally separate between filename(s) and >file contents. Having to do filesystem metatata transactions for an >fsync intended to sync contents

Re: [HACKERS] An attempt to reduce WALWriteLock contention

On 12/22/2016 04:00 PM, Kuntal Ghosh wrote: Hello all, > ... > \t select wait_event_type, wait_event from pg_stat_activity where pid != pg_backend_pid(); \watch 0.5 HEAD 48642 LWLockNamed | WALWriteLock With Patch -- 31889 LWLockNamed

Re: [HACKERS] Getting rid of "unknown error" in dblink and postgres_fdw

On 12/22/2016 06:55 AM, Tom Lane wrote: > Joe Conway writes: >> On 12/21/2016 09:20 PM, Tom Lane wrote: >>> I see that you need to pass the PGconn into dblink_res_error() now, but >>> what's the point of the new "bool fail" parameter? > >> That part isn't new -- we added it sometime prior to 9.2:

Re: [HACKERS] [COMMITTERS] pgsql: Implement table partitioning.

On Wed, Dec 21, 2016 at 8:00 PM, Amit Langote wrote: > On 2016/12/22 0:31, Robert Haas wrote: >> On Tue, Dec 20, 2016 at 12:22 PM, Alvaro Herrera >> wrote: >>> Robert Haas wrote: Implement table partitioning. >>> >>> I thought it was odd to use rd_rel->reloftype as a boolean in >>> ATExecAtt

Re: [HACKERS] Potential data loss of 2PC files

On Thu, Dec 22, 2016 at 12:39 PM, Andres Freund wrote: > It makes more sense of you mentally separate between filename(s) and file > contents. Having to do filesystem metatata transactions for an fsync > intended to sync contents would be annoying... I thought that's why there's fdatasync. --

Re: [HACKERS] Potential data loss of 2PC files

On December 22, 2016 5:50:38 PM GMT+01:00, Robert Haas wrote: >On Wed, Dec 21, 2016 at 8:30 PM, Michael Paquier > wrote: >> Hi all, >> >> 2PC files are created using RecreateTwoPhaseFile() in two places >currently: >> - at replay on a XLOG_XACT_PREPARE record. >> - At checkpoint with CheckPoint

Re: [HACKERS] dblink get_connect_string() passes FDW option "updatable" to the connect string, connection fails.

On 12/18/2016 02:47 PM, Corey Huinker wrote: > On Sun, Dec 18, 2016 at 4:57 PM, Michael Paquier wrote: >> On Mon, Dec 19, 2016 at 6:48 AM, Joe Conway wrote: >>> Maybe if "CREATE FOREIGN DATA WRAPPER" had a way to specify that the FDW >>> supports a libpq connection it would make sense to allows oth

Re: [HACKERS] [PATCH] Transaction traceability - txid_status(bigint)

On Thu, Dec 22, 2016 at 12:12 AM, Craig Ringer wrote: > On 22 December 2016 at 09:55, Craig Ringer wrote: > >> Updated. >> >> If you think it's better to just take XidGenLock again, let me know. > > Here's a further update that merges in Robert's changes from the patch > posted upthread. This pa

Re: [HACKERS] multi-level partitions and partition-wise joins

On Wed, Dec 21, 2016 at 11:31 PM, Ashutosh Bapat wrote: > Given the scenario described above, it looks like we have to retain > partition hierarchy in the form of inheritance hierarchy in order to > implement partition-wise joins for multi-leveled partition tables. Is > that the right thing to do?

Re: [HACKERS] Potential data loss of 2PC files

On Wed, Dec 21, 2016 at 8:30 PM, Michael Paquier wrote: > Hi all, > > 2PC files are created using RecreateTwoPhaseFile() in two places currently: > - at replay on a XLOG_XACT_PREPARE record. > - At checkpoint with CheckPointTwoPhase(). > > Now RecreateTwoPhaseFile() is careful to call pg_fsync() t

Re: [HACKERS] Fix checkpoint skip logic on idle systems by tracking LSN progress

Hi, On 2016-12-21 13:28:54 -0800, Andres Freund wrote: > A mime-type of invalid/octet-stream? That's an, uh, odd choice. > > Working on committing this (tomorrow morning, not tonight). There's > some relatively minor things I want to change: > > - I don't like the name XLogSetFlags() - it's compl

Re: [HACKERS] Write Ahead Logging for Hash Indexes

On Mon, Dec 5, 2016 at 2:46 AM, Amit Kapila wrote: >> I'll review after that, since I have other things to review meanwhile. > > Attached, please find the rebased patch attached with this e-mail. > There is no fundamental change in patch except for adapting the new > locking strategy in squeeze op

Re: [HACKERS] patch: function xmltable

Pavel Stehule wrote: > another update - lot of cleaning Ah, the tupledesc stuff in this one appears much more reasonable to me. -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailin

Re: [HACKERS] SET NOT NULL [NOT VALID / CONCURRENTLY]?

On Wed, Dec 21, 2016 at 7:55 PM, Joel Jacobson wrote: > Attached is the function SET_NOT_NULL(_Schema name, _Table name, _Column > name) which does the following: > > 1. LOCK TABLE %I.%I IN ACCESS EXCLUSIVE MODE > just like the normal DDL commands would do > > 2. SELECT EXISTS (SELECT 1 FROM %I.%

Re: [HACKERS] Hang in pldebugger after git commit : 98a64d0

On Wed, Dec 21, 2016 at 9:34 PM, Amit Kapila wrote: > On Wed, Dec 21, 2016 at 9:46 PM, Robert Haas wrote: >> On Tue, Dec 20, 2016 at 11:32 PM, Amit Kapila >> wrote: >>> Ashutosh Sharma has helped to test that pldebugger issue is fixed with >>> attached version. >> >> Committed. >> > > Thanks! >

Re: [HACKERS] postgres_fdw bug in 9.6

On Wed, Dec 21, 2016 at 11:04 AM, Ashutosh Bapat wrote: > Some review comments > > 1. postgres_fdw doesn't push down semi and anti joins so you may want to > discount these two too. > + jointype == JOIN_SEMI || > + jointype == JOIN_ANTI); But in the future, it might. We shoul

Re: [HACKERS] Vacuum: allow usage of more than 1GB of work mem

On Thu, Dec 22, 2016 at 12:15 PM, Anastasia Lubennikova wrote: > The following review has been posted through the commitfest application: > make installcheck-world: tested, failed > Implements feature: not tested > Spec compliant: not tested > Documentation:not tested

Re: [HACKERS] Vacuum: allow usage of more than 1GB of work mem

The following review has been posted through the commitfest application: make installcheck-world: tested, failed Implements feature: not tested Spec compliant: not tested Documentation:not tested Hi, I haven't read through the thread yet, just tried to apply the patch

Re: [HACKERS] Minor correction in alter_table.sgml

* Tom Lane (t...@sss.pgh.pa.us) wrote: > Alvaro Herrera writes: > >> I had considered removing those but thought that some users might think > >> that they're only "altering" one table and therefore felt it made sense > >> to keep those explicitly listed. > > > I agree with Stephen; it's not crys

[HACKERS] An attempt to reduce WALWriteLock contention

Hello all, In a recent post[1] by Robert, wait events for different LWLOCKS have been analyzed. The results clearly indicate a significant lock contention overhead on WAL Write locks. To get an idea of this overhead, we did the following two tests. 1. Hacked the code to comment out WAL write and

Re: [HACKERS] Getting rid of "unknown error" in dblink and postgres_fdw

Joe Conway writes: > On 12/21/2016 09:20 PM, Tom Lane wrote: >> I see that you need to pass the PGconn into dblink_res_error() now, but >> what's the point of the new "bool fail" parameter? > That part isn't new -- we added it sometime prior to 9.2: Oh! I misread the patch --- something about a

Re: [HACKERS] Minor correction in alter_table.sgml

Alvaro Herrera writes: >> I had considered removing those but thought that some users might think >> that they're only "altering" one table and therefore felt it made sense >> to keep those explicitly listed. > I agree with Stephen; it's not crystal clear from the user's POV that > the command is

Re: [HACKERS] Minor correction in alter_table.sgml

Stephen Frost wrote: > Tom, > > * Tom Lane (t...@sss.pgh.pa.us) wrote: > > BTW, so far as the HEAD version of this patch goes, I notice that > > ATTACH PARTITION and DETACH PARTITION were recently added to the > > list of exceptions. But they're not exceptions according to this > > wording: they

Re: [HACKERS] Minor correction in alter_table.sgml

Tom, * Tom Lane (t...@sss.pgh.pa.us) wrote: > Stephen Frost writes: > > * Tom Lane (t...@sss.pgh.pa.us) wrote: > >> So maybe something like > >> > >> All the forms of ALTER TABLE that act on a single table, > >> except RENAME and SET SCHEMA, can be combined into a > >> list of multiple alteratio

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

Hi, The attached results show that: (a) master shows the same zig-zag behavior - No idea why this wasn't observed on the previous runs. (b) group_update actually seems to improve the situation, because the performance keeps stable up to 72 clients, while on master the fluctuation starts way ea

Re: [HACKERS] pg_background contrib module proposal

Hi all, As we have discussed previously, we need to rework this patch as a client of Peter Eisentraut's background sessions code[1]. Attaching trial version patch to discussed possible design and api. We could have following APIs : • pg_background_launch : This function start and stores new back

Re: [HACKERS] Fix checkpoint skip logic on idle systems by tracking LSN progress

On Thu, Dec 22, 2016 at 3:10 AM, Andres Freund wrote: > On 2016-12-21 16:35:28 -0500, Robert Haas wrote: >> On Wed, Dec 21, 2016 at 4:28 PM, Andres Freund wrote: >> > - Similarly I don't like the name "progress LSN" much. What does >> > "progress" really mean in that". Maybe "consistency LSN"?

Re: [HACKERS] Proposal for CSN based snapshots

On Wed, Aug 24, 2016 at 2:24 PM, Heikki Linnakangas wrote: > > And here are the results on the 72 core machine (thanks again, Alexander!). > The test setup was the same as on the 32-core machine, except that I ran it > with more clients since the system has more CPU cores. In summary, in the > bes

Re: [HACKERS] Declarative partitioning - another take

On 2016/12/22 1:50, Robert Haas wrote: > On Wed, Dec 21, 2016 at 5:33 AM, Amit Langote > wrote: >> Breaking changes into multiple commits/patches does not seem to work for >> adding regression tests. So, I've combined multiple patches into a single >> patch which is now patch 0002 in the attached