Re: [HACKERS] Partition-wise aggregation/grouping

On Thu, Apr 27, 2017 at 4:53 PM, Antonin Houska wrote: > Robert Haas wrote: > > > On Wed, Apr 26, 2017 at 6:28 AM, Antonin Houska wrote: > > > Attached is a diff that contains both patches merged. This is just to > prove my > > >

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

On Fri, Apr 28, 2017 at 1:32 AM, Robert Haas wrote: > On Thu, Apr 27, 2017 at 3:41 AM, Ashutosh Bapat > wrote: >> The third goal requires that the partition bounds be compared based on >> the partition keys present in the equi-join. While

Re: [HACKERS] [COMMITTERS] pgsql: Replication lag tracking for walsenders

On Mon, Apr 24, 2017 at 2:53 PM, Tom Lane wrote: > Thomas Munro writes: >> Every machine sees the LSN moving backwards, but the code path that >> had the assertion only reached if it decides to interpolate, which is >> timing dependent: there

Re: [HACKERS] Transition tables for triggers on foreign tables and views

On Wed, Apr 26, 2017 at 11:17:05AM +1200, Thomas Munro wrote: > My colleague Prabhat Sahu reported off list that transition tables > don't work for views. I probably should have thought about that when > I fixed something similar for partitioned tables, and after some > experimentation I see that

Re: [HACKERS] some review comments on logical rep code

On Fri, Apr 28, 2017 at 01:55:48PM +0900, Masahiko Sawada wrote: > On Fri, Apr 28, 2017 at 1:42 PM, Noah Misch wrote: > > On Fri, Apr 28, 2017 at 06:37:09AM +0900, Fujii Masao wrote: > >> Pushed. Thanks! > > > > Does this close the open item, or is there more to do? > > There

Re: [HACKERS] some review comments on logical rep code

On Fri, Apr 28, 2017 at 1:42 PM, Noah Misch wrote: > On Fri, Apr 28, 2017 at 06:37:09AM +0900, Fujii Masao wrote: >> Pushed. Thanks! > > Does this close the open item, or is there more to do? There is only one item remaining, and the patch is attached on here[1]. I guess

Re: [HACKERS] scram and \password

On Fri, Apr 21, 2017 at 11:04:14PM +0300, Heikki Linnakangas wrote: > I'll continue reviewing the rest of the patch on Monday, but [...] This PostgreSQL 10 open item is past due for your status update. Kindly send a status update within 24 hours, and include a date for your subsequent status

Re: [HACKERS] pg_dump emits ALTER TABLE ONLY partitioned_table

Noah, * Noah Misch (n...@leadboat.com) wrote: > On Sun, Apr 23, 2017 at 11:58:23PM +, Stephen Frost wrote: > > The status is simply that I've been considering Robert's comments regarding > > the documentation and have had a busy weekend. I'll provide an update > > tomorrow. > > This

Re: [HACKERS] snapbuild woes

On April 27, 2017 9:34:44 PM PDT, Noah Misch wrote: >On Fri, Apr 21, 2017 at 10:36:21PM -0700, Andres Freund wrote: >> On 2017-04-17 21:16:57 -0700, Andres Freund wrote: >> > I've since the previous update reviewed Petr's patch, which he >since has >> > updated over the

Re: [HACKERS] some review comments on logical rep code

On Fri, Apr 28, 2017 at 06:37:09AM +0900, Fujii Masao wrote: > Pushed. Thanks! Does this close the open item, or is there more to do? -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] snapbuild woes

On Fri, Apr 21, 2017 at 10:36:21PM -0700, Andres Freund wrote: > On 2017-04-17 21:16:57 -0700, Andres Freund wrote: > > I've since the previous update reviewed Petr's patch, which he since has > > updated over the weekend. I'll do another round tomorrow, and will see > > how it looks. I think we

Re: [HACKERS] pg_dump emits ALTER TABLE ONLY partitioned_table

On Sun, Apr 23, 2017 at 11:58:23PM +, Stephen Frost wrote: > Noah, all, > > On Sun, Apr 23, 2017 at 19:52 Noah Misch wrote: > > > On Sat, Apr 22, 2017 at 01:14:08PM -0700, Noah Misch wrote: > > > On Thu, Apr 20, 2017 at 09:53:28PM -0400, Stephen Frost wrote: > > > > *

Re: [HACKERS] PG 10 release notes

On Tue, Apr 25, 2017 at 10:31 AM, Bruce Momjian wrote: > I have committed the first draft of the Postgres 10 release notes. They > are current as of two days ago, and I will keep them current. Please > give me any feedback you have. > Related to a following item in release

[HACKERS] proposal psql \gdesc

Hi Sometimes I have to solve the result types of some query. It is invisible in psql. You have to materialize table or you have to create view. Now, when we can enhance \g command, we can introduce query describing some like select a, b from foo \gdesc | type | length | collation |

Re: [HACKERS] PG 10 release notes

On Thu, Apr 27, 2017 at 7:53 PM, Marko Tiikkaja wrote: > On Thu, Apr 27, 2017 at 4:13 PM, Bruce Momjian wrote: >> >> On Thu, Apr 27, 2017 at 08:00:28AM +0530, Amit Kapila wrote: >> > > Oh, so non-correlated subqueries can be run in parallel. Yes, that is >> > >

Re: [HACKERS] [PostgreSQL 10] default of hot_standby should be "on"?

On Thu, Apr 27, 2017 at 11:05 PM, Huong Dangminh wrote: >> >>> I would refrain from doing that, having some parameters listed in the >> >>> tests makes the intention behind those perl routines clear. >> > >> > Hmm, you've got a point. But when we changed the default

Re: [HACKERS] subscription worker doesn't start immediately on eabled

On Thu, Apr 27, 2017 at 12:51 AM, Fujii Masao wrote: > On Wed, Apr 26, 2017 at 4:03 PM, Kyotaro HORIGUCHI > wrote: >> At Tue, 25 Apr 2017 14:45:03 -0400, Peter Eisentraut >> wrote in >>

Re: [HACKERS] Interval for launching the table sync worker

On Fri, Apr 28, 2017 at 4:00 AM, Peter Eisentraut wrote: > On 4/27/17 06:47, Petr Jelinek wrote: >> One thing I am missing in your patch however is cleanup of entries for >> relations that finished sync. I wonder if it would be enough to just >> destroy the hash

Re: [HACKERS] Crash when partition column specified twice

On 2017/04/27 12:36, Amit Langote wrote: > Noticed that a crash occurs if a column is specified twice when creating a > partition: > > create table p (a int) partition by list (a); > > -- crashes > create table p1 partition of parent ( > a not null, > a default 1 > ) for values in (1); > >

Re: [HACKERS] tablesync patch broke the assumption that logical rep depends on?

On Thu, Apr 27, 2017 at 4:01 AM, Tom Lane wrote: > Jeff Janes writes: > > This gives me compiler warning: > > launcher.c: In function 'logicalrep_worker_launch': > > launcher.c:257: warning: 'slot' may be used uninitialized in this > function > > Yeah,

Re: [HACKERS] Declarative partitioning - another take

On Thu, Apr 27, 2017 at 10:30:54AM +0900, Amit Langote wrote: > On 2017/04/27 1:52, Robert Haas wrote: > > On Tue, Apr 25, 2017 at 10:34 PM, Amit Langote > > wrote: > >> FWIW, I too prefer the latter, that is, fire only the parent's triggers. > >> In that case,

Re: [HACKERS] subscription worker doesn't start immediately on eabled

On Thu, Apr 27, 2017 at 6:32 PM, Kyotaro HORIGUCHI wrote: > At Thu, 27 Apr 2017 00:51:03 +0900, Fujii Masao wrote > in

Re: [HACKERS] some review comments on logical rep code

On Thu, Apr 27, 2017 at 5:37 PM, Petr Jelinek wrote: > On 26/04/17 18:36, Fujii Masao wrote: >> On Thu, Apr 27, 2017 at 1:28 AM, Fujii Masao wrote: >>> On Wed, Apr 26, 2017 at 3:47 PM, Kyotaro HORIGUCHI >>>

Re: [HACKERS] [PROPOSAL] Use SnapshotAny in get_actual_variable_range

Andres Freund writes: > On 2017-04-27 17:22:25 -0400, Tom Lane wrote: >> How so? Shouldn't the indexscan go back and mark such tuples dead in >> the index, such that they'd be visited this way only once? If that's >> not happening, maybe we should try to fix it. > One way

Re: [HACKERS] [PROPOSAL] Use SnapshotAny in get_actual_variable_range

On 2017-04-27 17:22:25 -0400, Tom Lane wrote: > > Yep, and I've seen that turn into a serious problem in production. > > How so? Shouldn't the indexscan go back and mark such tuples dead in > the index, such that they'd be visited this way only once? If that's > not happening, maybe we should

Re: [HACKERS] [PROPOSAL] Use SnapshotAny in get_actual_variable_range

Robert Haas writes: > On Thu, Apr 27, 2017 at 4:08 AM, Dmitriy Sarafannikov > wrote: >> I'd like to propose to search min and max value in index with SnapshotAny in >> get_actual_variable_range function. > +1 from me, but Tom rejected that

Re: [HACKERS] Unportable implementation of background worker start

Andres Freund writes: > On 2017-04-27 16:35:29 -0400, Tom Lane wrote: >> It looks like it might be sufficient to do "#ifdef EPOLL_CLOEXEC" >> in latch.c, rather than bothering with a full-blown configure check. > Yea, that sounds worth trying. Wonder if we need to care about

Re: [HACKERS] Unportable implementation of background worker start

On 2017-04-27 16:35:29 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2017-04-26 17:05:39 -0400, Tom Lane wrote: > >> I went ahead and changed the call to epoll_create into epoll_create1. > >> I'm not too concerned about loss of portability there --- it seems > >>

Re: [HACKERS] frogmouth failures

On 2017-04-27 16:30:35 -0400, Tom Lane wrote: > Andrew Dunstan writes: > > I've been trying to track down the cause of recent failures at the "make > > check" stage on frogmouth, a 32-bit Windows/Mingw instance running on XP. > > I've been wondering about that

Re: [HACKERS] frogmouth failures

On 04/27/2017 04:30 PM, Tom Lane wrote: > Andrew Dunstan writes: >> I've been trying to track down the cause of recent failures at the "make >> check" stage on frogmouth, a 32-bit Windows/Mingw instance running on XP. > I've been wondering about that too. > >>

Re: [HACKERS] Unportable implementation of background worker start

Andres Freund writes: > On 2017-04-26 17:05:39 -0400, Tom Lane wrote: >> I went ahead and changed the call to epoll_create into epoll_create1. >> I'm not too concerned about loss of portability there --- it seems >> unlikely that many people are still using ten-year-old glibc,

Re: [HACKERS] frogmouth failures

Andrew Dunstan writes: > I've been trying to track down the cause of recent failures at the "make > check" stage on frogmouth, a 32-bit Windows/Mingw instance running on XP. I've been wondering about that too. > Then I tried running (offline mode) the serial

Re: [HACKERS] [PROPOSAL] Use SnapshotAny in get_actual_variable_range

On Thu, Apr 27, 2017 at 4:08 AM, Dmitriy Sarafannikov wrote: > I'd like to propose to search min and max value in index with SnapshotAny in > get_actual_variable_range function. +1 from me, but Tom rejected that approach last time. > But if we delete many rows from

Re: [HACKERS] Declarative partitioning - another take

On Wed, Apr 26, 2017 at 9:30 PM, Amit Langote wrote: >> Do we need to update the documentation? > > Yes, I think we should. How about as in the attached? Looks reasonable, but I was thinking you might also update the section which contrasts inheritance-based

Re: [HACKERS] Adding support for Default partition in partitioning

On Thu, Apr 27, 2017 at 3:15 PM, Sven R. Kunze wrote: > On 27.04.2017 15:07, Robert Haas wrote: >> On Thu, Apr 27, 2017 at 8:49 AM, Rahila Syed >> wrote: >>> >>> +1 for CREATE TABLE..PARTITION OF...DEFAULT syntax. >>> I think substituting DEFAULT for FOR

[HACKERS] frogmouth failures

I've been trying to track down the cause of recent failures at the "make check" stage on frogmouth, a 32-bit Windows/Mingw instance running on XP. I couldn't see any obvious reason for the failures, and a reboot didn't cure the problem. Then I tried running (offline mode) the serial schedule

Re: [HACKERS] identity columns

On Thu, Apr 27, 2017 at 3:42 PM, Peter Eisentraut wrote: > On 4/27/17 10:03, Robert Haas wrote: >>> So we could make up new syntax >>> >>> ALTER TABLE t1 ALTER COLUMN c1 SET GENERATED ALWAYS AS IDENTITY; >>> >>> and let that be set-or-add, but then the argument

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

On Thu, Apr 27, 2017 at 3:41 AM, Ashutosh Bapat wrote: > The third goal requires that the partition bounds be compared based on > the partition keys present in the equi-join. While matching the > partitions to be joined, the partition bounds corresponding the base

Re: [HACKERS] identity columns

On 4/27/17 10:03, Robert Haas wrote: >> So we could make up new syntax >> >> ALTER TABLE t1 ALTER COLUMN c1 SET GENERATED ALWAYS AS IDENTITY; >> >> and let that be set-or-add, but then the argument for staying within the >> SQL standard goes out the window. > > What does the SQL standard actually

Re: [HACKERS] Adding support for Default partition in partitioning

Hi, On Apr 27, 2017 18:37, "Robert Haas" wrote: > > > Are you also working on extending this to work with range > partitioning? Because I think that would be good to do. > > > Currently I am working on review comments and bug fixes for the default list partitioning

Re: [HACKERS] Interval for launching the table sync worker

On 27/04/17 21:00, Peter Eisentraut wrote: > On 4/27/17 06:47, Petr Jelinek wrote: >> One thing I am missing in your patch however is cleanup of entries for >> relations that finished sync. I wonder if it would be enough to just >> destroy the hash when we get to empty list. > > I had omitted

Re: [HACKERS] Adding support for Default partition in partitioning

On 27.04.2017 15:07, Robert Haas wrote: On Thu, Apr 27, 2017 at 8:49 AM, Rahila Syed wrote: +1 for CREATE TABLE..PARTITION OF...DEFAULT syntax. I think substituting DEFAULT for FOR VALUES is appropriate as both cases are mutually exclusive. Just to make sound a little

Re: [HACKERS] Logical replication in the same cluster

On 4/27/17 04:08, Petr Jelinek wrote: > Note that the workaround for all of this is not all that complex, you do > same thing (create slot manually) you'd do for physical replication with > slots. Maybe we should just document this issue for now. -- Peter Eisentraut

Re: [HACKERS] Interval for launching the table sync worker

On 4/27/17 06:47, Petr Jelinek wrote: > One thing I am missing in your patch however is cleanup of entries for > relations that finished sync. I wonder if it would be enough to just > destroy the hash when we get to empty list. I had omitted that because the amount of memory "leaked" is not much,

Re: [HACKERS] PG 10 release notes

Bruce Momjian writes: > I have committed the first draft of the Postgres 10 release notes. They > are current as of two days ago, and I will keep them current. Please > give me any feedback you have. I noticed a few niggles with the links in "my" entires, patch attached.

Re: [HACKERS] [PATCH] Incremental sort

On Thu, Apr 27, 2017 at 5:06 PM, Robert Haas wrote: > On Wed, Apr 26, 2017 at 11:39 AM, Alexander Korotkov > wrote: > > But I'd like to make incremental sort not slower than quicksort in case > of > > presorted data. New idea about it comes to

Re: [HACKERS] PG 10 release notes

On Thu, Apr 27, 2017 at 4:13 PM, Bruce Momjian wrote: > On Thu, Apr 27, 2017 at 08:00:28AM +0530, Amit Kapila wrote: > > > Oh, so non-correlated subqueries can be run in parallel. Yes, that is > > > something we should have in the release notes. How is this? > > > > > >

Re: [HACKERS] pg_dump emits ALTER TABLE ONLY partitioned_table

Amit, * Amit Langote (langote_amit...@lab.ntt.co.jp) wrote: > >> So to summarize what the patches do (some of these were posted earlier) > >> > >> 0002: pg_dump: Do not emit WITH OPTIONS keywords with partition's columns > > > > I'm trying to understand why this is also different. At least on

Re: [HACKERS] PG 10 release notes

On Thu, Apr 27, 2017 at 08:14:34AM +0530, Amit Kapila wrote: > > > > Improve efficiency of hash index growth (Amit Kapila, Mithun Cy) > > > > The first two commits b0f18cb77, 30df93f69 are done as preliminary > work to "Add write-ahead logging support to

Re: [HACKERS] PG 10 release notes

On Thu, Apr 27, 2017 at 08:00:28AM +0530, Amit Kapila wrote: > > Oh, so non-correlated subqueries can be run in parallel. Yes, that is > > something we should have in the release notes. How is this? > > > > Author: Robert Haas > > 2017-02-14 [5e6d8d2bb]

Re: [HACKERS] [PATCH] Incremental sort

On Wed, Apr 26, 2017 at 11:39 AM, Alexander Korotkov wrote: > But I'd like to make incremental sort not slower than quicksort in case of > presorted data. New idea about it comes to my mind. Since cause of > incremental sort slowness in this case is too frequent reset

Re: [HACKERS] [PostgreSQL 10] default of hot_standby should be "on"?

> >>> I would refrain from doing that, having some parameters listed in the > >>> tests makes the intention behind those perl routines clear. > > > > Hmm, you've got a point. But when we changed the default values > > related to replication we dropped some explicitly settings from the > >

Re: [HACKERS] identity columns

On Wed, Apr 26, 2017 at 10:03 PM, Peter Eisentraut wrote: > On 4/23/17 16:58, Robert Haas wrote: >> I agree that ADD is a little odd here, but it doesn't seem terrible. >> But why do we need it? Instead of: >> >> ADD GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY

Re: [HACKERS] Adding support for Default partition in partitioning

On Thu, Apr 27, 2017 at 8:49 AM, Rahila Syed wrote: >>I suspect it could be done as of now, but I'm a little worried that it >>might create grammar conflicts in the future as we extend the syntax >>further. If we use CREATE TABLE ... PARTITION OF .. DEFAULT, then the

Re: [HACKERS] PG 10 release notes

Fabien COELHO wrote: >Fix psql \p to always print what would be executed by \g or \w (Daniel >Vérité) > >Previously \p didn't properly print the reverted-to command after a >buffer contents reset. CLARIFY? > > The fix is linked to the change introduced by Tom when >

Re: [HACKERS] Adding support for Default partition in partitioning

>I suspect it could be done as of now, but I'm a little worried that it >might create grammar conflicts in the future as we extend the syntax >further. If we use CREATE TABLE ... PARTITION OF .. DEFAULT, then the >word DEFAULT appears in the same position where we'd normally have FOR >VALUES, and

Re: [HACKERS] Interval for launching the table sync worker

On 25/04/17 19:54, Peter Eisentraut wrote: > I feel it's getting a bit late for reworkings of this extent, also > considering the marginal nature of the problem we are trying to fix. My > patch from April 18 is very localized and gets the job done. > > I think this is still a good direction to

Re: [HACKERS] subscription worker doesn't start immediately on eabled

At Thu, 27 Apr 2017 00:51:03 +0900, Fujii Masao wrote in

Re: [HACKERS] PG 10 release notes

Hello Bruce, I have committed the first draft of the Postgres 10 release notes. They are current as of two days ago, and I will keep them current. Please give me any feedback you have. About: """ Fix psql \p to always print what would be executed by \g or \w (Daniel Vérité)

Re: [HACKERS] some review comments on logical rep code

On 26/04/17 18:36, Fujii Masao wrote: > On Thu, Apr 27, 2017 at 1:28 AM, Fujii Masao wrote: >> On Wed, Apr 26, 2017 at 3:47 PM, Kyotaro HORIGUCHI >> wrote: >>> At Wed, 26 Apr 2017 14:31:12 +0900, Masahiko Sawada >>>

Re: [HACKERS] pg_basebackup: Allow use of arbitrary compression program

Hi All, I have a working prototype now, but there is one aspect I haven't been able to find the best solution for. The CLI interface so far has the following new added option: -C, --compressprog=PRG use supplied external program for compression An example usage would be: pg_basebackup

Re: [HACKERS] Logical replication in the same cluster

On 27/04/17 04:50, Tom Lane wrote: > Peter Eisentraut writes: If that's a predictable deadlock, I think a minimum expectation is that the system should notice it and throw an error, not just hang. > >> We had some discussions early on about detecting

Re: [HACKERS] pg_dump emits ALTER TABLE ONLY partitioned_table

Hi Stephen, On 2017/04/26 23:31, Stephen Frost wrote: >>> I looked through >>> pg_get_partkeydef() and it didn't seem to be particularly expensive to >>> run, though evidently it doesn't handle being passed an OID that it >>> doesn't expect very cleanly: >>> >>> =# select pg_get_partkeydef(oid)

[HACKERS] [PROPOSAL] Use SnapshotAny in get_actual_variable_range

Hi hackers,I'd like to propose to search min and max value in index with SnapshotAny in get_actual_variable_range function.Current implementation scans index with SnapshotDirty which accepts uncommitted rows and rejects dead rows.In a code there is a comment about this:  /*   * In

Re: [HACKERS] [PostgreSQL 10] default of hot_standby should be "on"?

On Thu, Apr 27, 2017 at 4:33 PM, Masahiko Sawada wrote: > On Thu, Apr 27, 2017 at 1:58 PM, Huong Dangminh > wrote: >>> On Thu, Apr 27, 2017 at 11:48 AM, Masahiko Sawada >>> wrote: >>> > Thank you for updating the patch.

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

On Wed, Apr 26, 2017 at 9:30 PM, Robert Haas wrote: > On Mon, Apr 24, 2017 at 7:06 AM, Ashutosh Bapat > wrote: >> This assumes that datums in partition bounds have one to one mapping >> with the partitions, which isn't true for list

Re: [HACKERS] [PostgreSQL 10] default of hot_standby should be "on"?

On Thu, Apr 27, 2017 at 1:58 PM, Huong Dangminh wrote: >> On Thu, Apr 27, 2017 at 11:48 AM, Masahiko Sawada >> wrote: >> > Thank you for updating the patch. Also maybe we can update line in >> > PostgresNode.pm where hot_standby is set to on

Re: [HACKERS] Vacuum: allow usage of more than 1GB of work mem

On Fri, Apr 21, 2017 at 6:24 AM, Claudio Freire wrote: > On Wed, Apr 12, 2017 at 4:35 PM, Robert Haas wrote: >> On Tue, Apr 11, 2017 at 4:38 PM, Claudio Freire >> wrote: >>> In essence, the patch as it is proposed, doesn't

Re: [HACKERS] scram and \password

On 04/26/2017 07:57 PM, Tom Lane wrote: Robert Haas writes: On Wed, Apr 26, 2017 at 6:22 AM, Heikki Linnakangas wrote: * If algorithm is not given explicitly, PQencryptPasswordConn() queries "SHOW password_encryption", and uses that. That is