Re: [HACKERS] some review comments on logical rep code

2017-04-25 Thread Masahiko Sawada
On Wed, Apr 26, 2017 at 12:35 PM, Petr Jelinek wrote: > On 26/04/17 01:01, Fujii Masao wrote: >> On Mon, Apr 24, 2017 at 7:57 PM, Kyotaro HORIGUCHI >> wrote: >>> Hello, >>> >>> At Mon, 24 Apr 2017 11:18:32 +0900, Masahiko Sawada

Re: [HACKERS] Adding support for Default partition in partitioning

2017-04-25 Thread Ashutosh Bapat
On Tue, Apr 25, 2017 at 11:23 PM, Robert Haas wrote: > On Tue, Apr 25, 2017 at 1:20 AM, Ashutosh Bapat > wrote: >>> I suspect it could be done as of now, but I'm a little worried that it >>> might create grammar conflicts in the future as

Re: [HACKERS] Foreign Join pushdowns not working properly for outer joins

2017-04-25 Thread Ashutosh Bapat
On Tue, Apr 25, 2017 at 7:42 PM, Peter Eisentraut wrote: > On 4/24/17 22:50, Peter Eisentraut wrote: >> On 4/14/17 00:24, Ashutosh Bapat wrote: >>> This looks better. Here are patches for master and 9.6. >>> Since join pushdown was supported in 9.6 the patch

Re: [HACKERS] PG_TRY & PG_CATCH in FDW development

2017-04-25 Thread Abbas Butt
Thanks for the reply. On Tue, Apr 25, 2017 at 7:45 PM, Tom Lane wrote: > Abbas Butt writes: > > What is happening for me is that PG_RE_THROW takes me to PG_TRY in the > same > > function and then PG_TRY jumps to PG_CATCH where PG_RE_THROW again

Re: [HACKERS] PG 10 release notes

2017-04-25 Thread Amit Kapila
On Tue, Apr 25, 2017 at 7:45 PM, Bruce Momjian wrote: > On Tue, Apr 25, 2017 at 09:38:55AM +0530, Rafia Sabih wrote: >> On Tue, Apr 25, 2017 at 9:15 AM, Tom Lane wrote: >> > Andres Freund writes: >> >> On 2017-04-24 23:37:42 -0400, Bruce

Re: [HACKERS] some review comments on logical rep code

2017-04-25 Thread Petr Jelinek
On 26/04/17 01:01, Fujii Masao wrote: > On Mon, Apr 24, 2017 at 7:57 PM, Kyotaro HORIGUCHI > wrote: >> Hello, >> >> At Mon, 24 Apr 2017 11:18:32 +0900, Masahiko Sawada >> wrote in

Re: [HACKERS] Declarative partitioning - another take

2017-04-25 Thread Amit Langote
Hi, Thanks for testing. On 2017/04/25 19:03, Rajkumar Raghuwanshi wrote: > Thanks for looking into it. I have applied fixes and checked for triggers. > I could see difference in behaviour of statement triggers for INSERT and > UPDATE, for insert only root partition triggers are getting fired but

Re: [HACKERS] Declarative partitioning - another take

2017-04-25 Thread Amit Langote
On 2017/04/26 3:58, Robert Haas wrote: > On Mon, Apr 24, 2017 at 6:43 AM, Amit Langote > wrote: >> The reason it doesn't work is that we do not allocate ResultRelInfos for >> partitioned tables (not even for the root partitioned table in the >> update/delete cases),

Re: [HACKERS] PG 10 release notes

2017-04-25 Thread Michael Paquier
On Wed, Apr 26, 2017 at 10:56 AM, Bruce Momjian wrote: > First, I don't think RFC references belong in the release notes, let > alone RFC links. > > Second, there seems to be some confusion over what SCRAM-SHA-256 gives > us over MD5. I think there are a few benefits: > > o

Re: [HACKERS] PG 10 release notes

2017-04-25 Thread Bruce Momjian
On Tue, Apr 25, 2017 at 07:17:20PM -0700, Andres Freund wrote: > On 2017-04-25 22:13:00 -0400, Bruce Momjian wrote: > > On Tue, Apr 25, 2017 at 06:40:08PM -0700, Andres Freund wrote: > > > On 2017-04-25 21:19:41 -0400, Bruce Momjian wrote: > > > > On Tue, Apr 25, 2017 at 06:51:47AM +0200, Petr

Re: [HACKERS] PG 10 release notes

2017-04-25 Thread Bruce Momjian
On Tue, Apr 25, 2017 at 06:40:08PM -0700, Andres Freund wrote: > On 2017-04-25 21:19:41 -0400, Bruce Momjian wrote: > > On Tue, Apr 25, 2017 at 06:51:47AM +0200, Petr Jelinek wrote: > > > Or the ability of logical decoding to follow timeline switches. > > > > When you say "logical decoding", you

Re: [HACKERS] StandbyRecoverPreparedTransactions recovers subtrans links incorrectly

2017-04-25 Thread Michael Paquier
On Wed, Apr 26, 2017 at 4:53 AM, Simon Riggs wrote: > On 25 April 2017 at 16:28, Tom Lane wrote: >> Simon Riggs writes: >>> I can't see any reason now why overwriteOK should exist at all. I'm >>> guessing that the whole

Re: [HACKERS] PG 10 release notes

2017-04-25 Thread Andres Freund
On 2017-04-25 22:13:00 -0400, Bruce Momjian wrote: > On Tue, Apr 25, 2017 at 06:40:08PM -0700, Andres Freund wrote: > > On 2017-04-25 21:19:41 -0400, Bruce Momjian wrote: > > > On Tue, Apr 25, 2017 at 06:51:47AM +0200, Petr Jelinek wrote: > > > > Or the ability of logical decoding to follow

Re: [HACKERS] PG 10 release notes

2017-04-25 Thread Andres Freund
On 2017-04-25 21:19:41 -0400, Bruce Momjian wrote: > On Tue, Apr 25, 2017 at 06:51:47AM +0200, Petr Jelinek wrote: > > Or the ability of logical decoding to follow timeline switches. > > When you say "logical decoding", you don't mean contrib/test_decoding? No. test_decoding is just an example

Re: [HACKERS] PG 10 release notes

2017-04-25 Thread Bruce Momjian
On Wed, Apr 26, 2017 at 11:06:03AM +0900, Michael Paquier wrote: > On Wed, Apr 26, 2017 at 10:56 AM, Bruce Momjian wrote: > > First, I don't think RFC references belong in the release notes, let > > alone RFC links. > > > > Second, there seems to be some confusion over what

Re: [HACKERS] Dropping a partitioned table takes too long

2017-04-25 Thread Amit Langote
Hi, On 2017/04/25 20:07, 高增琦 wrote: > > 2017-04-25 15:07 GMT+08:00 Amit Langote : > >> $SUBJECT, if the table has, say, 2000 partitions. >> >> The main reason seems to be that RelationBuildPartitionDesc() will be >> called that many times within the same

Re: [HACKERS] Dropping a partitioned table takes too long

2017-04-25 Thread Amit Langote
On 2017/04/25 20:55, Ashutosh Bapat wrote: > On Tue, Apr 25, 2017 at 12:37 PM, Amit Langote > wrote: >> $SUBJECT, if the table has, say, 2000 partitions. >> >> The main reason seems to be that RelationBuildPartitionDesc() will be >> called that many times within the

Re: [HACKERS] PG 10 release notes

2017-04-25 Thread Bruce Momjian
On Wed, Apr 26, 2017 at 09:02:51AM +0900, Michael Paquier wrote: > On Wed, Apr 26, 2017 at 12:20 AM, Bruce Momjian wrote: > > On Tue, Apr 25, 2017 at 02:39:40PM +0900, Michael Paquier wrote: > >> > >> Add SCRAM-SHA-256 > >> support for password negotiation and storage (Michael

Re: [HACKERS] logical replication and PANIC during shutdown checkpoint in publisher

2017-04-25 Thread Michael Paquier
On Wed, Apr 26, 2017 at 3:17 AM, Peter Eisentraut wrote: > On 4/21/17 00:11, Michael Paquier wrote: >> Hmm. I have been actually looking at this solution and I am having >> doubts regarding its robustness. In short this would need to be >> roughly a two-step

Re: [HACKERS] PG 10 release notes

2017-04-25 Thread Bruce Momjian
On Tue, Apr 25, 2017 at 06:51:47AM +0200, Petr Jelinek wrote: > I also wonder if ability to run SQL queries on walsender connected to a > database is worth mentioning (replication=database kind of connection). > > Or the ability of logical decoding to follow timeline switches. When you say

Re: [HACKERS] Re: logical replication and PANIC during shutdown checkpoint in publisher

2017-04-25 Thread Michael Paquier
On Wed, Apr 26, 2017 at 4:26 AM, Peter Eisentraut wrote: > On 4/20/17 11:30, Peter Eisentraut wrote: >> We have a possible solution but need to work out a patch. Let's say >> next check-in on Monday. > > Update: We have a patch that looks promising, but we

Re: [HACKERS] PG 10 release notes

2017-04-25 Thread Tsunakawa, Takayuki
From: 'Bruce Momjian' [mailto:br...@momjian.us] > > I forgot to point out one thing. > > > > Allow libpq to connect to multiple specified host names (Robert Haas) > > libpq will connect with the first responsive host name. > > > > According to the following CF entry and my memory, > > > >

Re: [HACKERS] PG 10 release notes

2017-04-25 Thread 'Bruce Momjian'
On Wed, Apr 26, 2017 at 12:26:33AM +, Tsunakawa, Takayuki wrote: > Hello, Bruce > > I forgot to point out one thing. > > Allow libpq to connect to multiple specified host names (Robert Haas) > libpq will connect with the first responsive host name. > > According to the following CF entry

Re: [HACKERS] Cached plans and statement generalization

2017-04-25 Thread Tsunakawa, Takayuki
From: pgsql-hackers-ow...@postgresql.org > [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Konstantin > Knizhnik > Well, first of all I want to share results I already get: pgbench with default > parameters, scale 10 and one connection: > > So autoprepare is as efficient as explicit

Re: [HACKERS] Separation walsender & normal backends

2017-04-25 Thread Andres Freund
On 2017-04-26 08:41:46 +0800, Craig Ringer wrote: > I'd very much like to reduce the amount of magic global juggling done > by the walsender, unify the XLogRead paths, unify the timeline > following logic for physical walsenders, logical walsenders and > logical decoding on normal backends, allow

Re: [HACKERS] scram and \password

2017-04-25 Thread Michael Paquier
On Wed, Apr 26, 2017 at 12:26 AM, Heikki Linnakangas wrote: > Yeah, there is that. But we simply cannot change the signature of an > existing function. It would not only produce compile-time errors when > building old applications, which would arguably be a good thing, but it >

Re: [HACKERS] Separation walsender & normal backends

2017-04-25 Thread Craig Ringer
On 26 April 2017 at 02:36, Andres Freund wrote: > For > logical rep we'd alternatively add more complexity because we'd need > both replication and non-replication connections (to stream changes, to > copy tables, to query config), which'd also complicate administration >

Re: [HACKERS] [PostgreSQL 10] default of hot_standby should be "on"?

2017-04-25 Thread Craig Ringer
On 26 April 2017 at 08:30, Huong Dangminh wrote: > Default for hot_standby parameter should be "on" from PostgreSQL 10? > > In PostgreSQL 10, -w option is default for [pg_ctl start]. > So in order to start standby we have to setting hot_standby to "on" or > start

[HACKERS] [PostgreSQL 10] default of hot_standby should be "on"?

2017-04-25 Thread Huong Dangminh
Hi, Default for hot_standby parameter should be "on" from PostgreSQL 10? In PostgreSQL 10, -w option is default for [pg_ctl start]. So in order to start standby we have to setting hot_standby to "on" or start standby with -W option. Change hot_standby to "on" will fix this inconvenience.

Re: [HACKERS] PG 10 release notes

2017-04-25 Thread Tsunakawa, Takayuki
Hello, Bruce I forgot to point out one thing. Allow libpq to connect to multiple specified host names (Robert Haas) libpq will connect with the first responsive host name. According to the following CF entry and my memory, https://commitfest.postgresql.org/12/879/ Authors mithun cy

Re: [HACKERS] PG 10 release notes

2017-04-25 Thread Michael Paquier
On Wed, Apr 26, 2017 at 12:20 AM, Bruce Momjian wrote: > On Tue, Apr 25, 2017 at 02:39:40PM +0900, Michael Paquier wrote: >> >> Add SCRAM-SHA-256 >> support for password negotiation and storage (Michael >> Paquier, Heikki Linnakangas) >> >> >> This proves better security than

Re: [HACKERS] question: data file update when pg_basebackup in progress

2017-04-25 Thread Michael Paquier
On Wed, Apr 26, 2017 at 1:45 AM, David G. Johnston wrote: > The first write to a page after a checkpoint is always recorded in the WAL > as a full page write. Every WAL file since the checkpoint must also be > copied to the backed up system. The replay of those WAL

[HACKERS] Transition tables for triggers on foreign tables and views

2017-04-25 Thread Thomas Munro
Hi hackers, My colleague Prabhat Sahu reported off list that transition tables don't work for views. I probably should have thought about that when I fixed something similar for partitioned tables, and after some experimentation I see that this is also broken for foreign tables. For foreign

Re: [HACKERS] Cached plans and statement generalization

2017-04-25 Thread Serge Rielau
via Newton Mail [https://cloudmagic.com/k/d/mailapp?ct=dx=9.4.52=10.11.6=email_footer_2] On Tue, Apr 25, 2017 at 3:48 PM, Doug Doole wrote: It's not always that simple, at least in postgres, unless you disregard search_path. Consider e.g. cases like CREATE SCHEMA a;

Re: [HACKERS] some review comments on logical rep code

2017-04-25 Thread Fujii Masao
On Mon, Apr 24, 2017 at 7:57 PM, Kyotaro HORIGUCHI wrote: > Hello, > > At Mon, 24 Apr 2017 11:18:32 +0900, Masahiko Sawada > wrote in >> >> BEGIN; >> >> ALTER

Re: [HACKERS] Cached plans and statement generalization

2017-04-25 Thread Finnerty, Jim
On 4/25/17, 6:34 PM, "pgsql-hackers-ow...@postgresql.org on behalf of Andres Freund" wrote: It's not always that simple, at least in postgres, unless you disregard search_path. Consider e.g. cases like CREATE

Re: [HACKERS] Cached plans and statement generalization

2017-04-25 Thread Doug Doole
> > (FWIW, on this list we don't do top-quotes) > I know. Forgot and just did "reply all". My bad. It's not always that simple, at least in postgres, unless you disregard > search_path. Consider e.g. cases like > > CREATE SCHEMA a; > CREATE SCHEMA b; > CREATE TABLE a.foobar(somecol int); > SET

Re: [HACKERS] Cached plans and statement generalization

2017-04-25 Thread David G. Johnston
On Tue, Apr 25, 2017 at 3:24 PM, David Fetter wrote: > I don't have an exploit yet. What concerns me is attackers' access to > what is in essence the ability to poke at RULEs when they only have > privileges to read. > ​If they want to see how it works they can read the

Re: [HACKERS] Cached plans and statement generalization

2017-04-25 Thread Andres Freund
Hi, (FWIW, on this list we don't do top-quotes) On 2017-04-25 22:21:22 +, Doug Doole wrote: > Plan invalidation was no different than for any SQL statement. DB2 keeps a > list of the objects the statement depends on. If any of the objects changes > in an incompatible way the plan is

Re: [HACKERS] Cached plans and statement generalization

2017-04-25 Thread David Fetter
On Tue, Apr 25, 2017 at 11:35:21PM +0300, Konstantin Knizhnik wrote: > On 04/25/2017 07:54 PM, David Fetter wrote: > > On Tue, Apr 25, 2017 at 06:11:09PM +0300, Konstantin Knizhnik wrote: > > > On 24.04.2017 21:43, Andres Freund wrote: > > > > Hi, > > > > > > > > On 2017-04-24 11:46:02 +0300,

Re: [HACKERS] Cached plans and statement generalization

2017-04-25 Thread Doug Doole
Plan invalidation was no different than for any SQL statement. DB2 keeps a list of the objects the statement depends on. If any of the objects changes in an incompatible way the plan is invalidated and kicked out of the cache. I suspect what is more interesting is plan lookup. DB2 has something

Re: [HACKERS] Separation walsender & normal backends

2017-04-25 Thread Andres Freund
On 2017-04-25 23:24:40 +0200, Petr Jelinek wrote: > On 25/04/17 17:13, Fujii Masao wrote: > > On Tue, Apr 25, 2017 at 11:34 PM, Tom Lane wrote: > >> Andres Freund writes: > >>> I've for a while suspected that the separation & duplication of > >>>

Re: [HACKERS] Cached plans and statement generalization

2017-04-25 Thread Andres Freund
On 2017-04-25 21:11:08 +, Doug Doole wrote: > When I did this in DB2, I didn't use the parser - it was too expensive. I > just tokenized the statement and used some simple rules to bypass the > invalid cases. For example, if I saw the tokens "ORDER" and "BY" then I'd > disallow replacement

Re: [HACKERS] Separation walsender & normal backends

2017-04-25 Thread David G. Johnston
On Tue, Apr 25, 2017 at 2:24 PM, Petr Jelinek wrote: > On 25/04/17 17:13, Fujii Masao wrote: > > On Tue, Apr 25, 2017 at 11:34 PM, Tom Lane wrote: > > OTOH, I believe that logical replication is still useful even without > > initial table sync

Re: [HACKERS] Separation walsender & normal backends

2017-04-25 Thread Petr Jelinek
On 25/04/17 17:13, Fujii Masao wrote: > On Tue, Apr 25, 2017 at 11:34 PM, Tom Lane wrote: >> Andres Freund writes: >>> I've for a while suspected that the separation & duplication of >>> infrastructure between walsenders and normal backends isn't nice. >>

Re: [HACKERS] Cached plans and statement generalization

2017-04-25 Thread Doug Doole
When I did this in DB2, I didn't use the parser - it was too expensive. I just tokenized the statement and used some simple rules to bypass the invalid cases. For example, if I saw the tokens "ORDER" and "BY" then I'd disallow replacement replacement until I hit the end of the current subquery or

Re: [HACKERS] StandbyRecoverPreparedTransactions recovers subtrans links incorrectly

2017-04-25 Thread Andres Freund
On 2017-04-25 21:22:44 +0100, Simon Riggs wrote: > On 24 April 2017 at 19:59, Andres Freund wrote: > > > I don't think that's generally true. > > If you think that, from a risk perspective it is enough for me to > continue to investigate and I have been doing that. I'm not

Re: [HACKERS] Cached plans and statement generalization

2017-04-25 Thread Konstantin Knizhnik
On 04/25/2017 11:40 PM, Serge Rielau wrote: On Apr 25, 2017, at 1:37 PM, Konstantin Knizhnik > wrote: SELECT ‘hello’::CHAR(10) || ‘World’, 5 + 6; You can substitute ‘hello’, ‘World’, 5, and 6. But not 10. I am substituting

Re: [HACKERS] Cached plans and statement generalization

2017-04-25 Thread Serge Rielau
> On Apr 25, 2017, at 1:37 PM, Konstantin Knizhnik > wrote: >> >> SELECT ‘hello’::CHAR(10) || ‘World’, 5 + 6; >> >> You can substitute ‘hello’, ‘World’, 5, and 6. But not 10. > > I am substituting only string literals. So the query above will be > transformed to

Re: [HACKERS] Cached plans and statement generalization

2017-04-25 Thread Konstantin Knizhnik
On 04/25/2017 08:09 PM, Serge Rielau wrote: On Tue, Apr 25, 2017 at 9:45 AM, Konstantin Knizhnik wrote: On 25.04.2017 19:12, Serge Rielau wrote: On Apr 25, 2017, at 8:11 AM, Konstantin Knizhnik

Re: [HACKERS] Cached plans and statement generalization

2017-04-25 Thread Konstantin Knizhnik
On 04/25/2017 07:54 PM, David Fetter wrote: On Tue, Apr 25, 2017 at 06:11:09PM +0300, Konstantin Knizhnik wrote: On 24.04.2017 21:43, Andres Freund wrote: Hi, On 2017-04-24 11:46:02 +0300, Konstantin Knizhnik wrote: So what I am thinking now is implicit query caching. If the same query with

Re: [HACKERS] StandbyRecoverPreparedTransactions recovers subtrans links incorrectly

2017-04-25 Thread Simon Riggs
On 24 April 2017 at 19:59, Andres Freund wrote: > I don't think that's generally true. If you think that, from a risk perspective it is enough for me to continue to investigate and I have been doing that. As I said before I thought I had found a problem. I'm suggesting we

Re: [HACKERS] StandbyRecoverPreparedTransactions recovers subtrans links incorrectly

2017-04-25 Thread Simon Riggs
On 25 April 2017 at 16:28, Tom Lane wrote: > Simon Riggs writes: >> I can't see any reason now why overwriteOK should exist at all. I'm >> guessing that the whole "overwriteOK" idea was an incorrect response >> to xids appearing where they shouldn't

Re: [HACKERS] scram and \password

2017-04-25 Thread Magnus Hagander
On Tue, Apr 25, 2017 at 8:29 PM, Tom Lane wrote: > Robert Haas writes: > > On Tue, Apr 25, 2017 at 11:26 AM, Heikki Linnakangas > wrote: > >> A) Have PQencryptPassword() return an md5 hash. > >> > >> B) Have PQencryptPassword() return

Re: [HACKERS] tablesync patch broke the assumption that logical rep depends on?

2017-04-25 Thread Petr Jelinek
On 22/04/17 22:09, Petr Jelinek wrote: > On 21/04/17 16:31, Petr Jelinek wrote: >> On 21/04/17 16:23, Peter Eisentraut wrote: >>> On 4/21/17 10:11, Petr Jelinek wrote: On 21/04/17 16:09, Peter Eisentraut wrote: > On 4/20/17 14:29, Petr Jelinek wrote: >> +/* Find unused

Re: [HACKERS] PG 10 release notes

2017-04-25 Thread Bruce Momjian
On Tue, Apr 25, 2017 at 03:17:29PM -0400, Tels wrote: > > I think all that was missing was "time": > > > > By default planning and execution time is display by > > EXPLAIN ANALYZE and not display in other cases. > > The new EXPLAIN option SUMMARY allows > > explicit

Re: [HACKERS] Interval for launching the table sync worker

2017-04-25 Thread Peter Eisentraut
On 4/20/17 15:36, Peter Eisentraut wrote: > On 4/19/17 23:02, Noah Misch wrote: >> This PostgreSQL 10 open item is past due for your status update. Kindly send >> a status update within 24 hours, and include a date for your subsequent >> status >> update. Refer to the policy on open item

Re: [HACKERS] PG 10 release notes

2017-04-25 Thread Bruce Momjian
On Tue, Apr 25, 2017 at 08:12:05PM +0200, Petr Jelinek wrote: > On 25/04/17 17:01, Bruce Momjian wrote: > > > >> I also wonder if ability to run SQL queries on walsender connected to a > >> database is worth mentioning (replication=database kind of connection). > > > > Uh, why would that be

Re: [HACKERS] Re: logical replication and PANIC during shutdown checkpoint in publisher

2017-04-25 Thread Peter Eisentraut
On 4/20/17 11:30, Peter Eisentraut wrote: > On 4/19/17 23:04, Noah Misch wrote: >> This PostgreSQL 10 open item is past due for your status update. Kindly send >> a status update within 24 hours, and include a date for your subsequent >> status >> update. Refer to the policy on open item

Re: [HACKERS] PG 10 release notes

2017-04-25 Thread Tels
On Tue, April 25, 2017 1:21 pm, Bruce Momjian wrote: > On Tue, Apr 25, 2017 at 01:06:05PM -0400, Tels wrote: >> Moin, >> >> On Mon, April 24, 2017 9:31 pm, Bruce Momjian wrote: >> > I have committed the first draft of the Postgres 10 release notes. >> They >> > are current as of two days ago, and

Re: [HACKERS] Declarative partitioning - another take

2017-04-25 Thread Robert Haas
On Mon, Apr 24, 2017 at 6:43 AM, Amit Langote wrote: > The reason it doesn't work is that we do not allocate ResultRelInfos for > partitioned tables (not even for the root partitioned table in the > update/delete cases), because the current implementation assumes

Re: [HACKERS] PG 10 release notes

2017-04-25 Thread Claudio Freire
On Tue, Apr 25, 2017 at 2:45 PM, Bruce Momjian wrote: > On Tue, Apr 25, 2017 at 10:37:48AM -0700, Andres Freund wrote: >> On 2017-04-25 13:11:32 -0400, Bruce Momjian wrote: >> > I don't think this warrants inclusion in the release notes for reasons >> > already discussed. The

Re: [HACKERS] subscription worker doesn't start immediately on eabled

2017-04-25 Thread Peter Eisentraut
On 4/6/17 08:24, Kyotaro HORIGUCHI wrote: > The attached patch wakes up launcher when a subscription is > enabled. This fails when a subscription is enabled immedaitely > after disabling but it won't be a matter. committed, thanks -- Peter Eisentraut http://www.2ndQuadrant.com/

Re: [HACKERS] Separation walsender & normal backends

2017-04-25 Thread Andres Freund
On 2017-04-25 10:34:20 -0400, Tom Lane wrote: > Andres Freund writes: > > I've for a while suspected that the separation & duplication of > > infrastructure between walsenders and normal backends isn't nice. > > I think we should consider a more radical solution: trying to

Re: [HACKERS] scram and \password

2017-04-25 Thread Tom Lane
Robert Haas writes: > On Tue, Apr 25, 2017 at 11:26 AM, Heikki Linnakangas wrote: >> A) Have PQencryptPassword() return an md5 hash. >> >> B) Have PQencryptPassword() return a SCRAM verifier >> >> C) Have PQencryptPassword() return a SCRAM verifier if

Re: [HACKERS] logical replication and PANIC during shutdown checkpoint in publisher

2017-04-25 Thread Peter Eisentraut
On 4/21/17 00:11, Michael Paquier wrote: > Hmm. I have been actually looking at this solution and I am having > doubts regarding its robustness. In short this would need to be > roughly a two-step process: > - In PostmasterStateMachine(), SIGUSR2 is sent to the checkpoint to > make it call

Re: [HACKERS] scram and \password

2017-04-25 Thread Robert Haas
On Tue, Apr 25, 2017 at 11:26 AM, Heikki Linnakangas wrote: > algorithm as argument. But there are open decisions on what the old > PQencryptPassword() function should do, and also what the new function > should do by default, if you don't specify an algorithm: > > A) Have

Re: [HACKERS] PG 10 release notes

2017-04-25 Thread Petr Jelinek
On 25/04/17 17:01, Bruce Momjian wrote: > >> I also wonder if ability to run SQL queries on walsender connected to a >> database is worth mentioning (replication=database kind of connection). > > Uh, why would that be important to users? Because every tool that uses logical decoding to capture

Re: [HACKERS] PG 10 release notes

2017-04-25 Thread Andres Freund
Hi, On 2017-04-25 13:39:07 -0400, Bruce Momjian wrote: > Understood, but the question is whether the release notes are the right > place to educate users of something that will no longer be a problem. I think it's the *prime* place for it. It obviously doesn't matter if you're not affected by

Re: [HACKERS] Interval for launching the table sync worker

2017-04-25 Thread Peter Eisentraut
I feel it's getting a bit late for reworkings of this extent, also considering the marginal nature of the problem we are trying to fix. My patch from April 18 is very localized and gets the job done. I think this is still a good direction to investigate, but if we have to extend the hash table

Re: [HACKERS] Adding support for Default partition in partitioning

2017-04-25 Thread Robert Haas
On Tue, Apr 25, 2017 at 1:20 AM, Ashutosh Bapat wrote: >> I suspect it could be done as of now, but I'm a little worried that it >> might create grammar conflicts in the future as we extend the syntax >> further. If we use CREATE TABLE ... PARTITION OF ..

Re: [HACKERS] PG 10 release notes

2017-04-25 Thread Bruce Momjian
On Tue, Apr 25, 2017 at 01:44:09PM -0400, Tom Lane wrote: > Bruce Momjian writes: > > On Tue, Apr 25, 2017 at 10:00:52AM -0700, Andres Freund wrote: > >> at this point, you can see, we've squarely left O(N) country, and > >> entered the vast O(N^2) waste. > > > OK, I got it

Re: [HACKERS] PG 10 release notes

2017-04-25 Thread Bruce Momjian
On Tue, Apr 25, 2017 at 10:37:48AM -0700, Andres Freund wrote: > On 2017-04-25 13:11:32 -0400, Bruce Momjian wrote: > > I don't think this warrants inclusion in the release notes for reasons > > already discussed. The vacuum truncation operation is a rare one and > > an implementation detail. >

Re: [HACKERS] Interval for launching the table sync worker

2017-04-25 Thread Peter Eisentraut
On 4/21/17 09:59, Petr Jelinek wrote: > Rereading the code again, it's actually not bug as we update the rstate > to what syncworker says, but it's obviously confusing so probably still > worth to commit that. We don't have the syncworker->relmutex at that point, so it's probably better to read

Re: [HACKERS] PG 10 release notes

2017-04-25 Thread Tom Lane
Bruce Momjian writes: > On Tue, Apr 25, 2017 at 10:00:52AM -0700, Andres Freund wrote: >> at this point, you can see, we've squarely left O(N) country, and >> entered the vast O(N^2) waste. > OK, I got it now. :-) Here is the new item: > Improve table creation speed

Re: [HACKERS] PG 10 release notes

2017-04-25 Thread Bruce Momjian
On Tue, Apr 25, 2017 at 02:31:50PM -0300, Claudio Freire wrote: > >> Author: Álvaro Herrera, loosely based on a submission by Claudio Freire > >> Discussion: > >> https://postgr.es/m/cagtbqpa6nfgo_6g_y_7zqx8l9gchdsqkydo1tguh791z6py...@mail.gmail.com > > > > I don't think this warrants

Re: [HACKERS] PG 10 release notes

2017-04-25 Thread Andres Freund
On 2017-04-25 13:11:32 -0400, Bruce Momjian wrote: > On Tue, Apr 25, 2017 at 01:37:13PM -0300, Claudio Freire wrote: > > The truncate scan has been measured to be five times faster than without > > this patch (that was on a slow disk, but it shouldn't hurt on fast > > disks.) > > > >

Re: [HACKERS] PG 10 release notes

2017-04-25 Thread Claudio Freire
On Tue, Apr 25, 2017 at 2:11 PM, Bruce Momjian wrote: > On Tue, Apr 25, 2017 at 01:37:13PM -0300, Claudio Freire wrote: >> > I think it has been pretty common to accumulate a lot of such changes >> > into generic entries like, say, "speedups for hash joins". More detail >> >

Re: [HACKERS] PG 10 release notes

2017-04-25 Thread Bruce Momjian
On Tue, Apr 25, 2017 at 10:00:52AM -0700, Andres Freund wrote: > at this point, you can see, we've squarely left O(N) country, and > entered the vast O(N^2) waste. > > 10 109ms > 10 100 22ms > 10 1000162ms > 10 1 1497ms >

Re: [HACKERS] PG 10 release notes

2017-04-25 Thread Bruce Momjian
On Tue, Apr 25, 2017 at 01:06:05PM -0400, Tels wrote: > Moin, > > On Mon, April 24, 2017 9:31 pm, Bruce Momjian wrote: > > I have committed the first draft of the Postgres 10 release notes. They > > are current as of two days ago, and I will keep them current. Please > > give me any feedback

Re: [HACKERS] pgbench tap tests & minor fixes

2017-04-25 Thread Fabien COELHO
Hello Robert, 1. This patch makes assorted cosmetic and non-cosmetic changes to pgbench.c. That is not expected for a testing patch. Indeed, cosmetic changes should be avoided. If those changes need to be made because they are bug fixes or whatever, Yep, this is the case, minor bugs,

Re: [HACKERS] pg_dump emits ALTER TABLE ONLY partitioned_table

2017-04-25 Thread Robert Haas
On Tue, Apr 25, 2017 at 12:26 PM, Stephen Frost wrote: > Interesting. Seems like the question is really what we mean by "ONLY" > here. For my 2c, at least, if we can check that all of the partitions > already have the constraint enforced, such that the only thing we're >

Re: [HACKERS] PG 10 release notes

2017-04-25 Thread Bruce Momjian
On Tue, Apr 25, 2017 at 01:37:13PM -0300, Claudio Freire wrote: > > I think it has been pretty common to accumulate a lot of such changes > > into generic entries like, say, "speedups for hash joins". More detail > > than that simply isn't useful to end users; and as a rule, our release > > notes

Re: [HACKERS] Cached plans and statement generalization

2017-04-25 Thread Serge Rielau
On Tue, Apr 25, 2017 at 9:45 AM, Konstantin Knizhnik wrote: On 25.04.2017 19:12, Serge Rielau wrote: On Apr 25, 2017, at 8:11 AM, Konstantin Knizhnik < k.knizh...@postgrespro.ru [k.knizh...@postgrespro.ru] > wrote: Another problem is caused by using integer literals

Re: [HACKERS] PG 10 release notes

2017-04-25 Thread Tels
Moin, On Mon, April 24, 2017 9:31 pm, Bruce Momjian wrote: > I have committed the first draft of the Postgres 10 release notes. They > are current as of two days ago, and I will keep them current. Please > give me any feedback you have. Thank you! Here is one thing I noticed: "By default

Re: [HACKERS] PG 10 release notes

2017-04-25 Thread Andres Freund
On 2017-04-25 10:10:07 -0400, Bruce Momjian wrote: > On Mon, Apr 24, 2017 at 08:52:05PM -0700, Andres Freund wrote: > > On 2017-04-24 23:45:06 -0400, Tom Lane wrote: > > Oh, I completely agree with accumulating related changes, and that > > code-level details aren't useful. I think we skipped

Re: [HACKERS] Cached plans and statement generalization

2017-04-25 Thread David Fetter
On Tue, Apr 25, 2017 at 06:11:09PM +0300, Konstantin Knizhnik wrote: > On 24.04.2017 21:43, Andres Freund wrote: > > Hi, > > > > On 2017-04-24 11:46:02 +0300, Konstantin Knizhnik wrote: > > > So what I am thinking now is implicit query caching. If the same query > > > with > > > different

Re: [HACKERS] Cached plans and statement generalization

2017-04-25 Thread Konstantin Knizhnik
On 25.04.2017 19:12, Serge Rielau wrote: On Apr 25, 2017, at 8:11 AM, Konstantin Knizhnik > wrote: Another problem is caused by using integer literals in context where parameters can not be used, for example "order by 1”. You will

Re: [HACKERS] question: data file update when pg_basebackup in progress

2017-04-25 Thread David G. Johnston
On Tue, Apr 25, 2017 at 9:08 AM, Rui Hai Jiang wrote: > When pg_basebackup is launched, a checkpoint is created first, then all > files are transferred to the pg_basebackup client. Is it possible that a > data page(say page-N) in a data file is changed after the checkpoint

Re: [HACKERS] pgbench tap tests & minor fixes

2017-04-25 Thread Fabien COELHO
Hello Nikolay, - I agree to add a generic command TestLib & a wrapper in PostgresNode, instead of having pgbench specific things in the later, then call them from pgbench test script. - I still think that moving the pgbench scripts inside the test script is a bad idea (tm). My sum up

Re: [HACKERS] PG 10 release notes

2017-04-25 Thread Claudio Freire
On Tue, Apr 25, 2017 at 12:45 AM, Tom Lane wrote: > Andres Freund writes: >> On 2017-04-24 23:37:42 -0400, Bruce Momjian wrote: >>> I remember seeing those and those are normally details I do not put in >>> the release notes as there isn't a clear user

Re: [HACKERS] pg_dump emits ALTER TABLE ONLY partitioned_table

2017-04-25 Thread Stephen Frost
Robert, * Robert Haas (robertmh...@gmail.com) wrote: > On Mon, Apr 24, 2017 at 9:17 AM, Stephen Frost wrote: > > I wonder why the restriction is there, which is probably part of the > > reason that I'm thinking of phrasing the documentation that way. > > > > Beyond a matter

Re: [HACKERS] Cached plans and statement generalization

2017-04-25 Thread Serge Rielau
> On Apr 25, 2017, at 8:11 AM, Konstantin Knizhnik > wrote: > Another problem is caused by using integer literals in context where > parameters can not be used, for example "order by 1”. You will also need to deal with modifiers in types such as VARCHAR(10). Not

Re: [HACKERS] pgbench tap tests & minor fixes

2017-04-25 Thread Fabien COELHO
Hello, Nope. TestLib does not know about PostgresNode, the idea is rather that PostgresNode knows and wraps around TestLib when needed. Actually as I look at this part, I feeling an urge to rewrite this code, and change it so, that all command_like calls were called in a context of certain

[HACKERS] question: data file update when pg_basebackup in progress

2017-04-25 Thread Rui Hai Jiang
Hello, I'm checking how the pg_basebackup works and I got a question(maybe there are no such issues): When pg_basebackup is launched, a checkpoint is created first, then all files are transferred to the pg_basebackup client. Is it possible that a data page(say page-N) in a data file is

Re: [HACKERS] Quorum commit for multiple synchronous replication.

2017-04-25 Thread Fujii Masao
On Tue, Apr 25, 2017 at 5:41 PM, Kyotaro HORIGUCHI wrote: > At Tue, 25 Apr 2017 09:22:59 +0900, Masahiko Sawada > wrote in >> >> Please observe the policy on open item

Re: [HACKERS] Unportable implementation of background worker start

2017-04-25 Thread Tom Lane
=?utf-8?Q?R=C3=A9mi_Zara?= writes: >> Le 25 avr. 2017 à 01:47, Tom Lane a écrit : >> It looks like coypu is going to need manual intervention (ie, kill -9 >> on the leftover postmaster) to get unwedged :-(. That's particularly >> disturbing because it

Re: [HACKERS] Crash observed during the start of the Postgres process

2017-04-25 Thread K S, Sandhya (Nokia - IN/Bangalore)
Hello, Did you get a chance to take a look into the issue? Please consider it with high priority. We will be awaiting your inputs. Regards, Sandhya _ From: K S, Sandhya (Nokia - IN/Bangalore) Sent: Thursday, April 20, 2017 1:36 PM To:

Re: [HACKERS] [BUGS] Crash observed during the start of the Postgres process

2017-04-25 Thread K S, Sandhya (Nokia - IN/Bangalore)
Hi Merlin, Below is the log captured when the crash was encountered. STATEMENT: select count(1) from pg_ls_dir(current_setting('data_directory')) where pg_ls_dir = 'backup_label' LOG: 0: duration: 4.083 ms LOCATION: exec_simple_query, postgres.c:1145 DEBUG: 0: shmem_exit(0): 7

Re: [HACKERS] pg_dump emits ALTER TABLE ONLY partitioned_table

2017-04-25 Thread Stephen Frost
Amit, * Amit Langote (langote_amit...@lab.ntt.co.jp) wrote: > I think why getPartitions() is separate from getInherits() and then > flagPartitions() separate from flagInhTables() is because I thought > originally that mixing the two would be undesirable. In the partitioning > case,

  1   2   >