Re: Postgres 11 release notes

[Bruce Momjian]

On Wed, Oct 10, 2018 at 06:13:56PM -0400, Bruce Momjian wrote:
> On Wed, Oct 10, 2018 at 06:06:01PM -0400, Bruce Momjian wrote:
> > On Wed, Oct 10, 2018 at 12:55:33PM -0700, Andres Freund wrote:
> > > Several people argued for introducing it, you're the only one that was
> > > against it. Adrien, Amit Kapila, Peter Geoghegan, and I all said we
> > > think that kind of thing should be included.
> > 
> > Then you need to start a new thread and argue that we need to include
> > more optimizer items in the release notes.  My point was that I was
> > using the existing criteria, and if we want to change it, we have to do
> > it consistently, not just jam in one item.
> > 
> > Once we agree we can go back and see what we missed in PG 11.
> 
> Years ago someone said they wanted more optimizer items in the release
> notes, so I did it, and then many more people said they didn't want it
> --- I don't want to repeat the experience.

Thinking some more, if this is this _only_ optimizer improvement you
think I missed, let's just add it and call it done, and we will not
re-litigate the criteria used for the release notes.

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

+ As you are, so once was I.  As I am, so you will be. +
+  Ancient Roman grave inscription +

Re: Postgres 11 release notes

[Bruce Momjian]

On Wed, Oct 10, 2018 at 06:06:01PM -0400, Bruce Momjian wrote:
> On Wed, Oct 10, 2018 at 12:55:33PM -0700, Andres Freund wrote:
> > Several people argued for introducing it, you're the only one that was
> > against it. Adrien, Amit Kapila, Peter Geoghegan, and I all said we
> > think that kind of thing should be included.
> 
> Then you need to start a new thread and argue that we need to include
> more optimizer items in the release notes.  My point was that I was
> using the existing criteria, and if we want to change it, we have to do
> it consistently, not just jam in one item.
> 
> Once we agree we can go back and see what we missed in PG 11.

Years ago someone said they wanted more optimizer items in the release
notes, so I did it, and then many more people said they didn't want it
--- I don't want to repeat the experience.

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

+ As you are, so once was I.  As I am, so you will be. +
+  Ancient Roman grave inscription +

Re: Postgres 11 release notes

[Bruce Momjian]

On Wed, Oct 10, 2018 at 12:55:33PM -0700, Andres Freund wrote:
> Hi,
> 
> On 2018-10-08 21:28:02 -0400, Bruce Momjian wrote:
> > On Mon, Oct  8, 2018 at 05:44:32PM +0200, Adrien NAYRAT wrote:
> > > On 10/8/18 5:20 PM, Bruce Momjian wrote:
> > > >Uh, where are you looking?  We don't rebuild the beta docs until the
> > > >next beta release, but you can see the changes in the developer docs:
> > > >
> > > > https://www.postgresql.org/docs/devel/static/release-11.html
> > > 
> > > I looked in doc/src/sgml/release-11.sgml
> > > 
> > > And even on https://www.postgresql.org/docs/devel/static/release-11.html I
> > > don't see mention of 1f6d515a67
> > 
> > I did not think there was sufficient interest in adding this item.  If
> > we add it we need to discuss adding all similar items.
> 
> Several people argued for introducing it, you're the only one that was
> against it. Adrien, Amit Kapila, Peter Geoghegan, and I all said we
> think that kind of thing should be included.

Then you need to start a new thread and argue that we need to include
more optimizer items in the release notes.  My point was that I was
using the existing criteria, and if we want to change it, we have to do
it consistently, not just jam in one item.

Once we agree we can go back and see what we missed in PG 11.

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

+ As you are, so once was I.  As I am, so you will be. +
+  Ancient Roman grave inscription +

Re: Postgres 11 release notes

[Andres Freund]

Hi,

On 2018-10-08 21:28:02 -0400, Bruce Momjian wrote:
> On Mon, Oct  8, 2018 at 05:44:32PM +0200, Adrien NAYRAT wrote:
> > On 10/8/18 5:20 PM, Bruce Momjian wrote:
> > >Uh, where are you looking?  We don't rebuild the beta docs until the
> > >next beta release, but you can see the changes in the developer docs:
> > >
> > >   https://www.postgresql.org/docs/devel/static/release-11.html
> > 
> > I looked in doc/src/sgml/release-11.sgml
> > 
> > And even on https://www.postgresql.org/docs/devel/static/release-11.html I
> > don't see mention of 1f6d515a67
> 
> I did not think there was sufficient interest in adding this item.  If
> we add it we need to discuss adding all similar items.

Several people argued for introducing it, you're the only one that was
against it. Adrien, Amit Kapila, Peter Geoghegan, and I all said we
think that kind of thing should be included.

Greetings,

Andres Freund

Re: Postgres 11 release notes

[Bruce Momjian]

On Mon, Oct  8, 2018 at 05:44:32PM +0200, Adrien NAYRAT wrote:
> On 10/8/18 5:20 PM, Bruce Momjian wrote:
> >Uh, where are you looking?  We don't rebuild the beta docs until the
> >next beta release, but you can see the changes in the developer docs:
> >
> > https://www.postgresql.org/docs/devel/static/release-11.html
> 
> I looked in doc/src/sgml/release-11.sgml
> 
> And even on https://www.postgresql.org/docs/devel/static/release-11.html I
> don't see mention of 1f6d515a67

I did not think there was sufficient interest in adding this item.  If
we add it we need to discuss adding all similar items.

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

+ As you are, so once was I.  As I am, so you will be. +
+  Ancient Roman grave inscription +

Re: Postgres 11 release notes

[Adrien NAYRAT]

On 10/8/18 5:20 PM, Bruce Momjian wrote:

Uh, where are you looking?  We don't rebuild the beta docs until the
next beta release, but you can see the changes in the developer docs:

https://www.postgresql.org/docs/devel/static/release-11.html


I looked in doc/src/sgml/release-11.sgml

And even on https://www.postgresql.org/docs/devel/static/release-11.html 
I don't see mention of 1f6d515a67


Thanks

Re: Postgres 11 release notes

[Bruce Momjian]

On Mon, Oct  8, 2018 at 12:05:03PM +0200, Adrien NAYRAT wrote:
> On 9/20/18 8:47 AM, Adrien Nayrat wrote:
> >On 8/25/18 11:24 PM, Peter Geoghegan wrote:
> >>On Sat, Aug 25, 2018 at 2:18 PM, Bruce Momjian  wrote:
> I think that's less "our" logic and more yours, that has become
> established because you've done most of the major release notes for a
> long time. I'm not trying to say that that's wrong or anything, just
> >>>
> >>>I don't do my work in a vacuum.  My behavior is based on what feedback I
> >>>have gotten from the community on what to include/exclude.
> >>
> >>FWIW, I agree with what Andres said about planner changes and the release 
> >>notes.
> >>
> >
> >Hello,
> >
> >Here is the patch that mention 1f6d515a67. Note I am not sure about my 
> >explanation.
> >
> >
> >Thanks
> >
> 
> Hi,
> 
> It seems this change has not been added to release notes? Should I change
> something in my path?

Uh, where are you looking?  We don't rebuild the beta docs until the
next beta release, but you can see the changes in the developer docs:

https://www.postgresql.org/docs/devel/static/release-11.html

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

+ As you are, so once was I.  As I am, so you will be. +
+  Ancient Roman grave inscription +

Re: Postgres 11 release notes

[Adrien NAYRAT]

On 9/20/18 8:47 AM, Adrien Nayrat wrote:

On 8/25/18 11:24 PM, Peter Geoghegan wrote:

On Sat, Aug 25, 2018 at 2:18 PM, Bruce Momjian  wrote:

I think that's less "our" logic and more yours, that has become
established because you've done most of the major release notes for a
long time. I'm not trying to say that that's wrong or anything, just


I don't do my work in a vacuum.  My behavior is based on what feedback I
have gotten from the community on what to include/exclude.


FWIW, I agree with what Andres said about planner changes and the release notes.



Hello,

Here is the patch that mention 1f6d515a67. Note I am not sure about my 
explanation.


Thanks



Hi,

It seems this change has not been added to release notes? Should I 
change something in my path?


Thanks,

Re: Postgres 11 release notes

[Jonathan S. Katz]

On 10/5/18 5:22 PM, Bruce Momjian wrote:
> On Fri, Oct  5, 2018 at 03:28:41PM -0400, Jonathan Katz wrote:
>> On 9/27/18 9:21 PM, Amit Langote wrote:
>>> Yeah, "any other partition" is what the existing description uses too, so:
>>>
>>> Having a "default" partition for storing data that does not match any
>>> other partition
>>
>> Sorry for the slow turnaround on this. Excellent suggestion. I have
>> updated the patches, please see attached.
> 
> Patch applied:
> 
>   
> https://git.postgresql.org/pg/commitdiff/6eb612fea9d080f2ae77ecb7091e73dc9f298c97
> 

Thank you!

Jonathan



signature.asc
Description: OpenPGP digital signature

Re: Postgres 11 release notes

[Bruce Momjian]

On Fri, Oct  5, 2018 at 03:28:41PM -0400, Jonathan Katz wrote:
> On 9/27/18 9:21 PM, Amit Langote wrote:
> > Yeah, "any other partition" is what the existing description uses too, so:
> > 
> > Having a "default" partition for storing data that does not match any
> > other partition
> 
> Sorry for the slow turnaround on this. Excellent suggestion. I have
> updated the patches, please see attached.

Patch applied:


https://git.postgresql.org/pg/commitdiff/6eb612fea9d080f2ae77ecb7091e73dc9f298c97

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

+ As you are, so once was I.  As I am, so you will be. +
+  Ancient Roman grave inscription +

Re: Postgres 11 release notes

[Jonathan S. Katz]

On 9/27/18 9:21 PM, Amit Langote wrote:
> On 2018/09/27 23:24, Alvaro Herrera wrote:
>> On 2018-Sep-27, Amit Langote wrote:
>>
>>> Sorry I couldn't reply sooner, but the following of your proposed text
>>> needs to be updated a bit:
>>>
>>> +   
>>> +
>>> + Having a "default" partition for storing data that does not match 
>>> a
>>> + partition key
>>> +
>>> +   
>>>
>>> I think "does not match a partition key" is not accurate.  Description of
>>> default partitions further below in the release notes says this:
>>>
>>> "The default partition can store rows that don't match any of the other
>>> defined partitions, and is searched accordingly."
>>>
>>> So, we could perhaps write it as:
>>>
>>> Having a "default" partition for storing data that does not match any of
>>> the remaining partitions
>>
>> Yeah, I agree that "a partition key" is not the right term to use there
>> (and that term is used in the press release text also).  However I don't
>> think "remaining" is the right word there either, because it sounds as
>> if you're removing something.
>>
>> For the Spanish translation of the press release, we ended up using the
>> equivalent of "for the data that does not match any other partition".
> 
> Yeah, "any other partition" is what the existing description uses too, so:
> 
> Having a "default" partition for storing data that does not match any
> other partition

Sorry for the slow turnaround on this. Excellent suggestion. I have
updated the patches, please see attached.

Thanks,

Jonathan
From 6fee4bd74a3ce32b47a9b53ad9d5536d069b6f56 Mon Sep 17 00:00:00 2001
From: "Jonathan S. Katz" 
Date: Sat, 15 Sep 2018 10:43:44 -0400
Subject: [PATCH 1/2] Updates to major improvements section of release notes.

---
 doc/src/sgml/release-11.sgml | 41 +
 1 file changed, 29 insertions(+), 12 deletions(-)

diff --git a/doc/src/sgml/release-11.sgml b/doc/src/sgml/release-11.sgml
index ae65431bbe..f9d93b734e 100644
--- a/doc/src/sgml/release-11.sgml
+++ b/doc/src/sgml/release-11.sgml
@@ -6,7 +6,7 @@
 
   
Release date:
-   2018-??-?? (CURRENT AS OF 2018-07-30)
+   2018-10-?? (CURRENT AS OF 2018-09-20)
   
 
   
@@ -22,7 +22,7 @@
 
 
  
-  Major improvements to partitioning:
+  Improvements to partitioning functionality, including:
   

 
@@ -37,9 +37,8 @@


 
- Improved SELECT query performance due to
- enhanced partition elimination during query processing and
- execution
+ Improved SELECT performance from enhanced partition
+ elimination strategies during query processing and execution
 


@@ -48,29 +47,37 @@
  KEY, indexes, and triggers on partitioned tables
 

+   
+
+ Having a "default" partition for storing data that does not match any
+ of the remaining partitions
+
+   
   
  
 
 
 
  
-  Improvements to parallelism:
+  Improvements to parallelism, including:
   

 
- Parallelized hash joins
+ B-tree indexes can now be built in parallel with
+ CREATE INDEX
 


 
- Parallelized CREATE INDEX for B-tree indexes
+ Parallelized CREATE TABLE .. AS,
+ CREATE MATERIALIZED VIEW, and certain
+ queries using UNION
 


 
- Parallelized CREATE TABLE .. AS,
- CREATE MATERIALIZED VIEW, and certain
- queries using UNION
+ Performance improvements for parallelized hash joins and parallelized
+ sequential scans
 

   
@@ -79,7 +86,10 @@
 
 
  
-  SQL stored procedures, with support for embedded transactions
+  SQL stored procedures that support embedded transactions. Stored
+  procedures can be created with 
+  CREATE PROCEDURE and executed with
+  CALL
  
 
 
@@ -99,6 +109,13 @@
  
 
 
+
+ 
+  Covering indexes, which can be utilized using the
+  INCLUDE clause of CREATE INDEX
+ 
+
+
 
  
   Many other useful performance improvements, including making
-- 
2.14.3 (Apple Git-98)

From a1922a4a590655ac9544fd41460a998fd975523b Mon Sep 17 00:00:00 2001
From: "Jonathan S. Katz" 
Date: Sat, 15 Sep 2018 11:01:35 -0400
Subject: [PATCH 2/2] Remove or modify placeholders for v11 release notes.

---
 doc/src/sgml/release-11.sgml | 18 --
 1 file changed, 4 insertions(+), 14 deletions(-)

diff --git a/doc/src/sgml/release-11.sgml b/doc/src/sgml/release-11.sgml
index f9d93b734e..ca42f28cc9 100644
--- a/doc/src/sgml/release-11.sgml
+++ b/doc/src/sgml/release-11.sgml
@@ -196,10 +196,6 @@
 would be dumped without such specifications if the database locale
 and encoding matched the old cluster's defaults.

-
-   
-DID I GET 

Re: Postgres 11 release notes

[Bruce Momjian]

On Sat, Sep 29, 2018 at 04:28:13PM +0900, Amit Langote wrote:
> > The default partition can store rows that don't match any of the
> > other defined partitions, and is searched accordingly.
> 
> I was commenting on the Jonathan's patch upthread [1] whereby he
> proposed to add a line about the default partition feature in the
> major partitioning enhancements section at the top.
> 
> The existing text that you'd added when creating the release notes is fine.
> 
> > The press release?
> 
> Yeah, Alvaro pointed out that the press release has inaccurate text
> about the default partition, but that's a separate issue, which I was
> unaware of.

Oh, OK, thanks.

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

+ As you are, so once was I.  As I am, so you will be. +
+  Ancient Roman grave inscription +

Re: Postgres 11 release notes

[Amit Langote]

On Fri, Sep 28, 2018 at 7:24 PM Bruce Momjian  wrote:
> On Fri, Sep 28, 2018 at 10:21:16AM +0900, Amit Langote wrote:
> > On 2018/09/27 23:24, Alvaro Herrera wrote:
> > > On 2018-Sep-27, Amit Langote wrote:
> > >
> > >> Sorry I couldn't reply sooner, but the following of your proposed text
> > >> needs to be updated a bit:
> > >>
> > >> +   
> > >> +
> > >> + Having a "default" partition for storing data that does not 
> > >> match a
> > >> + partition key
> > >> +
> > >> +   
> > >>
> > >> I think "does not match a partition key" is not accurate.  Description of
> > >> default partitions further below in the release notes says this:
> > >>
> > >> "The default partition can store rows that don't match any of the other
> > >> defined partitions, and is searched accordingly."
> > >>
> > >> So, we could perhaps write it as:
> > >>
> > >> Having a "default" partition for storing data that does not match any of
> > >> the remaining partitions
> > >
> > > Yeah, I agree that "a partition key" is not the right term to use there
> > > (and that term is used in the press release text also).  However I don't
> > > think "remaining" is the right word there either, because it sounds as
> > > if you're removing something.
> > >
> > > For the Spanish translation of the press release, we ended up using the
> > > equivalent of "for the data that does not match any other partition".
> >
> > Yeah, "any other partition" is what the existing description uses too, so:
> >
> > Having a "default" partition for storing data that does not match any
> > other partition
>
> Uh, what text are you talkinga about?  I see this text in the release
> notes since May:
>
> The default partition can store rows that don't match any of the
> other defined partitions, and is searched accordingly.

I was commenting on the Jonathan's patch upthread [1] whereby he
proposed to add a line about the default partition feature in the
major partitioning enhancements section at the top.

The existing text that you'd added when creating the release notes is fine.

> The press release?

Yeah, Alvaro pointed out that the press release has inaccurate text
about the default partition, but that's a separate issue, which I was
unaware of.

Thanks,
Amit

[1] 
https://www.postgresql.org/message-id/dcdbb9e9-cde2-fb78-fdb6-76d672d08dbc%40postgresql.org

Re: Postgres 11 release notes

[Bruce Momjian]

On Fri, Sep 28, 2018 at 10:21:16AM +0900, Amit Langote wrote:
> On 2018/09/27 23:24, Alvaro Herrera wrote:
> > On 2018-Sep-27, Amit Langote wrote:
> > 
> >> Sorry I couldn't reply sooner, but the following of your proposed text
> >> needs to be updated a bit:
> >>
> >> +   
> >> +
> >> + Having a "default" partition for storing data that does not 
> >> match a
> >> + partition key
> >> +
> >> +   
> >>
> >> I think "does not match a partition key" is not accurate.  Description of
> >> default partitions further below in the release notes says this:
> >>
> >> "The default partition can store rows that don't match any of the other
> >> defined partitions, and is searched accordingly."
> >>
> >> So, we could perhaps write it as:
> >>
> >> Having a "default" partition for storing data that does not match any of
> >> the remaining partitions
> > 
> > Yeah, I agree that "a partition key" is not the right term to use there
> > (and that term is used in the press release text also).  However I don't
> > think "remaining" is the right word there either, because it sounds as
> > if you're removing something.
> >
> > For the Spanish translation of the press release, we ended up using the
> > equivalent of "for the data that does not match any other partition".
> 
> Yeah, "any other partition" is what the existing description uses too, so:
> 
> Having a "default" partition for storing data that does not match any
> other partition

Uh, what text are you talkinga about?  I see this text in the release
notes since May:

The default partition can store rows that don't match any of the
other defined partitions, and is searched accordingly.

The press release?

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

+ As you are, so once was I.  As I am, so you will be. +
+  Ancient Roman grave inscription +

Re: Postgres 11 release notes

[Amit Langote]

On 2018/09/27 23:24, Alvaro Herrera wrote:
> On 2018-Sep-27, Amit Langote wrote:
> 
>> Sorry I couldn't reply sooner, but the following of your proposed text
>> needs to be updated a bit:
>>
>> +   
>> +
>> + Having a "default" partition for storing data that does not match a
>> + partition key
>> +
>> +   
>>
>> I think "does not match a partition key" is not accurate.  Description of
>> default partitions further below in the release notes says this:
>>
>> "The default partition can store rows that don't match any of the other
>> defined partitions, and is searched accordingly."
>>
>> So, we could perhaps write it as:
>>
>> Having a "default" partition for storing data that does not match any of
>> the remaining partitions
> 
> Yeah, I agree that "a partition key" is not the right term to use there
> (and that term is used in the press release text also).  However I don't
> think "remaining" is the right word there either, because it sounds as
> if you're removing something.
>
> For the Spanish translation of the press release, we ended up using the
> equivalent of "for the data that does not match any other partition".

Yeah, "any other partition" is what the existing description uses too, so:

Having a "default" partition for storing data that does not match any
other partition

Thanks,
Amit

Re: Postgres 11 release notes

[Alvaro Herrera]

On 2018-Sep-27, Amit Langote wrote:

> Sorry I couldn't reply sooner, but the following of your proposed text
> needs to be updated a bit:
> 
> +   
> +
> + Having a "default" partition for storing data that does not match a
> + partition key
> +
> +   
> 
> I think "does not match a partition key" is not accurate.  Description of
> default partitions further below in the release notes says this:
> 
> "The default partition can store rows that don't match any of the other
> defined partitions, and is searched accordingly."
> 
> So, we could perhaps write it as:
> 
> Having a "default" partition for storing data that does not match any of
> the remaining partitions

Yeah, I agree that "a partition key" is not the right term to use there
(and that term is used in the press release text also).  However I don't
think "remaining" is the right word there either, because it sounds as
if you're removing something.

For the Spanish translation of the press release, we ended up using the
equivalent of "for the data that does not match any other partition".

-- 
Álvaro Herrerahttps://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: Postgres 11 release notes

[Amit Langote]

Hi,

On 2018/09/17 1:07, Jonathan S. Katz wrote:
> On 9/16/18 11:36 AM, Bruce Momjian wrote:
>> On Sat, Sep 15, 2018 at 11:06:04AM -0400, Jonathan Katz wrote:
>>> @@ -2414,12 +2408,8 @@ same commits as above
>>>  The option --create-slot creates
>>>  the named replication slot (--slot)
>>>  when the WAL streaming method
>>> -(--wal-method=stream) is used.
>>> -   
>>> -
>>> -   
>>> -IS IT CLEAR FROM THE DOCS THAT THE REPLICATION SLOT IS NOT
>>> -TEMPORARY?
>>> +(--wal-method=stream) is used. The replication 
>>> slot is
>>> +available until it is explicitly dropped.
>>> 
>>>
>>
>> I have clarified the documentation for this option.  This additional
>> release note text is not necessary.
> 
> Thanks for updating it in the docs.
> 
> I can make arguments both ways about including/not including that in the
> release notes, but I've attached a patch that drops the clarification.

Sorry I couldn't reply sooner, but the following of your proposed text
needs to be updated a bit:

+   
+
+ Having a "default" partition for storing data that does not match a
+ partition key
+
+   

I think "does not match a partition key" is not accurate.  Description of
default partitions further below in the release notes says this:

"The default partition can store rows that don't match any of the other
defined partitions, and is searched accordingly."

So, we could perhaps write it as:

Having a "default" partition for storing data that does not match any of
the remaining partitions

Thanks,
Amit

Re: Postgres 11 release notes

[Adrien Nayrat]

On 8/25/18 11:24 PM, Peter Geoghegan wrote:
> On Sat, Aug 25, 2018 at 2:18 PM, Bruce Momjian  wrote:
>>> I think that's less "our" logic and more yours, that has become
>>> established because you've done most of the major release notes for a
>>> long time. I'm not trying to say that that's wrong or anything, just
>>
>> I don't do my work in a vacuum.  My behavior is based on what feedback I
>> have gotten from the community on what to include/exclude.
> 
> FWIW, I agree with what Andres said about planner changes and the release 
> notes.
> 

Hello,

Here is the patch that mention 1f6d515a67. Note I am not sure about my 
explanation.


Thanks

-- 
Adrien

diff --combined doc/src/sgml/release-11.sgml
index 53b6fa339c,ae65431bbe..00
--- a/doc/src/sgml/release-11.sgml
+++ b/doc/src/sgml/release-11.sgml
@@@ -1056,18 -1056,6 +1056,18 @@@ same commits as abov
  

  
 +  
 +
 +
 +   
 +Push LIMIT through subqueries to underlying sort (if there is no filtering
 +predicate or SRF).
 +   
 +
 +  
 +
   
  
  


signature.asc
Description: OpenPGP digital signature

Re: Postgres 11 release notes

[Jonathan S. Katz]

On 9/16/18 11:36 AM, Bruce Momjian wrote:
> On Sat, Sep 15, 2018 at 11:06:04AM -0400, Jonathan Katz wrote:
>> @@ -2414,12 +2408,8 @@ same commits as above
>>  The option --create-slot creates
>>  the named replication slot (--slot)
>>  when the WAL streaming method
>> -(--wal-method=stream) is used.
>> -   
>> -
>> -   
>> -IS IT CLEAR FROM THE DOCS THAT THE REPLICATION SLOT IS NOT
>> -TEMPORARY?
>> +(--wal-method=stream) is used. The replication 
>> slot is
>> +available until it is explicitly dropped.
>> 
>>
> 
> I have clarified the documentation for this option.  This additional
> release note text is not necessary.

Thanks for updating it in the docs.

I can make arguments both ways about including/not including that in the
release notes, but I've attached a patch that drops the clarification.

Thanks,

Jonathan
From c0a40e7831b19f75178a45d8db00f5e91d606e33 Mon Sep 17 00:00:00 2001
From: "Jonathan S. Katz" 
Date: Sat, 15 Sep 2018 10:43:44 -0400
Subject: [PATCH 1/2] Updates to major improvements section of release notes.

---
 doc/src/sgml/release-11.sgml | 45 ++--
 1 file changed, 31 insertions(+), 14 deletions(-)

diff --git a/doc/src/sgml/release-11.sgml b/doc/src/sgml/release-11.sgml
index 684d34c091..f81dd929e3 100644
--- a/doc/src/sgml/release-11.sgml
+++ b/doc/src/sgml/release-11.sgml
@@ -6,7 +6,7 @@
 
   
Release date:
-   2018-??-?? (CURRENT AS OF 2018-07-30)
+   2018-10-?? (CURRENT AS OF 2018-09-20)
   
 
   
@@ -22,7 +22,7 @@
 
 
  
-  Major improvements to partitioning:
+  Improvements to partitioning functionality, including:
   

 
@@ -37,9 +37,8 @@


 
- Improved SELECT query performance due to
- enhanced partition elimination during query processing and
- execution
+ Improved SELECT performance from enhanced partition
+ elimination strategies during query processing and execution
 


@@ -48,29 +47,37 @@
  KEY, indexes, and triggers on partitioned tables
 

+   
+
+ Having a "default" partition for storing data that does not match a
+ partition key
+
+   
   
  
 
 
 
  
-  Improvements to parallelism:
+  Improvements to parallelism, including:
   

 
- Parallelized hash joins
+ B-tree indexes can now be built in parallel with
+ CREATE INDEX
 


 
- Parallelized CREATE INDEX for B-tree indexes
+ Parallelized CREATE TABLE .. AS,
+ CREATE MATERIALIZED VIEW, and certain
+ queries using UNION
 


 
- Parallelized CREATE TABLE .. AS,
- CREATE MATERIALIZED VIEW, and certain
- queries using UNION
+ Performance improvements for parallelized hash joins and parallelized
+ sequential scans
 

   
@@ -79,14 +86,17 @@
 
 
  
-  SQL stored procedures, with support for embedded transactions
+  SQL stored procedures that support embedded transactions. Stored
+  procedures can be created with 
+  CREATE PROCEDURE and executed with
+  CALL
  
 
 
 
  
-  JIT compilation of some SQL code, including support for fast evaluation
-  of expressions
+  Introduction of just-in-time (JIT) compilation
+  during query execution
  
 
 
@@ -99,6 +109,13 @@
  
 
 
+
+ 
+  Covering indexes, which can be utilized using the
+  INCLUDE clause of CREATE INDEX
+ 
+
+
 
  
   Many other useful performance improvements, including making
-- 
2.14.3 (Apple Git-98)

From 8babed4b4535bae8b46b45bec43c8117efbcb332 Mon Sep 17 00:00:00 2001
From: "Jonathan S. Katz" 
Date: Sat, 15 Sep 2018 11:01:35 -0400
Subject: [PATCH 2/2] Remove or modify placeholders for v11 release notes.

---
 doc/src/sgml/release-11.sgml | 18 --
 1 file changed, 4 insertions(+), 14 deletions(-)

diff --git a/doc/src/sgml/release-11.sgml b/doc/src/sgml/release-11.sgml
index f81dd929e3..953bbd2df9 100644
--- a/doc/src/sgml/release-11.sgml
+++ b/doc/src/sgml/release-11.sgml
@@ -196,10 +196,6 @@
 would be dumped without such specifications if the database locale
 and encoding matched the old cluster's defaults.

-
-   
-DID I GET EVERYTHING?
-   
   
 
   
@@ -614,8 +610,7 @@
 The new command ALTER
 INDEX ATTACH PARTITION allows indexes to be
 attached to partitions.  This does not behave as a global index
-since the contents are private to each index.  WARN WHEN USING
-AN EXISTING INDEX?
+since the contents are private to each index.

   
 
@@ -924,7 +919,7 @@ same commits as above

 
   

Re: Postgres 11 release notes

[Bruce Momjian]

On Sat, Sep 15, 2018 at 11:06:04AM -0400, Jonathan Katz wrote:
> @@ -2414,12 +2408,8 @@ same commits as above
>  The option --create-slot creates
>  the named replication slot (--slot)
>  when the WAL streaming method
> -(--wal-method=stream) is used.
> -   
> -
> -   
> -IS IT CLEAR FROM THE DOCS THAT THE REPLICATION SLOT IS NOT
> -TEMPORARY?
> +(--wal-method=stream) is used. The replication slot 
> is
> +available until it is explicitly dropped.
> 
>

I have clarified the documentation for this option.  This additional
release note text is not necessary.

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

+ As you are, so once was I.  As I am, so you will be. +
+  Ancient Roman grave inscription +

Re: Postgres 11 release notes

[Jonathan S. Katz]

On 9/16/18 11:14 AM, Bruce Momjian wrote:
> On Sat, Sep 15, 2018 at 11:06:04AM -0400, Jonathan Katz wrote:
>> 
>>  
>> - Partitioning by a hash key
>> + Partitioning by a hash key (a.k.a. "hash partitioning")
> 
> I question whether a.k.a (also known as) is familiar enough to our
> readers to appear in the releaes notes.  I think the "a.k.a." can be
> simply removed.

Fine by me. I've reattached the patches.

Thanks,

Jonathan

From c0a40e7831b19f75178a45d8db00f5e91d606e33 Mon Sep 17 00:00:00 2001
From: "Jonathan S. Katz" 
Date: Sat, 15 Sep 2018 10:43:44 -0400
Subject: [PATCH 1/2] Updates to major improvements section of release notes.

---
 doc/src/sgml/release-11.sgml | 45 ++--
 1 file changed, 31 insertions(+), 14 deletions(-)

diff --git a/doc/src/sgml/release-11.sgml b/doc/src/sgml/release-11.sgml
index 684d34c091..f81dd929e3 100644
--- a/doc/src/sgml/release-11.sgml
+++ b/doc/src/sgml/release-11.sgml
@@ -6,7 +6,7 @@
 
   
Release date:
-   2018-??-?? (CURRENT AS OF 2018-07-30)
+   2018-10-?? (CURRENT AS OF 2018-09-20)
   
 
   
@@ -22,7 +22,7 @@
 
 
  
-  Major improvements to partitioning:
+  Improvements to partitioning functionality, including:
   

 
@@ -37,9 +37,8 @@


 
- Improved SELECT query performance due to
- enhanced partition elimination during query processing and
- execution
+ Improved SELECT performance from enhanced partition
+ elimination strategies during query processing and execution
 


@@ -48,29 +47,37 @@
  KEY, indexes, and triggers on partitioned tables
 

+   
+
+ Having a "default" partition for storing data that does not match a
+ partition key
+
+   
   
  
 
 
 
  
-  Improvements to parallelism:
+  Improvements to parallelism, including:
   

 
- Parallelized hash joins
+ B-tree indexes can now be built in parallel with
+ CREATE INDEX
 


 
- Parallelized CREATE INDEX for B-tree indexes
+ Parallelized CREATE TABLE .. AS,
+ CREATE MATERIALIZED VIEW, and certain
+ queries using UNION
 


 
- Parallelized CREATE TABLE .. AS,
- CREATE MATERIALIZED VIEW, and certain
- queries using UNION
+ Performance improvements for parallelized hash joins and parallelized
+ sequential scans
 

   
@@ -79,14 +86,17 @@
 
 
  
-  SQL stored procedures, with support for embedded transactions
+  SQL stored procedures that support embedded transactions. Stored
+  procedures can be created with 
+  CREATE PROCEDURE and executed with
+  CALL
  
 
 
 
  
-  JIT compilation of some SQL code, including support for fast evaluation
-  of expressions
+  Introduction of just-in-time (JIT) compilation
+  during query execution
  
 
 
@@ -99,6 +109,13 @@
  
 
 
+
+ 
+  Covering indexes, which can be utilized using the
+  INCLUDE clause of CREATE INDEX
+ 
+
+
 
  
   Many other useful performance improvements, including making
-- 
2.14.3 (Apple Git-98)

From 5d86d059faa3a85626acdddf8429a197cc787a60 Mon Sep 17 00:00:00 2001
From: "Jonathan S. Katz" 
Date: Sat, 15 Sep 2018 11:01:35 -0400
Subject: [PATCH 2/2] Remove or modify placeholders for v11 release notes.

---
 doc/src/sgml/release-11.sgml | 21 ++---
 1 file changed, 6 insertions(+), 15 deletions(-)

diff --git a/doc/src/sgml/release-11.sgml b/doc/src/sgml/release-11.sgml
index f81dd929e3..700fd02090 100644
--- a/doc/src/sgml/release-11.sgml
+++ b/doc/src/sgml/release-11.sgml
@@ -196,10 +196,6 @@
 would be dumped without such specifications if the database locale
 and encoding matched the old cluster's defaults.

-
-   
-DID I GET EVERYTHING?
-   
   
 
   
@@ -614,8 +610,7 @@
 The new command ALTER
 INDEX ATTACH PARTITION allows indexes to be
 attached to partitions.  This does not behave as a global index
-since the contents are private to each index.  WARN WHEN USING
-AN EXISTING INDEX?
+since the contents are private to each index.

   
 
@@ -924,7 +919,7 @@ same commits as above

 

-This reduces the likelihood of serialization conflicts.  ACCURATE?
+This reduces the likelihood of serialization conflicts.

   
 
@@ -1991,7 +1986,6 @@ same commits as above
 CALLs or in nested PL/pgSQL DO and
 CALL blocks that only contain other PL/pgSQL
 DO and CALL blocks.
-ACCURATE?

   
 
@@ -2414,12 +2408,8 @@ same commits as above
 The option 

Re: Postgres 11 release notes

[Bruce Momjian]

On Sat, Sep 15, 2018 at 11:06:04AM -0400, Jonathan Katz wrote:
> 
>  
> - Partitioning by a hash key
> + Partitioning by a hash key (a.k.a. "hash partitioning")

I question whether a.k.a (also known as) is familiar enough to our
readers to appear in the releaes notes.  I think the "a.k.a." can be
simply removed.

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

+ As you are, so once was I.  As I am, so you will be. +
+  Ancient Roman grave inscription +

Re: Postgres 11 release notes

[Jonathan S. Katz]

Hi,

On 9/12/18 1:34 PM, Michael Banck wrote:
> Hi,
> 
> On Fri, May 11, 2018 at 11:08:52AM -0400, Bruce Momjian wrote:
>> I have committed the first draft of the Postgres 11 release notes.  I
>> will add more markup soon.  You can view the most current version here:
>>
>>  http://momjian.us/pgsql_docs/release-11.html
>  
> The first item of section 'E.1.3.10. Server Applications' is mine and
> has this TODO (though maybe that should be better marked up?) text:
> 
> |IS IT CLEAR FROM THE DOCS THAT THE REPLICATION SLOT IS NOT TEMPORARY?
> 
> So I guess we need to decide on that before release. The current doc is
> AFAIK:
> 
> | This option causes the replication slot specified by the
> | option --slot to be created before starting the
> | backup.  In this case, an error is raised if the slot already exists.
> 
> I guess it does not hurt to have something like "to be (permanently)
> created before starting", but what do others think?  Is it clear enough?

Attached are two patches.

The first modifies the major improvements section to align with what we
have discovered during the beta period + the general advocacy push. It
also modifies the "last updated date" + narrows down a prospective
release month.

The second modifies/removes the questions/placeholders in the release
notes, such as Michael's comment above (which I agreed with and added a
one sentence explanation after).

Thanks,

Jonathan
From 2733f0df599aeda48408a3d27b796e36fbdd6fc3 Mon Sep 17 00:00:00 2001
From: "Jonathan S. Katz" 
Date: Sat, 15 Sep 2018 10:43:44 -0400
Subject: [PATCH 1/2] Updates to major improvements section of release notes.

---
 doc/src/sgml/release-11.sgml | 47 ++--
 1 file changed, 32 insertions(+), 15 deletions(-)

diff --git a/doc/src/sgml/release-11.sgml b/doc/src/sgml/release-11.sgml
index 684d34c091..9ed34e72e8 100644
--- a/doc/src/sgml/release-11.sgml
+++ b/doc/src/sgml/release-11.sgml
@@ -6,7 +6,7 @@
 
   
Release date:
-   2018-??-?? (CURRENT AS OF 2018-07-30)
+   2018-10-?? (CURRENT AS OF 2018-09-20)
   
 
   
@@ -22,11 +22,11 @@
 
 
  
-  Major improvements to partitioning:
+  Improvements to partitioning functionality, including:
   

 
- Partitioning by a hash key
+ Partitioning by a hash key (a.k.a. "hash partitioning")
 


@@ -37,9 +37,8 @@


 
- Improved SELECT query performance due to
- enhanced partition elimination during query processing and
- execution
+ Improved SELECT performance from enhanced partition
+ elimination strategies during query processing and execution
 


@@ -48,29 +47,37 @@
  KEY, indexes, and triggers on partitioned tables
 

+   
+
+ Having a "default" partition for storing data that does not match a
+ partition key
+
+   
   
  
 
 
 
  
-  Improvements to parallelism:
+  Improvements to parallelism, including:
   

 
- Parallelized hash joins
+ B-tree indexes can now be built in parallel with
+ CREATE INDEX
 


 
- Parallelized CREATE INDEX for B-tree indexes
+ Parallelized CREATE TABLE .. AS,
+ CREATE MATERIALIZED VIEW, and certain
+ queries using UNION
 


 
- Parallelized CREATE TABLE .. AS,
- CREATE MATERIALIZED VIEW, and certain
- queries using UNION
+ Performance improvements for parallelized hash joins and parallelized
+ sequential scans
 

   
@@ -79,14 +86,17 @@
 
 
  
-  SQL stored procedures, with support for embedded transactions
+  SQL stored procedures that support embedded transactions. Stored
+  procedures can be created with 
+  CREATE PROCEDURE and executed with
+  CALL
  
 
 
 
  
-  JIT compilation of some SQL code, including support for fast evaluation
-  of expressions
+  Introduction of just-in-time (JIT) compilation
+  during query execution
  
 
 
@@ -99,6 +109,13 @@
  
 
 
+
+ 
+  Covering indexes, which can be utilized using the
+  INCLUDE clause of CREATE INDEX
+ 
+
+
 
  
   Many other useful performance improvements, including making
-- 
2.14.3 (Apple Git-98)

From feb08e3747e5dc043e21b60e9e681f44a0562896 Mon Sep 17 00:00:00 2001
From: "Jonathan S. Katz" 
Date: Sat, 15 Sep 2018 11:01:35 -0400
Subject: [PATCH 2/2] Remove or modify placeholders for v11 release notes.

---
 doc/src/sgml/release-11.sgml | 21 ++---
 1 file changed, 6 insertions(+), 15 deletions(-)

diff --git a/doc/src/sgml/release-11.sgml b/doc/src/sgml/release-11.sgml
index 9ed34e72e8..e8af023140 100644
--- a/doc/src/sgml/release-11.sgml
+++ b/doc/src/sgml/release-11.sgml

Re: Postgres 11 release notes

[Michael Banck]

Hi,

On Fri, May 11, 2018 at 11:08:52AM -0400, Bruce Momjian wrote:
> I have committed the first draft of the Postgres 11 release notes.  I
> will add more markup soon.  You can view the most current version here:
> 
>   http://momjian.us/pgsql_docs/release-11.html
 
The first item of section 'E.1.3.10. Server Applications' is mine and
has this TODO (though maybe that should be better marked up?) text:

|IS IT CLEAR FROM THE DOCS THAT THE REPLICATION SLOT IS NOT TEMPORARY?

So I guess we need to decide on that before release. The current doc is
AFAIK:

| This option causes the replication slot specified by the
| option --slot to be created before starting the
| backup.  In this case, an error is raised if the slot already exists.

I guess it does not hurt to have something like "to be (permanently)
created before starting", but what do others think?  Is it clear enough?


Michael

-- 
Michael Banck
Projektleiter / Senior Berater
Tel.: +49 2166 9901-171
Fax:  +49 2166 9901-100
Email: michael.ba...@credativ.de

credativ GmbH, HRB Mönchengladbach 12080
USt-ID-Nummer: DE204566209
Trompeterallee 108, 41189 Mönchengladbach
Geschäftsführung: Dr. Michael Meskes, Jörg Folz, Sascha Heuer

Unser Umgang mit personenbezogenen Daten unterliegt
folgenden Bestimmungen: https://www.credativ.de/datenschutz

Re: Postgres 11 release notes

[Amit Kapila]

On Sun, Aug 26, 2018 at 12:17 AM Bruce Momjian  wrote:
>
> On Sat, Aug 25, 2018 at 11:42:35AM -0700, Andres Freund wrote:
> >
> >
> > On August 25, 2018 11:41:11 AM PDT, Bruce Momjian  wrote:
> > >On Sun, Aug 12, 2018 at 11:21:26AM +0200, Adrien Nayrat wrote:
> > >> Hello,
> > >>
> > >> It seems this feature is missing in releases notes ?
> > >>
> > >> commit 1f6d515a67ec98194c23a5db25660856c9aab944
> > >> Author: Robert Haas 
> > >> Date:   Mon Aug 21 14:43:01 2017 -0400
> > >>
> > >> Push limit through subqueries to underlying sort, where possible.
> > >>
> > >> Douglas Doole, reviewed by Ashutosh Bapat and by me.  Minor
> > >formatting
> > >> change by me.
> > >>
> > >> Discussion:
> > >>
> > >http://postgr.es/m/cade5jyluugneeusyw6q_4mzfytxhxavcqmgasf0yiy8zdgg...@mail.gmail.com
> > >
> > >I looked into this and it is usually a detail we don't have in the
> > >release notes.  It isn't generally interesting enough to user.
> >
> > This seems quite relevant. Both because it addresses concerns, but also can 
> > lead to a worse plan.

+1.

>
> Well, our normal logical is whether the average user would adjust their
> behavior based on this change, or whether it is user visible.
>

In general, I think this is okay because otherwise providing a lot of
information which most users won't care doesn't serve any purpose.
OTOH, I think it is not easy to decide which features to ignore.  I
think it won't be important to mention some of the code refactoring
commits or minor improvements in the code here and there, but we can't
say that about any of the performance or planner improvements.  Sure,
we might not want to mention everything, but it is better we discuss
such features before ignoring it.  I agree that it will be more work
for preparing release notes, but in the end, we might save some time
by not arguing later about which feature is important and which is
not.

-- 
With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com

Re: Postgres 11 release notes

[Peter Geoghegan]

On Sat, Aug 25, 2018 at 2:18 PM, Bruce Momjian  wrote:
>> I think that's less "our" logic and more yours, that has become
>> established because you've done most of the major release notes for a
>> long time. I'm not trying to say that that's wrong or anything, just
>
> I don't do my work in a vacuum.  My behavior is based on what feedback I
> have gotten from the community on what to include/exclude.

FWIW, I agree with what Andres said about planner changes and the release notes.

-- 
Peter Geoghegan

Re: Postgres 11 release notes

[Bruce Momjian]

On Sat, Aug 25, 2018 at 12:17:17PM -0700, Andres Freund wrote:
> Hi,
> 
> On 2018-08-25 14:47:20 -0400, Bruce Momjian wrote:
> > Well, our normal logical is whether the average user would adjust their
> > behavior based on this change, or whether it is user visible.  I thought
> > it was a contrived-enough query that this was not the case, but I would
> > be interested to hear what others think.
> 
> I think that's less "our" logic and more yours, that has become
> established because you've done most of the major release notes for a
> long time. I'm not trying to say that that's wrong or anything, just

I don't do my work in a vacuum.  My behavior is based on what feedback I
have gotten from the community on what to include/exclude.

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

+ As you are, so once was I.  As I am, so you will be. +
+  Ancient Roman grave inscription +

Re: Postgres 11 release notes

[Andres Freund]

Hi,

On 2018-08-25 14:47:20 -0400, Bruce Momjian wrote:
> Well, our normal logical is whether the average user would adjust their
> behavior based on this change, or whether it is user visible.  I thought
> it was a contrived-enough query that this was not the case, but I would
> be interested to hear what others think.

I think that's less "our" logic and more yours, that has become
established because you've done most of the major release notes for a
long time. I'm not trying to say that that's wrong or anything, just
that a lot of those principles grew organically without explicit
decisions.

I also think that the need of userbase have shifted - it used to be that
a lot more people couldn't migrate to PG for lack of features, but these
days we have a much larger userbase that has to migrate from version to
version on a regular basis.

Planner changes, in my experience, are *the* major hurdle in doing so
(leaving aside things like the 8.3 casting changes), because they have
the potential to drastically change performance. In a largely
unpredictable way.  So one of the first things you do after encountering
an issue while testing an issue during a migration, is to look through
the release notes whether that query could be affected by a documented
change.

Greetings,

Andres Freund

Re: Postgres 11 release notes

[Bruce Momjian]

On Sat, Aug 25, 2018 at 11:42:35AM -0700, Andres Freund wrote:
> 
> 
> On August 25, 2018 11:41:11 AM PDT, Bruce Momjian  wrote:
> >On Sun, Aug 12, 2018 at 11:21:26AM +0200, Adrien Nayrat wrote:
> >> Hello,
> >> 
> >> It seems this feature is missing in releases notes ?
> >> 
> >> commit 1f6d515a67ec98194c23a5db25660856c9aab944
> >> Author: Robert Haas 
> >> Date:   Mon Aug 21 14:43:01 2017 -0400
> >> 
> >> Push limit through subqueries to underlying sort, where possible.
> >> 
> >> Douglas Doole, reviewed by Ashutosh Bapat and by me.  Minor
> >formatting
> >> change by me.
> >> 
> >> Discussion:
> >>
> >http://postgr.es/m/cade5jyluugneeusyw6q_4mzfytxhxavcqmgasf0yiy8zdgg...@mail.gmail.com
> >
> >I looked into this and it is usually a detail we don't have in the
> >release notes.  It isn't generally interesting enough to user.
> 
> This seems quite relevant. Both because it addresses concerns, but also can 
> lead to a worse plan.

Well, our normal logical is whether the average user would adjust their
behavior based on this change, or whether it is user visible.  I thought
it was a contrived-enough query that this was not the case, but I would
be interested to hear what others think.

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

+ As you are, so once was I.  As I am, so you will be. +
+  Ancient Roman grave inscription +

Re: Postgres 11 release notes

[Andres Freund]

On August 25, 2018 11:41:11 AM PDT, Bruce Momjian  wrote:
>On Sun, Aug 12, 2018 at 11:21:26AM +0200, Adrien Nayrat wrote:
>> Hello,
>> 
>> It seems this feature is missing in releases notes ?
>> 
>> commit 1f6d515a67ec98194c23a5db25660856c9aab944
>> Author: Robert Haas 
>> Date:   Mon Aug 21 14:43:01 2017 -0400
>> 
>> Push limit through subqueries to underlying sort, where possible.
>> 
>> Douglas Doole, reviewed by Ashutosh Bapat and by me.  Minor
>formatting
>> change by me.
>> 
>> Discussion:
>>
>http://postgr.es/m/cade5jyluugneeusyw6q_4mzfytxhxavcqmgasf0yiy8zdgg...@mail.gmail.com
>
>I looked into this and it is usually a detail we don't have in the
>release notes.  It isn't generally interesting enough to user.

This seems quite relevant. Both because it addresses concerns, but also can 
lead to a worse plan.

Andres
-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

Re: Postgres 11 release notes

[Bruce Momjian]

On Sun, Aug 12, 2018 at 11:21:26AM +0200, Adrien Nayrat wrote:
> Hello,
> 
> It seems this feature is missing in releases notes ?
> 
> commit 1f6d515a67ec98194c23a5db25660856c9aab944
> Author: Robert Haas 
> Date:   Mon Aug 21 14:43:01 2017 -0400
> 
> Push limit through subqueries to underlying sort, where possible.
> 
> Douglas Doole, reviewed by Ashutosh Bapat and by me.  Minor formatting
> change by me.
> 
> Discussion:
> http://postgr.es/m/cade5jyluugneeusyw6q_4mzfytxhxavcqmgasf0yiy8zdgg...@mail.gmail.com

I looked into this and it is usually a detail we don't have in the
release notes.  It isn't generally interesting enough to user.

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

+ As you are, so once was I.  As I am, so you will be. +
+  Ancient Roman grave inscription +

Re: Postgres 11 release notes

[Masahiko Sawada]

On Fri, Aug 10, 2018 at 7:10 PM, Alexander Korotkov
 wrote:
> On Fri, Aug 10, 2018 at 8:08 AM Masahiko Sawada 
> wrote:
>>
>> I found that the release note says "Add pgtrgm function
>> strict_word_similarity() to compute the similarity of whole words" but
>> I think "pgtrgm" should be "pg_trgm". Attached patch fixes it.
>
>
> Right.  Pushed, thanks!
>

Thank you!

Regards,

--
Masahiko Sawada
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center

Re: Postgres 11 release notes

[Adrien Nayrat]

Hello,

It seems this feature is missing in releases notes ?

commit 1f6d515a67ec98194c23a5db25660856c9aab944
Author: Robert Haas 
Date:   Mon Aug 21 14:43:01 2017 -0400

Push limit through subqueries to underlying sort, where possible.

Douglas Doole, reviewed by Ashutosh Bapat and by me.  Minor formatting
change by me.

Discussion:
http://postgr.es/m/cade5jyluugneeusyw6q_4mzfytxhxavcqmgasf0yiy8zdgg...@mail.gmail.com

Regards,



signature.asc
Description: OpenPGP digital signature

Re: Postgres 11 release notes

[Alexander Korotkov]

On Fri, Aug 10, 2018 at 8:08 AM Masahiko Sawada 
wrote:

> I found that the release note says "Add pgtrgm function
> strict_word_similarity() to compute the similarity of whole words" but
> I think "pgtrgm" should be "pg_trgm". Attached patch fixes it.
>

Right.  Pushed, thanks!

--
Alexander Korotkov
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company

Re: Postgres 11 release notes

[Masahiko Sawada]

Hi,

I found that the release note says "Add pgtrgm function
strict_word_similarity() to compute the similarity of whole words" but
I think "pgtrgm" should be "pg_trgm". Attached patch fixes it.

Regards,

--
Masahiko Sawada
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center


fix_11_release_note.patch
Description: Binary data

Re: Postgres 11 release notes

[Tom Lane]

"Jonathan S. Katz"  writes:
> On Jun 30, 2018, at 5:52 PM, Tom Lane  wrote:
>> Mmm, yeah, I suppose it should say "all framing options" rather than
>> implying that we've implemented every other window-related frammish
>> there is in the spec.

> +1. Attached patch that does exactly that.

Pushed, thanks.

regards, tom lane

Re: Postgres 11 release notes

[Andres Freund]

On 2018-07-09 16:52:11 +0200, David Fetter wrote:
> On Mon, Jul 09, 2018 at 10:36:30AM -0400, Bruce Momjian wrote:
> > On Sun, Jul  8, 2018 at 10:28:15PM -0700, Andres Freund wrote:
> > > On 2018-07-09 14:18:14 +0900, Tatsuro Yamada wrote:
> > > > Hi Bruce,
> > > > 
> > > > > I expect a torrent of feedback.;-)
> > > > 
> > > > Could you add this fix to the release note because this change affects
> > > > an extension developer using the hook function.
> > > > It would be better to know the change by reading the release note, not 
> > > > compilation error.
> > > > 
> > > > 
> > > >
> > > > Add QueryEnvironment to ExplainOneQuery_hook's parameter list
> > > > (Tatsuro Yamada, Thomas Munro)
> > > >
> > > > 
> > > > Discussion: 
> > > > https://postgr.es/m/890e8dd9-c1c7-a422-6892-874f5eaee...@lab.ntt.co.jp
> > > > 
> > > > I guess "E.1.3.11. Source Code" or "E.1.3.12. Additional Modules" 
> > > > sections are
> > > > suitable for putting the message in the release note.
> > > 
> > > We adapt plenty of functions signatures without listing them
> > > individually in the release notes. I don't think randomly selecting one
> > > relatively uncommonly used hook is a good way to attack that.
> > 
> > Agreed.  Ideally we would have a wiki page that lists _all_ the hooks,
> > and what release they were added.
> 
> If we're talking about ideals, all our public APIs including the hooks
> should be in the official documentation and have man pages.

FWIW, at this point in time I'd pretty strenuously object to a rule
requiring all hooks / all public functions to be documented. I think the
development velocity penalty would be far greater than the benefit.

That's however *NOT* to say, that we shouldn't document individual API
that we expect to be somewhat stable / frequently externally used (say
the PL interface, C trigger interface).

Greetings,

Andres Freund

Re: Postgres 11 release notes

[Tatsuro Yamada]

On 2018/07/09 23:52, David Fetter wrote:

On Mon, Jul 09, 2018 at 10:36:30AM -0400, Bruce Momjian wrote:

On Sun, Jul  8, 2018 at 10:28:15PM -0700, Andres Freund wrote:

On 2018-07-09 14:18:14 +0900, Tatsuro Yamada wrote:

Hi Bruce,


I expect a torrent of feedback.;-)


Could you add this fix to the release note because this change affects
an extension developer using the hook function.
It would be better to know the change by reading the release note, not 
compilation error.



 Add QueryEnvironment to ExplainOneQuery_hook's parameter list
 (Tatsuro Yamada, Thomas Munro)


Discussion: 
https://postgr.es/m/890e8dd9-c1c7-a422-6892-874f5eaee...@lab.ntt.co.jp

I guess "E.1.3.11. Source Code" or "E.1.3.12. Additional Modules" sections are
suitable for putting the message in the release note.


We adapt plenty of functions signatures without listing them
individually in the release notes. I don't think randomly selecting one
relatively uncommonly used hook is a good way to attack that.


Agreed.  Ideally we would have a wiki page that lists _all_ the hooks,
and what release they were added.


Did you mean the page is pdf file?
If so, it is not updated from 2012 because it's a presentation material.
I couldn't find other lists by using "hooks" in search box on wiki.

  https://wiki.postgresql.org/images/e/e3/Hooks_in_postgresql.pdf



If we're talking about ideals, all our public APIs including the hooks
should be in the official documentation and have man pages.


I think so too.
It is useful information for newbe developer and extension developer.

Thanks,
Tatsuro Yamada

Re: Postgres 11 release notes

[David Fetter]

On Mon, Jul 09, 2018 at 10:36:30AM -0400, Bruce Momjian wrote:
> On Sun, Jul  8, 2018 at 10:28:15PM -0700, Andres Freund wrote:
> > On 2018-07-09 14:18:14 +0900, Tatsuro Yamada wrote:
> > > Hi Bruce,
> > > 
> > > > I expect a torrent of feedback.;-)
> > > 
> > > Could you add this fix to the release note because this change affects
> > > an extension developer using the hook function.
> > > It would be better to know the change by reading the release note, not 
> > > compilation error.
> > > 
> > > 
> > >
> > > Add QueryEnvironment to ExplainOneQuery_hook's parameter list
> > > (Tatsuro Yamada, Thomas Munro)
> > >
> > > 
> > > Discussion: 
> > > https://postgr.es/m/890e8dd9-c1c7-a422-6892-874f5eaee...@lab.ntt.co.jp
> > > 
> > > I guess "E.1.3.11. Source Code" or "E.1.3.12. Additional Modules" 
> > > sections are
> > > suitable for putting the message in the release note.
> > 
> > We adapt plenty of functions signatures without listing them
> > individually in the release notes. I don't think randomly selecting one
> > relatively uncommonly used hook is a good way to attack that.
> 
> Agreed.  Ideally we would have a wiki page that lists _all_ the hooks,
> and what release they were added.

If we're talking about ideals, all our public APIs including the hooks
should be in the official documentation and have man pages.

Best,
David.
-- 
David Fetter  http://fetter.org/
Phone: +1 415 235 3778

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate

Re: Postgres 11 release notes

[Bruce Momjian]

On Sun, Jul  8, 2018 at 10:28:15PM -0700, Andres Freund wrote:
> On 2018-07-09 14:18:14 +0900, Tatsuro Yamada wrote:
> > Hi Bruce,
> > 
> > > I expect a torrent of feedback.;-)
> > 
> > Could you add this fix to the release note because this change affects
> > an extension developer using the hook function.
> > It would be better to know the change by reading the release note, not 
> > compilation error.
> > 
> > 
> >
> > Add QueryEnvironment to ExplainOneQuery_hook's parameter list
> > (Tatsuro Yamada, Thomas Munro)
> >
> > 
> > Discussion: 
> > https://postgr.es/m/890e8dd9-c1c7-a422-6892-874f5eaee...@lab.ntt.co.jp
> > 
> > I guess "E.1.3.11. Source Code" or "E.1.3.12. Additional Modules" sections 
> > are
> > suitable for putting the message in the release note.
> 
> We adapt plenty of functions signatures without listing them
> individually in the release notes. I don't think randomly selecting one
> relatively uncommonly used hook is a good way to attack that.

Agreed.  Ideally we would have a wiki page that lists _all_ the hooks,
and what release they were added.

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

+ As you are, so once was I.  As I am, so you will be. +
+  Ancient Roman grave inscription +

Re: Postgres 11 release notes

[Andres Freund]

On 2018-07-09 14:18:14 +0900, Tatsuro Yamada wrote:
> Hi Bruce,
> 
> > I expect a torrent of feedback.;-)
> 
> Could you add this fix to the release note because this change affects
> an extension developer using the hook function.
> It would be better to know the change by reading the release note, not 
> compilation error.
> 
> 
>
> Add QueryEnvironment to ExplainOneQuery_hook's parameter list
> (Tatsuro Yamada, Thomas Munro)
>
> 
> Discussion: 
> https://postgr.es/m/890e8dd9-c1c7-a422-6892-874f5eaee...@lab.ntt.co.jp
> 
> I guess "E.1.3.11. Source Code" or "E.1.3.12. Additional Modules" sections are
> suitable for putting the message in the release note.

We adapt plenty of functions signatures without listing them
individually in the release notes. I don't think randomly selecting one
relatively uncommonly used hook is a good way to attack that.

Greetings,

Andres Freund

Re: Postgres 11 release notes

[Tatsuro Yamada]

Hi Bruce,

> I expect a torrent of feedback.;-)

Could you add this fix to the release note because this change affects
an extension developer using the hook function.
It would be better to know the change by reading the release note, not 
compilation error.


   
Add QueryEnvironment to ExplainOneQuery_hook's parameter list
(Tatsuro Yamada, Thomas Munro)
   

Discussion: 
https://postgr.es/m/890e8dd9-c1c7-a422-6892-874f5eaee...@lab.ntt.co.jp

I guess "E.1.3.11. Source Code" or "E.1.3.12. Additional Modules" sections are
suitable for putting the message in the release note.

Thanks,
Tatsuro Yamada
NTT Open Source Software Center

Re: Postgres 11 release notes

[Jonathan S. Katz]

> On Jun 30, 2018, at 5:52 PM, Tom Lane  wrote:
> 
> Yaroslav  writes:
>> I've noticed the following in v11 release notes:
>> "Window functions now support all options shown in the SQL:2011 standard,
>> including RANGE distance PRECEDING/FOLLOWING, GROUPS mode, and frame
>> exclusion options"
> 
>> But, as far as I see, IGNORE NULLS option for lead, lag, etc. is still not
>> supported. May be, the item could be reformulated?
> 
> Mmm, yeah, I suppose it should say "all framing options" rather than
> implying that we've implemented every other window-related frammish
> there is in the spec.

+1. Attached patch that does exactly that.

Jonathan



0001-window-docs.patch
Description: Binary data

Re: Postgres 11 release notes

[Tom Lane]

Yaroslav  writes:
> I've noticed the following in v11 release notes:
> "Window functions now support all options shown in the SQL:2011 standard,
> including RANGE distance PRECEDING/FOLLOWING, GROUPS mode, and frame
> exclusion options"

> But, as far as I see, IGNORE NULLS option for lead, lag, etc. is still not
> supported. May be, the item could be reformulated?

Mmm, yeah, I suppose it should say "all framing options" rather than
implying that we've implemented every other window-related frammish
there is in the spec.

regards, tom lane

Re: Postgres 11 release notes

[Yaroslav]

Hello.

I've noticed the following in v11 release notes:
"Window functions now support all options shown in the SQL:2011 standard,
including RANGE distance PRECEDING/FOLLOWING, GROUPS mode, and frame
exclusion options"

But, as far as I see, IGNORE NULLS option for lead, lag, etc. is still not
supported. May be, the item could be reformulated?

--
WBR, Yaroslav Schekin




-
WBR, Yaroslav Schekin.
--
Sent from: http://www.postgresql-archive.org/PostgreSQL-hackers-f1928748.html

Re: Postgres 11 release notes

[Jonathan S. Katz]

> On Jun 30, 2018, at 2:48 PM, Daniel Gustafsson  wrote:
> 
>> On 30 Jun 2018, at 20:11, Brad DeJong  wrote:
>> 
>> I suggest the following edits for the PostgreSQL beta 2 release announcement.
>> I did not provide the edits as a patch because I can't find the source of 
>> /about/news/1867.
> 
> The news item is stored in the postgresql.org database, and not in the code, 
> so
> there is no file to patch.  CC:ing -www where website changes are discussed.

The "CHANGES SINCE BETA 1” was applied earlier thanks to a separate report.

I applied wording fixes for the 3 bulleted items listed. I did not see any 
reason to
change the opening line as well as intro paragraph to the “CHANGES SINCE BETA 1”
section.

Thanks for reporting!

Jonathan

Re: Postgres 11 release notes

[Daniel Gustafsson]

> On 30 Jun 2018, at 20:11, Brad DeJong  wrote:
> 
> I suggest the following edits for the PostgreSQL beta 2 release announcement.
> I did not provide the edits as a patch because I can't find the source of 
> /about/news/1867.

The news item is stored in the postgresql.org database, and not in the code, so
there is no file to patch.  CC:ing -www where website changes are discussed.

cheers ./daniel

Re: Postgres 11 release notes

[Brad DeJong]

I suggest the following edits for the PostgreSQL beta 2 release
announcement.
I did not provide the edits as a patch because I can't find the source of
/about/news/1867.

- This release contains ... as well as bug fixes that were reported ...
+ This release contains ... as well as fixes for bugs that were reported ...

- we do not advise for you to run
+ we do not advise you to run
or
+ we do not advise running

- CHANGES SINCE BETA 2
+ CHANGES SINCE BETA 1

- There have been many bug fixes for PostgreSQL 11 reported during
- the Beta 1 period and applied to the Beta 2 release. Several bug fixes
- reported for version 10 or earlier that also affected version 11 are
- included in the Beta 2 release.
+ Fixes for many of the bugs reported during the PostgreSQL 11 Beta 1
+ period have been applied to the Beta 2 release. Fixes for several bugs
+ reported for version 10 or earlier that affected version 11 have also
+ been included in the Beta 2 release.
or, if you prefer active voice
+ PostgreSQL 11 Beta 2 contains fixes for many of the bugs that were
+ reported during the Beta 1 period and for several bugs reported
+ for version 10 or earlier that affected version 11.

- ... eliminating a needless additional partition constraint checks ...
+ ... eliminating needless additional partition constraint checks ...

- including returning NULL if slot is not advanced
+ including returning NULL if the slot is not advanced

- Fix returning accurate results with ...
This phrasing makes it sound like you didn't want the accurate results so
you fixed them. :)
Not being familiar with the fix in question, I do not have a suggestion as
to how it should be rephrased.

Re: Postgres 11 release notes

[Alexander Korotkov]

On Tue, Jun 19, 2018 at 1:40 PM Alexander Korotkov
 wrote:
> On Tue, Jun 19, 2018 at 12:15 PM Alexander Korotkov
>  wrote:
> > On Sat, Jun 16, 2018 at 3:57 PM Darafei "Komяpa" Praliaskouski
> >  wrote:
> > >>
> > >> > I'm not sure it is usefull in release notes since it is more about 
> > >> > API, and not
> > >> > user-facing change. Just in case.
> > >> > GiST opclasses now can omit compress and decompress functions. If 
> > >> > compress
> > >> > function is omited, IndexOnlyScan is enabled for opclass without any 
> > >> > extra
> > >> > change.
> > >> > https://github.com/postgres/postgres/commit/
> > >> > d3a4f89d8a3e500bd7c0b7a8a8a5ce1b47859128
> > >>
> > >> Uh, we do have this for SP-GiST:
> > >>
> > >> Allow SP-GiST indexes to optionally use compression (Teodor 
> > >> Sigaev,
> > >> Heikki Linnakangas, Alexander Korotkov, Nikita Glukhov)
> > >>
> > >> I am unclear how far downt the API stack I should go in documenting
> > >> changes like this.
> > >
> > >
> > > It is also a bit misleading - the idea in that change is that now index 
> > > representation can be a lossy version of actual data type (a box instead 
> > > of polygon as a referende, so a changelog entry can tell "Allow SP-GiST 
> > > index creation for polygon datatype.").  There is no "decompression" for 
> > > such thing. "compression" sounds like gzip for me in user-facing context.
> >
> > +1 that current wording looks confusing.  But I think we need to
> > highlight that we have general SP-GiST improvement, not just support
> > for particular datatype.  So, I propose following wording: "Allow
> > SP-GiST to use lossy representation of leaf keys, and add SP-GiST
> > support for polygon type using that".
>
> Oh, I missed that we have separate release notes entry for polygons
> indexing.  Then second part of sentence isn't needed, it should be
> just "Allow SP-GiST to use lossy representation of leaf keys".  If no
> objections, I'm going to commit that altogether with fixes by
> Liudmila.

Wording improvement is pushed.  Typo fixes are already pushed by Magnus.

--
Alexander Korotkov
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company

Re: Postgres 11 release notes

[Alexander Korotkov]

On Tue, Jun 19, 2018 at 6:18 PM Daniel Gustafsson  wrote:
> > On 19 Jun 2018, at 12:40, Alexander Korotkov  
> > wrote:
> >
> > On Tue, Jun 19, 2018 at 12:15 PM Alexander Korotkov
> >  wrote:
> >> On Sat, Jun 16, 2018 at 3:57 PM Darafei "Komяpa" Praliaskouski
> >>  wrote:
> 
> > I'm not sure it is usefull in release notes since it is more about API, 
> > and not
> > user-facing change. Just in case.
> > GiST opclasses now can omit compress and decompress functions. If 
> > compress
> > function is omited, IndexOnlyScan is enabled for opclass without any 
> > extra
> > change.
> > https://github.com/postgres/postgres/commit/
> > d3a4f89d8a3e500bd7c0b7a8a8a5ce1b47859128
> 
>  Uh, we do have this for SP-GiST:
> 
> Allow SP-GiST indexes to optionally use compression (Teodor 
>  Sigaev,
> Heikki Linnakangas, Alexander Korotkov, Nikita Glukhov)
> 
>  I am unclear how far downt the API stack I should go in documenting
>  changes like this.
> >>>
> >>>
> >>> It is also a bit misleading - the idea in that change is that now index 
> >>> representation can be a lossy version of actual data type (a box instead 
> >>> of polygon as a referende, so a changelog entry can tell "Allow SP-GiST 
> >>> index creation for polygon datatype.").  There is no "decompression" for 
> >>> such thing. "compression" sounds like gzip for me in user-facing context.
> >>
> >> +1 that current wording looks confusing.  But I think we need to
> >> highlight that we have general SP-GiST improvement, not just support
> >> for particular datatype.  So, I propose following wording: "Allow
> >> SP-GiST to use lossy representation of leaf keys, and add SP-GiST
> >> support for polygon type using that".
> >
> > Oh, I missed that we have separate release notes entry for polygons
> > indexing.  Then second part of sentence isn't needed, it should be
> > just "Allow SP-GiST to use lossy representation of leaf keys".  If no
> > objections, I'm going to commit that altogether with fixes by
> > Liudmila.
>
> Speaking of typos, I think I spotted two more: s/features is/feature is/.
> Fixed in the attached diff.

Committed, thanks!

--
Alexander Korotkov
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company

Re: Postgres 11 release notes

[Daniel Gustafsson]

> On 19 Jun 2018, at 12:40, Alexander Korotkov  
> wrote:
> 
> On Tue, Jun 19, 2018 at 12:15 PM Alexander Korotkov
>  wrote:
>> On Sat, Jun 16, 2018 at 3:57 PM Darafei "Komяpa" Praliaskouski
>>  wrote:
 
> I'm not sure it is usefull in release notes since it is more about API, 
> and not
> user-facing change. Just in case.
> GiST opclasses now can omit compress and decompress functions. If compress
> function is omited, IndexOnlyScan is enabled for opclass without any extra
> change.
> https://github.com/postgres/postgres/commit/
> d3a4f89d8a3e500bd7c0b7a8a8a5ce1b47859128
 
 Uh, we do have this for SP-GiST:
 
Allow SP-GiST indexes to optionally use compression (Teodor Sigaev,
Heikki Linnakangas, Alexander Korotkov, Nikita Glukhov)
 
 I am unclear how far downt the API stack I should go in documenting
 changes like this.
>>> 
>>> 
>>> It is also a bit misleading - the idea in that change is that now index 
>>> representation can be a lossy version of actual data type (a box instead of 
>>> polygon as a referende, so a changelog entry can tell "Allow SP-GiST index 
>>> creation for polygon datatype.").  There is no "decompression" for such 
>>> thing. "compression" sounds like gzip for me in user-facing context.
>> 
>> +1 that current wording looks confusing.  But I think we need to
>> highlight that we have general SP-GiST improvement, not just support
>> for particular datatype.  So, I propose following wording: "Allow
>> SP-GiST to use lossy representation of leaf keys, and add SP-GiST
>> support for polygon type using that".
> 
> Oh, I missed that we have separate release notes entry for polygons
> indexing.  Then second part of sentence isn't needed, it should be
> just "Allow SP-GiST to use lossy representation of leaf keys".  If no
> objections, I'm going to commit that altogether with fixes by
> Liudmila.

Speaking of typos, I think I spotted two more: s/features is/feature is/.
Fixed in the attached diff.

cheers ./daniel



relnotes11typo.diff
Description: Binary data

Re: Postgres 11 release notes

[Alexander Korotkov]

On Tue, Jun 19, 2018 at 12:15 PM Alexander Korotkov
 wrote:
> On Sat, Jun 16, 2018 at 3:57 PM Darafei "Komяpa" Praliaskouski
>  wrote:
> >>
> >> > I'm not sure it is usefull in release notes since it is more about API, 
> >> > and not
> >> > user-facing change. Just in case.
> >> > GiST opclasses now can omit compress and decompress functions. If 
> >> > compress
> >> > function is omited, IndexOnlyScan is enabled for opclass without any 
> >> > extra
> >> > change.
> >> > https://github.com/postgres/postgres/commit/
> >> > d3a4f89d8a3e500bd7c0b7a8a8a5ce1b47859128
> >>
> >> Uh, we do have this for SP-GiST:
> >>
> >> Allow SP-GiST indexes to optionally use compression (Teodor Sigaev,
> >> Heikki Linnakangas, Alexander Korotkov, Nikita Glukhov)
> >>
> >> I am unclear how far downt the API stack I should go in documenting
> >> changes like this.
> >
> >
> > It is also a bit misleading - the idea in that change is that now index 
> > representation can be a lossy version of actual data type (a box instead of 
> > polygon as a referende, so a changelog entry can tell "Allow SP-GiST index 
> > creation for polygon datatype.").  There is no "decompression" for such 
> > thing. "compression" sounds like gzip for me in user-facing context.
>
> +1 that current wording looks confusing.  But I think we need to
> highlight that we have general SP-GiST improvement, not just support
> for particular datatype.  So, I propose following wording: "Allow
> SP-GiST to use lossy representation of leaf keys, and add SP-GiST
> support for polygon type using that".

Oh, I missed that we have separate release notes entry for polygons
indexing.  Then second part of sentence isn't needed, it should be
just "Allow SP-GiST to use lossy representation of leaf keys".  If no
objections, I'm going to commit that altogether with fixes by
Liudmila.

--
Alexander Korotkov
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company


spgist-compress-release-notes.patch
Description: Binary data

Re: Postgres 11 release notes

[Justin Pryzby]

On Sat, Jun 09, 2018 at 01:35:19PM -0700, David G. Johnston wrote:
> On Fri, May 11, 2018 at 8:08 AM, Bruce Momjian  wrote:
> 
> > I have committed the first draft of the Postgres 11 release notes.  I
> > will add more markup soon.  You can view the most current version here:
> >
> > http://momjian.us/pgsql_docs/release-11.html
> 
> """
> Also, if any table mentioned in VACUUM uses a column list, then ANALYZE
> keyword must be supplied; previously ANALYZE was implied in such cases.
> """
> 
> Should that be mentioned in the compatibility notes?

+1

Note also from 921059bd66c7fb1230c705d3b1a65940800c4cbb 


This is okay in v10 but rejected in v11b1:  

  
postgres=# VACUUM ANALYZE VERBOSE t;

  
ERROR:  syntax error at or near "VERBOSE"   

  

Justin

Re: Postgres 11 release notes

[Jonathan S. Katz]

> On Jun 9, 2018, at 4:35 PM, David G. Johnston  
> wrote:
> 
> On Fri, May 11, 2018 at 8:08 AM, Bruce Momjian  > wrote:
> I have committed the first draft of the Postgres 11 release notes.  I
> will add more markup soon.  You can view the most current version here:
> 
> http://momjian.us/pgsql_docs/release-11.html 
> 
> 
> ​Some thoughts:​
> 
> ​As a major ​item:
> 
> JIT compilation of some SQL code, including support for fast evaluation of 
> expressions
> 
> 
> leaves me wondering what the general benefit is to the user.  Also, spelling 
> out "Just-In-Time (JIT)" here seems warranted.

+1 for spelling it out.

Looking through past release notes[1][2], we’ve typically left the context for
the features press releases[3].

Now, as the beta press release (feature-explanation oriented) typically differs 
from
the major version press release (use-case/context oriented), perhaps we could
incorporate some of the elements from the beta press release into the
“Major Features” section. It would be a departure from the release notes of 
recent
memory, but perhaps it would be more helpful.

Thoughts?

Jonathan

[1] https://www.postgresql.org/docs/11/static/release-10.html 

[2] https://www.postgresql.org/docs/11/static/release-9-6.html 

[3] https://www.postgresql.org/about/news/1855/ 

Re: Postgres 11 release notes

[David G. Johnston]

On Fri, May 11, 2018 at 8:08 AM, Bruce Momjian  wrote:

> I have committed the first draft of the Postgres 11 release notes.  I
> will add more markup soon.  You can view the most current version here:
>
> http://momjian.us/pgsql_docs/release-11.html


​Some thoughts:​

​As a major ​item:

JIT compilation of some SQL code, including support for fast evaluation of
expressions


leaves me wondering what the general benefit is to the user.  Also,
spelling out "Just-In-Time (JIT)" here seems warranted.


"""
No longer retain WAL that spans two checkpoints (Simon Riggs)

The retention of WAL records for only one checkpoint is required.
"""

Maybe: "Servers now only ensure that a single checkpoint record is present
in the locally retained WAL.  Previously it would ensure at least two
checkpoints were available for recovery."


"""
Also, if any table mentioned in VACUUM uses a column list, then ANALYZE
keyword must be supplied; previously ANALYZE was implied in such cases.
"""

Should that be mentioned in the compatibility notes?


"""
Add the ability to define PL/pgSQL record types as not null, constant, or
with initial values (Tom Lane)
"""

Found the commit message for this one - should we back-patch a
documentation change indicating that this doesn't work prior to v11?

David J.

Re: Postgres 11 release notes

[Bruce Momjian]

On Tue, May 22, 2018 at 11:31:08PM -0400, Tom Lane wrote:
> Bruce Momjian  writes:
> > On Fri, May 18, 2018 at 10:28:12AM -0400, Tom Lane wrote:
> >> While we're all griping about omissions from the release notes ...
> >> I think you should have documented that we fixed plpgsql to cope
> >> (or cope better, at least) with intrasession changes in the rowtypes
> >> of composite-type variables.
> 
> > It was fixed in this commit, right?
> > commit 4b93f57999a2ca9b9c9e573ea32ab1aeaa8bf496
> 
> That was the main one, anyway.

OK, added.  Applied patch attached.

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

+ As you are, so once was I.  As I am, so you will be. +
+  Ancient Roman grave inscription +
diff --git a/doc/src/sgml/release-11.sgml b/doc/src/sgml/release-11.sgml
new file mode 100644
index 05fd23f..7599c6f
*** a/doc/src/sgml/release-11.sgml
--- b/doc/src/sgml/release-11.sgml
*** same commits as above
*** 1982,1987 
--- 1982,2004 
  

  
+ 
+
+ Allow PL/pgSQL to handle changes to composite types (e.g. record,
+ row) that happen between the first and later function executions
+ in the same session (Tom Lane)
+
+ 
+
+ Previously such circumstances generated errors.
+
+   
+ 
+   
+ 
  

Re: Postgres 11 release notes

[Tom Lane]

Bruce Momjian  writes:
> On Fri, May 18, 2018 at 10:28:12AM -0400, Tom Lane wrote:
>> While we're all griping about omissions from the release notes ...
>> I think you should have documented that we fixed plpgsql to cope
>> (or cope better, at least) with intrasession changes in the rowtypes
>> of composite-type variables.

> It was fixed in this commit, right?
>   commit 4b93f57999a2ca9b9c9e573ea32ab1aeaa8bf496

That was the main one, anyway.

regards, tom lane

Re: Postgres 11 release notes

[Bruce Momjian]

On Wed, May 23, 2018 at 02:06:15PM +1200, David Rowley wrote:
> On 23 May 2018 at 13:18, Bruce Momjian  wrote:
> > On Mon, May 21, 2018 at 07:34:18PM +1200, David Rowley wrote:
> >> I've been working a bit in this area over the past few weeks and with
> >> PG11 I measured a single INSERT into a 10k RANGE partitioned table at
> >> just 84 tps (!), while inserting the same row into a non-partitioned
> >> table was about 11.1k tps. I have patches locally that take this up to
> >> ~9.8k tps, which I'll submit for PG12. I'm unsure if we should be
> >
> > Yikes!  I think the question is whether we need to _remove_ the item I
> > just posted that is already in the release notes:
> >
> > Allow faster partition elimination during query processing (Amit
> > Langote, David Rowley, Dilip Kumar)
> >
> > This speeds access to partitioned tables with many partitions.
> 
> Well, partition elimination/pruning and tuple routing are not the same
> thing. Pruning saves us scanning partitions that can't contain
> matching tuples, whereas routing finds a home for a specific tuple.

I just suspected they would use the same algorithm.

> Amit's work to improve partition elimination certainly is much faster
> than constraint exclusion. It's not the last thing we'll ever do to
> speed up the planning of queries for partitioned tables but it is a
> very good start, and without it, run-time pruning would not be
> possible.
> 
> I'd say the release notes in this regard don't claim anything that's
> untrue. They look fine to me. Thanks for working on them!

OK.

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

+ As you are, so once was I.  As I am, so you will be. +
+  Ancient Roman grave inscription +

Re: Postgres 11 release notes

[Bruce Momjian]

On Wed, May 23, 2018 at 11:55:29AM +1000, Haribabu Kommi wrote:
> Yes, security labels also gets dumped only with --create option.

OK.

> Can you update the author name as (Haribabu Kommi) to make it consistent
> with other entries.

Done.  Applied patch attached.

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

+ As you are, so once was I.  As I am, so you will be. +
+  Ancient Roman grave inscription +
diff --git a/doc/src/sgml/release-11.sgml b/doc/src/sgml/release-11.sgml
new file mode 100644
index c02bf0e..05fd23f
*** a/doc/src/sgml/release-11.sgml
--- b/doc/src/sgml/release-11.sgml
***
*** 169,175 
 
  pg_dump and
  pg_restore, without
! --clean, no longer dump/restore database comments
  and security labels.
 
  
--- 169,175 
 
  pg_dump and
  pg_restore, without
! --create, no longer dump/restore database comments
  and security labels.
 
  
***
*** 287,293 
 
  Have libpq's PQhost()
! always return the actual connected host (Hari Babu)
 
  
 
--- 287,293 
 
  Have libpq's PQhost()
! always return the actual connected host (Haribabu Kommi)
 
  
 

Re: Postgres 11 release notes

[David Rowley]

On 23 May 2018 at 13:18, Bruce Momjian  wrote:
> On Mon, May 21, 2018 at 07:34:18PM +1200, David Rowley wrote:
>> I've been working a bit in this area over the past few weeks and with
>> PG11 I measured a single INSERT into a 10k RANGE partitioned table at
>> just 84 tps (!), while inserting the same row into a non-partitioned
>> table was about 11.1k tps. I have patches locally that take this up to
>> ~9.8k tps, which I'll submit for PG12. I'm unsure if we should be
>
> Yikes!  I think the question is whether we need to _remove_ the item I
> just posted that is already in the release notes:
>
> Allow faster partition elimination during query processing (Amit
> Langote, David Rowley, Dilip Kumar)
>
> This speeds access to partitioned tables with many partitions.

Well, partition elimination/pruning and tuple routing are not the same
thing. Pruning saves us scanning partitions that can't contain
matching tuples, whereas routing finds a home for a specific tuple.

Amit's work to improve partition elimination certainly is much faster
than constraint exclusion. It's not the last thing we'll ever do to
speed up the planning of queries for partitioned tables but it is a
very good start, and without it, run-time pruning would not be
possible.

I'd say the release notes in this regard don't claim anything that's
untrue. They look fine to me. Thanks for working on them!

-- 
 David Rowley   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training & Services

Re: Postgres 11 release notes

[Haribabu Kommi]

On Wed, May 23, 2018 at 11:34 AM, Bruce Momjian  wrote:

> On Mon, May 21, 2018 at 06:28:36PM +1000, Haribabu Kommi wrote:
> > On Sat, May 12, 2018 at 1:08 AM, Bruce Momjian  wrote:
> >
> > I have committed the first draft of the Postgres 11 release notes.  I
> > will add more markup soon.  You can view the most current version
> here:
> >
> > http://momjian.us/pgsql_docs/release-11.html
> >
> >
> > Thanks for preparing the release notes.
> >
> > >Have pg_dump dump all aspects of a database (Haribabu Kommi)
> > >
> > >pg_dump and pg_restore, without --clean, no longer dump/restore database
> > > comments and security labels.
> >
> > There is small change in option name, the option to print database
> comments is
> > --create not --clean.
>
> Uh, what about security labels?
>

Yes, security labels also gets dumped only with --create option.


> > >Have libpq's PQhost() always return the actual connected host (Hari
> Babu)
> > >
> > >Previously PQhost() often returned the supplied host parameters, which
> could
> > contain
> > >several hosts. The same is true of PQport(), which now returns the
> actual port
> > number,
> > >not the multiple supplied port numbers. ACCURATE?
> >
> > Now PQhost() can return hostaddr also if host is not available. How about
> > changing as below?
> >
> > Have libpq's PQhost() always return the actual connected host/hostaddr
> > (Haribabu Kommi)
> >
> > hostaddr is returned, when there is no host available with the connected
> host.
>
> OK, here is the new text:
>
> Have libpq's  linkend="libpq-pqhost">PQhost()
> always return the actual connected host (Hari Babu)
>

Can you update the author name as (Haribabu Kommi) to make it consistent
with other entries.


> Previously PQhost() often returned the
> supplied host parameters, which could contain several hosts.
> ->  It will now also return the host's IP address if the host name was
> ->  not supplied.  The same is true of PQport(),
> which now returns the actual port number, not the multiple supplied
> port numbers.


That looks good to me.

Regards,
Haribabu Kommi
Fujitsu Australia

Re: Postgres 11 release notes

[Amit Langote]

On 2018/05/23 10:36, Bruce Momjian wrote:
> On Wed, May 23, 2018 at 10:28:41AM +0900, Amit Langote wrote:
>>> Uh, we already have this in the release notes:
>>>
>>> Allow faster partition elimination during query processing (Amit
>>> Langote, David Rowley, Dilip Kumar)
>>>
>>> This speeds access to partitioned tables with many partitions.
>>>
>>> Do you want me to add the git commit hash to this release note entry?
>>
>> I suppose you meant the above as an entry for performance improvement of
>> partition "pruning".  The commit I quoted is concerned with making "tuple
>> routing" a bit faster, but as David said that's not making it as fast as
>> it could really be.  So, we should hold off from touting it as an
>> improvement at this point and I have to agree.  Sorry for the noise.
> 
> OK, no problem.  So _finding_ the rows is faster, but adding rows to
> partitioned tables with many partitions is still slow, got it.

Yeah.

To be honest, even _finding_ is not yet performing the best it could be,
which I guess David would chime in to say.  We know what needs to be fixed
to get the close-to-ideal performance there, but didn't have time to do it
for PG 11.  To clarify, what changed is that we replaced constraint
exclusion, which has to consider the partition constraint of *all*
partitions individually, as an algorithm for partition pruning by a faster
alternative that only looks at the parent table's partition descriptor.
That gives a good boost but that's not the end of it.  Moreover, addition
of this new pruning algorithm enabled the development of execution time
pruning which is a completely new feature.

Thanks,
Amit

Re: Postgres 11 release notes

[Bruce Momjian]

On Wed, May 23, 2018 at 01:37:22PM +1200, Thomas Munro wrote:
> On Sat, May 19, 2018 at 2:28 AM, Tom Lane  wrote:
> > While we're all griping about omissions from the release notes ...
> 
> Hi Bruce,
> 
>
> Add support for ARMv8 hardware
> CRC calculations (Yuqi Gu, Heikki Linnakangas)
>
> 
> Could I please ask for a third author credit for this one?  See
> commits 1c72ec6 (and follow-up a7a7387) which extended ARM CRC support
> to non-Linux systems.

Done.

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

+ As you are, so once was I.  As I am, so you will be. +
+  Ancient Roman grave inscription +

Re: Postgres 11 release notes

[Thomas Munro]

On Sat, May 19, 2018 at 2:28 AM, Tom Lane  wrote:
> While we're all griping about omissions from the release notes ...

Hi Bruce,

   
Add support for ARMv8 hardware
CRC calculations (Yuqi Gu, Heikki Linnakangas)
   

Could I please ask for a third author credit for this one?  See
commits 1c72ec6 (and follow-up a7a7387) which extended ARM CRC support
to non-Linux systems.

-- 
Thomas Munro
http://www.enterprisedb.com

Re: [Sender Address Forgery]Re: Postgres 11 release notes

[Bruce Momjian]

On Wed, May 23, 2018 at 10:28:41AM +0900, Amit Langote wrote:
> > Uh, we already have this in the release notes:
> > 
> > Allow faster partition elimination during query processing (Amit
> > Langote, David Rowley, Dilip Kumar)
> > 
> > This speeds access to partitioned tables with many partitions.
> > 
> > Do you want me to add the git commit hash to this release note entry?
> 
> I suppose you meant the above as an entry for performance improvement of
> partition "pruning".  The commit I quoted is concerned with making "tuple
> routing" a bit faster, but as David said that's not making it as fast as
> it could really be.  So, we should hold off from touting it as an
> improvement at this point and I have to agree.  Sorry for the noise.

OK, no problem.  So _finding_ the rows is faster, but adding rows to
partitioned tables with many partitions is still slow, got it.

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

+ As you are, so once was I.  As I am, so you will be. +
+  Ancient Roman grave inscription +

Re: Postgres 11 release notes

[Bruce Momjian]

On Mon, May 21, 2018 at 06:28:36PM +1000, Haribabu Kommi wrote:
> On Sat, May 12, 2018 at 1:08 AM, Bruce Momjian  wrote:
> 
> I have committed the first draft of the Postgres 11 release notes.  I
> will add more markup soon.  You can view the most current version here:
> 
>         http://momjian.us/pgsql_docs/release-11.html
> 
> 
> Thanks for preparing the release notes.
> 
> >Have pg_dump dump all aspects of a database (Haribabu Kommi)
> >
> >pg_dump and pg_restore, without --clean, no longer dump/restore database
> > comments and security labels.
> 
> There is small change in option name, the option to print database comments is
> --create not --clean.

Uh, what about security labels?

> >Have libpq's PQhost() always return the actual connected host (Hari Babu)
> >
> >Previously PQhost() often returned the supplied host parameters, which could
> contain
> >several hosts. The same is true of PQport(), which now returns the actual 
> >port
> number,
> >not the multiple supplied port numbers. ACCURATE?
> 
> Now PQhost() can return hostaddr also if host is not available. How about
> changing as below?
> 
> Have libpq's PQhost() always return the actual connected host/hostaddr
> (Haribabu Kommi)
> 
> hostaddr is returned, when there is no host available with the connected host.

OK, here is the new text:

Have libpq's PQhost()
always return the actual connected host (Hari Babu)

Previously PQhost() often returned the
supplied host parameters, which could contain several hosts.
->  It will now also return the host's IP address if the host name was
->  not supplied.  The same is true of PQport(),
which now returns the actual port number, not the multiple supplied
port numbers.

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

+ As you are, so once was I.  As I am, so you will be. +
+  Ancient Roman grave inscription +

Re: [Sender Address Forgery]Re: Postgres 11 release notes

[Amit Langote]

On 2018/05/23 10:16, Bruce Momjian wrote:
> On Sat, May 19, 2018 at 12:58:04AM +0900, Amit Langote wrote:
>> Hi Bruce.
>>
>> On Tue, May 15, 2018 at 12:46 PM, Amit Langote
>>  wrote:
>>> On 2018/05/15 5:30, Bruce Momjian wrote:
 I like it, done.
>>>
>>> Thank you.
>>
>> I wonder what you think about including this little performance item:
>>
>> https://www.postgresql.org/message-id/e1eotsq-0005v0...@gemulon.postgresql.org
>>
>> especially considering the part of the commit message which states
>>
>> ...Still, testing shows
>> that this makes single-row inserts significantly faster on a table
>> with many partitions without harming the bulk-insert case.
>>
>> I recall seeing those inserts being as much as 2x faster as partition
>> count grows beyond hundreds.  One might argue that we should think
>> about publicizing this only after we've dealt with the
>> lock-all-partitions issue that's also mentioned in the commit message
>> which is still a significant portion of the time spent and I'm totally
>> fine with that.
> 
> Uh, we already have this in the release notes:
> 
> Allow faster partition elimination during query processing (Amit
> Langote, David Rowley, Dilip Kumar)
> 
> This speeds access to partitioned tables with many partitions.
> 
> Do you want me to add the git commit hash to this release note entry?

I suppose you meant the above as an entry for performance improvement of
partition "pruning".  The commit I quoted is concerned with making "tuple
routing" a bit faster, but as David said that's not making it as fast as
it could really be.  So, we should hold off from touting it as an
improvement at this point and I have to agree.  Sorry for the noise.

Thanks,
Amit

Re: Postgres 11 release notes

[Bruce Momjian]

On Mon, May 21, 2018 at 07:34:18PM +1200, David Rowley wrote:
> On 19 May 2018 at 03:58, Amit Langote  wrote:
> > I wonder what you think about including this little performance item:
> >
> > https://www.postgresql.org/message-id/e1eotsq-0005v0...@gemulon.postgresql.org
> >
> > especially considering the part of the commit message which states
> >
> > ...Still, testing shows
> > that this makes single-row inserts significantly faster on a table
> > with many partitions without harming the bulk-insert case.
> >
> > I recall seeing those inserts being as much as 2x faster as partition
> > count grows beyond hundreds.  One might argue that we should think
> > about publicizing this only after we've dealt with the
> > lock-all-partitions issue that's also mentioned in the commit message
> > which is still a significant portion of the time spent and I'm totally
> > fine with that.
> 
> While I do think that was a good change, I do think there's much still
> left to do to speed up usage of partitioned tables with many
> partitions.
> 
> I've been working a bit in this area over the past few weeks and with
> PG11 I measured a single INSERT into a 10k RANGE partitioned table at
> just 84 tps (!), while inserting the same row into a non-partitioned
> table was about 11.1k tps. I have patches locally that take this up to
> ~9.8k tps, which I'll submit for PG12. I'm unsure if we should be

Yikes!  I think the question is whether we need to _remove_ the item I
just posted that is already in the release notes:

Allow faster partition elimination during query processing (Amit
Langote, David Rowley, Dilip Kumar)

This speeds access to partitioned tables with many partitions.

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

+ As you are, so once was I.  As I am, so you will be. +
+  Ancient Roman grave inscription +

Re: Postgres 11 release notes

[Bruce Momjian]

On Sat, May 19, 2018 at 12:58:04AM +0900, Amit Langote wrote:
> Hi Bruce.
> 
> On Tue, May 15, 2018 at 12:46 PM, Amit Langote
>  wrote:
> > On 2018/05/15 5:30, Bruce Momjian wrote:
> >> I like it, done.
> >
> > Thank you.
> 
> I wonder what you think about including this little performance item:
> 
> https://www.postgresql.org/message-id/e1eotsq-0005v0...@gemulon.postgresql.org
> 
> especially considering the part of the commit message which states
> 
> ...Still, testing shows
> that this makes single-row inserts significantly faster on a table
> with many partitions without harming the bulk-insert case.
> 
> I recall seeing those inserts being as much as 2x faster as partition
> count grows beyond hundreds.  One might argue that we should think
> about publicizing this only after we've dealt with the
> lock-all-partitions issue that's also mentioned in the commit message
> which is still a significant portion of the time spent and I'm totally
> fine with that.

Uh, we already have this in the release notes:

Allow faster partition elimination during query processing (Amit
Langote, David Rowley, Dilip Kumar)

This speeds access to partitioned tables with many partitions.

Do you want me to add the git commit hash to this release note entry?

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

+ As you are, so once was I.  As I am, so you will be. +
+  Ancient Roman grave inscription +

Re: Postgres 11 release notes

[Bruce Momjian]

On Fri, May 18, 2018 at 10:28:12AM -0400, Tom Lane wrote:
> While we're all griping about omissions from the release notes ...
> I think you should have documented that we fixed plpgsql to cope
> (or cope better, at least) with intrasession changes in the rowtypes
> of composite-type variables.  See bug #15203 for the latest in a long
> line of user complaints about that --- so it's not like this is
> something users don't care about.

It was fixed in this commit, right?

commit 4b93f57999a2ca9b9c9e573ea32ab1aeaa8bf496
Author: Tom Lane 
Date:   Tue Feb 13 18:52:21 2018 -0500

Make plpgsql use its DTYPE_REC code paths for composite-type 
variables.

Sure I can add it, once you confirm.

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

+ As you are, so once was I.  As I am, so you will be. +
+  Ancient Roman grave inscription +

RE: Postgres 11 release notes

[Ideriha, Takeshi]

>-Original Message-
>From: Bruce Momjian [mailto:br...@momjian.us]
>I have committed the first draft of the Postgres 11 release notes.  I will add 
>more
>markup soon.  You can view the most current version here:

Hi, there is a small typo.

I think "These function" should be "These functions".
(At the next sentece "these functions" is used.) 

Regards,
Ideriha, Takeshi


release_note_typo.patch
Description: release_note_typo.patch

Re: Postgres 11 release notes

[Jonathan S. Katz]

> On May 21, 2018, at 12:38 PM, Tom Lane  wrote:
> 
> "Jonathan S. Katz"  writes:
>> Per feedback from many asynchronous threads, attached is the proposed
>> patch for the list of major features. I also expect a torrent of feedback. I
>> will have a corresponding press release for Beta 1 in the very near future.
>> This being my first doc patch proposal, I tried to follow the formatting I
>> saw in the existing SGML. Please let me know if I can make improvements.
> 
> Pushed with some very minor editorialization.

Thanks! I do agree with your editorial, the main goal now is to give exposure 
to features for testing.

Jonathan

Re: Postgres 11 release notes

[Tom Lane]

"Jonathan S. Katz"  writes:
> Per feedback from many asynchronous threads, attached is the proposed
> patch for the list of major features. I also expect a torrent of feedback. I
> will have a corresponding press release for Beta 1 in the very near future.
> This being my first doc patch proposal, I tried to follow the formatting I
> saw in the existing SGML. Please let me know if I can make improvements.

Pushed with some very minor editorialization.

regards, tom lane

Re: Postgres 11 release notes

[Amit Langote]

On Mon, May 21, 2018 at 4:34 PM, David Rowley
 wrote:
> On 19 May 2018 at 03:58, Amit Langote  wrote:
>> I wonder what you think about including this little performance item:
>>
>> https://www.postgresql.org/message-id/e1eotsq-0005v0...@gemulon.postgresql.org
>>
>> especially considering the part of the commit message which states
>>
>> ...Still, testing shows
>> that this makes single-row inserts significantly faster on a table
>> with many partitions without harming the bulk-insert case.
>>
>> I recall seeing those inserts being as much as 2x faster as partition
>> count grows beyond hundreds.  One might argue that we should think
>> about publicizing this only after we've dealt with the
>> lock-all-partitions issue that's also mentioned in the commit message
>> which is still a significant portion of the time spent and I'm totally
>> fine with that.
>
> While I do think that was a good change, I do think there's much still
> left to do to speed up usage of partitioned tables with many
> partitions.
>
> I've been working a bit in this area over the past few weeks and with
> PG11 I measured a single INSERT into a 10k RANGE partitioned table at
> just 84 tps (!), while inserting the same row into a non-partitioned
> table was about 11.1k tps. I have patches locally that take this up to
> ~9.8k tps, which I'll submit for PG12. I'm unsure if we should be
> shouting anything from the rooftops about the work done in this area
> for PG11, since it's still got a long way to go still before the
> feature is usable with higher numbers of partitions. I do think your
> change was a good one to make, but I just don't want users to think
> that we're done here when we all know that much work remains.
>
> If we're going to add an item in the release notes about this then I
> wouldn't object, providing it could be done in a way that indicates
> we've not finished here yet, but if that's the case then maybe it's
> better to say nothing at all.

You're right, it surely isn't time yet to make fanfare about this.  I
will look forward to being able to review your patches. :-)

Thanks,
Amit

Re: Postgres 11 release notes

[Haribabu Kommi]

On Sat, May 12, 2018 at 1:08 AM, Bruce Momjian  wrote:

> I have committed the first draft of the Postgres 11 release notes.  I
> will add more markup soon.  You can view the most current version here:
>
> http://momjian.us/pgsql_docs/release-11.html
>

Thanks for preparing the release notes.

>Have pg_dump dump all aspects of a database (Haribabu Kommi)
>
>pg_dump and pg_restore, without --clean, no longer dump/restore database
> comments and security labels.

There is small change in option name, the option to print database comments
is --create not --clean.


>Have libpq's PQhost() always return the actual connected host (Hari Babu)
>
>Previously PQhost() often returned the supplied host parameters, which
could contain
>several hosts. The same is true of PQport(), which now returns the actual
port number,
>not the multiple supplied port numbers. ACCURATE?

Now PQhost() can return hostaddr also if host is not available. How about
changing as below?

Have libpq's PQhost() always return the actual connected host/hostaddr
(Haribabu Kommi)

hostaddr is returned, when there is no host available with the connected
host.


Regards,
Haribabu Kommi
Fujitsu Australia

Re: Postgres 11 release notes

[David Rowley]

On 19 May 2018 at 03:58, Amit Langote  wrote:
> I wonder what you think about including this little performance item:
>
> https://www.postgresql.org/message-id/e1eotsq-0005v0...@gemulon.postgresql.org
>
> especially considering the part of the commit message which states
>
> ...Still, testing shows
> that this makes single-row inserts significantly faster on a table
> with many partitions without harming the bulk-insert case.
>
> I recall seeing those inserts being as much as 2x faster as partition
> count grows beyond hundreds.  One might argue that we should think
> about publicizing this only after we've dealt with the
> lock-all-partitions issue that's also mentioned in the commit message
> which is still a significant portion of the time spent and I'm totally
> fine with that.

While I do think that was a good change, I do think there's much still
left to do to speed up usage of partitioned tables with many
partitions.

I've been working a bit in this area over the past few weeks and with
PG11 I measured a single INSERT into a 10k RANGE partitioned table at
just 84 tps (!), while inserting the same row into a non-partitioned
table was about 11.1k tps. I have patches locally that take this up to
~9.8k tps, which I'll submit for PG12. I'm unsure if we should be
shouting anything from the rooftops about the work done in this area
for PG11, since it's still got a long way to go still before the
feature is usable with higher numbers of partitions. I do think your
change was a good one to make, but I just don't want users to think
that we're done here when we all know that much work remains.

If we're going to add an item in the release notes about this then I
wouldn't object, providing it could be done in a way that indicates
we've not finished here yet, but if that's the case then maybe it's
better to say nothing at all.

-- 
 David Rowley   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training & Services

Re: Postgres 11 release notes

[Amit Langote]

Hi Bruce.

On Tue, May 15, 2018 at 12:46 PM, Amit Langote
 wrote:
> On 2018/05/15 5:30, Bruce Momjian wrote:
>> I like it, done.
>
> Thank you.

I wonder what you think about including this little performance item:

https://www.postgresql.org/message-id/e1eotsq-0005v0...@gemulon.postgresql.org

especially considering the part of the commit message which states

...Still, testing shows
that this makes single-row inserts significantly faster on a table
with many partitions without harming the bulk-insert case.

I recall seeing those inserts being as much as 2x faster as partition
count grows beyond hundreds.  One might argue that we should think
about publicizing this only after we've dealt with the
lock-all-partitions issue that's also mentioned in the commit message
which is still a significant portion of the time spent and I'm totally
fine with that.

Thanks,
Amit

Re: Postgres 11 release notes

[Jonathan S. Katz]

> On May 11, 2018, at 11:08 AM, Bruce Momjian  wrote:
> 
> I have committed the first draft of the Postgres 11 release notes.  I
> will add more markup soon.  You can view the most current version here:
> 
>   http://momjian.us/pgsql_docs/release-11.html
> 
> I expect a torrent of feedback.  ;-)

Per feedback from many asynchronous threads, attached is the proposed
patch for the list of major features. I also expect a torrent of feedback. I
will have a corresponding press release for Beta 1 in the very near future.

This being my first doc patch proposal, I tried to follow the formatting I
saw in the existing SGML. Please let me know if I can make improvements.

Thanks,

Jonathan




pg11features.diff
Description: Binary data

Re: Postgres 11 release notes

[Tom Lane]

While we're all griping about omissions from the release notes ...
I think you should have documented that we fixed plpgsql to cope
(or cope better, at least) with intrasession changes in the rowtypes
of composite-type variables.  See bug #15203 for the latest in a long
line of user complaints about that --- so it's not like this is
something users don't care about.

regards, tom lane

Re: Postgres 11 release notes

[Bruce Momjian]

On Fri, May 18, 2018 at 10:56:25AM +0500, Andrey Borodin wrote:
> H, Bruce!
> 
> 
> 11 мая 2018 г., в 20:08, Bruce Momjian  написал(а):
> 
> I expect a torrent of feedback.  ;-)
> 
> 
> I'm not sure it is usefull in release notes since it is more about API, and 
> not
> user-facing change. Just in case.
> GiST opclasses now can omit compress and decompress functions. If compress
> function is omited, IndexOnlyScan is enabled for opclass without any extra
> change.
> https://github.com/postgres/postgres/commit/
> d3a4f89d8a3e500bd7c0b7a8a8a5ce1b47859128

Uh, we do have this for SP-GiST:

Allow SP-GiST indexes to optionally use compression (Teodor Sigaev,
Heikki Linnakangas, Alexander Korotkov, Nikita Glukhov)

I am unclear how far downt the API stack I should go in documenting
changes like this.

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

+ As you are, so once was I.  As I am, so you will be. +
+  Ancient Roman grave inscription +

Re: Postgres 11 release notes

[Amit Kapila]

On Tue, May 15, 2018 at 2:11 AM, Bruce Momjian  wrote:
> On Mon, May 14, 2018 at 05:34:54PM +0530, Dilip Kumar wrote:
>>
>> On Fri, May 11, 2018 at 8:38 PM, Bruce Momjian  wrote:
>>
>> I have committed the first draft of the Postgres 11 release notes.  I
>> will add more markup soon.  You can view the most current version here:
>>
>> http://momjian.us/pgsql_docs/release-11.html
>>
>> I expect a torrent of feedback.  ;-)
>>
>> On a personal note, I want to apologize for not adequately championing
>> two features that I think have strategic significance but didn't make it
>> into Postgres 11:  parallel FDW access and improved multi-variate
>> statistics.  I hope to do better job during Postgres 12.
>>
>> --
>>   Bruce Momjian  http://momjian.us
>>   EnterpriseDB http://enterprisedb.com
>>
>> + As you are, so once was I.  As I am, so you will be. +
>> +  Ancient Roman grave inscription +
>>
>>
>>
>> I think the below commit is missed in the release notes?
>>
>> 5edc63bda68a77c4d38f0cbeae1c4271f9ef4100
>>
>> Committer: Robert Haas   2017-11-10 13:50:50
>>
>> Account for the effect of lossy pages when costing bitmap scans.
>>
>> Dilip Kumar, reviewed by Alexander Kumenkov, Amul Sul, and me.
>> Some final adjustments by me.
>>
>> As part of this commit, we are also accounting for the lossy pages during
>> bitmap costing.  This will consider
>> the effect of work_mem while selecting the bitmap heap scan path.
>
> Uh, that seems very exotic to mention, sorry.  Others have opinions?
>

This patch improves the plan selection in many cases, see the
discussion of this patch[1][2].  The change in plan leads to
significant performance improvements in a number of TPC-H queries at
various settings.  I think it is worth considering to add this in the
release notes.

[1] - 
https://www.postgresql.org/message-id/CAFiTN-uL%3DrQtvt9zFnLV9khXODhEyJTvC4TB135HSK1%3DYdFAxQ%40mail.gmail.com
[2] - 
https://www.postgresql.org/message-id/9daaf288-e67f-f349-965f-3a8c6e0819ae%40postgrespro.ru


-- 
With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com

Re: Postgres 11 release notes

[Andrey Borodin]

H, Bruce!

> 11 мая 2018 г., в 20:08, Bruce Momjian  написал(а):
> 
> I expect a torrent of feedback.  ;-)


I'm not sure it is usefull in release notes since it is more about API, and not 
user-facing change. Just in case.
GiST opclasses now can omit compress and decompress functions. If compress 
function is omited, IndexOnlyScan is enabled for opclass without any extra 
change.
https://github.com/postgres/postgres/commit/d3a4f89d8a3e500bd7c0b7a8a8a5ce1b47859128
 


Best regards, Andrey Borodin.

Re: Postgres 11 release notes

[Michael Paquier]

On Thu, May 17, 2018 at 02:23:00PM -0400, Bruce Momjian wrote:
> On Thu, May 17, 2018 at 10:35:53PM +0900, Michael Paquier wrote:
> > Hi Bruce,
> > 
> > Here is some bonus feedback.
> > 
> > On Fri, May 11, 2018 at 11:08:52AM -0400, Bruce Momjian wrote:
> > > I expect a torrent of feedback.  ;-)
> > 
> > I have just noticed that this entry does not have the correct author
> > (guess who?):
> 
> Fixed.  (I think I guessed right.)

Thanks, Bruce.  It looks that your guess is right.
--
Michael


signature.asc
Description: PGP signature

Re: Postgres 11 release notes

[Bruce Momjian]

On Thu, May 17, 2018 at 10:35:53PM +0900, Michael Paquier wrote:
> Hi Bruce,
> 
> Here is some bonus feedback.
> 
> On Fri, May 11, 2018 at 11:08:52AM -0400, Bruce Momjian wrote:
> > I expect a torrent of feedback.  ;-)
> 
> I have just noticed that this entry does not have the correct author
> (guess who?):

Fixed.  (I think I guessed right.)

>
> Add libpq option to support channel binding when using  linkend="auth-password">SCRAM
> authentication (Peter Eisentraut)
>
> 
> I think that there should be two different entries in the release notes
> for channel binding:
> 1) The new connection parameter in libpq which allows to control the
> channel binding name, as well as to decide if it should be disabled.
> I would think that this is better placed within the section for client
> interface changes.

Well, I tend to put items in the first section that applies, and in this
case, you are right that the API is libpq but the feature is
authentication.  We know we are going to need to adjust this feature,
so let's see where it ends up and let's revisit it.

> 2) Channel binding itself, which should be part of the authentication
> section.
> 
>
> Have libpq's  linkend="libpq-pqhost">PQhost()
> always return the actual connected host (Hari Babu)
>
> Should this be added as well in the section "Client interfaces"?

Well, again, using the rules above, the PQhost item goes into the first
section where it fits, and incompatibility is the first such section. 
There are other items in incompatibility that could be moved to lower,
but again, we want the incompatibilities to all be in the same place.

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

+ As you are, so once was I.  As I am, so you will be. +
+  Ancient Roman grave inscription +

Re: NaNs in numeric_power (was Re: Postgres 11 release notes)

[Tom Lane]

Huong Dangminh  writes:
> Thank you. The patch looks fine to me.
> Also, I have done the "make check" in Windows and Linux environment with no 
> problem.

Pushed, thanks for reviewing/testing.

regards, tom lane

Re: Postgres 11 release notes

[Bruce Momjian]

On Thu, May 17, 2018 at 09:48:54PM +0900, Michael Paquier wrote:
> On Wed, May 16, 2018 at 09:09:22PM -0400, Bruce Momjian wrote:
> > On Thu, May 17, 2018 at 09:56:49AM +0900, Michael Paquier wrote:
> >> On Wed, May 16, 2018 at 08:20:49PM -0400, Bruce Momjian wrote:
> >>> SCRAM-with-binding is the first password method that attempts to avoid
> >>> man-in-the-middle attacks, and therefore is much less likely to be able
> >>> to trust what the endpoints supports.  I think it is really the
> >>> channel_binding_mode that we want to control at the client.  The lesser
> >>> modes are much more reasonable to use an automatic best-supported
> >>> negotiation, which is what we do now.
> >> 
> >> Noted.  Which means that the parameter is ignored when using a non-SSL
> >> connection, as well as when the server tries to enforce the use of
> >> anything else than SCRAM.
> > 
> > Uh, a man-in-the-middle could prevent SSL or ask for a different
> > password authentication method and then channel binding would not be
> > used.  I think when you say you want channel binding, you have to fail
> > if you don't get it.
> 
> I am not exactly sure what is the result we are looking for here, so I
> am adding for now an open item which refers to this part of the thread.
> Please note that I am fine to spend cycles if needed to address any
> issues and/or concerns.  Let's the discussion continue for now.

Agreed, and I just posted a more detailed email about when
authentication downgrades are possible.

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

+ As you are, so once was I.  As I am, so you will be. +
+  Ancient Roman grave inscription +

Re: Postgres 11 release notes

[Bruce Momjian]

On Wed, May 16, 2018 at 09:09:22PM -0400, Bruce Momjian wrote:
> > > FYI, I think the server could also require channel binding for SCRAM. We
> > > already have scram-sha-256 in pg_hba.conf, and I think
> > > scram-sha-256-plus would be reasonable.
> > 
> > Noted as well.  There is of course the question of v10 libpq versions
> > which don't support channel binding, but if an admin is willing to set
> > up scram-sha-256-plus in pg_hba.conf then he can request his users to
> > update his drivers/libs as well.
> 
> Yes, I don't see a way around it.  Once you accept that someone in the
> middle can change what you request undetected, then you can't do 
> fallback.  Imagine a man-in-the-middle with TLS where the
> man-in-the-middle allows the two end-points to negotiate the shared
> secret, but the man-in-the-middle forces a weak cipher.  This is what is
> happening with Postgres when the man-in-the-middle forces a weaker
> authentication method.

Technically, you can do automatic fallback if the fallback has
man-in-the-middle and downgrade protection.  Technically, because TLS is
already active when we start authentication negotiation, we don't need
downgrade protection as long as we have man-in-the-middle protection.

Unfortunately, only SCRAM with channel binding and sslmode=verify-full
have man-in-the-middle protection.  Therefore, downgrading from SCRAM
with channel binding to SCRAM without channel binding, MD5, or
'password' is only safe if sslmode=verify-full is enabled.  Technically
MD5, or 'password' has weak or non-existent replay protection, but if we
are requiring TLS, then that doesn't really matter, assuming we have
man-in-the-middle protection.

I don't know if falling back from SCRAM with channel binding to a lesser
authentication methods only if sslmode=verify-full is enabled is really
helpful to anyone since it requires certificate installation.

TLS has similar downgrade issues:


http://www.educatedguesswork.org/2012/07/problems_with_secure_upgrade_t.html

but many of its downgrade options have downgrade protection, and you
don't lose man-in-the-middle protection by downgrading. 
Man-in-the-middle protection via certificate checking happens
independent of the TLS version being used, which is not the case for
Postgres authentication downgrade options.

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

+ As you are, so once was I.  As I am, so you will be. +
+  Ancient Roman grave inscription +

Re: Postgres 11 release notes

[Magnus Hagander]

On Thu, May 17, 2018 at 2:56 AM, Michael Paquier 
wrote:

> On Wed, May 16, 2018 at 08:20:49PM -0400, Bruce Momjian wrote:
> > SCRAM-with-binding is the first password method that attempts to avoid
> > man-in-the-middle attacks, and therefore is much less likely to be able
> > to trust what the endpoints supports.  I think it is really the
> > channel_binding_mode that we want to control at the client.  The lesser
> > modes are much more reasonable to use an automatic best-supported
> > negotiation, which is what we do now.
>
> Noted.  Which means that the parameter is ignored when using a non-SSL
> connection, as well as when the server tries to enforce the use of
> anything else than SCRAM.
>

(apologies if this was covered earlier, as I'm entering late into the
discussion)

"ignored" in combination with a security parameter is generally a very very
red flag.

If the client requests channel binding and ends up using a non encrypted
connection, surely the correct thing to do is fail the connection, rather
than downgrade the authentication?

We should really make sure we don't re-implement something as silly as our
current "sslmode=prefer", because it makes no sense. From the client side
perspective, there really only needs to be two choices -- "enforce channel
binding at level " or "meh, I don't care". In the "meh, I don't care"
mode, go with whatever the server picks (through enforcement in pg_hba.conf
for example).


> FYI, I think the server could also require channel binding for SCRAM. We
> > already have scram-sha-256 in pg_hba.conf, and I think
> > scram-sha-256-plus would be reasonable.
>
> Noted as well.  There is of course the question of v10 libpq versions
> which don't support channel binding, but if an admin is willing to set
> up scram-sha-256-plus in pg_hba.conf then he can request his users to
> update his drivers/libs as well.
>

Yes. And they *should* fail if they don't upgrade. That's what requirement
means... :)


What's the take of others?  Magnus, Stephen or Heikki perhaps (you've
> been the most involved with SCRAM early talks)?
>

Saw it by luck. It would probably be better if it wasn't hidden deep in a
thread about release notes.



-- 
 Magnus Hagander
 Me: https://www.hagander.net/ 
 Work: https://www.redpill-linpro.com/ 

Re: Postgres 11 release notes

[Michael Paquier]

Hi Bruce,

Here is some bonus feedback.

On Fri, May 11, 2018 at 11:08:52AM -0400, Bruce Momjian wrote:
> I expect a torrent of feedback.  ;-)

I have just noticed that this entry does not have the correct author
(guess who?):
   
Add libpq option to support channel binding when using SCRAM
authentication (Peter Eisentraut)
   

I think that there should be two different entries in the release notes
for channel binding:
1) The new connection parameter in libpq which allows to control the
channel binding name, as well as to decide if it should be disabled.
I would think that this is better placed within the section for client
interface changes.
2) Channel binding itself, which should be part of the authentication
section.

   
Have libpq's PQhost()
always return the actual connected host (Hari Babu)
   
Should this be added as well in the section "Client interfaces"?
--
Michael


signature.asc
Description: PGP signature

Re: Postgres 11 release notes

[Michael Paquier]

On Wed, May 16, 2018 at 09:09:22PM -0400, Bruce Momjian wrote:
> On Thu, May 17, 2018 at 09:56:49AM +0900, Michael Paquier wrote:
>> On Wed, May 16, 2018 at 08:20:49PM -0400, Bruce Momjian wrote:
>>> SCRAM-with-binding is the first password method that attempts to avoid
>>> man-in-the-middle attacks, and therefore is much less likely to be able
>>> to trust what the endpoints supports.  I think it is really the
>>> channel_binding_mode that we want to control at the client.  The lesser
>>> modes are much more reasonable to use an automatic best-supported
>>> negotiation, which is what we do now.
>> 
>> Noted.  Which means that the parameter is ignored when using a non-SSL
>> connection, as well as when the server tries to enforce the use of
>> anything else than SCRAM.
> 
> Uh, a man-in-the-middle could prevent SSL or ask for a different
> password authentication method and then channel binding would not be
> used.  I think when you say you want channel binding, you have to fail
> if you don't get it.

I am not exactly sure what is the result we are looking for here, so I
am adding for now an open item which refers to this part of the thread.
Please note that I am fine to spend cycles if needed to address any
issues and/or concerns.  Let's the discussion continue for now.
--
Michael


signature.asc
Description: PGP signature

RE: NaNs in numeric_power (was Re: Postgres 11 release notes)

[Huong Dangminh]

Hi, 

> From: Tom Lane [mailto:t...@sss.pgh.pa.us]
> David Rowley  writes:
> > On 16 May 2018 at 02:01, Tom Lane  wrote:
> >> I'm not particularly fussed about getting credit for that.  However,
> >> looking again at how that patch series turned out --- ie, that we
> >> ensured POSIX behavior for NaNs only in HEAD --- I wonder whether we
> >> shouldn't do what was mentioned in the commit log for 6bdf1303, and
> >> teach numeric_pow() about these same special cases.
> >> It seems like it would be more consistent to change both functions
> >> for v11, rather than letting that other shoe drop in some future
> >> major release.
> 
> > I'm inclined to agree. It's hard to imagine these two functions
> > behaving differently in regards to NaN input is useful to anyone.
> 

Thank you. The patch looks fine to me.
Also, I have done the "make check" in Windows and Linux environment with no 
problem.


Thanks and best regards,
---
Dang Minh Huong
NEC Solution Innovators, Ltd.
http://www.nec-solutioninnovators.co.jp/en/

Re: Postgres 11 release notes

[Amit Kapila]

On Thu, May 17, 2018 at 5:54 AM, Bruce Momjian  wrote:
>
> Oh, sorry, changed to:
>
> Allow single-evaluation queries, e.g. WHERE
> clause aggregate queries, and functions in the target list to be
> parallelized (Amit Kapila, Robert Haas)
>

LGTM.  Thanks.


-- 
With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com

Re: Postgres 11 release notes

[Bruce Momjian]

On Thu, May 17, 2018 at 09:56:49AM +0900, Michael Paquier wrote:
> On Wed, May 16, 2018 at 08:20:49PM -0400, Bruce Momjian wrote:
> > SCRAM-with-binding is the first password method that attempts to avoid
> > man-in-the-middle attacks, and therefore is much less likely to be able
> > to trust what the endpoints supports.  I think it is really the
> > channel_binding_mode that we want to control at the client.  The lesser
> > modes are much more reasonable to use an automatic best-supported
> > negotiation, which is what we do now.
> 
> Noted.  Which means that the parameter is ignored when using a non-SSL
> connection, as well as when the server tries to enforce the use of
> anything else than SCRAM.

Uh, a man-in-the-middle could prevent SSL or ask for a different
password authentication method and then channel binding would not be
used.  I think when you say you want channel binding, you have to fail
if you don't get it.

> > FYI, I think the server could also require channel binding for SCRAM. We
> > already have scram-sha-256 in pg_hba.conf, and I think
> > scram-sha-256-plus would be reasonable.
> 
> Noted as well.  There is of course the question of v10 libpq versions
> which don't support channel binding, but if an admin is willing to set
> up scram-sha-256-plus in pg_hba.conf then he can request his users to
> update his drivers/libs as well.

Yes, I don't see a way around it.  Once you accept that someone in the
middle can change what you request undetected, then you can't do 
fallback.  Imagine a man-in-the-middle with TLS where the
man-in-the-middle allows the two end-points to negotiate the shared
secret, but the man-in-the-middle forces a weak cipher.  This is what is
happening with Postgres when the man-in-the-middle forces a weaker
authentication method.

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

+ As you are, so once was I.  As I am, so you will be. +
+  Ancient Roman grave inscription +

Re: Postgres 11 release notes

[Michael Paquier]

On Wed, May 16, 2018 at 08:20:49PM -0400, Bruce Momjian wrote:
> SCRAM-with-binding is the first password method that attempts to avoid
> man-in-the-middle attacks, and therefore is much less likely to be able
> to trust what the endpoints supports.  I think it is really the
> channel_binding_mode that we want to control at the client.  The lesser
> modes are much more reasonable to use an automatic best-supported
> negotiation, which is what we do now.

Noted.  Which means that the parameter is ignored when using a non-SSL
connection, as well as when the server tries to enforce the use of
anything else than SCRAM.

> FYI, I think the server could also require channel binding for SCRAM. We
> already have scram-sha-256 in pg_hba.conf, and I think
> scram-sha-256-plus would be reasonable.

Noted as well.  There is of course the question of v10 libpq versions
which don't support channel binding, but if an admin is willing to set
up scram-sha-256-plus in pg_hba.conf then he can request his users to
update his drivers/libs as well.

What's the take of others?  Magnus, Stephen or Heikki perhaps (you've
been the most involved with SCRAM early talks)?
--
Michael


signature.asc
Description: PGP signature

Re: Postgres 11 release notes

[Bruce Momjian]

On Wed, May 16, 2018 at 10:08:18AM -0400, Alvaro Herrera wrote:
> On 2018-May-15, Bruce Momjian wrote:
> 
> > On Wed, May 16, 2018 at 09:01:35AM +0900, Tatsuo Ishii wrote:
> > > There's a small typo.
> > > 
> > > > Add support for with huge(large) pages on Windows (Takayuki Tsunakawa, 
> > > > Thomas Munro) 
> > > 
> > > I think a space between "huge" and "(large)" is needed.
> > 
> > Done, URL updated.
> 
> I'm not sure why we say "huge (large) pages".  The natural term for
> Windows is "large-pages",
> https://msdn.microsoft.com/en-us/library/windows/desktop/aa366720(v=vs.85).aspx
> so I think we should use that terminology.  Maybe something like
> 
> Add support for large pages on Windows (Takayuki 
> Tsunakawa, Thomas Munro)
> This is controlled by the huge_pages configuration parameter.

OK, done.

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

+ As you are, so once was I.  As I am, so you will be. +
+  Ancient Roman grave inscription +

Re: Postgres 11 release notes

[Bruce Momjian]

On Wed, May 16, 2018 at 01:50:04PM +0300, Heikki Linnakangas wrote:
> >* Allow bytes to be specified for server variable sizes (Beena Emerson)
> >
> >The specification is "B".
> 
> I had to dig the commit in the git history to figure out what this is. I'd
> suggest rewording this:
> 
> * Allow server options related to memory and file sizes, to be specified as
> number of bytes.
> 
> The new unit is "B". This is in addition to "kB", "MB", "GB" and "TB", which
> were accepted previously.

OK, good, done.

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

+ As you are, so once was I.  As I am, so you will be. +
+  Ancient Roman grave inscription +

Re: Postgres 11 release notes

[Bruce Momjian]

On Wed, May 16, 2018 at 09:55:05AM +0530, Amit Kapila wrote:
> On Wed, May 16, 2018 at 12:17 AM, Bruce Momjian  wrote:
> > On Tue, May 15, 2018 at 08:45:07AM +0530, Amit Kapila wrote:
> >> No, it is not like that.  We divide the scan among workers and each
> >> worker should perform projection of the rows it scanned (after
> >> applying filter).  Now, if the expensive functions are part of target
> >> lists, then we can push the computation of expensive functions (as
> >> part of target list) in workers which will divide the work.
> >>
> >> >  Really?  Do
> >> > we run each column in its own worker or do we split the result set into
> >> > parts and run those in parallel?  How do we know, just the function call
> >> > costs?
> >> >
> >>
> >> The function's cost can be determined via pg_proc->procost.  For this
> >> particular case, you can refer the call graph -
> >> create_pathtarget->set_pathtarget_cost_width->cost_qual_eval_node->cost_qual_eval_walker->get_func_cost
> >>
> >> >  I can admit I never saw that coming.
> >> >
> >>
> >> I think the use case becomes interesting with parallel query because
> >> now you can divide such cost among workers.
> >>
> >> Feel free to ask more questions if above doesn't clarify the usage of
> >> these features.
> >
> > OK, I have added the following release note item for both of these:
> >
> > 2017-11-16 [e89a71fb4] Pass InitPlan values to workers via Gather (Merge).
> > 2018-03-29 [3f90ec859] Postpone generate_gather_paths for topmost scan/join 
> > rel
> > 2018-03-29 [11cf92f6e] Rewrite the code that applies scan/join targets to 
> > paths
> >
> > Allow single-evaluation queries, e.g. FROM
> > clause queries, and functions in the target list to be
> > parallelized (Amit Kapila, Robert Haas)
> >
> 
> Sorry, but it is not clear to me what you intend to say by "e.g.
> FROM clause queries"?   What I could imagine is
> something like "e.g. queries in WHERE clause that
> return aggregate value"

Oh, sorry, changed to:

Allow single-evaluation queries, e.g. WHERE
clause aggregate queries, and functions in the target list to be
parallelized (Amit Kapila, Robert Haas)

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

+ As you are, so once was I.  As I am, so you will be. +
+  Ancient Roman grave inscription +

Re: Postgres 11 release notes

[Bruce Momjian]

On Wed, May 16, 2018 at 08:59:23PM +0900, Michael Paquier wrote:
> On Wed, May 16, 2018 at 01:09:07PM +0300, Heikki Linnakangas wrote:
> > I have to agree with Bruce, that it's pretty useless to implement channel
> > binding, if there is no way to require it in libpq. IMHO that must be
> > fixed.
> 
> Wouldn't we want to also do something for the case where a client is
> willing to use SCRAM but that the server forces back MD5?  In which
> case, one possibility is a connection parameter like the following,
> named say authmethod:
> - An empty value is equivalent to the current behavior, and is the
> default.
> - 'scram' means that client is willing to use SCRAM, which would cause a
> failure if server attempts to enforce MD5.
> - 'scram-plus' means that client enforces SCRAM and channel binding.
> 
> Or we could just have a channel_binding_mode, which has a "require"
> value like sslmode, and "prefer" mode, which is the default and the
> current behavior...  Still what to do with MD5 requests in this case?

Just to reiterate, MD5 and SCRAM-less-binding is designed to avoid
packet _replay_.  It assumes no man-in-the-middle has adjusted what is
supported by the two endpoints.

SCRAM-with-binding is the first password method that attempts to avoid
man-in-the-middle attacks, and therefore is much less likely to be able
to trust what the endpoints supports.  I think it is really the
channel_binding_mode that we want to control at the client.  The lesser
modes are much more reasonable to use an automatic best-supported
negotiation, which is what we do now.

FYI, I think the server could also require channel binding for SCRAM. We
already have scram-sha-256 in pg_hba.conf, and I think
scram-sha-256-plus would be reasonable.

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

+ As you are, so once was I.  As I am, so you will be. +
+  Ancient Roman grave inscription +

Re: Postgres 11 release notes

[Bruce Momjian]

On Wed, May 16, 2018 at 01:22:45PM +0900, Michael Paquier wrote:
> On Mon, May 14, 2018 at 08:45:44PM -0400, Bruce Momjian wrote:
> > What TLS does is to mix the offered ciphers into the negotiation hash so
> > a man-in-the-middle can't pretend it doesn't support something.  Could
> > we do something like that here?
> 
> I have to admit that I don't quite follow here, the shape of channel
> binding data is decided by RFC 5929, so we need to stick with it.

OK, let me explain.  First, there are two man-in-the-middle types of
attacks.  The first one allows the two legitimate ends of the TLS
connection to negotiate the shared secret between themselves, but
injects or changes some of the packets before the shared secret is
agreed upon, perhaps to downgrade the strength of the protocol options. 
TLS prevents this type of man-in-the-middle attack by hashing the shared
secret with a hash of all of the TLS negotiation messages previously
sent.  Each side who knows the shared secret can verify that no messages
were changed;  see:

  
https://security.stackexchange.com/questions/115284/how-can-an-attacker-downgrade-modify-the-cipher-suites-when-they-are-maced-fre

The second type of man-in-the-middle attack is where the
man-in-the-middle is negotiating the shared secret separately with the
two end points.  There is no way to detect this without having some
shared secret between the two valid end points.  One shared secret is
the password hashed with the shared secret, which is what tls-unique
uses.  The second uses the server certificate hashed with the shared
secret.  This is now documented in Postgres:

   In SCRAM with tls-unique
   channel binding, the shared secret negotiated during the SSL session
   is mixed into the user-supplied password hash.  The shared secret
   is partly chosen by the server, but not directly transmitted, making
   it impossible for a fake server to create an SSL connection with the
   client that has the same shared secret it has with the real server.
   SCRAM with tls-server-end-point
   mixes a hash of the server's certificate into the user-supplied password
   hash.  While a fake server can retransmit the real server's certificate,
   it doesn't have access to the private key matching that certificate, and
   therefore cannot prove it is the owner, causing SSL connection failure.

TLS prevents a man-in-the-middle from injecting invalid packets. 
However, it does not prevent a man-in-the-middle attack with separeate
shared secret negotiation unless certificates are used.

The current problem under discussion is the same as that of the
man-in-the-middle packet change/injection attack in that the
man-in-the-middle can change the options reported as supported by the
Postgres server, before the password is sent.  The most efficient fix
for this would be for the all Postgres protocol messages up to the point
where the authentication was chosen to be hashed with the password and
sent to the server.  In that way, if a man-in-the-middle changed the
server-reported supported authentication, the server would identify this
and refuse the connection.

The problem is that current and past-version PG authentication methods
don't have any protocol downgrade protection, and it is hard to add it
unless you just remove support for old protocols.  I think the only
reasonable solution is to allow the client and/or server to force
channel binding.

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

+ As you are, so once was I.  As I am, so you will be. +
+  Ancient Roman grave inscription +

Re: NaNs in numeric_power (was Re: Postgres 11 release notes)

[Dean Rasheed]

On 16 May 2018 at 14:44, Tom Lane  wrote:
> Dean Rasheed  writes:
>> In the case 1 ^ NaN = 1, what should the result scale be?
>
> The result is exact, so I don't see a reason to be worried about its
> scale.  Messing with the scale would also require expending even
> more code on what is, in the end, a corner case that no users have
> expressed concern about.
>

OK, fine. Not really worth worrying about.

Regards,
Dean