Re: [HACKERS] PG 10 release notes

[Tsunakawa, Takayuki]

From: pgsql-hackers-ow...@postgresql.org
> [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Tom Lane
> TBH, I think that's another reason for not release-noting it.  There's no
> concrete change to point to, and so it's hard to figure out what to say.
> I'm not even very sure that we should be encouraging people to change
> existing shared_buffers settings; the experiments that Takayuki-san did
> were only on Windows 10, so we don't really know that changing that would
> be a good idea on older Windows versions.

In fact, when I ran pgbench on Windows 2008 for some other unrelated reason, I 
found larger shared buffers (4GB, 8GB) was effective, too.

I didn't know pure documentation changes are not listed on the release notes.  
But I suggest listing them (of course, excluding mere typos), because the 
documentation is also important for users as well as programs.  The 
documentation is also part of software product.  If the documented behavior and 
the actual one differs and the user is wondering which is correct, he can know 
the answer only from the release note when the mismatch is fixed.  I think the 
documentation changes are more useful for users than, for example, the 
following items:

E.1.3.11. Source Code
Improve behavior of pgindent (Piotr Stefaniak, Tom Lane)
Allow WaitLatchOrSocket() to wait for socket connection on Windows (Andres 
Freund)
Overhaul documentation build process (Alexander Lakhin)
Use XSLT to build the PostgreSQL documentation (Peter Eisentraut)


Regards
Takayuki Tsunakawa







-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PG 10 release notes

[Joshua D. Drake]

On 09/19/2017 09:32 AM, 'Bruce Momjian' wrote:

On Tue, Sep 19, 2017 at 12:30:01PM -0400, Tom Lane wrote:

"'Bruce Momjian'"  writes:

On Tue, Sep 19, 2017 at 12:22:39PM -0400, Tom Lane wrote:

We don't normally release-note documentation changes.  If this
wasn't purely a documentation change, then I was probably in error
to decide it didn't need to be in the notes.



It was purely a documentation change, but it was a documented change in a
long-standing and popular practice of not using too many shared buffers
on Windows, so I thought it wise to add it.


Well, if the intent of the note was to encourage people to raise
shared_buffers, it didn't do a very good job of that as written,
because I sure didn't understand it that way.


Do you have any suggestions since it is not a code change that I can
point to?  My guess is that the limitation was removed years ago, but we
only found out recently.


My guess is that the limitation was removed as of 9.3 with the work Haas 
did with shared buffers. Thus, yes it was years ago. I think that 
listing it regardless of the documentation change could be useful. 
Something like:


"""
Better support for large shared_buffers configurations including the 
Windows platform. Users are encouraged to review their shared_buffer 
settings against the size of their active data set and reconfigure 
appropriately.

"""

It is pretty much practitioner given that if your active data set can 
fit in shared_buffers and you aren't going to adversely affect the 
ability for the system to operate, that you should configure a high 
setting. I have seen settings as much as 96GB doing wonderfully in 
production.


JD






--
Command Prompt, Inc. || http://the.postgres.company/ || @cmdpromptinc

PostgreSQL Centered full stack support, consulting and development.
Advocate: @amplifypostgres || Learn: https://pgconf.us
* Unless otherwise stated, opinions are my own.   *


--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PG 10 release notes

[Tom Lane]

"'Bruce Momjian'"  writes:
> On Tue, Sep 19, 2017 at 12:30:01PM -0400, Tom Lane wrote:
>> Well, if the intent of the note was to encourage people to raise
>> shared_buffers, it didn't do a very good job of that as written,
>> because I sure didn't understand it that way.

> Do you have any suggestions since it is not a code change that I can
> point to?  My guess is that the limitation was removed years ago, but we
> only found out recently.

TBH, I think that's another reason for not release-noting it.  There's
no concrete change to point to, and so it's hard to figure out what
to say.  I'm not even very sure that we should be encouraging people
to change existing shared_buffers settings; the experiments that
Takayuki-san did were only on Windows 10, so we don't really know that
changing that would be a good idea on older Windows versions.

regards, tom lane


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PG 10 release notes

['Bruce Momjian']

On Tue, Sep 19, 2017 at 12:30:01PM -0400, Tom Lane wrote:
> "'Bruce Momjian'"  writes:
> > On Tue, Sep 19, 2017 at 12:22:39PM -0400, Tom Lane wrote:
> >> We don't normally release-note documentation changes.  If this
> >> wasn't purely a documentation change, then I was probably in error
> >> to decide it didn't need to be in the notes.
> 
> > It was purely a documentation change, but it was a documented change in a
> > long-standing and popular practice of not using too many shared buffers
> > on Windows, so I thought it wise to add it.
> 
> Well, if the intent of the note was to encourage people to raise
> shared_buffers, it didn't do a very good job of that as written,
> because I sure didn't understand it that way.

Do you have any suggestions since it is not a code change that I can
point to?  My guess is that the limitation was removed years ago, but we
only found out recently.

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

+ As you are, so once was I.  As I am, so you will be. +
+  Ancient Roman grave inscription +


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PG 10 release notes

[Tom Lane]

"'Bruce Momjian'"  writes:
> On Tue, Sep 19, 2017 at 12:22:39PM -0400, Tom Lane wrote:
>> We don't normally release-note documentation changes.  If this
>> wasn't purely a documentation change, then I was probably in error
>> to decide it didn't need to be in the notes.

> It was purely a documentation change, but it was a documented change in a
> long-standing and popular practice of not using too many shared buffers
> on Windows, so I thought it wise to add it.

Well, if the intent of the note was to encourage people to raise
shared_buffers, it didn't do a very good job of that as written,
because I sure didn't understand it that way.

regards, tom lane


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PG 10 release notes

['Bruce Momjian']

On Tue, Sep 19, 2017 at 12:22:39PM -0400, Tom Lane wrote:
> "'Bruce Momjian'"  writes:
> > I am sure Tom can explain his reasoning.
> 
> We don't normally release-note documentation changes.  If this
> wasn't purely a documentation change, then I was probably in error
> to decide it didn't need to be in the notes.

It was purely a documentation change, but it was a documented change in a
long-standing and popular practice of not using too many shared buffers
on Windows, so I thought it wise to add it.

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

+ As you are, so once was I.  As I am, so you will be. +
+  Ancient Roman grave inscription +


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PG 10 release notes

[Tom Lane]

"'Bruce Momjian'"  writes:
> I am sure Tom can explain his reasoning.

We don't normally release-note documentation changes.  If this
wasn't purely a documentation change, then I was probably in error
to decide it didn't need to be in the notes.

regards, tom lane


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PG 10 release notes

['Bruce Momjian']

On Thu, Sep 14, 2017 at 03:13:50AM +, Tsunakawa, Takayuki wrote:
> It's embarrassing to ask about such a trivial thing, but I noticed
> the following line was missing in the latest release note, which was
> originally in Bruce's website:
>
> Remove documented restriction about using large shared buffers on
> Windows (Takayuki Tsunakawa)
>
> Is this intended?

I don't know.  The original text was:

Remove documented restriction about using large shared buffers on
   Windows (Takayuki Tsunakawa)

and was removed in this commit:

commit 749eceff4a1f9740391b126c81af9fd4bf3b1eaa
Author: Tom Lane 
Date:   Sun Jul 9 20:11:21 2017 -0400

Doc: desultory copy-editing for v10 release notes.

Improve many item descriptions, improve markup, relocate some items
that seemed to be in the wrong section.

I am sure Tom can explain his reasoning.

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

+ As you are, so once was I.  As I am, so you will be. +
+  Ancient Roman grave inscription +


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PG 10 release notes

[Tsunakawa, Takayuki]

It's embarrassing to ask about such a trivial thing, but I noticed the 
following line was missing in the latest release note, which was originally in 
Bruce's website:

Remove documented restriction about using large shared buffers on Windows 
(Takayuki Tsunakawa)

Is this intended?

Regards
Takayuki Tsunakawa




-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PG 10 release notes

[Adrien Nayrat]

On 09/12/2017 01:50 AM, Bruce Momjian wrote:
> I don't know, but I suggest you read this email thread from April to get
> an idea of how performance items are handled:
> 
>   
> https://www.postgresql.org/message-id/flat/20170425013144.GA7513%40momjian.us#20170425013144.ga7...@momjian.us

Oh, sorry, I missed your explanation :

https://www.postgresql.org/message-id/20170425033742.GG7513%40momjian.us
> I remember seeing those and those are normally details I do not put in
> the release notes as there isn't a clear user experience change except
> "Postgres is faster".  Yeah, a bummer, and I can change my filter, but
> it would require discussion.


>From an user experience, IMHO, it is a huge improvement. I thought it could be
mentioned in "General Performance section". But I do not know your habits.

If it was not mentioned in previous releases, I understand we do not mention in
v10's releases notes.

Regards,



-- 
Adrien NAYRAT




signature.asc
Description: OpenPGP digital signature

Re: [HACKERS] PG 10 release notes

[Thomas Munro]

On Tue, Sep 12, 2017 at 11:57 AM, Bruce Momjian  wrote:
> On Tue, Aug  1, 2017 at 08:53:51AM +1200, Thomas Munro wrote:
>> "Add AFTER trigger transition tables to record changed rows (Kevin Grittner)"
>>
>> Any chance I could ask for a secondary author credit here?
>
> Sure, done.

Thanks.  Can you take it out again?  (Just kidding!)

-- 
Thomas Munro
http://www.enterprisedb.com


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PG 10 release notes

[Bruce Momjian]

On Fri, Sep  1, 2017 at 05:39:31PM +0900, Masahiko Sawada wrote:
> Hi all,
> 
> On Tue, Aug 1, 2017 at 5:53 AM, Thomas Munro
>  wrote:
> > On Tue, Apr 25, 2017 at 1:31 PM, Bruce Momjian  wrote:
> >> I have committed the first draft of the Postgres 10 release notes.  They
> >> are current as of two days ago, and I will keep them current.  Please
> >> give me any feedback you have.
> >
> > Hi Bruce,
> >
> > "Add AFTER trigger transition tables to record changed rows (Kevin 
> > Grittner)"
> >
> > Any chance I could ask for a secondary author credit here?  While I
> > started out as a reviewer and I understand that we don't list those, I
> > finished up writing quite a lot of lines of committed code for this
> > (see commits 1add0b15, 8c55244a, c46c0e52, 501ed02c, f32d57fd,
> > 9e6104c6, 29fd3d9d, 304007d9, 5ebeb579) and was already listed as a
> > co-author by Kevin in the original commits (59702716, 18ce3a4a).  Of
> > course I wish I'd identified and fixed all of those things *before*
> > the original commits, but that's how it played out...
> >
> 
> It might be too late but I found that the following entry is
> categorized in Monitoring. But in PostgreSQL 9.6 release note, the
> feature related to default role is categorized in Permissions
> Management. I think the adding new default roles can be categorized in
> the same category to not confuse users and personally it's more
> suitable.
> 
> "Add default monitoring roles (Dave Page)"

We don't have a "Permissions Management" category in PG 10 and unless we
have at least three items to add in there, I don't think it is worth
adding it.

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

+ As you are, so once was I.  As I am, so you will be. +
+  Ancient Roman grave inscription +


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PG 10 release notes

[Bruce Momjian]

On Tue, Aug  1, 2017 at 08:53:51AM +1200, Thomas Munro wrote:
> On Tue, Apr 25, 2017 at 1:31 PM, Bruce Momjian  wrote:
> > I have committed the first draft of the Postgres 10 release notes.  They
> > are current as of two days ago, and I will keep them current.  Please
> > give me any feedback you have.
> 
> Hi Bruce,
> 
> "Add AFTER trigger transition tables to record changed rows (Kevin Grittner)"
> 
> Any chance I could ask for a secondary author credit here?  While I
> started out as a reviewer and I understand that we don't list those, I
> finished up writing quite a lot of lines of committed code for this
> (see commits 1add0b15, 8c55244a, c46c0e52, 501ed02c, f32d57fd,
> 9e6104c6, 29fd3d9d, 304007d9, 5ebeb579) and was already listed as a
> co-author by Kevin in the original commits (59702716, 18ce3a4a).  Of
> course I wish I'd identified and fixed all of those things *before*
> the original commits, but that's how it played out...

Sure, done.

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

+ As you are, so once was I.  As I am, so you will be. +
+  Ancient Roman grave inscription +


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PG 10 release notes

[Bruce Momjian]

On Sat, Sep  9, 2017 at 12:39:43PM +0200, Adrien Nayrat wrote:
> On 07/13/2017 04:36 PM, Adrien Nayrat wrote:
> > Hello hackers,
> > 
> > From: Peter Geoghegan 
> >> Date: Wed, 5 Jul 2017 15:19:57 -0700
> >> Subject: Re: [BUGS] BUG #14722: Segfault in tuplesort_heap_siftup, 32 bit 
> >> overflow
> >> On pgsql-b...@postgresql.org
> > 
> > On 07/06/2017 12:19 AM, Peter Geoghegan wrote:
> >> In Postgres 10, tuplesort external sort run merging became much faster
> >> following commit 24598337c8d. It might be noticeable if such a machine
> >> were using Postgres 10 [...]
> > 
> > Should-we mention this improvement in release notes?
> > 
> > Regards,
> > 
> 
> Hello,
> 
> After seeing theses slides (especially 52) :
> https://speakerdeck.com/peterg/sort-hash-pgconfus-2017
...
> 
> 
> 
> Should we mention it ?

I don't know, but I suggest you read this email thread from April to get
an idea of how performance items are handled:


https://www.postgresql.org/message-id/flat/20170425013144.GA7513%40momjian.us#20170425013144.ga7...@momjian.us

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

+ As you are, so once was I.  As I am, so you will be. +
+  Ancient Roman grave inscription +


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PG 10 release notes

[Bruce Momjian]

On Mon, Sep 11, 2017 at 06:30:58PM -0400, Tom Lane wrote:
> Bruce Momjian  writes:
> > On Fri, Jun  2, 2017 at 04:05:44PM -0500, Jim Nasby wrote:
> >> Can you change the attribution on
> >> Allow PL/Tcl functions to return composite types and sets
> >> to Karl Lehenbauer?
> 
> > Done and backpatched.  Sorry for the delay.
> 
> I don't see this pushed to the repo?

Sorry, pushed now.

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

+ As you are, so once was I.  As I am, so you will be. +
+  Ancient Roman grave inscription +


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PG 10 release notes

[Tom Lane]

Bruce Momjian  writes:
> On Fri, Jun  2, 2017 at 04:05:44PM -0500, Jim Nasby wrote:
>> Can you change the attribution on
>> Allow PL/Tcl functions to return composite types and sets
>> to Karl Lehenbauer?

> Done and backpatched.  Sorry for the delay.

I don't see this pushed to the repo?

regards, tom lane


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PG 10 release notes

[Bruce Momjian]

On Fri, Jun  2, 2017 at 04:05:44PM -0500, Jim Nasby wrote:
> On 4/24/17 8:31 PM, Bruce Momjian wrote:
> >I have committed the first draft of the Postgres 10 release notes.  They
> >are current as of two days ago, and I will keep them current.  Please
> >give me any feedback you have.
> >
> >The only unusual thing is that this release has ~180 items while most
> >recent release have had ~220.  The pattern I see that there are more
> >large features in this release than previous ones.
> 
> Can you change the attribution on
> 
> Allow PL/Tcl functions to return composite types and sets
> 
> to Karl Lehenbauer? He actually wrote the original patch; I just helped to
> get it through the community (something that FlightAware paid for). I didn't
> realize at the time that you could change the listed Author in the
> commitfest.

Done and backpatched.  Sorry for the delay.

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

+ As you are, so once was I.  As I am, so you will be. +
+  Ancient Roman grave inscription +


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PG 10 release notes

[Adrien Nayrat]

On 07/13/2017 04:36 PM, Adrien Nayrat wrote:
> Hello hackers,
> 
> From: Peter Geoghegan 
>> Date: Wed, 5 Jul 2017 15:19:57 -0700
>> Subject: Re: [BUGS] BUG #14722: Segfault in tuplesort_heap_siftup, 32 bit 
>> overflow
>> On pgsql-b...@postgresql.org
> 
> On 07/06/2017 12:19 AM, Peter Geoghegan wrote:
>> In Postgres 10, tuplesort external sort run merging became much faster
>> following commit 24598337c8d. It might be noticeable if such a machine
>> were using Postgres 10 [...]
> 
> Should-we mention this improvement in release notes?
> 
> Regards,
> 

Hello,

After seeing theses slides (especially 52) :
https://speakerdeck.com/peterg/sort-hash-pgconfus-2017

I noticed several commits which improves performance of hash tables. Commit's
messages mentions performance improvements for bitmap scans, hash aggregation
and (according to Peter Geoghegan's conference) hash join :



commit b30d3ea824c5ccba43d3e942704f20686e7dbab8
Author: Andres Freund 
Date:   Fri Oct 14 16:05:30 2016 -0700

Add a macro templatized hashtable.
[...]
In queries where these use up a large fraction of the time, this
has been measured to lead to performance improvements of over 100%.



commit 75ae538bc3168bf44475240d4e0487ee2f3bb376
Author: Andres Freund 
Date:   Fri Oct 14 16:05:30 2016 -0700

Use more efficient hashtable for tidbitmap.c to speed up bitmap scans.
[...]
For bitmap scan heavy queries speedups of over 100% have been measured.



commit 5dfc198146b49ce7ecc8a1fc9d5e171fb75f6ba5
Author: Andres Freund 
Date:   Fri Oct 14 17:22:51 2016 -0700

Use more efficient hashtable for execGrouping.c to speed up hash 
aggregation.
[...]
Improvements of over 120% have been   measured.



Should we mention it ?

Regards,

-- 
Adrien NAYRAT

http://dalibo.com - http://dalibo.org



signature.asc
Description: OpenPGP digital signature

Re: [HACKERS] PG 10 release notes

[Masahiko Sawada]

Hi all,

On Tue, Aug 1, 2017 at 5:53 AM, Thomas Munro
 wrote:
> On Tue, Apr 25, 2017 at 1:31 PM, Bruce Momjian  wrote:
>> I have committed the first draft of the Postgres 10 release notes.  They
>> are current as of two days ago, and I will keep them current.  Please
>> give me any feedback you have.
>
> Hi Bruce,
>
> "Add AFTER trigger transition tables to record changed rows (Kevin Grittner)"
>
> Any chance I could ask for a secondary author credit here?  While I
> started out as a reviewer and I understand that we don't list those, I
> finished up writing quite a lot of lines of committed code for this
> (see commits 1add0b15, 8c55244a, c46c0e52, 501ed02c, f32d57fd,
> 9e6104c6, 29fd3d9d, 304007d9, 5ebeb579) and was already listed as a
> co-author by Kevin in the original commits (59702716, 18ce3a4a).  Of
> course I wish I'd identified and fixed all of those things *before*
> the original commits, but that's how it played out...
>

It might be too late but I found that the following entry is
categorized in Monitoring. But in PostgreSQL 9.6 release note, the
feature related to default role is categorized in Permissions
Management. I think the adding new default roles can be categorized in
the same category to not confuse users and personally it's more
suitable.

"Add default monitoring roles (Dave Page)"

Thought?

Regards,

--
Masahiko Sawada
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PG 10 release notes

[Thomas Munro]

On Tue, Apr 25, 2017 at 1:31 PM, Bruce Momjian  wrote:
> I have committed the first draft of the Postgres 10 release notes.  They
> are current as of two days ago, and I will keep them current.  Please
> give me any feedback you have.

Hi Bruce,

"Add AFTER trigger transition tables to record changed rows (Kevin Grittner)"

Any chance I could ask for a secondary author credit here?  While I
started out as a reviewer and I understand that we don't list those, I
finished up writing quite a lot of lines of committed code for this
(see commits 1add0b15, 8c55244a, c46c0e52, 501ed02c, f32d57fd,
9e6104c6, 29fd3d9d, 304007d9, 5ebeb579) and was already listed as a
co-author by Kevin in the original commits (59702716, 18ce3a4a).  Of
course I wish I'd identified and fixed all of those things *before*
the original commits, but that's how it played out...

-- 
Thomas Munro
http://www.enterprisedb.com


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PG 10 release notes

[Adrien Nayrat]

On 07/13/2017 04:36 PM, Adrien Nayrat wrote:
> Hello hackers,
> 
> From: Peter Geoghegan 
>> Date: Wed, 5 Jul 2017 15:19:57 -0700
>> Subject: Re: [BUGS] BUG #14722: Segfault in tuplesort_heap_siftup, 32 bit 
>> overflow
>> On pgsql-b...@postgresql.org
> 
> On 07/06/2017 12:19 AM, Peter Geoghegan wrote:
>> In Postgres 10, tuplesort external sort run merging became much faster
>> following commit 24598337c8d. It might be noticeable if such a machine
>> were using Postgres 10 [...]
> 
> Should-we mention this improvement in release notes?
> 
> Regards,
> 

Patch attached.

Magic sorting is not my cup of tea, so I only put "Improve external sort". Maybe
we should add an explanation.



Regards,

-- 
Adrien NAYRAT

http://dalibo.com - http://dalibo.org
diff --git a/doc/src/sgml/release-10.sgml b/doc/src/sgml/release-10.sgml
index debaa80..e654c17 100644
--- a/doc/src/sgml/release-10.sgml
+++ b/doc/src/sgml/release-10.sgml
@@ -856,6 +856,16 @@
 
   
 
+   
+Improve external sort (Peter Geoghegan)
+   
+
+  
+
+  
+



signature.asc
Description: OpenPGP digital signature

Re: [HACKERS] PG 10 release notes

[Adrien Nayrat]

Hello hackers,

From: Peter Geoghegan 
> Date: Wed, 5 Jul 2017 15:19:57 -0700
> Subject: Re: [BUGS] BUG #14722: Segfault in tuplesort_heap_siftup, 32 bit 
> overflow
> On pgsql-b...@postgresql.org

On 07/06/2017 12:19 AM, Peter Geoghegan wrote:
> In Postgres 10, tuplesort external sort run merging became much faster
> following commit 24598337c8d. It might be noticeable if such a machine
> were using Postgres 10 [...]

Should-we mention this improvement in release notes?

Regards,

-- 
Adrien NAYRAT

http://dalibo.com - http://dalibo.org



signature.asc
Description: OpenPGP digital signature

Re: [HACKERS] PG 10 release notes

[Bruce Momjian]

On Wed, Jun  7, 2017 at 03:18:49PM +1000, Neha Khatri wrote:
> 
> On Mon, May 15, 2017 at 12:45 PM, Bruce Momjian  wrote:
> 
> On Thu, May 11, 2017 at 11:50:03PM -0400, Tom Lane wrote:
> > Michael Paquier  writes:
> > > Bruce, the release notes do not mention yet that support for cleartext
> > > passwords is removed. This has been done recently by Heikki in
> > > eb61136d. This has as consequence that CREATE ROLE PASSWORD
> > > UNENCRYPTED returns an error, and that password_encryption loses its
> > > value 'off' compared to last releases.
> >
> > The release notes only say that they're current through 4-22.  The
> > usual process is to update them in batches every so often.  It'd be
> > great if Bruce has time to do another round before Monday, but the
> > current situation is not really broken.
> 
> Done.  Applied patch attached.
> 
> 
> 
> The patch introduced one release note item twice in 
> https://www.postgresql.org/
> docs/10/static/release-10.html :
> 
> 
>   • Rename WAL-related functions and views to use lsn instead of location 
> (David Rowley)
> 
>   • RenameWAL-related functions and views to use lsn instead of location 
> (David
> Rowley)
> 
> Perhaps just one entry is good.

Yes, this has been fixed already, see:

https://www.postgresql.org/docs/devel/static/release-10.html

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

+ As you are, so once was I.  As I am, so you will be. +
+  Ancient Roman grave inscription +


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PG 10 release notes

[Neha Khatri]

On Mon, May 15, 2017 at 12:45 PM, Bruce Momjian  wrote:

> On Thu, May 11, 2017 at 11:50:03PM -0400, Tom Lane wrote:
> > Michael Paquier  writes:
> > > Bruce, the release notes do not mention yet that support for cleartext
> > > passwords is removed. This has been done recently by Heikki in
> > > eb61136d. This has as consequence that CREATE ROLE PASSWORD
> > > UNENCRYPTED returns an error, and that password_encryption loses its
> > > value 'off' compared to last releases.
> >
> > The release notes only say that they're current through 4-22.  The
> > usual process is to update them in batches every so often.  It'd be
> > great if Bruce has time to do another round before Monday, but the
> > current situation is not really broken.
>
> Done.  Applied patch attached.
>
>
The patch introduced one release note item twice in
https://www.postgresql.org/docs/10/static/release-10.html :


   -

   Rename WAL-related functions and views to use lsn instead of location (David
   Rowley)
   -

   RenameWAL-related functions and views to use lsn instead of location (David
   Rowley)

Perhaps just one entry is good.

Regards,
Neha

Re: [HACKERS] PG 10 release notes

[Jim Nasby]

On 4/24/17 8:31 PM, Bruce Momjian wrote:

I have committed the first draft of the Postgres 10 release notes.  They
are current as of two days ago, and I will keep them current.  Please
give me any feedback you have.

The only unusual thing is that this release has ~180 items while most
recent release have had ~220.  The pattern I see that there are more
large features in this release than previous ones.


Can you change the attribution on

Allow PL/Tcl functions to return composite types and sets

to Karl Lehenbauer? He actually wrote the original patch; I just helped 
to get it through the community (something that FlightAware paid for). I 
didn't realize at the time that you could change the listed Author in 
the commitfest.

--
Jim C. Nasby, Data Architect   j...@nasby.net
512.569.9461 (cell) http://jim.nasby.net


--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PG 10 release notes

[Euler Taveira]

2017-05-04 22:26 GMT-03:00 Andres Freund :

> At least I was a bit more optimistic that we'd get a decent json plugin
> into care soon-ish.  While there was some initial work towards that, it
> unfortunately stalled at some point.
>
> That was my initial idea but there wasn't some interest at that time.
wal2json is used in some replication solutions but also for audit and
change capture solution. I recently received some patches/suggestions from
Daniele Varrazzo [1].


> Euler, are you perchance interested in working towards that? ;)
>

Sure. I can submit it for the next CF.


[1]
https://www.postgresql.org/message-id/CA%2Bmi_8bJ_uPr67j-6mbin537DVvfk%3DbOhmWneyBRfbZu89q0tw%40mail.gmail.com


-- 
   Euler Taveira   Timbira -
http://www.timbira.com.br/
   PostgreSQL: Consultoria, Desenvolvimento, Suporte 24x7 e Treinamento

Re: [HACKERS] PG 10 release notes

[Bruce Momjian]

On Thu, May 11, 2017 at 11:50:03PM -0400, Tom Lane wrote:
> Michael Paquier  writes:
> > Bruce, the release notes do not mention yet that support for cleartext
> > passwords is removed. This has been done recently by Heikki in
> > eb61136d. This has as consequence that CREATE ROLE PASSWORD
> > UNENCRYPTED returns an error, and that password_encryption loses its
> > value 'off' compared to last releases.
> 
> The release notes only say that they're current through 4-22.  The
> usual process is to update them in batches every so often.  It'd be
> great if Bruce has time to do another round before Monday, but the
> current situation is not really broken.

Done.  Applied patch attached.

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

+ As you are, so once was I.  As I am, so you will be. +
+  Ancient Roman grave inscription +
diff --git a/doc/src/sgml/release-10.sgml b/doc/src/sgml/release-10.sgml
new file mode 100644
index 6497641..4f953dc
*** a/doc/src/sgml/release-10.sgml
--- b/doc/src/sgml/release-10.sgml
***
*** 128,133 
--- 128,155 
  
  
   
+  
+   Rename WAL-related functions and views to use 
lsn
+   instead of location (David Rowley)
+  
+ 
+ 
+ 
+  
+  
+   RenameWAL-related functions and views to use lsn
+   instead of location (David Rowley)
+  
+ 
+ 
+ 
+  
+ 
+  Remove the ability to store unencrypted passwords on the server
+  (Heikki Linnakangas)
+ 
+ 
+ 
+  The server-side variable 
+  no longer supports off or plain.
+  The UNENCRYPTED option is no longer supported for
+  CREATE/ALTER USER ... PASSSWORD.  Similarly, the
+  --unencrypted has been removed from createuser.
+  The default for password_encryption is still
+  md5, and users migrating passwords from older systems
+  will have them stored encrypted by default in this release.
+ 
+ 
+ 
+ 
+  
+   
+Add function PQencryptPasswordConn()
+to allow creation of more types of encrypted passwords on the
+client-side (Michael Paquier, Heikki Linnakangas)
+   
+ 
+   
+Previously only MD5 passwords could be created using PQencryptPassword().
+This new function can also create SCRAM-SHA-256 passwords.
+   
+  
+ 
+  
+   
***
*** 2847,2852 
--- 2912,2919 


 Push aggregates to foreign data wrapper servers, where possible
***
*** 2858,2864 
 from the foreign data wrapper server, and offloads
 aggregate computation from the requesting server.  The postgres_fdw is able to
!perform this optimization.

   
  
--- 2925,2932 
 from the foreign data wrapper server, and offloads
 aggregate computation from the requesting server.  The postgres_fdw is able to
!perform this optimization.  There are also improvements in
!pushing down joins involving extensions.

   
  

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PG 10 release notes

[Tom Lane]

Michael Paquier  writes:
> Bruce, the release notes do not mention yet that support for cleartext
> passwords is removed. This has been done recently by Heikki in
> eb61136d. This has as consequence that CREATE ROLE PASSWORD
> UNENCRYPTED returns an error, and that password_encryption loses its
> value 'off' compared to last releases.

The release notes only say that they're current through 4-22.  The
usual process is to update them in batches every so often.  It'd be
great if Bruce has time to do another round before Monday, but the
current situation is not really broken.

regards, tom lane


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PG 10 release notes

[Michael Paquier]

On Mon, May 8, 2017 at 10:50 PM, Bruce Momjian  wrote:
> [other stuff]

Bruce, the release notes do not mention yet that support for cleartext
passwords is removed. This has been done recently by Heikki in
eb61136d. This has as consequence that CREATE ROLE PASSWORD
UNENCRYPTED returns an error, and that password_encryption loses its
value 'off' compared to last releases.
-- 
Michael


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PG 10 release notes

[Masahiko Sawada]

On Mon, May 8, 2017 at 10:50 PM, Bruce Momjian  wrote:
> On Mon, May  8, 2017 at 12:18:54PM +0900, Masahiko Sawada wrote:
>> On Fri, May 5, 2017 at 8:38 AM, Bruce Momjian  wrote:
>> > On Fri, Apr 28, 2017 at 01:12:34PM +0900, Masahiko Sawada wrote:
>> >> On Tue, Apr 25, 2017 at 10:31 AM, Bruce Momjian  wrote:
>> >> > I have committed the first draft of the Postgres 10 release notes.  They
>> >> > are current as of two days ago, and I will keep them current.  Please
>> >> > give me any feedback you have.
>> >> >
>> >>
>> >> Related to a following item in release note, oldestxmin is also
>> >> reported by VACUUM VERBOSE and autovacuum , which is introduced by
>> >> commit 9eb344faf54a849898d9be012ddfa8204cfeb57c (author is Simon).
>> >>
>> >> * Have VACUUM VERBOSE report the number of skipped frozen pages
>> >> (Masahiko Sawada)
>> >>
>> >> Could we add this item to the release note?
>> >
>> > OK, adjusted text below and commit added above this item:
>> >
>> >Have VACUUM VERBOSE report
>> >the number of skipped frozen pages and oldest xmin (Masahiko Sawada)
>> >
>>
>> Thank you! But actually main author of this item is Simon, I didn't
>> involved in it. Maybe we can have it as a separate item.
>
> I just added Simon's name to the same item.
>

Thank you!

Regards,

--
Masahiko Sawada
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PG 10 release notes

[Bruce Momjian]

On Mon, May  8, 2017 at 12:18:54PM +0900, Masahiko Sawada wrote:
> On Fri, May 5, 2017 at 8:38 AM, Bruce Momjian  wrote:
> > On Fri, Apr 28, 2017 at 01:12:34PM +0900, Masahiko Sawada wrote:
> >> On Tue, Apr 25, 2017 at 10:31 AM, Bruce Momjian  wrote:
> >> > I have committed the first draft of the Postgres 10 release notes.  They
> >> > are current as of two days ago, and I will keep them current.  Please
> >> > give me any feedback you have.
> >> >
> >>
> >> Related to a following item in release note, oldestxmin is also
> >> reported by VACUUM VERBOSE and autovacuum , which is introduced by
> >> commit 9eb344faf54a849898d9be012ddfa8204cfeb57c (author is Simon).
> >>
> >> * Have VACUUM VERBOSE report the number of skipped frozen pages
> >> (Masahiko Sawada)
> >>
> >> Could we add this item to the release note?
> >
> > OK, adjusted text below and commit added above this item:
> >
> >Have VACUUM VERBOSE report
> >the number of skipped frozen pages and oldest xmin (Masahiko Sawada)
> >
> 
> Thank you! But actually main author of this item is Simon, I didn't
> involved in it. Maybe we can have it as a separate item.

I just added Simon's name to the same item.

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

+ As you are, so once was I.  As I am, so you will be. +
+  Ancient Roman grave inscription +


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PG 10 release notes

[Masahiko Sawada]

On Fri, May 5, 2017 at 8:38 AM, Bruce Momjian  wrote:
> On Fri, Apr 28, 2017 at 01:12:34PM +0900, Masahiko Sawada wrote:
>> On Tue, Apr 25, 2017 at 10:31 AM, Bruce Momjian  wrote:
>> > I have committed the first draft of the Postgres 10 release notes.  They
>> > are current as of two days ago, and I will keep them current.  Please
>> > give me any feedback you have.
>> >
>>
>> Related to a following item in release note, oldestxmin is also
>> reported by VACUUM VERBOSE and autovacuum , which is introduced by
>> commit 9eb344faf54a849898d9be012ddfa8204cfeb57c (author is Simon).
>>
>> * Have VACUUM VERBOSE report the number of skipped frozen pages
>> (Masahiko Sawada)
>>
>> Could we add this item to the release note?
>
> OK, adjusted text below and commit added above this item:
>
>Have VACUUM VERBOSE report
>the number of skipped frozen pages and oldest xmin (Masahiko Sawada)
>

Thank you! But actually main author of this item is Simon, I didn't
involved in it. Maybe we can have it as a separate item.

Regards,

--
Masahiko Sawada
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PG 10 release notes

[Bruce Momjian]

On Thu, May  4, 2017 at 07:01:17PM -0300, Alvaro Herrera wrote:
> Thanks for doing this, looks great.  A few notes:
> 
>   
>
>   
>Add the ability to compute a correlation ratio and the number of
>distinct values on several columns (Tomas Vondra, David Rowley)
>   
> 
> I think this should be worded in terms of "extended data statistics" or
> such.  I think your proposed wording is a bit obscure.  How about
> something like "Add extended statistics to improve query planning".
> Also, I'd add myself as co-author, with Tomas' permission.

Already done.

>   
>
>
> Cause BRIN index summarization to happen more
> aggressively (lvaro Herrera)
>
> 
>
> Specifically, summarize the previous page range when a new page
> range is created.
>
>   
> 
> I think it should be pointed out that this is optional and defaults to
> off.  Maybe start with "add option to ..."  (I wonder if it's worth
> creating a linkable area in the docs that explain this feature, so that
> we could have a link in the relnotes entry).

Wow, looking at the patch, there is a lot more going on.  I have updated
the docs but please let me know if there is more needed.

Also, can ALTER INDEX change the index summariziation?  Should it?

Also, when you remove summarization, when is it re-added?  Why would you
remove summarization rather than just re-add it.  I must be missing
something.

>   
>
>
> Add function brin_desummarize_range() to remove
> BRIN summarization of a specified range (lvaro
> Herrera)
>
> 
>
> This allows future BRIN index summarization to be
> more compact.  CLARIFY
>
>   
> 
> The point of this change is that if data has changed (delete, update) in
> a way that could use tighter min/max limits, it would be worthwhile to
> remove the previous BRIN tuple so that a new one is created so that
> future scans can skip scanning the range.  Maybe word it something like
> "This is useful if UPDATE and DELETE have changed the data to tighter
> limits; a future BRIN index entry will be more accurate"?

OK, updated, but please let me know if more changes are needed.

>  
>   
>   
>Add support for converting XML-formatted data into a row
>set (Pavel Stehule, lvaro Herrera)
>   
> XMLTABLE is a sql-standard-specified construct, so we should mention it
> by name in the leading paragraph more prominently ("add support for the
> XMLTABLE standard feature" or something like that); also, I think it

Done.

> should be in the Queries section, not Functions.

Done.


>
> Add GUC 
> to limit the number of worker processes that can be used for
> parallelism (Julien Rouhaud)
>
> 
>
> This can be set lower than  linkend="guc-max-worker-processes"> to reserve worker processes
> for non-parallel purposes.
>
> 
> Strange last phrase.  I suggest "...to reserve worker processes for
> purposes other than parallel queries".  Also in the leading para, "for
> parallelism" should probably be "for query parallelism".

Done.

> I think commit e306df7f9 ("Full Text Search support for JSON
> and JSONB") does definitely not belong to section Indexes; I
> think Data Types is a better fit.

Moved.

> I think commits 67dc4ccbb and 1753b1b02 (pg_sequence and pg_sequences)
> should be listed in the opposite order.  Maybe merge them into one?

Merged.

>
> This proves better security than the existing md5
> negotiation and storage method.
>
> You probably mean "provides" not "proves".

Fixed.

>
> Allow SSL configuration to be updated at
> SIGHUP (Andreas Karlsson, Tom Lane)
>
> SIGHUP seems much too technical.  How about "... to be updated during
> configurating reload"

Yes, updated.

> I think the entry for commits a734fd5d1 and dafa0848d does not belong in
> the reliability section.  In fact I wonder if it's necessary to list
> this one at all.

Yes, I removed it.  We can always re-add it later.

> 
>  Increase the maximum configurable WAL size to 1
>  gigabyte (Beena Emerson)
> 
> "WAL segment size" perhaps, and note that this is useful to reduce
> frequency of archive_command?

Yes, updated.

>
> Also add -nosync option to disable fsync.
>
> Misspelled --no-sync.

Fixed.

>
> Add log options for pg_ctl wait (--wait)
> and no-wait (--no-wait) (Vik Fearing)
>
> Mispelled "long options".

Fixed.

Applied patch attached.

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

+ As you are, so once was I.  As I am, so you will be. +
+  Ancient Roman grave inscription +
diff --git a/doc/src/sgml/release-10.sgml 

Re: [HACKERS] PG 10 release notes

[Bruce Momjian]

On Fri, May  5, 2017 at 02:28:50PM +0200, Petr Jelinek wrote:
> On 05/05/17 02:46, Bruce Momjian wrote:
> > I am getting tired of saying this.  When I am writing the release notes,
> > I am trying to figure out how it affects our shipped code, and the only
> > "decoding" I know of is test_decoding.  My message was this:
> 
> I actually think the main misunderstanding here comes from the
> test_decoding name interpretation. The test_decoding does not decode
> anything and there are no external "decoders". The decoding happens in
> core, the decoding just provides C API for plugins to consume the
> decoded info (ie, heap tuples and similar). The test_decoding *tests*
> the decoding API and the external projects use the decoding API as well
> but they don't really decode anything, their role is of filtering and
> deciding the wire protocol mostly.

Yes, I thought test_decoding did decoding and not just use a
backend-internal decoder.

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

+ As you are, so once was I.  As I am, so you will be. +
+  Ancient Roman grave inscription +


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PG 10 release notes

[Petr Jelinek]

On 05/05/17 02:46, Bruce Momjian wrote:
> On Thu, May  4, 2017 at 05:09:40PM -0700, Andres Freund wrote:
 I would not in any way refer to logical decoding as being only a proof
 of concept, even before logical replication.
>>>
>>> The community ships a reliable logical _encoding_, and a test logical
>>> _decoding_.
>>
>> Yes, so what?  What you said is "I didn't think logical decoding was
>> really more than a proof-of-concept until now", which is plainly wrong,
>> given I know a significant number of users using it in production.  Some
>> of them are well known & large enterprises, and it's used to enable
>> critical things.
> 
> I am getting tired of saying this.  When I am writing the release notes,
> I am trying to figure out how it affects our shipped code, and the only
> "decoding" I know of is test_decoding.  My message was this:

I actually think the main misunderstanding here comes from the
test_decoding name interpretation. The test_decoding does not decode
anything and there are no external "decoders". The decoding happens in
core, the decoding just provides C API for plugins to consume the
decoded info (ie, heap tuples and similar). The test_decoding *tests*
the decoding API and the external projects use the decoding API as well
but they don't really decode anything, their role is of filtering and
deciding the wire protocol mostly.

-- 
  Petr Jelinek  http://www.2ndQuadrant.com/
  PostgreSQL Development, 24x7 Support, Training & Services


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PG 10 release notes

[Petr Jelinek]

On 05/05/17 03:26, Andres Freund wrote:
> On 2017-05-04 18:23:38 -0700, Peter Geoghegan wrote:
>> On Thu, May 4, 2017 at 6:08 PM, Robert Haas  wrote:
 E.g. to power amazon's data migration service (yes, that scares me).
>>>
>>> If you recall, I did predict prior to commit that test_decoding would
>>> get put into production use regardless of the name.
>>
>> I thought you were completely wrong when you said that. Lesson
>> learned, I suppose.
> 
> At least I was a bit more optimistic that we'd get a decent json plugin
> into care soon-ish.  While there was some initial work towards that, it
> unfortunately stalled at some point.
> 
> Euler, are you perchance interested in working towards that? ;)
> 

I am happy to review any work in this area BTW.

-- 
  Petr Jelinek  http://www.2ndQuadrant.com/
  PostgreSQL Development, 24x7 Support, Training & Services


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PG 10 release notes

[Andres Freund]

On 2017-05-04 18:23:38 -0700, Peter Geoghegan wrote:
> On Thu, May 4, 2017 at 6:08 PM, Robert Haas  wrote:
> >> E.g. to power amazon's data migration service (yes, that scares me).
> >
> > If you recall, I did predict prior to commit that test_decoding would
> > get put into production use regardless of the name.
> 
> I thought you were completely wrong when you said that. Lesson
> learned, I suppose.

At least I was a bit more optimistic that we'd get a decent json plugin
into care soon-ish.  While there was some initial work towards that, it
unfortunately stalled at some point.

Euler, are you perchance interested in working towards that? ;)

- Andres


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PG 10 release notes

[Peter Geoghegan]

On Thu, May 4, 2017 at 6:08 PM, Robert Haas  wrote:
>> E.g. to power amazon's data migration service (yes, that scares me).
>
> If you recall, I did predict prior to commit that test_decoding would
> get put into production use regardless of the name.

I thought you were completely wrong when you said that. Lesson
learned, I suppose.


-- 
Peter Geoghegan

VMware vCenter Server
https://www.vmware.com/


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PG 10 release notes

[Robert Haas]

On Thu, May 4, 2017 at 8:46 PM, Bruce Momjian  wrote:
> On Thu, May  4, 2017 at 05:09:40PM -0700, Andres Freund wrote:
>> > > I would not in any way refer to logical decoding as being only a proof
>> > > of concept, even before logical replication.
>> >
>> > The community ships a reliable logical _encoding_, and a test logical
>> > _decoding_.
>>
>> Yes, so what?  What you said is "I didn't think logical decoding was
>> really more than a proof-of-concept until now", which is plainly wrong,
>> given I know a significant number of users using it in production.  Some
>> of them are well known & large enterprises, and it's used to enable
>> critical things.
>
> I am getting tired of saying this.  When I am writing the release notes,
> I am trying to figure out how it affects our shipped code, and the only
> "decoding" I know of is test_decoding.

If you run 'git show --stat b89e151054a05f0f6d356ca52e3b725dd0505e53',
you will see that it includes a test_decoding module (which is a
sample logical decoding output plugin) and a tremendous pile of
changes to src/backend/replication/logical (which is the core logical
decoding infrastructure).  The latter is a larger volume of changes
than the former.  It would perhaps be fair to describe test_decoding
as a proof-of-concept, but it is not fair or correct to describe the
core infrastructure that way.  Anyway, they're separate things.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PG 10 release notes

[Andres Freund]

On 2017-05-04 21:08:41 -0400, Robert Haas wrote:
> On Thu, May 4, 2017 at 8:09 PM, Andres Freund  wrote:
> > On Mon, May  1, 2017 at 08:02:46AM -0400, Robert Haas wrote:
> >> Even test_decoding is (perhaps regrettably) being used to build production 
> >> solutions.
> >
> > E.g. to power amazon's data migration service (yes, that scares me).
>
> If you recall, I did predict prior to commit that test_decoding would
> get put into production use regardless of the name.

Yea.  I think we really should add a generic json plugin into core, I've
seen three (I think) distinct versions deployed into production now. One
of them derived from Euler's https://github.com/eulerto/wal2json, two
apparently developed largely from scratch (more efficient schema handling).

- Andres


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PG 10 release notes

[Robert Haas]

On Thu, May 4, 2017 at 8:09 PM, Andres Freund  wrote:
> On Mon, May  1, 2017 at 08:02:46AM -0400, Robert Haas wrote:
>> Even test_decoding is (perhaps regrettably) being used to build production 
>> solutions.
>
> E.g. to power amazon's data migration service (yes, that scares me).

If you recall, I did predict prior to commit that test_decoding would
get put into production use regardless of the name.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PG 10 release notes

[Andres Freund]

On 2017-05-04 20:46:24 -0400, Bruce Momjian wrote:
> On Thu, May  4, 2017 at 05:09:40PM -0700, Andres Freund wrote:
> > > > I would not in any way refer to logical decoding as being only a proof
> > > > of concept, even before logical replication.
> > > 
> > > The community ships a reliable logical _encoding_, and a test logical
> > > _decoding_.
> > 
> > Yes, so what?  What you said is "I didn't think logical decoding was
> > really more than a proof-of-concept until now", which is plainly wrong,
> > given I know a significant number of users using it in production.  Some
> > of them are well known & large enterprises, and it's used to enable
> > critical things.
> 
> I am getting tired of saying this.  When I am writing the release notes,
> I am trying to figure out how it affects our shipped code, and the only
> "decoding" I know of is test_decoding.

Then ask for input instead of saying uninformed stuff like "I didn't
think logical decoding was really more than a proof-of-concept".  That's
what people are complaining about, not that you're not
all-knowledgeable.  I agree with you - and wrote that previously - that
the specific change is not necessarily worthwhile to be mentioned on its
own, I don't see how that makes your various statements in this
subthread ok.


My message was this:
>   
> https://www.postgresql.org/message-id/a6d13cf7-fbf8-913c-2353-f149c6f95fdc%402ndquadrant.com
>   > 
>   >> Or the ability of logical decoding to follow timeline switches.
>   > 
>   > I didn't think logical decoding was really more than a 
> proof-of-concept
>   > until now.
> 
> Now, if the "encoding"

There's not really a "encoding" part of this. The decoding refers to
decoding the WAL into something externally usable.


> I assume it is this commit:

That, and
commit 24c5f1a103ce6656a5cb430d9a996c34e61ab2a5
Author: Alvaro Herrera 
Date:   2016-03-30 20:07:05 -0300

Enable logical slots to follow timeline switches

When decoding from a logical slot, it's necessary for xlog reading to be
able to read xlog from historical (i.e. not current) timelines;
otherwise, decoding fails after failover, because the archives are in
the historical timeline.  This is required to make "failover logical
slots" possible; it currently has no other use, although theoretically
it could be used by an extension that creates a slot on a standby and
continues to replay from the slot when the standby is promoted.

This commit includes a module in src/test/modules with functions to
manipulate the slots (which is not otherwise possible in SQL code) in
order to enable testing, and a new test in src/test/recovery to ensure
that the behavior is as expected.

Author: Craig Ringer
Reviewed-By: Oleksii Kliukin, Andres Freund, Petr Jelínek


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PG 10 release notes

[Robert Haas]

On Thu, May 4, 2017 at 7:56 PM, Bruce Momjian  wrote:
> The community ships a reliable logical _encoding_, and a test logical
> _decoding_.

As far as I am aware, logical encoding is a term you just made up,
because it's not referenced anywhere in the commit log or the source
tree, unlike logical decoding, which is referenced many times.

[rhaas pgsql]$ git log --grep='logical decoding' --oneline | wc -l
  59
[rhaas pgsql]$ git log --grep='logical encoding' --oneline | wc -l
   0
[rhaas pgsql]$ git grep 'logical decoding' | wc -l
 242
[rhaas pgsql]$ git grep 'logical encoding' | wc -l
   0

> This came up from discussion related to this item:
>
> the ability of logical decoding to follow timeline switches
>
> My point was that based on the text it is test_decoding that can do
> timeline switches, and is that significant enough to mention in the
> release notes?

Actually, that commit message says nothing about test_decoding.  You
seem to be confusing a logical decoding output plugin (of which
test_decoding is an example) with the core logical decoding facilities
(which are what got modified by this commit).  As Andres said, that's
like confusing postgres_fdw with the fact that PostgreSQL supports
foreign data wrappers.

> You can have all the emotional reactions you want.

I'll try to avoid emotional reactions on the list, but I think you
should try to understand how the features work if you're going to
write the release notes.  The email to which I responded was factually
incorrect, and what you said in the email to which I'm replying now
was, too.  If you want to keep insisting otherwise, you have that
right.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PG 10 release notes

[Bruce Momjian]

On Thu, May  4, 2017 at 05:09:40PM -0700, Andres Freund wrote:
> > > I would not in any way refer to logical decoding as being only a proof
> > > of concept, even before logical replication.
> > 
> > The community ships a reliable logical _encoding_, and a test logical
> > _decoding_.
> 
> Yes, so what?  What you said is "I didn't think logical decoding was
> really more than a proof-of-concept until now", which is plainly wrong,
> given I know a significant number of users using it in production.  Some
> of them are well known & large enterprises, and it's used to enable
> critical things.

I am getting tired of saying this.  When I am writing the release notes,
I am trying to figure out how it affects our shipped code, and the only
"decoding" I know of is test_decoding.  My message was this:


https://www.postgresql.org/message-id/a6d13cf7-fbf8-913c-2353-f149c6f95fdc%402ndquadrant.com
> 
>> Or the ability of logical decoding to follow timeline switches.
> 
> I didn't think logical decoding was really more than a 
proof-of-concept
> until now.

Now, if the "encoding" was changed so the external decoders can do more,
or the API we supply to the decoders has new abilities, that is fine to
mention, but someone needs to say that.  That is an "encoding" change,
right?

I said nothing about external decoders, only the user-interface decoder
we ship.

I assume it is this commit:

Author: Simon Riggs 
2017-03-22 [1148e22a8] Teach xlogreader to follow timeline switches

Teach xlogreader to follow timeline switches

Uses page-based mechanism to ensure we’re using the correct 
timeline.

Tests are included to exercise the functionality using a cold 
disk-level copy
of the master that's started up as a replica with slots intact, but 
the
intended use of the functionality is with later features.

Craig Ringer, reviewed by Simon Riggs and Andres Freund

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

+ As you are, so once was I.  As I am, so you will be. +
+  Ancient Roman grave inscription +


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PG 10 release notes

[Andres Freund]

On 2017-05-04 19:56:21 -0400, Bruce Momjian wrote:
> On Mon, May  1, 2017 at 08:02:46AM -0400, Robert Haas wrote:
> This came up from discussion related to this item:
> 
>   the ability of logical decoding to follow timeline switches
> 
> My point was that based on the text it is test_decoding that can do
> timeline switches, and is that significant enough to mention in the
> release notes?

FWIW, the relevant commit neither mentions nor modifies test_decoding
(It does add tests that *use* test_decoding, but that's it). Nor did
Petr's comment reference test_decoding.  So I don't see how "based on
the text it is test_decoding" follows.

- Andres


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PG 10 release notes

[Bruce Momjian]

On Thu, May  4, 2017 at 07:01:17PM -0300, Alvaro Herrera wrote:
> Thanks for doing this, looks great.  A few notes:
> 
>   
>
>   
>Add the ability to compute a correlation ratio and the number of
>distinct values on several columns (Tomas Vondra, David Rowley)
>   
> 
> I think this should be worded in terms of "extended data statistics" or
> such.  I think your proposed wording is a bit obscure.  How about
> something like "Add extended statistics to improve query planning".
> Also, I'd add myself as co-author, with Tomas' permission.

I have adjusted the text to add your term, and added your name:

   Add multi-column optimizer statistics to compute the correlation
   ratio and number of distinct values (Tomas Vondra, David Rowley,
   lvaro Herrera)

I think we have to mention the exact statistics collected because we
know that they are of limited usefulness (per Tomas) and full
multi-column statistics are needed and hopefully coming in PG 11.  If we
don't mention the details I am afraid people will be disappointed with PG
10 and will not be excited when they are more powerful in PG 11.  Any
better wording?

I will work on the other items you posted shortly.

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

+ As you are, so once was I.  As I am, so you will be. +
+  Ancient Roman grave inscription +


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PG 10 release notes

[Bruce Momjian]

On Thu, May  4, 2017 at 06:02:58PM -0300, Alvaro Herrera wrote:
> > I can't see how this can be added to an existing BRIN entry, so it would
> > have to be new.  The text would be:
> > 
> > Improve accuracy in determining if a BRIN index scan is beneficial
> > 
> > though this not something I would normally mention becuause most users
> > don't understand the optimizer choices and just assume it works.
> 
> The problem is that previously it was possible for a BRIN index to be
> created, and cause some queries to choose it which were faster using
> some other index (a btree).  The point is that with this change it
> becomes more credible to create BRIN indexes without fear that such
> queries are going to slow down.
> 
> Your proposed text sounds good to me.  Authors would be David Rowley and
> Emre Hasegeli.

OK, added, thanks:


 
+
+ Improve accuracy in determining if a BRIN index scan
+ is beneficial (David Rowley, Emre Hasegeli)
+
+   

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

+ As you are, so once was I.  As I am, so you will be. +
+  Ancient Roman grave inscription +


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PG 10 release notes

[Andres Freund]

On 2017-05-04 19:56:21 -0400, Bruce Momjian wrote:
> On Mon, May  1, 2017 at 08:02:46AM -0400, Robert Haas wrote:
> > On Tue, Apr 25, 2017 at 11:01 AM, Bruce Momjian  wrote:
> > >> Or the ability of logical decoding to follow timeline switches.
> > >
> > > I didn't think logical decoding was really more than a proof-of-concept
> > > until now.
> > 
> > /me searches for jaw on floor.
> > 
> > It sounds like you don't understand how logical decoding works.  There
> > are plugins -- fairly widely used, I think -- like
> > https://github.com/confluentinc/bottledwater-pg and
> > https://github.com/eulerto/wal2json which use the in-core
> > infrastructure to do very nifty things, much like there are foreign
> > data wrappers other than postgres_fdw.  Even test_decoding is (perhaps
> > regrettably) being used to build production solutions.  The point is
> > that most of the logic is in core; test_decoding or bottlewater or
> > wal2json are just small plugins that tap into that infrastructure.
> > 
> > I would not in any way refer to logical decoding as being only a proof
> > of concept, even before logical replication.
> 
> The community ships a reliable logical _encoding_, and a test logical
> _decoding_.

Yes, so what?  What you said is "I didn't think logical decoding was
really more than a proof-of-concept until now", which is plainly wrong,
given I know a significant number of users using it in production.  Some
of them are well known & large enterprises, and it's used to enable
critical things.


On Mon, May  1, 2017 at 08:02:46AM -0400, Robert Haas wrote:
> Even test_decoding is (perhaps regrettably) being used to build production 
> solutions.

E.g. to power amazon's data migration service (yes, that scares me).


> My point was that based on the text it is test_decoding that can do
> timeline switches, and is that significant enough to mention in the
> release notes?  Now, if it is that logical "encoding" now allows
> external logical decoding modules to handle timeline switches, that is
> different, but no one has said that yet.

The change has nothing to do with test_decoding.


Petr: The timeline change itself does, for the moment, not seem
particularly noteworthy to me - it's not really useful atm on its own?
To me it's more of infrastructure to add "logical decoding on standby"
next release?


> You can have all the emotional reactions you want.

Nice one.


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PG 10 release notes

[Bruce Momjian]

On Mon, May  1, 2017 at 10:20:38AM -0400, Robert Haas wrote:
> I'm pretty sure this is not the first year in which your policy of
> excluding certain performance-related items has met with opposition.
> I agree that there are some improvements that are sufficiently small
> and boring that they do not merit a mention, but (1) that's also true
> of non-performance items and (2) the fact that people keep complaining
> about performance items you excluded constitutes a discussion of
> changing your filter.
> 
> My own opinion is that the filter should not be particularly different
> for performance items vs. non-performance.  The question ought to be
> whether the change is significant enough that users are likely to
> care.  If you've got several people saying "hey, you forgot NN in
> the release notes!" it is probably a good bet that the change is
> significant enough that users will care about it.

Yes, the "do people care" filter is what I usually use.  When new
functionality is added, we usually mention it because someone usually
care, but for performance, the threshold is usually whether workloads,
even rare ones, would have a visible performance change.  It is
difficult to determine this from the release notes, which is why I
always need feedback on these items.

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

+ As you are, so once was I.  As I am, so you will be. +
+  Ancient Roman grave inscription +


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PG 10 release notes

[Bruce Momjian]

On Mon, May  1, 2017 at 08:02:46AM -0400, Robert Haas wrote:
> On Tue, Apr 25, 2017 at 11:01 AM, Bruce Momjian  wrote:
> >> Or the ability of logical decoding to follow timeline switches.
> >
> > I didn't think logical decoding was really more than a proof-of-concept
> > until now.
> 
> /me searches for jaw on floor.
> 
> It sounds like you don't understand how logical decoding works.  There
> are plugins -- fairly widely used, I think -- like
> https://github.com/confluentinc/bottledwater-pg and
> https://github.com/eulerto/wal2json which use the in-core
> infrastructure to do very nifty things, much like there are foreign
> data wrappers other than postgres_fdw.  Even test_decoding is (perhaps
> regrettably) being used to build production solutions.  The point is
> that most of the logic is in core; test_decoding or bottlewater or
> wal2json are just small plugins that tap into that infrastructure.
> 
> I would not in any way refer to logical decoding as being only a proof
> of concept, even before logical replication.

The community ships a reliable logical _encoding_, and a test logical
_decoding_.

This came up from discussion related to this item:

the ability of logical decoding to follow timeline switches

My point was that based on the text it is test_decoding that can do
timeline switches, and is that significant enough to mention in the
release notes?  Now, if it is that logical "encoding" now allows
external logical decoding modules to handle timeline switches, that is
different, but no one has said that yet.

You can have all the emotional reactions you want.

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

+ As you are, so once was I.  As I am, so you will be. +
+  Ancient Roman grave inscription +


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PG 10 release notes

[Bruce Momjian]

On Fri, Apr 28, 2017 at 01:12:34PM +0900, Masahiko Sawada wrote:
> On Tue, Apr 25, 2017 at 10:31 AM, Bruce Momjian  wrote:
> > I have committed the first draft of the Postgres 10 release notes.  They
> > are current as of two days ago, and I will keep them current.  Please
> > give me any feedback you have.
> >
> 
> Related to a following item in release note, oldestxmin is also
> reported by VACUUM VERBOSE and autovacuum , which is introduced by
> commit 9eb344faf54a849898d9be012ddfa8204cfeb57c (author is Simon).
> 
> * Have VACUUM VERBOSE report the number of skipped frozen pages
> (Masahiko Sawada)
> 
> Could we add this item to the release note?

OK, adjusted text below and commit added above this item:

   Have VACUUM VERBOSE report
   the number of skipped frozen pages and oldest xmin (Masahiko Sawada)

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

+ As you are, so once was I.  As I am, so you will be. +
+  Ancient Roman grave inscription +


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PG 10 release notes

[Bruce Momjian]

On Thu, Apr 27, 2017 at 04:05:09PM +0100, Dagfinn Ilmari Mannsåker wrote:
> Bruce Momjian  writes:
> 
> > I have committed the first draft of the Postgres 10 release notes.  They
> > are current as of two days ago, and I will keep them current.  Please
> > give me any feedback you have.
> 
> I noticed a few niggles with the links in "my" entires, patch attached.

Thanks, applied.

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

+ As you are, so once was I.  As I am, so you will be. +
+  Ancient Roman grave inscription +


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PG 10 release notes

[Bruce Momjian]

On Thu, Apr 27, 2017 at 03:04:57PM +0200, Daniel Verite wrote:
>   Fabien COELHO wrote:
> 
> >Fix psql \p to always print what would be executed by \g or \w (Daniel
> >Vérité)
> > 
> >Previously \p didn't properly print the reverted-to command after a
> >buffer contents reset. CLARIFY?
> > 
> > The fix is linked to the change introduced by Tom when 
> > refactoring/cleaning up in e984ef586 (\if) which change psql's \p 
> > behavior.
> 
> That's my recollection as well. The "Previously" does not refer to 9.6,
> but to that commit.

Yes, adjusted.

> > I'm not sure how this should appear in the release notes. Maybe not at 
> > all, associated to the feature in which the behavioral change was 
> > introduced...
> 
> There is small change of behavior coming as a by-product of the
> introduction of  /if.../endif blocks.
> 
> When doing in 9.x:
> 
> select '1st buffer' \g
> followed by \e
> and editing with select '2nd buffer' (without ending the query)
> and then back in psql doing '\r' and '\p', the result is
> select '2nd buffer'
> 
> The same with v10 leads instead to
> select '1st buffer'
> 
> I'm not sure whether it's above the level of detail worth being mentioned
> in the release notes.

Wow, I am not sure how to even explain that.  ;-)  Let's see if we get
any user feedback on the change.

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

+ As you are, so once was I.  As I am, so you will be. +
+  Ancient Roman grave inscription +


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PG 10 release notes

[Bruce Momjian]

On Thu, Apr 27, 2017 at 10:43:34AM +0200, Fabien COELHO wrote:
> 
> Hello Bruce,
> 
> >I have committed the first draft of the Postgres 10 release notes.  They
> >are current as of two days ago, and I will keep them current.  Please
> >give me any feedback you have.
> 
> About:
> 
> """
>   Fix psql \p to always print what would be executed by \g or \w (Daniel
>   Vérité)
> 
>   Previously \p didn't properly print the reverted-to command after a
>   buffer contents reset. CLARIFY?
> """
> 
> The fix is linked to the change introduced by Tom when refactoring/cleaning
> up in e984ef586 (\if) which change psql's \p behavior.
> 
> This is not a user visible change version-to-version, it is more like a bug
> fix for a bug/behavioral change introduced within pg10 developement process
> itself.
> 
> I'm not sure how this should appear in the release notes. Maybe not at all,
> associated to the feature in which the behavioral change was introduced...

Agreed, I removed the item and moved those commits to the \if item.

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

+ As you are, so once was I.  As I am, so you will be. +
+  Ancient Roman grave inscription +


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PG 10 release notes

[Bruce Momjian]

On Thu, Apr 27, 2017 at 10:21:44AM +0500, Andrew Borodin wrote:
> Hi, Bruce!
> 
> 2017-04-25 6:31 GMT+05:00 Bruce Momjian :
> > The only unusual thing is that this release has ~180 items while most
> > recent release have had ~220.  The pattern I see that there are more
> > large features in this release than previous ones.
> 
> I'm not sure, but, probably, it worth mentioning this [1,2] in the
> E.1.3.1.2. Indexes section.
> Better tuple management on GiST page allows faster inserts and
> updates. (In numbers: 15% for randomized data,3x for ordered data in
> specific cases)
> 
> [1] https://commitfest.postgresql.org/10/661/
> [2] 
> https://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=b1328d78f88cdf4f7504004159e530b776f0de16

OK, got it.  Here is the new item:

   Author: Tom Lane 
   2016-09-09 [b1328d78f] Invent PageIndexTupleOverwrite, and
   teach BRIN and GiST

   Allow faster GiST inserts and updates by reusing
   index space more efficiently (Andrey Borodin)

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

+ As you are, so once was I.  As I am, so you will be. +
+  Ancient Roman grave inscription +


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PG 10 release notes

[Andres Freund]

On 2017-05-04 17:33:13 -0500, Merlin Moncure wrote:
> On Mon, May 1, 2017 at 7:02 AM, Robert Haas  wrote:
> > On Tue, Apr 25, 2017 at 11:01 AM, Bruce Momjian  wrote:
> >> I didn't think logical decoding was really more than a proof-of-concept
> >> until now.
> >
> > /me searches for jaw on floor.

Yea, this kind of argument is getting really old.  People, including
Robert and I in this thread, have spent tremendous amounts of work on
it.  Not fun to just get that discounted.


> > I would not in any way refer to logical decoding as being only a proof
> > of concept, even before logical replication.
> 
> That's fair, but I think I understand what Bruce was going for here.
> Data point: github third party modules are generally not approved for
> deployment in my organization so logical decoding from a production
> perspective does not exist (for me) until 10.0.  Point being, an
> important threshold has been crossed.

By that argument the extension APIs, which after all are what external
FDWs, external index types, postgis, and other extensions use, aren't a
feature of postgres.  Some sites not being able to use external
extensions is *one* argument for building some things into core, but
that doesn't mean the extension APIs don't exist or aren't features.

- Andres


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PG 10 release notes

[Andres Freund]

On 2017-04-25 15:29:01 -0400, Bruce Momjian wrote:
> Uh, the only logical decoding code that I know we ship pre-PG 10 is
> contrib/test_decoding/.

That's completely wrong.  src/backend/replication/logical/ is a a bit
bigger than that...

- Andres


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PG 10 release notes

[Merlin Moncure]

On Mon, May 1, 2017 at 7:02 AM, Robert Haas  wrote:
> On Tue, Apr 25, 2017 at 11:01 AM, Bruce Momjian  wrote:
>> I didn't think logical decoding was really more than a proof-of-concept
>> until now.
>
> /me searches for jaw on floor.
>
> I would not in any way refer to logical decoding as being only a proof
> of concept, even before logical replication.

That's fair, but I think I understand what Bruce was going for here.
Data point: github third party modules are generally not approved for
deployment in my organization so logical decoding from a production
perspective does not exist (for me) until 10.0.  Point being, an
important threshold has been crossed.

merlin


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PG 10 release notes

[Alvaro Herrera]

Thanks for doing this, looks great.  A few notes:

  
   
  
   Add the ability to compute a correlation ratio and the number of
   distinct values on several columns (Tomas Vondra, David Rowley)
  

I think this should be worded in terms of "extended data statistics" or
such.  I think your proposed wording is a bit obscure.  How about
something like "Add extended statistics to improve query planning".
Also, I'd add myself as co-author, with Tomas' permission.

  
   
   
Cause BRIN index summarization to happen more
aggressively (lvaro Herrera)
   

   
Specifically, summarize the previous page range when a new page
range is created.
   
  

I think it should be pointed out that this is optional and defaults to
off.  Maybe start with "add option to ..."  (I wonder if it's worth
creating a linkable area in the docs that explain this feature, so that
we could have a link in the relnotes entry).


  
   
   
Add function brin_desummarize_range() to remove
BRIN summarization of a specified range (lvaro
Herrera)
   

   
This allows future BRIN index summarization to be
more compact.  CLARIFY
   
  

The point of this change is that if data has changed (delete, update) in
a way that could use tighter min/max limits, it would be worthwhile to
remove the previous BRIN tuple so that a new one is created so that
future scans can skip scanning the range.  Maybe word it something like
"This is useful if UPDATE and DELETE have changed the data to tighter
limits; a future BRIN index entry will be more accurate"?


 
  
  
   Add support for converting XML-formatted data into a row
   set (Pavel Stehule, lvaro Herrera)
  
XMLTABLE is a sql-standard-specified construct, so we should mention it
by name in the leading paragraph more prominently ("add support for the
XMLTABLE standard feature" or something like that); also, I think it
should be in the Queries section, not Functions.


   
Add GUC 
to limit the number of worker processes that can be used for
parallelism (Julien Rouhaud)
   

   
This can be set lower than  to reserve worker processes
for non-parallel purposes.
   

Strange last phrase.  I suggest "...to reserve worker processes for
purposes other than parallel queries".  Also in the leading para, "for
parallelism" should probably be "for query parallelism".


I think commit e306df7f9 ("Full Text Search support for JSON
and JSONB") does definitely not belong to section Indexes; I
think Data Types is a better fit.


I think commits 67dc4ccbb and 1753b1b02 (pg_sequence and pg_sequences)
should be listed in the opposite order.  Maybe merge them into one?


   
This proves better security than the existing md5
negotiation and storage method.
   
You probably mean "provides" not "proves".


   
Allow SSL configuration to be updated at
SIGHUP (Andreas Karlsson, Tom Lane)
   
SIGHUP seems much too technical.  How about "... to be updated during
configurating reload"


I think the entry for commits a734fd5d1 and dafa0848d does not belong in
the reliability section.  In fact I wonder if it's necessary to list
this one at all.



 Increase the maximum configurable WAL size to 1
 gigabyte (Beena Emerson)

"WAL segment size" perhaps, and note that this is useful to reduce
frequency of archive_command?


   
Also add -nosync option to disable fsync.
   
Misspelled --no-sync.


   
Add log options for pg_ctl wait (--wait)
and no-wait (--no-wait) (Vik Fearing)
   
Mispelled "long options".


-- 
Álvaro Herrerahttps://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PG 10 release notes

[Alvaro Herrera]

Bruce Momjian wrote:
> On Tue, Apr 25, 2017 at 04:03:53PM +1200, David Rowley wrote:
> >  ..On 25 April 2017 at 13:31, Bruce Momjian  wrote:
> > > The only unusual thing is that this release has ~180 items while most
> > > recent release have had ~220.  The pattern I see that there are more
> > > large features in this release than previous ones.
> > 
> > Thanks for drafting this up.
> > 
> > I understand that it may have been filtered out, but I'd say that
> > 7e534adcdc70 is worth a mention.
> > 
> > Users creating BRIN indexes previously would have had to know
> > beforehand that the table would be sufficiently correlated on the
> > indexed columns for the index to be worthwhile, whereas now there's a
> > lesser need for the user to know this beforehand.
> 
> I can't see how this can be added to an existing BRIN entry, so it would
> have to be new.  The text would be:
> 
>   Improve accuracy in determining if a BRIN index scan is beneficial
> 
> though this not something I would normally mention becuause most users
> don't understand the optimizer choices and just assume it works.

The problem is that previously it was possible for a BRIN index to be
created, and cause some queries to choose it which were faster using
some other index (a btree).  The point is that with this change it
becomes more credible to create BRIN indexes without fear that such
queries are going to slow down.

Your proposed text sounds good to me.  Authors would be David Rowley and
Emre Hasegeli.

-- 
Álvaro Herrerahttps://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PG 10 release notes

[Alvaro Herrera]

Claudio Freire wrote:
> On Tue, Apr 25, 2017 at 2:45 PM, Bruce Momjian  wrote:

> > However, given your explanation, I have added the item:
> >
> >Improve speed of VACUUM's removal of trailing empty
> >heap pages (Alvaro Herrera)
> 
> That's enough for me, thanks.

Thanks!  I amended the entry by cons'ing Claudio's name as author.

-- 
Álvaro Herrerahttps://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PG 10 release notes

[Robert Haas]

On Mon, Apr 24, 2017 at 11:37 PM, Bruce Momjian  wrote:
>> I think that might also be because you skipped a few things that should
>> get their own entries.  I've not yet made a pass through your draft (and
>> won't for some days), but a quick search shows the draft to e.g miss:
>> b30d3ea824c5ccba43d3e942704f20686e7dbab8 - Add a macro templatized hashtable.
>> 75ae538bc3168bf44475240d4e0487ee2f3bb376 - Use more efficient hashtable for 
>> tidbitmap.c to speed up bitmap scans.
>> 5dfc198146b49ce7ecc8a1fc9d5e171fb75f6ba5 - Use more efficient hashtable for 
>> execGrouping.c to speed up hash aggregation.
>> fc4b3dea2950e4f6081f1ed2380f82c9efd672e0 - User narrower representative 
>> tuples in the hash-agg hashtable.
>> 8ed3f11bb045ad7a3607690be668dbd5b3cc31d7 - Perform one only projection to 
>> compute agg arguments.
>> (not that this needs to five entries)
>
> I remember seeing those and those are normally details I do not put in
> the release notes as there isn't a clear user experience change except
> "Postgres is faster".  Yeah, a bummer, and I can change my filter, but
> it would require discussion.

I'm pretty sure this is not the first year in which your policy of
excluding certain performance-related items has met with opposition.
I agree that there are some improvements that are sufficiently small
and boring that they do not merit a mention, but (1) that's also true
of non-performance items and (2) the fact that people keep complaining
about performance items you excluded constitutes a discussion of
changing your filter.

My own opinion is that the filter should not be particularly different
for performance items vs. non-performance.  The question ought to be
whether the change is significant enough that users are likely to
care.  If you've got several people saying "hey, you forgot NN in
the release notes!" it is probably a good bet that the change is
significant enough that users will care about it.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PG 10 release notes

[Joshua D. Drake]

On 05/01/2017 05:40 AM, Robert Haas wrote:

On Tue, Apr 25, 2017 at 9:56 PM, Bruce Momjian  wrote:

First, I don't think RFC references belong in the release notes, let
alone RFC links.


Why not?  I think RFC references are a great thing to include in the
release notes, and including links seems helpful, too.


I could see not including RFC references in a PR but release notes seem 
to be the perfect place for them.


JD


--
Command Prompt, Inc.  http://the.postgres.company/
+1-503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Everyone appreciates your honesty, until you are honest with them.


--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PG 10 release notes

[Robert Haas]

On Tue, Apr 25, 2017 at 9:56 PM, Bruce Momjian  wrote:
> First, I don't think RFC references belong in the release notes, let
> alone RFC links.

Why not?  I think RFC references are a great thing to include in the
release notes, and including links seems helpful, too.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PG 10 release notes

[Robert Haas]

On Tue, Apr 25, 2017 at 11:01 AM, Bruce Momjian  wrote:
>> Or the ability of logical decoding to follow timeline switches.
>
> I didn't think logical decoding was really more than a proof-of-concept
> until now.

/me searches for jaw on floor.

It sounds like you don't understand how logical decoding works.  There
are plugins -- fairly widely used, I think -- like
https://github.com/confluentinc/bottledwater-pg and
https://github.com/eulerto/wal2json which use the in-core
infrastructure to do very nifty things, much like there are foreign
data wrappers other than postgres_fdw.  Even test_decoding is (perhaps
regrettably) being used to build production solutions.  The point is
that most of the logic is in core; test_decoding or bottlewater or
wal2json are just small plugins that tap into that infrastructure.

I would not in any way refer to logical decoding as being only a proof
of concept, even before logical replication.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PG 10 release notes

[Masahiko Sawada]

On Tue, Apr 25, 2017 at 10:31 AM, Bruce Momjian  wrote:
> I have committed the first draft of the Postgres 10 release notes.  They
> are current as of two days ago, and I will keep them current.  Please
> give me any feedback you have.
>

Related to a following item in release note, oldestxmin is also
reported by VACUUM VERBOSE and autovacuum , which is introduced by
commit 9eb344faf54a849898d9be012ddfa8204cfeb57c (author is Simon).

* Have VACUUM VERBOSE report the number of skipped frozen pages
(Masahiko Sawada)

Could we add this item to the release note?

Regards,

--
Masahiko Sawada
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PG 10 release notes

[Amit Kapila]

On Thu, Apr 27, 2017 at 7:53 PM, Marko Tiikkaja  wrote:
> On Thu, Apr 27, 2017 at 4:13 PM, Bruce Momjian  wrote:
>>
>> On Thu, Apr 27, 2017 at 08:00:28AM +0530, Amit Kapila wrote:
>> > > Oh, so non-correlated subqueries can be run in parallel.  Yes, that is
>> > > something we should have in the release notes.  How is this?
>> > >
>> > > Author: Robert Haas 
>> > > 2017-02-14 [5e6d8d2bb] Allow parallel workers to execute
>> > > subplans.
>> > >
>> > > Allow non-correlated subqueries to be run in parallel (Amit
>> > > Kapila)
>> > >
>> >
>> > Looks good to me.
>
>
> It's not clear from this item what the previous behavior was.  How about
> adding something like "Correlated subqueries can not yet be parallelized."?
> (I'm guessing that's the change, anyway, please correct me if I'm mistaken.)
>

Previous to this change queries containing a reference to subplans
(correlated or un-correlated) doesn't get parallelised.



-- 
With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PG 10 release notes

[Dagfinn Ilmari Mannsåker]

Bruce Momjian  writes:

> I have committed the first draft of the Postgres 10 release notes.  They
> are current as of two days ago, and I will keep them current.  Please
> give me any feedback you have.

I noticed a few niggles with the links in "my" entires, patch attached.

Cheers,

ilmari

-- 
"A disappointingly low fraction of the human race is,
 at any given time, on fire." - Stig Sandbeck Mathisen

>From a32336f420a4fb25bdedeb0dc8ccf68503638e93 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Dagfinn=20Ilmari=20Manns=C3=A5ker?= 
Date: Thu, 27 Apr 2017 15:20:59 +0100
Subject: [PATCH] Fix some release note links

- Make max_pred_locks_per_page a link too
- The ALTER TYPE link was pointing to the ALTER TABLE page
---
 doc/src/sgml/release-10.sgml | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/doc/src/sgml/release-10.sgml b/doc/src/sgml/release-10.sgml
index de2129758c..6f49618d07 100644
--- a/doc/src/sgml/release-10.sgml
+++ b/doc/src/sgml/release-10.sgml
@@ -665,7 +665,7 @@

 The new settings are  and
-   max_pred_locks_per_page.
+.

   
 
@@ -1830,7 +1830,7 @@
   
 
   
-   This uses the syntax ALTER
+   This uses the syntax ALTER
TYPE ... RENAME VALUE.
   
  
-- 
2.11.0


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PG 10 release notes

[Marko Tiikkaja]

On Thu, Apr 27, 2017 at 4:13 PM, Bruce Momjian  wrote:

> On Thu, Apr 27, 2017 at 08:00:28AM +0530, Amit Kapila wrote:
> > > Oh, so non-correlated subqueries can be run in parallel.  Yes, that is
> > > something we should have in the release notes.  How is this?
> > >
> > > Author: Robert Haas 
> > > 2017-02-14 [5e6d8d2bb] Allow parallel workers to execute
> subplans.
> > >
> > > Allow non-correlated subqueries to be run in parallel (Amit
> Kapila)
> > >
> >
> > Looks good to me.


It's not clear from this item what the previous behavior was.  How about
adding something like "Correlated subqueries can not yet be parallelized."?
(I'm guessing that's the change, anyway, please correct me if I'm mistaken.)


.m

Re: [HACKERS] PG 10 release notes

[Bruce Momjian]

On Thu, Apr 27, 2017 at 08:14:34AM +0530, Amit Kapila wrote:
> 
> 
> 
>  Improve efficiency of hash index growth (Amit Kapila, Mithun Cy)
> 
>  
> 
> The first two commits b0f18cb77, 30df93f69 are done as preliminary
> work to "Add write-ahead logging support to hash indexes", so it seems
> inappropriate to add them here.  We can add it along with below item:
> 
>  
> 
>  Add write-ahead logging support to hash indexes (Amit Kapila)
> 

OK, commits moved.

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

+ As you are, so once was I.  As I am, so you will be. +
+  Ancient Roman grave inscription +


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PG 10 release notes

[Bruce Momjian]

On Thu, Apr 27, 2017 at 08:00:28AM +0530, Amit Kapila wrote:
> > Oh, so non-correlated subqueries can be run in parallel.  Yes, that is
> > something we should have in the release notes.  How is this?
> >
> > Author: Robert Haas 
> > 2017-02-14 [5e6d8d2bb] Allow parallel workers to execute subplans.
> >
> > Allow non-correlated subqueries to be run in parallel (Amit Kapila)
> >
> 
> Looks good to me.

OK, added.

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

+ As you are, so once was I.  As I am, so you will be. +
+  Ancient Roman grave inscription +


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PG 10 release notes

[Daniel Verite]

Fabien COELHO wrote:

>Fix psql \p to always print what would be executed by \g or \w (Daniel
>Vérité)
> 
>Previously \p didn't properly print the reverted-to command after a
>buffer contents reset. CLARIFY?
> 
> The fix is linked to the change introduced by Tom when 
> refactoring/cleaning up in e984ef586 (\if) which change psql's \p 
> behavior.

That's my recollection as well. The "Previously" does not refer to 9.6,
but to that commit.

> I'm not sure how this should appear in the release notes. Maybe not at 
> all, associated to the feature in which the behavioral change was 
> introduced...

There is small change of behavior coming as a by-product of the
introduction of  /if.../endif blocks.

When doing in 9.x:

select '1st buffer' \g
followed by \e
and editing with select '2nd buffer' (without ending the query)
and then back in psql doing '\r' and '\p', the result is
select '2nd buffer'

The same with v10 leads instead to
select '1st buffer'

I'm not sure whether it's above the level of detail worth being mentioned
in the release notes.


Best regards,
-- 
Daniel Vérité
PostgreSQL-powered mailer: http://www.manitou-mail.org
Twitter: @DanielVerite


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PG 10 release notes

[Fabien COELHO]

Hello Bruce,


I have committed the first draft of the Postgres 10 release notes.  They
are current as of two days ago, and I will keep them current.  Please
give me any feedback you have.


About:

"""
  Fix psql \p to always print what would be executed by \g or \w (Daniel
  Vérité)

  Previously \p didn't properly print the reverted-to command after a
  buffer contents reset. CLARIFY?
"""

The fix is linked to the change introduced by Tom when 
refactoring/cleaning up in e984ef586 (\if) which change psql's \p 
behavior.


This is not a user visible change version-to-version, it is more like a 
bug fix for a bug/behavioral change introduced within pg10 developement 
process itself.


I'm not sure how this should appear in the release notes. Maybe not at 
all, associated to the feature in which the behavioral change was 
introduced...


--
Fabien.
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PG 10 release notes

[Andrew Borodin]

Hi, Bruce!

2017-04-25 6:31 GMT+05:00 Bruce Momjian :
> The only unusual thing is that this release has ~180 items while most
> recent release have had ~220.  The pattern I see that there are more
> large features in this release than previous ones.

I'm not sure, but, probably, it worth mentioning this [1,2] in the
E.1.3.1.2. Indexes section.
Better tuple management on GiST page allows faster inserts and
updates. (In numbers: 15% for randomized data,3x for ordered data in
specific cases)

[1] https://commitfest.postgresql.org/10/661/
[2] 
https://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=b1328d78f88cdf4f7504004159e530b776f0de16

Best regards, Andrey Borodin.


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PG 10 release notes

[Amit Kapila]

On Tue, Apr 25, 2017 at 8:59 AM, Bruce Momjian  wrote:
> On Mon, Apr 24, 2017 at 11:05:41PM -0400, Bruce Momjian wrote:
>> > I think the above commit needs a separate mention, as this is a really
>> > huge step forward to control the size of hash indexes.
>>
>> Yes, it is unfotunate that the item is in the incompatibility item.  I
>> wonder if I should split out the need to rebuild the hash indexes and
>> keep it there and move this item into the "Index" section.
>
> Done, items split.
>




 Improve efficiency of hash index growth (Amit Kapila, Mithun Cy)

 

The first two commits b0f18cb77, 30df93f69 are done as preliminary
work to "Add write-ahead logging support to hash indexes", so it seems
inappropriate to add them here.  We can add it along with below item:

 

 Add write-ahead logging support to hash indexes (Amit Kapila)




-- 
With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PG 10 release notes

[Amit Kapila]

On Wed, Apr 26, 2017 at 8:46 PM, Bruce Momjian  wrote:
> On Wed, Apr 26, 2017 at 07:38:05PM +0530, Amit Kapila wrote:
>> >> I have already mentioned the commit id (5e6d8d2b).  Text can be "Allow
>> >> queries containing subplans to execute in parallel".  We should also
>> >> mention in some way that this applies only when the query contains
>> >> uncorrelated subplan.
>> >
>> > Sorry but I don't know what that means, and if I don't know, others
>> > might not either.
>> >
>>
>> regression=# explain (costs off) select count(*) from tenk1 where
>> (two, four) not in (select hundred, thousand from tenk2 where
>> thousand > 100);
>>   QUERY PLAN
>> --
>>  Finalize Aggregate
>>->  Gather
>>  Workers Planned: 2
>>  ->  Partial Aggregate
>>->  Parallel Seq Scan on tenk1
>>  Filter: (NOT (hashed SubPlan 1))
>>  SubPlan 1
>>->  Seq Scan on tenk2
>>  Filter: (thousand > 100)
>> (9 rows)
>>
>
> Oh, so non-correlated subqueries can be run in parallel.  Yes, that is
> something we should have in the release notes.  How is this?
>
> Author: Robert Haas 
> 2017-02-14 [5e6d8d2bb] Allow parallel workers to execute subplans.
>
> Allow non-correlated subqueries to be run in parallel (Amit Kapila)
>

Looks good to me.


-- 
With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PG 10 release notes

[Bruce Momjian]

On Wed, Apr 26, 2017 at 07:38:05PM +0530, Amit Kapila wrote:
> >> I have already mentioned the commit id (5e6d8d2b).  Text can be "Allow
> >> queries containing subplans to execute in parallel".  We should also
> >> mention in some way that this applies only when the query contains
> >> uncorrelated subplan.
> >
> > Sorry but I don't know what that means, and if I don't know, others
> > might not either.
> >
> 
> regression=# explain (costs off) select count(*) from tenk1 where
> (two, four) not in (select hundred, thousand from tenk2 where
> thousand > 100);
>   QUERY PLAN
> --
>  Finalize Aggregate
>->  Gather
>  Workers Planned: 2
>  ->  Partial Aggregate
>->  Parallel Seq Scan on tenk1
>  Filter: (NOT (hashed SubPlan 1))
>  SubPlan 1
>->  Seq Scan on tenk2
>  Filter: (thousand > 100)
> (9 rows)
> 

Oh, so non-correlated subqueries can be run in parallel.  Yes, that is
something we should have in the release notes.  How is this?

Author: Robert Haas 
2017-02-14 [5e6d8d2bb] Allow parallel workers to execute subplans.

Allow non-correlated subqueries to be run in parallel (Amit Kapila)

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

+ As you are, so once was I.  As I am, so you will be. +
+  Ancient Roman grave inscription +


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PG 10 release notes

[Amit Kapila]

On Tue, Apr 25, 2017 at 7:19 PM, Bruce Momjian  wrote:
> On Tue, Apr 25, 2017 at 09:00:45AM +0530, Amit Kapila wrote:
>> On Tue, Apr 25, 2017 at 8:35 AM, Bruce Momjian  wrote:
>> > On Tue, Apr 25, 2017 at 08:30:50AM +0530, Amit Kapila wrote:
>> >> On Tue, Apr 25, 2017 at 7:01 AM, Bruce Momjian  wrote:
>> >> > I have committed the first draft of the Postgres 10 release notes.  They
>> >> > are current as of two days ago, and I will keep them current.  Please
>> >> > give me any feedback you have.
>> >> >
>> >>
>> >> Some of the items which I feel could be added:
>> >>
>> >> 5e6d8d2bbbcace304450b309e79366c0da4063e4
>> >> Allow parallel workers to execute subplans.
>> >
>> > Uh, can you show me the commit on that and give some text ideas?
>> >
>>
>> I have already mentioned the commit id (5e6d8d2b).  Text can be "Allow
>> queries containing subplans to execute in parallel".  We should also
>> mention in some way that this applies only when the query contains
>> uncorrelated subplan.
>
> Sorry but I don't know what that means, and if I don't know, others
> might not either.
>

Let me try to explain by example:

Without this feature, the queries that refer subplans will be executed
serially like below:

regression=# explain (costs off) select count(*) from tenk1 where
(two, four) not in (select hundred, thousand from tenk2 where thousand
> 100);
QUERY PLAN
--
 Aggregate
   ->  Seq Scan on tenk1
 Filter: (NOT (hashed SubPlan 1))
 SubPlan 1
   ->  Seq Scan on tenk2
 Filter: (thousand > 100)
(6 rows)

After this feature, the queries that refer subplans can use parallel
plans like below:

regression=# explain (costs off) select count(*) from tenk1 where
(two, four) not in (select hundred, thousand from tenk2 where
thousand > 100);
  QUERY PLAN
--
 Finalize Aggregate
   ->  Gather
 Workers Planned: 2
 ->  Partial Aggregate
   ->  Parallel Seq Scan on tenk1
 Filter: (NOT (hashed SubPlan 1))
 SubPlan 1
   ->  Seq Scan on tenk2
 Filter: (thousand > 100)
(9 rows)


Now, it won't use parallelism if there is correlated subplan like below:

Seq Scan on t1
   Filter: (SubPlan 1)
   SubPlan 1
   ->  Result
 One-Time Filter: (t1.k = 0)
 ->  Parallel Seq Scan on t2

In this plan difference is that SubPlan refers to outer relation t1.

Do the above examples helps in understanding the feature?

-- 
With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PG 10 release notes

[Bruce Momjian]

On Wed, Apr 26, 2017 at 09:34:02AM +0530, Amit Kapila wrote:
> >> Just wondering if the mention of commit
> >> 0414b26bac09379a4cbf1fbd847d1cee2293c5e4 is missed? Not sure if this
> >> requires a separate entry or could be merged with -- Support parallel
> >> btree index scans.
> >
> > This item was merged into the general parallel index scan item:
> >
> 
> Okay, but then shouldn't we add Rafia's name as well to that release
> notes entry as she is the author of one of the merged item (commit
> 0414b26bac09379a4cbf1fbd847d1cee2293c5e4) which has an independent
> value.

Oh, yes, added.  I think in a quick look I saw Rafia Sabih as the same
as Rahila Syed and thought the name was already listed. Sorry to both of
them.

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

+ As you are, so once was I.  As I am, so you will be. +
+  Ancient Roman grave inscription +


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PG 10 release notes

[Bruce Momjian]

On Tue, Apr 25, 2017 at 09:56:58PM -0400, Bruce Momjian wrote:
> > SCRAM-SHA-256 improves deficiencies of MD5 password hashing by
> > preventing any kind of pass-the-hash vulnerabilities, where a user
> > would be able to connect to a PostgreSQL instance by just knowing the
> > hash of a password and not the password itself.
> 
> First, I don't think RFC references belong in the release notes, let
> alone RFC links.
> 
> Second, there seems to be some confusion over what SCRAM-SHA-256 gives
> us over MD5.  I think there are a few benefits:
> 
> o  packets cannot be replayed as easily, i.e. md5 replayed random salt
> packets with a 50% probability after 16k sessions
   ^^^

Sorry, it is after 64k sessions.  The rule of thumb is that if you
choose from 2^x random values, you have a 50% chance of repeating one
after 2^(x/2) numbers.

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

+ As you are, so once was I.  As I am, so you will be. +
+  Ancient Roman grave inscription +


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PG 10 release notes

[Amit Kapila]

On Tue, Apr 25, 2017 at 7:45 PM, Bruce Momjian  wrote:
> On Tue, Apr 25, 2017 at 09:38:55AM +0530, Rafia Sabih wrote:
>> On Tue, Apr 25, 2017 at 9:15 AM, Tom Lane  wrote:
>> > Andres Freund  writes:
>> >> On 2017-04-24 23:37:42 -0400, Bruce Momjian wrote:
>> >>> I remember seeing those and those are normally details I do not put in
>> >>> the release notes as there isn't a clear user experience change except
>> >>> "Postgres is faster".  Yeah, a bummer, and I can change my filter, but
>> >>> it would require discussion.
>> >
>> >> I think "postgres is faster" is one of the bigger user demands, so I
>> >> don't think that policy makes much sense.  A large number of the changes
>> >> over the next few releases will focus solely on that.  Nor do I think
>> >> past release notes particularly filtered such changes out.
>> >
>> > I think it has been pretty common to accumulate a lot of such changes
>> > into generic entries like, say, "speedups for hash joins".  More detail
>> > than that simply isn't useful to end users; and as a rule, our release
>> > notes are too long anyway.
>> >
>> > regards, tom lane
>> >
>> >
>> Just wondering if the mention of commit
>> 0414b26bac09379a4cbf1fbd847d1cee2293c5e4 is missed? Not sure if this
>> requires a separate entry or could be merged with -- Support parallel
>> btree index scans.
>
> This item was merged into the general parallel index scan item:
>

Okay, but then shouldn't we add Rafia's name as well to that release
notes entry as she is the author of one of the merged item (commit
0414b26bac09379a4cbf1fbd847d1cee2293c5e4) which has an independent
value.

-- 
With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PG 10 release notes

[Michael Paquier]

On Wed, Apr 26, 2017 at 10:56 AM, Bruce Momjian  wrote:
> First, I don't think RFC references belong in the release notes, let
> alone RFC links.
>
> Second, there seems to be some confusion over what SCRAM-SHA-256 gives
> us over MD5.  I think there are a few benefits:
>
> o  packets cannot be replayed as easily, i.e. md5 replayed random salt
> packets with a 50% probability after 16k sessions
> o  hard to re-use SCRAM-SHA-256 string if disclosed vs. simple for md5
> o  harder to brute-force trying all possible strings to find a matching
> hash
>
> So if you tell users that SCRAM-SHA-256 is better than MD5 only because
> of one of those, they will not realize that three benefits of changing
> to SCRAM-SHA-256.  I might have even missed some benefits.

If the release notes keep a general tone, perhaps it would be better
to mention as well that SCRAM is the recommended password-based
authentication method moving forward?
-- 
Michael


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PG 10 release notes

[Bruce Momjian]

On Tue, Apr 25, 2017 at 07:17:20PM -0700, Andres Freund wrote:
> On 2017-04-25 22:13:00 -0400, Bruce Momjian wrote:
> > On Tue, Apr 25, 2017 at 06:40:08PM -0700, Andres Freund wrote:
> > > On 2017-04-25 21:19:41 -0400, Bruce Momjian wrote:
> > > > On Tue, Apr 25, 2017 at 06:51:47AM +0200, Petr Jelinek wrote:
> > > > > Or the ability of logical decoding to follow timeline switches.
> > > > 
> > > > When you say "logical decoding", you don't mean contrib/test_decoding?
> > > 
> > > No.  test_decoding is just an example output plugin ("formatting" the
> > > changes), but there's in-production users of logical decoding out
> > > there.  So features adding to logical decoding in general, are worth to
> > > be mention in general.
> > 
> > Yes, but I am still trying to find out what changed because we don't
> > ship much that does "decoding" except contrib/test_decoding.  Are you
> > saying logical information is included in WAL?
> 
> We ship an extension API [1] [2] that allows to use decoding.  And that
> the internals of that API now knows how to deal with timeline changes.
> You could e.g. compare it with the FDW API now providing the ability to
> do aggregate pushdown or such.

Yes, if we changed the developer-visible API we should mention that in
the release notes.

> > > >> I also wonder if ability to run SQL queries on walsender connected to a
> > > >> database is worth mentioning (replication=database kind of connection).
> 
> Yes, but we're still discussing the feature's exact look nearby.

OK, please keep us up-to-date on what needs to be added.

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

+ As you are, so once was I.  As I am, so you will be. +
+  Ancient Roman grave inscription +


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PG 10 release notes

[Bruce Momjian]

On Tue, Apr 25, 2017 at 06:40:08PM -0700, Andres Freund wrote:
> On 2017-04-25 21:19:41 -0400, Bruce Momjian wrote:
> > On Tue, Apr 25, 2017 at 06:51:47AM +0200, Petr Jelinek wrote:
> > > Or the ability of logical decoding to follow timeline switches.
> > 
> > When you say "logical decoding", you don't mean contrib/test_decoding?
> 
> No.  test_decoding is just an example output plugin ("formatting" the
> changes), but there's in-production users of logical decoding out
> there.  So features adding to logical decoding in general, are worth to
> be mention in general.

Yes, but I am still trying to find out what changed because we don't
ship much that does "decoding" except contrib/test_decoding.  Are you
saying logical information is included in WAL?

> > Do you mean the WAL logical stream now has timeline information and 3rd
> > party software should know about that?
> 
> It's less about knowing about it, and more about being empowering new 
> usecases.
> 
> But, TBH, in this case I'm not sure what the point would be - given
> there's no way to sanely create and use such slots, I don't think the
> timeline following has advantages on its own, it's imo more of a
> stepping stone.

OK, makes sense.  Let me know if we should add:

> >> I also wonder if ability to run SQL queries on walsender connected to a
> >> database is worth mentioning (replication=database kind of connection).

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

+ As you are, so once was I.  As I am, so you will be. +
+  Ancient Roman grave inscription +


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PG 10 release notes

[Andres Freund]

On 2017-04-25 22:13:00 -0400, Bruce Momjian wrote:
> On Tue, Apr 25, 2017 at 06:40:08PM -0700, Andres Freund wrote:
> > On 2017-04-25 21:19:41 -0400, Bruce Momjian wrote:
> > > On Tue, Apr 25, 2017 at 06:51:47AM +0200, Petr Jelinek wrote:
> > > > Or the ability of logical decoding to follow timeline switches.
> > > 
> > > When you say "logical decoding", you don't mean contrib/test_decoding?
> > 
> > No.  test_decoding is just an example output plugin ("formatting" the
> > changes), but there's in-production users of logical decoding out
> > there.  So features adding to logical decoding in general, are worth to
> > be mention in general.
> 
> Yes, but I am still trying to find out what changed because we don't
> ship much that does "decoding" except contrib/test_decoding.  Are you
> saying logical information is included in WAL?

We ship an extension API [1] [2] that allows to use decoding.  And that
the internals of that API now knows how to deal with timeline changes.
You could e.g. compare it with the FDW API now providing the ability to
do aggregate pushdown or such.


> > >> I also wonder if ability to run SQL queries on walsender connected to a
> > >> database is worth mentioning (replication=database kind of connection).

Yes, but we're still discussing the feature's exact look nearby.

Greetings,

Andres Freund

[1] https://www.postgresql.org/docs/devel/static/logicaldecoding.html
[2] 
https://www.postgresql.org/docs/devel/static/logicaldecoding-output-plugin.html


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PG 10 release notes

[Andres Freund]

On 2017-04-25 21:19:41 -0400, Bruce Momjian wrote:
> On Tue, Apr 25, 2017 at 06:51:47AM +0200, Petr Jelinek wrote:
> > Or the ability of logical decoding to follow timeline switches.
> 
> When you say "logical decoding", you don't mean contrib/test_decoding?

No.  test_decoding is just an example output plugin ("formatting" the
changes), but there's in-production users of logical decoding out
there.  So features adding to logical decoding in general, are worth to
be mention in general.


> Do you mean the WAL logical stream now has timeline information and 3rd
> party software should know about that?

It's less about knowing about it, and more about being empowering new usecases.

But, TBH, in this case I'm not sure what the point would be - given
there's no way to sanely create and use such slots, I don't think the
timeline following has advantages on its own, it's imo more of a
stepping stone.

Greetings,

Andres Freund


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PG 10 release notes

[Bruce Momjian]

On Wed, Apr 26, 2017 at 11:06:03AM +0900, Michael Paquier wrote:
> On Wed, Apr 26, 2017 at 10:56 AM, Bruce Momjian  wrote:
> > First, I don't think RFC references belong in the release notes, let
> > alone RFC links.
> >
> > Second, there seems to be some confusion over what SCRAM-SHA-256 gives
> > us over MD5.  I think there are a few benefits:
> >
> > o  packets cannot be replayed as easily, i.e. md5 replayed random salt
> > packets with a 50% probability after 16k sessions
> > o  hard to re-use SCRAM-SHA-256 string if disclosed vs. simple for md5
> > o  harder to brute-force trying all possible strings to find a matching
> > hash
> >
> > So if you tell users that SCRAM-SHA-256 is better than MD5 only because
> > of one of those, they will not realize that three benefits of changing
> > to SCRAM-SHA-256.  I might have even missed some benefits.
> 
> If the release notes keep a general tone, perhaps it would be better
> to mention as well that SCRAM is the recommended password-based
> authentication method moving forward?

Well, we could add "MD5 users are encouraged to switch to
SCRAM-SHA-256".  Now whether we want to list this as something on the
SCRAM-SHA-256 description, or mention it as an incompatibility, or
under Migration.  I am not clear that MD5 is in such terrible shape that
this is warranted.

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

+ As you are, so once was I.  As I am, so you will be. +
+  Ancient Roman grave inscription +


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PG 10 release notes

[Bruce Momjian]

On Wed, Apr 26, 2017 at 09:02:51AM +0900, Michael Paquier wrote:
> On Wed, Apr 26, 2017 at 12:20 AM, Bruce Momjian  wrote:
> > On Tue, Apr 25, 2017 at 02:39:40PM +0900, Michael Paquier wrote:
> >> 
> >> Add SCRAM-SHA-256
> >> support for password negotiation and storage (Michael
> >> Paquier, Heikki Linnakangas)
> >> 
> >> 
> >> This proves better security than the existing 'md5' negotiation and
> >> storage method.
> >> 
> >> This is quite vague...
> >
> > Can you give me better text?  I can't think of any.
> 
> Sure, here is an idea:
> Add support for SASL authentication using protocol mechanism
> SCRAM-SHA-256 per RFC 5802 and 7677. (adding a reference to the RFCs
> with a link seems important to me).
> 
> SCRAM-SHA-256 improves deficiencies of MD5 password hashing by
> preventing any kind of pass-the-hash vulnerabilities, where a user
> would be able to connect to a PostgreSQL instance by just knowing the
> hash of a password and not the password itself.

First, I don't think RFC references belong in the release notes, let
alone RFC links.

Second, there seems to be some confusion over what SCRAM-SHA-256 gives
us over MD5.  I think there are a few benefits:

o  packets cannot be replayed as easily, i.e. md5 replayed random salt
packets with a 50% probability after 16k sessions

o  hard to re-use SCRAM-SHA-256 string if disclosed vs. simple for md5

o  harder to brute-force trying all possible strings to find a matching
hash

So if you tell users that SCRAM-SHA-256 is better than MD5 only because
of one of those, they will not realize that three benefits of changing
to SCRAM-SHA-256.  I might have even missed some benefits.
 
-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

+ As you are, so once was I.  As I am, so you will be. +
+  Ancient Roman grave inscription +


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PG 10 release notes

[Bruce Momjian]

On Tue, Apr 25, 2017 at 06:51:47AM +0200, Petr Jelinek wrote:
> I also wonder if ability to run SQL queries on walsender connected to a
> database is worth mentioning (replication=database kind of connection).
> 
> Or the ability of logical decoding to follow timeline switches.

When you say "logical decoding", you don't mean contrib/test_decoding? 
Do you mean the WAL logical stream now has timeline information and 3rd
party software should know about that?

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

+ As you are, so once was I.  As I am, so you will be. +
+  Ancient Roman grave inscription +


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PG 10 release notes

[Tsunakawa, Takayuki]

From: 'Bruce Momjian' [mailto:br...@momjian.us]
> > I forgot to point out one thing.
> >
> > Allow libpq to connect to multiple specified host names (Robert Haas)
> > libpq will connect with the first responsive host name.
> >
> > According to the following CF entry and my memory,
> >
> > https://commitfest.postgresql.org/12/879/
> >
> > Authors
> > mithun cy (mithun.cy)
> >
> > Reviewers
> > Gerdan Santos (gerdan), Takayuki Tsunakawa (maumau)Remove from
> > reviewers
> >
> > Committer
> > Robert Haas (rhaas)
> >
> > The author should probably be "(Mithun Cy, Robert Haas, Victor Wagner)"
> 
> I based the release note text on this commit:
> 
>   commit 274bb2b3857cc987cfa21d14775cae9b0dababa5
>   Author: Robert Haas 
>   Date:   Thu Nov 3 09:25:20 2016 -0400
> 
>   libpq: Allow connection strings and URIs to specify multiple
> hosts.
> 
>   It's also possible to specify a separate port for each host.
> 
>   Previously, we'd loop over every address returned by looking
> up the
>   host name; now, we'll try every address for every host name.
> 
>   Patch by me.  Victor Wagner wrote an earlier patch for this
> feature,
>   which I read, but I didn't use any of his code.  Review by Mithun
> Cy.
> 
> Was it wrong?

Oh, that commit.  I got it.

Regards
Takayuki Tsunakawa



-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PG 10 release notes

['Bruce Momjian']

On Wed, Apr 26, 2017 at 12:26:33AM +, Tsunakawa, Takayuki wrote:
> Hello, Bruce
> 
> I forgot to point out one thing.
> 
> Allow libpq to connect to multiple specified host names (Robert Haas) 
> libpq will connect with the first responsive host name. 
> 
> According to the following CF entry and my memory,
> 
> https://commitfest.postgresql.org/12/879/
> 
> Authors
> mithun cy (mithun.cy) 
> 
> Reviewers
> Gerdan Santos (gerdan), Takayuki Tsunakawa (maumau)Remove from reviewers 
> 
> Committer
> Robert Haas (rhaas)  
> 
> The author should probably be "(Mithun Cy, Robert Haas, Victor Wagner)"

I based the release note text on this commit:

commit 274bb2b3857cc987cfa21d14775cae9b0dababa5
Author: Robert Haas 
Date:   Thu Nov 3 09:25:20 2016 -0400

libpq: Allow connection strings and URIs to specify multiple hosts.

It's also possible to specify a separate port for each host.

Previously, we'd loop over every address returned by looking up the
host name; now, we'll try every address for every host name.

Patch by me.  Victor Wagner wrote an earlier patch for this feature,
which I read, but I didn't use any of his code.  Review by Mithun 
Cy.

Was it wrong?

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

+ As you are, so once was I.  As I am, so you will be. +
+  Ancient Roman grave inscription +


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PG 10 release notes

[Tsunakawa, Takayuki]

Hello, Bruce

I forgot to point out one thing.

Allow libpq to connect to multiple specified host names (Robert Haas) 
libpq will connect with the first responsive host name. 

According to the following CF entry and my memory,

https://commitfest.postgresql.org/12/879/

Authors
mithun cy (mithun.cy) 

Reviewers
Gerdan Santos (gerdan), Takayuki Tsunakawa (maumau)Remove from reviewers 

Committer
Robert Haas (rhaas)  

The author should probably be "(Mithun Cy, Robert Haas, Victor Wagner)"

Regards
Takayuki Tsunakawa





-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PG 10 release notes

[Michael Paquier]

On Wed, Apr 26, 2017 at 12:20 AM, Bruce Momjian  wrote:
> On Tue, Apr 25, 2017 at 02:39:40PM +0900, Michael Paquier wrote:
>> 
>> Add SCRAM-SHA-256
>> support for password negotiation and storage (Michael
>> Paquier, Heikki Linnakangas)
>> 
>> 
>> This proves better security than the existing 'md5' negotiation and
>> storage method.
>> 
>> This is quite vague...
>
> Can you give me better text?  I can't think of any.

Sure, here is an idea:
Add support for SASL authentication using protocol mechanism
SCRAM-SHA-256 per RFC 5802 and 7677. (adding a reference to the RFCs
with a link seems important to me).

SCRAM-SHA-256 improves deficiencies of MD5 password hashing by
preventing any kind of pass-the-hash vulnerabilities, where a user
would be able to connect to a PostgreSQL instance by just knowing the
hash of a password and not the password itself.
-- 
Michael


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PG 10 release notes

[Bruce Momjian]

On Tue, Apr 25, 2017 at 03:17:29PM -0400, Tels wrote:
> > I think all that was missing was "time":
> >
> > By default planning and execution time is display by
> > EXPLAIN ANALYZE and not display in other cases.
> > The new EXPLAIN option SUMMARY allows
> > explicit control of this.
> 
> Hm, no, I disagree, the the grammar is not right:
> 
> It should be: "X and Y _are_ display_ed_"
> 
> instead of:
> 
> "X and Y is display"
> 
> likewise "and not display_ed_ in other cases."

OK, got it:

By default planning and execution time are display by
EXPLAIN ANALYZE and are not display in other cases.
The new EXPLAIN option SUMMARY allows
explicit control of this.

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

+ As you are, so once was I.  As I am, so you will be. +
+  Ancient Roman grave inscription +


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PG 10 release notes

[Bruce Momjian]

On Tue, Apr 25, 2017 at 08:12:05PM +0200, Petr Jelinek wrote:
> On 25/04/17 17:01, Bruce Momjian wrote:
> > 
> >> I also wonder if ability to run SQL queries on walsender connected to a
> >> database is worth mentioning (replication=database kind of connection).
> > 
> > Uh, why would that be important to users?
> 
> Because every tool that uses logical decoding to capture changes does not
> need to open extra connection to fetch initial state.
> 
> (and we document much less user visible changes than this in the release
> notes)

Are you saying that 3rd party applications that use logical WAL records
need to know about that?  Can you provide some text?

> >> Or the ability of logical decoding to follow timeline switches.
> > 
> > I didn't think logical decoding was really more than a proof-of-concept
> > until now.
> > 
> 
> Huh?

Uh, the only logical decoding code that I know we ship pre-PG 10 is
contrib/test_decoding/.

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

+ As you are, so once was I.  As I am, so you will be. +
+  Ancient Roman grave inscription +


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PG 10 release notes

[Tels]

On Tue, April 25, 2017 1:21 pm, Bruce Momjian wrote:
> On Tue, Apr 25, 2017 at 01:06:05PM -0400, Tels wrote:
>> Moin,
>>
>> On Mon, April 24, 2017 9:31 pm, Bruce Momjian wrote:
>> > I have committed the first draft of the Postgres 10 release notes.
>> They
>> > are current as of two days ago, and I will keep them current.  Please
>> > give me any feedback you have.
>>
>> Thank you! Here is one thing I noticed:
>>
>> "By default planning and execution is display by EXPLAIN ANALYZE and not
>> display in other cases. The new EXPLAIN option SUMMARY allows explicit
>> control of this."
>>
>> The grammar sounds a bit off, I'd suggest:
>>
>> "By default planning and execution time are both displayed by EXPLAIN
>> ANALYZE and not displayed in other cases. The new EXPLAIN option SUMMARY
>> allows explicit control of this feature."
>
> I think all that was missing was "time":
>
> By default planning and execution time is display by
> EXPLAIN ANALYZE and not display in other cases.
> The new EXPLAIN option SUMMARY allows
> explicit control of this.

Hm, no, I disagree, the the grammar is not right:

It should be: "X and Y _are_ display_ed_"

instead of:

"X and Y is display"

likewise "and not display_ed_ in other cases."

Best wishes,

Tels


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PG 10 release notes

[Claudio Freire]

On Tue, Apr 25, 2017 at 2:45 PM, Bruce Momjian  wrote:
> On Tue, Apr 25, 2017 at 10:37:48AM -0700, Andres Freund wrote:
>> On 2017-04-25 13:11:32 -0400, Bruce Momjian wrote:
>> > I don't think this warrants inclusion in the release notes for reasons
>> > already discussed.  The vacuum truncation operation is a rare one and
>> > an implementation detail.
>>
>> I think that's backwards. The truncate operation is quite delicate
>> because it happens with AccessExclusiveLock held.  This regularly does
>> cause issues in production.  When users look for things they possibly
>> should update for, something like "performance improvents in final
>> vacuum phase" + oneliner is going to be a lot more interesting than,
>> say, "Add MONEY operators for multiplication and division with INT8
>> values".
>>
>> More and more users are going to be primarily interested in three
>> classes of new things in postgres: 1) performance 2) replication 3)
>> easier management. Arbitrarily excluding one of the major categories
>> from release notes isn't a useful policy, especially if we continue to
>> list new feature that'll effect no more than a handful of people.
>
> I disputed that my logic is arbitrary.  However, given your
> explanation, I have added the item:
>
>Improve speed of VACUUM's removal of trailing empty
>heap pages (Alvaro Herrera)

That's enough for me, thanks.


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PG 10 release notes

[Petr Jelinek]

On 25/04/17 17:01, Bruce Momjian wrote:
> 
>> I also wonder if ability to run SQL queries on walsender connected to a
>> database is worth mentioning (replication=database kind of connection).
> 
> Uh, why would that be important to users?

Because every tool that uses logical decoding to capture changes does no
need to open extra connection to fetch initial state.

(and we document much less user visible changes than this in the release
notes)

> 
>> Or the ability of logical decoding to follow timeline switches.
> 
> I didn't think logical decoding was really more than a proof-of-concept
> until now.
> 

Huh?

-- 
  Petr Jelinek  http://www.2ndQuadrant.com/
  PostgreSQL Development, 24x7 Support, Training & Services


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PG 10 release notes

[Andres Freund]

Hi,

On 2017-04-25 13:39:07 -0400, Bruce Momjian wrote:
> Understood, but the question is whether the release notes are the right
> place to educate users of something that will no longer be a problem.

I think it's the *prime* place for it.  It obviously doesn't matter if
you're not affected by $performance_problem, but if you are affected,
then it's quite likely going to be important.  Deciding whether to
migrate to a new version will often be a decision about a *lot* of work,
so it'll not be made lightly, and without a motivator.

If we go by that logic, why are we listing parallelism? Why are we
listing new planner/executor features that lead to faster plans?  The
reason we do, is that they're addressing concerns that users had, which
they need to know about.

> I am happy to adjust things to whatever the community wants, but, on the
> other hand I have a responsibility to be consistent what what they have
> wanted in the past.

Why?

Greetings,

Andres Freund


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PG 10 release notes

[Bruce Momjian]

On Tue, Apr 25, 2017 at 01:44:09PM -0400, Tom Lane wrote:
> Bruce Momjian  writes:
> > On Tue, Apr 25, 2017 at 10:00:52AM -0700, Andres Freund wrote:
> >> at this point, you can see, we've squarely left O(N) country, and
> >> entered the vast O(N^2) waste.
> 
> > OK, I got it now. :-)  Here is the new item:
> 
> > Improve table creation speed in sessions that reference many
> > relations (Aleksander Alekseev)
> 
> That's not specifically about table creation.  How about
> 
>   Reduce statistics tracking overhead in sessions that reference
>   many thousands of relations (Aleksander Alekseev)
> 
> I think it is fair, based on Andres' example, to quantify this as not
> being of much interest till you get to thousands of relations.  But
> it would matter for any action that creates a statistics report,
> whether it be scans, inserts, updates, deletes ...

OK, text updated.

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

+ As you are, so once was I.  As I am, so you will be. +
+  Ancient Roman grave inscription +


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers