Re: PG 13 release notes, first draft
On Tue, Sep 15, 2020 at 11:30 AM Tom Lane wrote: > Pushed, thanks for looking. I think that the Postgres 13 release notes should mention the enhancement to contrib/amcheck made by Alexander's commit d114cc53. I suggest something along the lines of: "Make the cross-level verification checks performed by contrib/amcheck's bt_index_parent_check() function more robust". -- Peter Geoghegan
Re: PG 13 release notes, first draft
On 9/15/20 2:30 PM, Tom Lane wrote: > "Jonathan S. Katz" writes: >> On 9/15/20 2:11 PM, Tom Lane wrote: >>> The other incompatibilities are only listed once, if they're minor. >>> I was just about to commit the attached. > >> Even better. +1 > > Pushed, thanks for looking. Thanks for modifying...though I have a gripe about it being labeled a gripe[1] ;) Though it gave me a good laugh... Jonathan [1] https://git.postgresql.org/pg/commitdiff/d42c6176446440b185fcb95c214b7e40d5758b60 signature.asc Description: OpenPGP digital signature
Re: PG 13 release notes, first draft
"Jonathan S. Katz" writes: > On 9/15/20 2:11 PM, Tom Lane wrote: >> The other incompatibilities are only listed once, if they're minor. >> I was just about to commit the attached. > Even better. +1 Pushed, thanks for looking. regards, tom lane
Re: PG 13 release notes, first draft
On 9/15/20 2:11 PM, Tom Lane wrote: > "Jonathan S. Katz" writes: >> On 9/15/20 1:05 PM, Tom Lane wrote: >>> After thinking a bit, I'm inclined to agree that we should move it >>> to "Incompatibilities". It is a core server change (third-party >>> extensions don't have a choice to opt out, as per postgis' issue), >>> and even if it's trivial, we have some even-more-trivial issues >>> listed there, like command tag changes. > >> How about this? > > The other incompatibilities are only listed once, if they're minor. > I was just about to commit the attached. Even better. +1 Jonathan signature.asc Description: OpenPGP digital signature
Re: PG 13 release notes, first draft
"Jonathan S. Katz" writes: > On 9/15/20 1:05 PM, Tom Lane wrote: >> After thinking a bit, I'm inclined to agree that we should move it >> to "Incompatibilities". It is a core server change (third-party >> extensions don't have a choice to opt out, as per postgis' issue), >> and even if it's trivial, we have some even-more-trivial issues >> listed there, like command tag changes. > How about this? The other incompatibilities are only listed once, if they're minor. I was just about to commit the attached. regards, tom lane diff --git a/doc/src/sgml/release-13.sgml b/doc/src/sgml/release-13.sgml index 1f130ca1fe..3fd97c9d10 100644 --- a/doc/src/sgml/release-13.sgml +++ b/doc/src/sgml/release-13.sgml @@ -270,6 +270,25 @@ Author: Tom Lane + + + + + Remove support for upgrading unpackaged (pre-9.1) extensions (Tom Lane) + + + + The FROM option + of CREATE + EXTENSION is no longer supported. Any installations + still using unpackaged extensions should upgrade them to a packaged + version before updating to PostgreSQL 13. + + + - - - Remove support for upgrading unpackaged (pre-9.1) extensions (Tom Lane) - - - - -
Re: PG 13 release notes, first draft
On 9/15/20 1:05 PM, Tom Lane wrote: > "Jonathan S. Katz" writes: >> On 9/15/20 12:05 PM, Tom Lane wrote: >>> Ah. OK, we can certainly extend it to mention that. How about >>> (not bothering with markup yet) >>> >>> Remove support for upgrading unpackaged (pre-9.1) extensions (Tom Lane) >>> + >>> +The FROM option of CREATE EXTENSION is no longer supported. > >> +1. > >> With that in place, I'm more ambivalent to whether or not it's mentioned >> in the incompatibilities section as well, though would lean towards >> having a mention of it there as it technically is one. But I don't feel >> too strongly about it. > > After thinking a bit, I'm inclined to agree that we should move it > to "Incompatibilities". It is a core server change (third-party > extensions don't have a choice to opt out, as per postgis' issue), > and even if it's trivial, we have some even-more-trivial issues > listed there, like command tag changes. How about this? Jonathan diff --git a/doc/src/sgml/release-13.sgml b/doc/src/sgml/release-13.sgml index 0ca970e452..bed56b0f3d 100644 --- a/doc/src/sgml/release-13.sgml +++ b/doc/src/sgml/release-13.sgml @@ -324,6 +324,19 @@ Author: Peter Geoghegan + + + + + Remove support for CREATE + EXTENSION ... FROM + (Tom Lane) + + + @@ -2941,6 +2954,12 @@ Author: Tom Lane Remove support for upgrading unpackaged (pre-9.1) extensions (Tom Lane) + + + The FROM option of + CREATE EXTENSION +is no longer supported. + signature.asc Description: OpenPGP digital signature
Re: PG 13 release notes, first draft
"Jonathan S. Katz" writes: > On 9/15/20 12:05 PM, Tom Lane wrote: >> Ah. OK, we can certainly extend it to mention that. How about >> (not bothering with markup yet) >> >> Remove support for upgrading unpackaged (pre-9.1) extensions (Tom Lane) >> + >> +The FROM option of CREATE EXTENSION is no longer supported. > +1. > With that in place, I'm more ambivalent to whether or not it's mentioned > in the incompatibilities section as well, though would lean towards > having a mention of it there as it technically is one. But I don't feel > too strongly about it. After thinking a bit, I'm inclined to agree that we should move it to "Incompatibilities". It is a core server change (third-party extensions don't have a choice to opt out, as per postgis' issue), and even if it's trivial, we have some even-more-trivial issues listed there, like command tag changes. regards, tom lane
Re: PG 13 release notes, first draft
On 9/15/20 12:05 PM, Tom Lane wrote: > "Jonathan S. Katz" writes: >> On 9/15/20 11:45 AM, Tom Lane wrote: >>> It is listed already in the "Additional Modules" section (about line 2940 >>> in release-13.sgml as of right now). > >> ...sort of. It talks about the feature, but does not talk about the >> syntax removal, which is what I was originally searching for in the >> release notes. > > Ah. OK, we can certainly extend it to mention that. How about > (not bothering with markup yet) > > Remove support for upgrading unpackaged (pre-9.1) extensions (Tom Lane) > + > +The FROM option of CREATE EXTENSION is no longer supported. +1. With that in place, I'm more ambivalent to whether or not it's mentioned in the incompatibilities section as well, though would lean towards having a mention of it there as it technically is one. But I don't feel too strongly about it. Jonathan signature.asc Description: OpenPGP digital signature
Re: PG 13 release notes, first draft
"Jonathan S. Katz" writes: > On 9/15/20 11:45 AM, Tom Lane wrote: >> It is listed already in the "Additional Modules" section (about line 2940 >> in release-13.sgml as of right now). > ...sort of. It talks about the feature, but does not talk about the > syntax removal, which is what I was originally searching for in the > release notes. Ah. OK, we can certainly extend it to mention that. How about (not bothering with markup yet) Remove support for upgrading unpackaged (pre-9.1) extensions (Tom Lane) + +The FROM option of CREATE EXTENSION is no longer supported. regards, tom lane
Re: PG 13 release notes, first draft
On 9/15/20 11:45 AM, Tom Lane wrote: > "Jonathan S. Katz" writes: >> On a different note, I became aware of this[1] and noticed that dropping >> "CREATE EXTENSION ... FROM" was not listed in the incompatibilities >> section, so proposing the attached. I have no strong opinions on the >> final wording, mainly wanted to get it listed. > > It is listed already in the "Additional Modules" section (about line 2940 > in release-13.sgml as of right now). ...sort of. It talks about the feature, but does not talk about the syntax removal, which is what I was originally searching for in the release notes. > I don't know Bruce's exact reasoning > for not including it in the "Incompatibilities" section, but I tend to > agree that it shouldn't be significant to any real-world user. I do tend agree with this intuitively but don't have any data to back it up either way. That said, we did modify the command and it would be good to at least mention the fact we dropped "FROM" somewhere in the release notes. It provides a good reference in case someone reports an "issue" in the future stemming from dropping the "FROM" keyword, we can say "oh it changed in PG13, see XYZ." (Speaking from having used the release notes to perform similar troubleshooting). > I think > that the postgis testing issue you reference is just an ancient test > case that they should drop as no longer relevant. Sure, and AIUI they're going to do that, mostly that was a reference point to the changed syntax. Jonathan signature.asc Description: OpenPGP digital signature
Re: PG 13 release notes, first draft
"Jonathan S. Katz" writes: > On a different note, I became aware of this[1] and noticed that dropping > "CREATE EXTENSION ... FROM" was not listed in the incompatibilities > section, so proposing the attached. I have no strong opinions on the > final wording, mainly wanted to get it listed. It is listed already in the "Additional Modules" section (about line 2940 in release-13.sgml as of right now). I don't know Bruce's exact reasoning for not including it in the "Incompatibilities" section, but I tend to agree that it shouldn't be significant to any real-world user. I think that the postgis testing issue you reference is just an ancient test case that they should drop as no longer relevant. regards, tom lane
Re: PG 13 release notes, first draft
On 9/15/20 9:49 AM, Jonathan S. Katz wrote: > On 9/15/20 5:22 AM, Masahiko Sawada wrote: >> On Tue, 15 Sep 2020 at 13:56, Peter Eisentraut >> wrote: >>> >>> On 2020-09-09 22:57, Jonathan S. Katz wrote: + + + Parallelized vacuuming of B-tree indexes + + >>> >>> I don't think B-tree indexes are relevant here. AFAICT, this feature >>> applies to all indexes. >>> >> >> Yes, parallel vacuum applies to all types of indexes provided by >> PostgreSQL binary, and other types of indexes also can use it. > > I'm not sure where I got B-tree from. I've attached a correction. On a different note, I became aware of this[1] and noticed that dropping "CREATE EXTENSION ... FROM" was not listed in the incompatibilities section, so proposing the attached. I have no strong opinions on the final wording, mainly wanted to get it listed. Thanks, Jonathan [1] https://trac.osgeo.org/postgis/ticket/4753 diff --git a/doc/src/sgml/release-13.sgml b/doc/src/sgml/release-13.sgml index 0ca970e452..8fbaff4623 100644 --- a/doc/src/sgml/release-13.sgml +++ b/doc/src/sgml/release-13.sgml @@ -324,6 +324,19 @@ Author: Peter Geoghegan + + + + + Remove support for CREATE + EXTENSION ... FROM + (Tom Lane) + + + signature.asc Description: OpenPGP digital signature
Re: PG 13 release notes, first draft
"Jonathan S. Katz" writes: > On 9/15/20 5:22 AM, Masahiko Sawada wrote: >> On Tue, 15 Sep 2020 at 13:56, Peter Eisentraut >> wrote: >>> I don't think B-tree indexes are relevant here. AFAICT, this feature >>> applies to all indexes. > I'm not sure where I got B-tree from. I've attached a correction. Right, pushed. I clarified the main entry for this a tad, too. regards, tom lane
Re: PG 13 release notes, first draft
On 9/15/20 5:22 AM, Masahiko Sawada wrote: > On Tue, 15 Sep 2020 at 13:56, Peter Eisentraut > wrote: >> >> On 2020-09-09 22:57, Jonathan S. Katz wrote: >>> + >>> + >>> + Parallelized vacuuming of B-tree indexes >>> + >>> + >> >> I don't think B-tree indexes are relevant here. AFAICT, this feature >> applies to all indexes. >> > > Yes, parallel vacuum applies to all types of indexes provided by > PostgreSQL binary, and other types of indexes also can use it. I'm not sure where I got B-tree from. I've attached a correction. Thanks, Jonathan diff --git a/doc/src/sgml/release-13.sgml b/doc/src/sgml/release-13.sgml index 0ca970e452..f7852c8618 100644 --- a/doc/src/sgml/release-13.sgml +++ b/doc/src/sgml/release-13.sgml @@ -37,7 +37,7 @@ - Parallelized vacuuming of B-tree indexes + Parallelized vacuuming of indexes signature.asc Description: OpenPGP digital signature
Re: PG 13 release notes, first draft
On Tue, 15 Sep 2020 at 13:56, Peter Eisentraut wrote: > > On 2020-09-09 22:57, Jonathan S. Katz wrote: > > + > > + > > + Parallelized vacuuming of B-tree indexes > > + > > + > > I don't think B-tree indexes are relevant here. AFAICT, this feature > applies to all indexes. > Yes, parallel vacuum applies to all types of indexes provided by PostgreSQL binary, and other types of indexes also can use it. Regards, -- Masahiko Sawadahttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Re: PG 13 release notes, first draft
On 2020-09-09 22:57, Jonathan S. Katz wrote: + + + Parallelized vacuuming of B-tree indexes + + I don't think B-tree indexes are relevant here. AFAICT, this feature applies to all indexes. -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Re: PG 13 release notes, first draft
Justin Pryzby writes: > Rebasing onto 3965de54e718600a4703233936e56a3202caf73f, I'm left with: Sorry, I hadn't seen that you submitted more updates. I pushed these with minor additional wordsmithing. regards, tom lane
Re: PG 13 release notes, first draft
On Mon, Sep 07, 2020 at 08:40:26AM -0500, Justin Pryzby wrote: Rebasing onto 3965de54e718600a4703233936e56a3202caf73f, I'm left with: diff --git a/doc/src/sgml/release-13.sgml b/doc/src/sgml/release-13.sgml index 8fffc6fe0a..69d143e10c 100644 --- a/doc/src/sgml/release-13.sgml +++ b/doc/src/sgml/release-13.sgml @@ -131,7 +131,7 @@ Author: Thomas Munro -SELECT round(sum(OLDVALUE / n::float)) FROM generate_series(1, OLDVALUE) s(n); +SELECT round(sum(OLDVALUE / n::float)) AS newvalue FROM generate_series(1, OLDVALUE) s(n); @@ -776,8 +776,8 @@ Author: Noah Misch --> -Allow skipping of WAL for full table writes if WAL writes to be skipped during a transaction +which creates or rewrites a relation if is minimal (Kyotaro Horiguchi) @@ -1007,7 +1007,7 @@ Author: Michael Paquier Add leader_pid to to report parallel worker ownership +linkend="pg-stat-activity-view"/> to report parallel worker's leader process (Julien Rouhaud) @@ -1262,8 +1262,8 @@ Author: Peter Eisentraut --> -Enable Unix-domain sockets -support on Windows (Peter Eisentraut) +Enable support for Unix-domain sockets +on Windows (Peter Eisentraut) @@ -1391,8 +1391,8 @@ Author: Fujii Masao --> - Allow WAL recovery to continue even if invalid - pages are referenced (Fujii Masao) + Allow recovery to continue even if invalid + pages are referenced by WAL (Fujii Masao)
Re: PG 13 release notes, first draft
"Jonathan S. Katz" writes: > One thing that did not make it through was this: > - 2020-XX-XX, CURRENT AS OF 2020-08-09 > + 2020-09-24, CURRENT AS OF 2020-09-09 Yeah, that's a placeholder to recall how far back to look for additional changes to the relnotes, so unless you already read the git history that far back and concluded nothing needed documenting, that was premature. (I've just about finished making those updates and making an editorial pass over the notes, so I will change it in a little bit.) regards, tom lane
Re: PG 13 release notes, first draft
On 9/10/20 1:14 PM, Tom Lane wrote: > "Jonathan S. Katz" writes: >> Attached is a proposal for the "major enhancements" section. I borrowed >> from the press release[1] but tried to stay true to the release notes >> format and listed out the enhancements that way. > > Pushed with some very minor wording tweaks. Thanks! The tweaks were minor enough that it took a few readthroughs to catch them. One thing that did not make it through was this: - 2020-XX-XX, CURRENT AS OF 2020-08-09 + 2020-09-24, CURRENT AS OF 2020-09-09 Is the plan to update that at a later date? Understandable if so, but wanted to check. Thanks, Jonathan signature.asc Description: OpenPGP digital signature
Re: PG 13 release notes, first draft
"Jonathan S. Katz" writes: > Attached is a proposal for the "major enhancements" section. I borrowed > from the press release[1] but tried to stay true to the release notes > format and listed out the enhancements that way. Pushed with some very minor wording tweaks. regards, tom lane
Re: PG 13 release notes, first draft
Hi, On 5/4/20 11:16 PM, Bruce Momjian wrote: > I have committed the first draft of the PG 13 release notes. You can > see them here: > > https://momjian.us/pgsql_docs/release-13.html > > It still needs markup, word wrap, and indenting. The community doc > build should happen in a few hours. Thank you again for compiling and maintaining the release notes through another major release cycle, I know it's no small undertaking! Attached is a proposal for the "major enhancements" section. I borrowed from the press release[1] but tried to stay true to the release notes format and listed out the enhancements that way. Open to suggestion, formatting changes, etc. Thanks! Jonathan [1] https://www.postgresql.org/message-id/3bd579f8-438a-ed1a-ee20-738292099aae%40postgresql.org diff --git a/doc/src/sgml/release-13.sgml b/doc/src/sgml/release-13.sgml index 68f9a0a9f1..95106921d7 100644 --- a/doc/src/sgml/release-13.sgml +++ b/doc/src/sgml/release-13.sgml @@ -6,29 +6,49 @@ Release date: - 2020-XX-XX, CURRENT AS OF 2020-08-09 + 2020-09-24, CURRENT AS OF 2020-09-09 Overview - Major enhancements in PostgreSQL 13 include: + PostgreSQL 13 contains many new features and + enhancements, including: - - - -TBD + + Space savings and performance gains from B-tree index deduplication + + + + + Improved response times for queries that use aggregates or partitions + + + + + Better query planning when using enhanced statistics + + + + + Parallelized vacuuming of B-tree indexes + + + + + Incremental sorting + - - - The above items are explained in more detail in the sections below. - + +and more. The above items and other new features of PostgreSQL 13 are +explained in more detail in the sections below. + signature.asc Description: OpenPGP digital signature
Re: PG 13 release notes, first draft
On Tue, May 12, 2020 at 10:28 AM Amit Langote wrote: > > On Tue, May 12, 2020 at 8:51 AM Bruce Momjian wrote: > > On Fri, May 8, 2020 at 12:07:09PM +0900, Amit Langote wrote: > > > I have attached a patch with my suggestions above. > > > > OK, I slightly modified the wording of your first change, patch > > attached. > > Thanks. I checked that what you committed looks fine. Sorry about not having reported this earlier, but I had noticed that the wording of the partitioned tables logical replication item isn't correct grammatically, which I noticed again while going through the webpage. Attached fixes it as follows: -to be automatically published. Addition/removal of partitions from -partitioned tables are automatically added/removed from publications. +to be automatically published. Adding/removing partitions from +a partitioned table automatically adds/removes them from publications. -- Amit Langote EnterpriseDB: http://www.enterprisedb.com release-13-wording.patch Description: Binary data
Re: PG 13 release notes, first draft
There's some obvious improvements for which I include a patch. >Add alternate version of jsonb_setI() with special NULL handling (Andrew >Dunstan) >The new function, jsonb_set_lax(), allows null new values to either set the >specified key to JSON null, delete the key, raise exception, or ignore the >operation. jsonb_set() raise *an* exception new null values? >IS 'return_target' CLEAR? I haven't used these before, but following the examples, it seems to return the "target" argument, unchanged. | Add function min_scale() that returns the number of digits to the right the decimal point that is required to represent the numeric value with full precision (Pavel Stehule) right *of* that *are* required ? |The old function names were kept for backward compatibility. DO WE HAVE NEW NAMES? => I think the docs are clear: > In releases of PostgreSQL before 13 there was no xid8 type, so variants of > these functions were provided that used bigint to represent a 64-bit XID, > with a correspondingly distinct snapshot data type txid_snapshot. These older > functions have txid in their names. They are still supported for backward > compatibility, but may be removed from a future release. See Table 9.76 > This improves performance for queries that access many object. The internal > List API has also been improved. many objects* |Allow skipping of WAL for full table writes if wal_level is minimal (Kyotaro Horiguchi) Allow WAL writes to be skipped... And: this is not related to full_page_writes. | Enable Unix-domain sockets support on Windows (Peter Eisentraut) Enable support for .. | Improve the performance when replaying DROP DATABASE commands when many tablespaces are in use (Fujii Masao) s/the// |Allow a sample of statements to be logged (Adrien Nayrat) Allow logging a sample of statements |Allow WAL receivers use a temporary replication slot if a permanent one is not specified (Peter Eisentraut, Sergei Kornilov) *to* use | Add leader_pid to pg_stat_activity to report parallel worker ownership (Julien Rouhaud) s/ownership/leader/ ? |Allow WAL recovery to continue even if invalid pages are referenced (Fujii Masao) remove "WAL" or say: >Allow recovery to continue even if invalid pages are referenced by WAL (Fujii >Masao) A few things I have't fixed in my patch: |Previously, this value was adjusted before effecting the number of concurrent requests. This value is now used directly. Conversion of old values to new ones can be done using: |SELECT round(sum(OLD / n::float)) FROM generate_series(1, OLD) s(n); I think the round() should be aliased, "AS new". I think "before effecting" is confusing, maybe say: | Previously, the effective value was computed internally from the user-supplied parameter... |Allow partitioned tables to be logically replicated via publications (Amit Langote) |Previously, partitions had to be replicated individually. Now partitioned tables can be published explicitly causing all partitions to be automatically published. Addition/removal of partitions from partitioned tables are automatically added/removed from publications. The CREATE PUBLICATION option publish_via_partition_root controls whether changes to partitions are published as their own or their ancestor's. => "causing any future partitions to be automatically published". "addition/removal .. are automatically" isn't right |Implement incremental sorting (James Coleman, Alexander Korotkov, Tomas Vondra) |If a result is already sorted by several leading keys, this allows for batch sorting of additional trailing keys because the previous keys are already equal. This is controlled by enable_incremental_sort. s/several/one or more/ remove "additional" ? remove "batch" ? maybe "of ONLY trailing keys" |Allow inserts to trigger autovacuum activity (Laurenz Albe, Darafei Praliaskouski) |This new behavior reduces the work necessary when the table needs to be frozen and allows pages to be set as all-visible. All-visible pages allow index-only scans to access fewer heap rows. There's a lot of "allow" here, but it sounds like relaxing a restriction when actually this is a new functionality. Maybe: | Allow autovacuum to be triggered by INSERTs. | ..and allows autovacuum to set pages as all-visible. I already mentioned a couple things back in May that still stand out;: |Add jsonpath .datetime() method (Nikita Glukhov, Teodor Sigaev, Oleg Bartunov, Alexander Korotkov) |This allows json values to be converted to timestamps, which can then be processed in jsonpath expressions. This also adds jsonpath functions that support time zone-aware output. timezone-aware or time-zone-aware, if you must. > Allow vacuum commands run by vacuumdb to operate in parallel mode (Masahiko > Sawada) => I think this is still going to be lost/misunderstood/confuse some people. vacuumdb already supports -j. This allows it to run vacuum(parallel N). So maybe say "...to process indexes in parallel". --
Re: PG 13 release notes, first draft
On Mon, Aug 3, 2020 at 11:35:50AM -0700, Peter Geoghegan wrote: > On Thu, Jul 30, 2020 at 10:45 AM Peter Geoghegan wrote: > > On Thu, Jul 30, 2020 at 10:32 AM Bruce Momjian wrote: > > > I came up with a more verbose documentation suggestion, attached. > > > > I'm okay with this. > > Are you going to push this soon, Bruce? Done. -- Bruce Momjian https://momjian.us EnterpriseDB https://enterprisedb.com The usefulness of a cup is in its emptiness, Bruce Lee
Re: PG 13 release notes, first draft
On Thu, Jul 30, 2020 at 10:45 AM Peter Geoghegan wrote: > On Thu, Jul 30, 2020 at 10:32 AM Bruce Momjian wrote: > > I came up with a more verbose documentation suggestion, attached. > > I'm okay with this. Are you going to push this soon, Bruce? -- Peter Geoghegan
Re: PG 13 release notes, first draft
On Thu, Jul 30, 2020 at 10:32 AM Bruce Momjian wrote: > I came up with a more verbose documentation suggestion, attached. I'm okay with this. Thanks -- Peter Geoghegan
Re: PG 13 release notes, first draft
On Wed, Jul 29, 2020 at 08:43:24PM -0700, David G. Johnston wrote: > On Wednesday, July 29, 2020, Peter Geoghegan wrote: > > On Wed, Jul 29, 2020 at 6:30 PM Bruce Momjian wrote: > > > There should be a note about this in the Postgres 13 release notes, > > > for the usual reasons. More importantly, the "Allow hash aggregation > > > to use disk storage for large aggregation result sets" feature should > > > reference the new GUC directly. Users should be advised that the GUC > > > may be useful in cases where they upgrade and experience a performance > > > regression linked to slower hash aggregation. Just including a > > > documentation link for the GUC would be very helpful. > > > > I came up with the attached patch. > > I was thinking something along like the following (after the existing > sentence about avoiding hash aggs in the planner): > > If you find that hash aggregation is slower than in previous major > releases of PostgreSQL, it may be useful to increase the value of > hash_mem_multiplier. This allows hash aggregation to use more memory > without affecting competing query operations that are generally less > likely to put any additional memory to good use. I came up with a more verbose documentation suggestion, attached. > How about adding wording for GROUP BY as well to cater to users who are more > comfortable thinking in terms of SQL statements instead of execution plans? Uh, it is unclear exactly what SQL generates what node types, so I want to avoid this. -- Bruce Momjian https://momjian.us EnterpriseDB https://enterprisedb.com The usefulness of a cup is in its emptiness, Bruce Lee diff --git a/doc/src/sgml/release-13.sgml b/doc/src/sgml/release-13.sgml new file mode 100644 index 18e6497..0b1051b *** a/doc/src/sgml/release-13.sgml --- b/doc/src/sgml/release-13.sgml *** Author: Jeff Davis Previously, hash aggregation was avoided if it was expected to use ! more than memory. --- 649,658 Previously, hash aggregation was avoided if it was expected to use ! more than memory. To reduce the ! likelihood of using disk storage for hash aggregation and attain ! behavior similar to previous Postgres releases, increase .
Re: PG 13 release notes, first draft
On Wednesday, July 29, 2020, Peter Geoghegan wrote: > On Wed, Jul 29, 2020 at 6:30 PM Bruce Momjian wrote: > > > There should be a note about this in the Postgres 13 release notes, > > > for the usual reasons. More importantly, the "Allow hash aggregation > > > to use disk storage for large aggregation result sets" feature should > > > reference the new GUC directly. Users should be advised that the GUC > > > may be useful in cases where they upgrade and experience a performance > > > regression linked to slower hash aggregation. Just including a > > > documentation link for the GUC would be very helpful. > > > > I came up with the attached patch. > > I was thinking something along like the following (after the existing > sentence about avoiding hash aggs in the planner): > > If you find that hash aggregation is slower than in previous major > releases of PostgreSQL, it may be useful to increase the value of > hash_mem_multiplier. This allows hash aggregation to use more memory > without affecting competing query operations that are generally less > likely to put any additional memory to good use. > > How about adding wording for GROUP BY as well to cater to users who are more comfortable thinking in terms of SQL statements instead of execution plans? David J.
Re: PG 13 release notes, first draft
On Wed, Jul 29, 2020 at 7:48 PM Bruce Momjian wrote: > Well, that seems to be repeating what is already in the docs for > hash_mem_multiplier, which I try to avoid. One other direction is to > put something in the incompatibilities section. Does that make sense? I would prefer to put it next to the hash agg item itself. It's more likely to be noticed there, and highlighting it a little seems warranted. OTOH, this may not be a problem at all for many individual users. Framing it as a tip rather than a compatibility item gets that across. -- Peter Geoghegan
Re: PG 13 release notes, first draft
On Wed, Jul 29, 2020 at 07:00:43PM -0700, Peter Geoghegan wrote: > On Wed, Jul 29, 2020 at 6:30 PM Bruce Momjian wrote: > > > There should be a note about this in the Postgres 13 release notes, > > > for the usual reasons. More importantly, the "Allow hash aggregation > > > to use disk storage for large aggregation result sets" feature should > > > reference the new GUC directly. Users should be advised that the GUC > > > may be useful in cases where they upgrade and experience a performance > > > regression linked to slower hash aggregation. Just including a > > > documentation link for the GUC would be very helpful. > > > > I came up with the attached patch. > > I was thinking something along like the following (after the existing > sentence about avoiding hash aggs in the planner): > > If you find that hash aggregation is slower than in previous major > releases of PostgreSQL, it may be useful to increase the value of > hash_mem_multiplier. This allows hash aggregation to use more memory > without affecting competing query operations that are generally less > likely to put any additional memory to good use. Well, that seems to be repeating what is already in the docs for hash_mem_multiplier, which I try to avoid. One other direction is to put something in the incompatibilities section. Does that make sense? -- Bruce Momjian https://momjian.us EnterpriseDB https://enterprisedb.com The usefulness of a cup is in its emptiness, Bruce Lee
Re: PG 13 release notes, first draft
On Wed, Jul 29, 2020 at 6:30 PM Bruce Momjian wrote: > > There should be a note about this in the Postgres 13 release notes, > > for the usual reasons. More importantly, the "Allow hash aggregation > > to use disk storage for large aggregation result sets" feature should > > reference the new GUC directly. Users should be advised that the GUC > > may be useful in cases where they upgrade and experience a performance > > regression linked to slower hash aggregation. Just including a > > documentation link for the GUC would be very helpful. > > I came up with the attached patch. I was thinking something along like the following (after the existing sentence about avoiding hash aggs in the planner): If you find that hash aggregation is slower than in previous major releases of PostgreSQL, it may be useful to increase the value of hash_mem_multiplier. This allows hash aggregation to use more memory without affecting competing query operations that are generally less likely to put any additional memory to good use. -- Peter Geoghegan
Re: PG 13 release notes, first draft
On Wed, Jul 29, 2020 at 03:34:22PM -0700, Peter Geoghegan wrote: > Hi Bruce, > > On Fri, Jun 26, 2020 at 3:24 PM Bruce Momjian wrote: > > Patch attached and applied to PG 13. > > I committed the hash_mem_multiplier GUC to Postgres 13 just now. > > There should be a note about this in the Postgres 13 release notes, > for the usual reasons. More importantly, the "Allow hash aggregation > to use disk storage for large aggregation result sets" feature should > reference the new GUC directly. Users should be advised that the GUC > may be useful in cases where they upgrade and experience a performance > regression linked to slower hash aggregation. Just including a > documentation link for the GUC would be very helpful. I came up with the attached patch. -- Bruce Momjian https://momjian.us EnterpriseDB https://enterprisedb.com The usefulness of a cup is in its emptiness, Bruce Lee diff --git a/doc/src/sgml/release-13.sgml b/doc/src/sgml/release-13.sgml new file mode 100644 index 18e6497..2586645 *** a/doc/src/sgml/release-13.sgml --- b/doc/src/sgml/release-13.sgml *** Author: Jeff Davis Previously, hash aggregation was avoided if it was expected to use ! more than memory. --- 649,657 Previously, hash aggregation was avoided if it was expected to use ! more than memory. To use more ! memory for hash query operations, increase .
Re: PG 13 release notes, first draft
Hi Bruce, On Fri, Jun 26, 2020 at 3:24 PM Bruce Momjian wrote: > Patch attached and applied to PG 13. I committed the hash_mem_multiplier GUC to Postgres 13 just now. There should be a note about this in the Postgres 13 release notes, for the usual reasons. More importantly, the "Allow hash aggregation to use disk storage for large aggregation result sets" feature should reference the new GUC directly. Users should be advised that the GUC may be useful in cases where they upgrade and experience a performance regression linked to slower hash aggregation. Just including a documentation link for the GUC would be very helpful. Thanks -- Peter Geoghegan
Re: PG 13 release notes, first draft
On Fri, Jun 26, 2020 at 04:23:26PM -0400, Alvaro Herrera wrote: > Reading Luis Carril's other entry in the relnotes, > > Allow pg_dump --include-foreign-data to dump data from foreign servers (Luis > Carril) > > It seems to suggest that --include-foreign-data existed previously, > which is not true. I would have worded it as "Add --include-foreign-data > option to pg_dump to allow dumping data from foreign servers". OK, pg_dump item about FOREIGN keyword removed from PG 13 release notes, and the above item clarified. Patch attached and applied to PG 13. -- Bruce Momjian https://momjian.us EnterpriseDB https://enterprisedb.com The usefulness of a cup is in its emptiness, Bruce Lee diff --git a/doc/src/sgml/release-13.sgml b/doc/src/sgml/release-13.sgml new file mode 100644 index 401c557..45d8f0b *** a/doc/src/sgml/release-13.sgml --- b/doc/src/sgml/release-13.sgml *** Author: Peter Eisentraut - - - - -Add FOREIGN to ALTER statements, -if appropriate (Luis Carril) - - - -WHAT IS THIS ABOUT? - - - --- 1534,1539 *** Author: Alvaro Herrera !Allow pg_dump !--include-foreign-data to dump data from foreign !servers (Luis Carril) --- 2398,2407 --> !Add pg_dump !option --include-foreign-data to dump data from !foreign servers (Luis Carril)
Re: PG 13 release notes, first draft
Reading Luis Carril's other entry in the relnotes, Allow pg_dump --include-foreign-data to dump data from foreign servers (Luis Carril) It seems to suggest that --include-foreign-data existed previously, which is not true. I would have worded it as "Add --include-foreign-data option to pg_dump to allow dumping data from foreign servers". -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Re: PG 13 release notes, first draft
On 2020-Jun-26, Bruce Momjian wrote: > On Fri, Jun 26, 2020 at 05:24:16PM +0900, Masahiko Sawada wrote: > > Author: Alvaro Herrera > > 2020-03-20 [4e6209134] pg_dump: Add FOREIGN to ALTER statements, if > > appropriate > > --> > > > > > >Add FOREIGN to ALTER > > statements, > >if appropriate (Luis Carril) > > > > IIUC this entry is about that pg_dump adds FOREIGN word to ALTER TABLE > > command. Please find the attached patch. > > OK, so if that is, what used to happen before? Did it still work > without the FOREIGN keyword? If so, I am thinking we should just remove > this item. I tend to agree, it's not a change significant enough to be documented in the relnotes, i think. -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Re: PG 13 release notes, first draft
On Fri, Jun 26, 2020 at 05:24:16PM +0900, Masahiko Sawada wrote: > Hi, > > I realized that PG 13 release note still has the following entry: > > > > >Add FOREIGN to ALTER statements, >if appropriate (Luis Carril) > > > >WHAT IS THIS ABOUT? > > > > > > IIUC this entry is about that pg_dump adds FOREIGN word to ALTER TABLE > command. Please find the attached patch. OK, so if that is, what used to happen before? Did it still work without the FOREIGN keyword? If so, I am thinking we should just remove this item. -- Bruce Momjian https://momjian.us EnterpriseDB https://enterprisedb.com The usefulness of a cup is in its emptiness, Bruce Lee
Re: PG 13 release notes, first draft
Hi, I realized that PG 13 release note still has the following entry: Add FOREIGN to ALTER statements, if appropriate (Luis Carril) WHAT IS THIS ABOUT? IIUC this entry is about that pg_dump adds FOREIGN word to ALTER TABLE command. Please find the attached patch. Regards, -- Masahiko Sawadahttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services pg13_release_note.patch Description: Binary data
Re: PG 13 release notes, first draft
On Mon, May 25, 2020 at 10:54:03AM +0200, Daniel Gustafsson wrote: > Spotted this in the release notes: > > >Add extension bool_plperl which transforms >SQL booleans to/from PL/Perl booleans (Ivan >Panchenko) WHERE IS THIS DOCUMENTED? > > > bool_plperl is documented in "44.1. PL/Perl Functions and Arguments", but not > with a separate section (which fwiw I don't disagree with). Linking there > from > the release notes entry would require some rewriting to make it fit; I would > just remove the placeholder question for this one. Thanks, done. -- Bruce Momjian https://momjian.us EnterpriseDB https://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
Re: PG 13 release notes, first draft
Spotted this in the release notes: Add extension bool_plperl which transforms SQL booleans to/from PL/Perl booleans (Ivan Panchenko) WHERE IS THIS DOCUMENTED? bool_plperl is documented in "44.1. PL/Perl Functions and Arguments", but not with a separate section (which fwiw I don't disagree with). Linking there from the release notes entry would require some rewriting to make it fit; I would just remove the placeholder question for this one. cheers ./daniel
Re: PG 13 release notes, first draft
Thanks, applied. --- On Mon, May 18, 2020 at 11:18:51AM +0200, Daniel Gustafsson wrote: > > On 5 May 2020, at 05:16, Bruce Momjian wrote: > > > > I have committed the first draft of the PG 13 release notes. You can > > see them here: > > Spotted a typo we probably should fix: s/PostgresSQL/PostgreSQL/ =) > > cheers ./daniel > > diff --git a/doc/src/sgml/release-13.sgml b/doc/src/sgml/release-13.sgml > index c39a6ad38e..7f864da162 100644 > --- a/doc/src/sgml/release-13.sgml > +++ b/doc/src/sgml/release-13.sgml > @@ -215,7 +215,7 @@ Author: Tom Lane > > >Remove support for defining operator > - classes using pre-PostgresSQL > + classes using pre-PostgreSQL >8.0 syntax (Daniel Gustafsson) > > > @@ -228,7 +228,7 @@ Author: Tom Lane > > >Remove support for defining foreign key > - constraints using pre-PostgresSQL > + constraints using pre-PostgreSQL >7.3 syntax (Daniel Gustafsson) > > > @@ -242,7 +242,7 @@ Author: Tom Lane > >Remove support for "opaque" linkend="sql-createtype">pseudo-types used by > - pre-PostgresSQL 7.3 servers (Daniel > + pre-PostgreSQL 7.3 servers (Daniel >Gustafsson) > > -- Bruce Momjian https://momjian.us EnterpriseDB https://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
Re: PG 13 release notes, first draft
> On 5 May 2020, at 05:16, Bruce Momjian wrote: > > I have committed the first draft of the PG 13 release notes. You can > see them here: Spotted a typo we probably should fix: s/PostgresSQL/PostgreSQL/ =) cheers ./daniel 13relnotes_postgressql.diff Description: Binary data
Re: PG 13 release notes, first draft
On Fri, May 15, 2020 at 09:54:55PM +0900, Fujii Masao wrote: > > Actually, it should be: > > > > > > > > because we are using the text from the link. > > Yes, this works. > > > See > > doc/src/sgml/README.links for details on xref links. Release notes > > updated. > > Thanks! > > > Odd I got no warning for this on 'make check'. > > I'm not sure why, but btw I got the message when I compiled the document on > Mac. I don't think I looked at the HTML build output, only the check one, so that might be the cause. -- Bruce Momjian https://momjian.us EnterpriseDB https://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
Re: PG 13 release notes, first draft
On 2020/05/15 21:29, Bruce Momjian wrote: On Fri, May 15, 2020 at 03:55:19PM +0900, Fujii Masao wrote: On 2020/05/05 12:16, Bruce Momjian wrote: I have committed the first draft of the PG 13 release notes. You can see them here: https://momjian.us/pgsql_docs/release-13.html It still needs markup, word wrap, and indenting. The community doc build should happen in a few hours. Many thanks for working on this! When I did "make html", I got the following message. Link element has no content and no Endterm. Nothing to show in the link to sepgsql "Allow to control access to the" in release-13.sgml seems to have caused this. Also I found it's converted into "Allow ??? to control access to the", i.e., ??? was used. - Allow to control access to the + Allow sepgsql to control access to the Shouldn't we change that as the above? Actually, it should be: because we are using the text from the link. Yes, this works. See doc/src/sgml/README.links for details on xref links. Release notes updated. Thanks! Odd I got no warning for this on 'make check'. I'm not sure why, but btw I got the message when I compiled the document on Mac. Regards, -- Fujii Masao Advanced Computing Technology Center Research and Development Headquarters NTT DATA CORPORATION
Re: PG 13 release notes, first draft
On Fri, May 15, 2020 at 03:55:19PM +0900, Fujii Masao wrote: > > > On 2020/05/05 12:16, Bruce Momjian wrote: > > I have committed the first draft of the PG 13 release notes. You can > > see them here: > > > > https://momjian.us/pgsql_docs/release-13.html > > > > It still needs markup, word wrap, and indenting. The community doc > > build should happen in a few hours. > > Many thanks for working on this! > > When I did "make html", I got the following message. > > Link element has no content and no Endterm. Nothing to show in the link > to sepgsql > > "Allow to control access to the" in release-13.sgml > seems to have caused this. Also I found it's converted into "Allow ??? to > control access to the", i.e., ??? was used. > > - Allow to control access to the > + Allow sepgsql to control access to the > > Shouldn't we change that as the above? Actually, it should be: because we are using the text from the link. See doc/src/sgml/README.links for details on xref links. Release notes updated. Odd I got no warning for this on 'make check'. -- Bruce Momjian https://momjian.us EnterpriseDB https://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
Re: PG 13 release notes, first draft
On 2020/05/05 12:16, Bruce Momjian wrote: I have committed the first draft of the PG 13 release notes. You can see them here: https://momjian.us/pgsql_docs/release-13.html It still needs markup, word wrap, and indenting. The community doc build should happen in a few hours. Many thanks for working on this! When I did "make html", I got the following message. Link element has no content and no Endterm. Nothing to show in the link to sepgsql "Allow to control access to the" in release-13.sgml seems to have caused this. Also I found it's converted into "Allow ??? to control access to the", i.e., ??? was used. - Allow to control access to the + Allow sepgsql to control access to the Shouldn't we change that as the above? Regards, -- Fujii Masao Advanced Computing Technology Center Research and Development Headquarters NTT DATA CORPORATION
Re: PG 13 release notes, first draft
On Thu, May 14, 2020 at 11:01:51PM +0200, Peter Eisentraut wrote: > On 2020-05-12 02:41, Justin Pryzby wrote: > > I'm not opposed to including it, but I think it's still true that the user > > doesn't need to know in advance that the error message will be additionally > > helpful in the event of corruption. If we were to include more "error" > > items, > > we might also include these: > > > > 71a8a4f6e36547bb060dbcc961ea9b57420f7190 Add backtrace support for error > > reporting > > This is actually a legitimate user-visible feature and should be listed > somewhere. On Thu, May 14, 2020 at 02:02:52PM -0700, Peter Geoghegan wrote: > +1 -- it's very handy. Plus it has user-facing knobs. That's already included: | Allow function call backtraces of errors to be logged (Peter Eisentraut, Álvaro Herrera) | Server variable backtrace_functions specifies which C functions should generate backtraces on error. I 1) I failed to double check my list; and, 2) intended for that to be interpretted as items which could be moved to a separate "error reporting" section of the release notes. -- Justin
Re: PG 13 release notes, first draft
On Thu, May 14, 2020 at 2:02 PM Peter Eisentraut wrote: > On 2020-05-12 02:41, Justin Pryzby wrote: > > I'm not opposed to including it, but I think it's still true that the user > > doesn't need to know in advance that the error message will be additionally > > helpful in the event of corruption. If we were to include more "error" > > items, > > we might also include these: > > > > 71a8a4f6e36547bb060dbcc961ea9b57420f7190 Add backtrace support for error > > reporting > > This is actually a legitimate user-visible feature and should be listed > somewhere. +1 -- it's very handy. Plus it has user-facing knobs. -- Peter Geoghegan
Re: PG 13 release notes, first draft
On 2020-05-12 02:41, Justin Pryzby wrote: I'm not opposed to including it, but I think it's still true that the user doesn't need to know in advance that the error message will be additionally helpful in the event of corruption. If we were to include more "error" items, we might also include these: 71a8a4f6e36547bb060dbcc961ea9b57420f7190 Add backtrace support for error reporting This is actually a legitimate user-visible feature and should be listed somewhere. -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Re: PG 13 release notes, first draft
On Thu, May 14, 2020 at 07:23:05AM +0200, Fabien COELHO wrote: > > Hello Bruce, > > > > > > * 34a0a81bfb > > > > > > > > We already have: > > > > > > > > Reformat tables containing function information for better > > > > clarity (Tom Lane) > > > > > > > > so it seems it is covered as part of this. > > > > > > AFAICR this one is not by the same author, and although the point was > > > about > > > better clarity, it was not about formating but rather about restructuring > > > text vs binary string function documentations. Then Tom reformatted the > > > result. > > > > Well, we were not even clear we should document changes in the functions > > section, so going into details of all the changes seems unwise. > > The restructuring was a significant change, and ISTM that another function > of the release note is also to implicitely thank contributors (their name is > appended, which does not bring any useful information about the feature from > a release note perspective) hence my suggestion to include this one, > the author of which is not Tom Lane. We list people's names next to items. We don't list items to list people's names, as far as I know of the policy. If you want to change that, you will need to start a new thread and get agreement. > > > > > * e829337d42 > > > > > > > > Uh, this is a doc link formatting addition. I think this falls into the > > > > error message logic, where it is nice when people want it, but they > > > > don't need to know about it ahead of time. > > > > > > [...] > > > > I don't see it. > > While reading again the sequence, ISTM that I did not understand your first > answer, so my answer was kind-of off topic, sorry. This is indeed "link > formatting addition", which helps making the libpq doc more usable. > Probably you do not need to know about it in advance, but I do not think > that it is a good reason not to include it: with the same argument, a > performance improvement would not need to be advertise, you'll see it when > you need it. The same holds for all non-functional improvements, and there > are many which are listed. Peformance items are listed only if they will produce a visible change in performance, or enable new workloads that were too slow in the past. > > > Possibly, but as the "THIS WAS NOT DOCUMENTED BEFORE?" question seemed to > > > still be in the release notes, I gathered that the information had not > > > reached its destination, hence the possible repetition. But maybe the > > > issue > > > is that this answer is not satisfactory. Sorry for the inconvenience. > > > > I removed it already based on feedback from someone else. > > Good. I looked at the online version which is off the latest commits by a > few hours. > > I'd consider moving "Upgrade to use DocBook 4.5 (Peter Eisentraut)" to the > doc section, maybe. Agreed, done. -- Bruce Momjian https://momjian.us EnterpriseDB https://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
Re: PG 13 release notes, first draft
At Wed, 13 May 2020 22:40:52 -0400, Bruce Momjian wrote in > On Thu, May 14, 2020 at 09:51:41AM +0900, Kyotaro Horiguchi wrote: > > At Wed, 13 May 2020 11:15:18 -0400, Bruce Momjian wrote > > in > > > On Wed, May 13, 2020 at 11:56:33AM +0900, Kyotaro Horiguchi wrote: > > It is just an more accurate (not an detailed) version of the > > previously proposed description. If we simplify that, I choose to > > remove explanation on wal_skip_threshold. > > > > How about this? > > > > WAL-logging is now skipped while all kinds of bulk-insertion, then > > relations are sync'ed to disk at commit. Previously this was done > > only for COPY operations, but the implementation had a bug that could > > cause data loss during crash recovery. > > OK, I went with this text, stating WAL "generation" is skipped: > > Allow skipping of WAL for full table writes if wal_level is 'minimal' > (Kyotaro Horiguchi) > > Relations larger than wal_skip_threshold will have their files > fsync'ed rather than generating WAL. Previously this was done > only for COPY operations, but the implementation had a bug that > could cause data loss during crash recovery. Although I can't help feeling it out-of-point a bit, it is right in apperarance. So, I don't object it. regards. -- Kyotaro Horiguchi NTT Open Source Software Center
Re: PG 13 release notes, first draft
Hello Bruce, * 34a0a81bfb We already have: Reformat tables containing function information for better clarity (Tom Lane) so it seems it is covered as part of this. AFAICR this one is not by the same author, and although the point was about better clarity, it was not about formating but rather about restructuring text vs binary string function documentations. Then Tom reformatted the result. Well, we were not even clear we should document changes in the functions section, so going into details of all the changes seems unwise. The restructuring was a significant change, and ISTM that another function of the release note is also to implicitely thank contributors (their name is appended, which does not bring any useful information about the feature from a release note perspective) hence my suggestion to include this one, the author of which is not Tom Lane. * e829337d42 Uh, this is a doc link formatting addition. I think this falls into the error message logic, where it is nice when people want it, but they don't need to know about it ahead of time. [...] I don't see it. While reading again the sequence, ISTM that I did not understand your first answer, so my answer was kind-of off topic, sorry. This is indeed "link formatting addition", which helps making the libpq doc more usable. Probably you do not need to know about it in advance, but I do not think that it is a good reason not to include it: with the same argument, a performance improvement would not need to be advertise, you'll see it when you need it. The same holds for all non-functional improvements, and there are many which are listed. Possibly, but as the "THIS WAS NOT DOCUMENTED BEFORE?" question seemed to still be in the release notes, I gathered that the information had not reached its destination, hence the possible repetition. But maybe the issue is that this answer is not satisfactory. Sorry for the inconvenience. I removed it already based on feedback from someone else. Good. I looked at the online version which is off the latest commits by a few hours. I'd consider moving "Upgrade to use DocBook 4.5 (Peter Eisentraut)" to the doc section, maybe. -- Fabien.
Re: PG 13 release notes, first draft
On Thu, May 14, 2020 at 09:51:41AM +0900, Kyotaro Horiguchi wrote: > At Wed, 13 May 2020 11:15:18 -0400, Bruce Momjian wrote in > > On Wed, May 13, 2020 at 11:56:33AM +0900, Kyotaro Horiguchi wrote: > It is just an more accurate (not an detailed) version of the > previously proposed description. If we simplify that, I choose to > remove explanation on wal_skip_threshold. > > How about this? > > WAL-logging is now skipped while all kinds of bulk-insertion, then > relations are sync'ed to disk at commit. Previously this was done > only for COPY operations, but the implementation had a bug that could > cause data loss during crash recovery. OK, I went with this text, stating WAL "generation" is skipped: Allow skipping of WAL for full table writes if wal_level is 'minimal' (Kyotaro Horiguchi) Relations larger than wal_skip_threshold will have their files fsync'ed rather than generating WAL. Previously this was done only for COPY operations, but the implementation had a bug that could cause data loss during crash recovery. -- Bruce Momjian https://momjian.us EnterpriseDB https://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
Re: PG 13 release notes, first draft
At Wed, 13 May 2020 11:15:18 -0400, Bruce Momjian wrote in > On Wed, May 13, 2020 at 11:56:33AM +0900, Kyotaro Horiguchi wrote: > > > > > > Allow skipping of WAL for new tables and indexes if wal_level is > > > 'minimal' (Kyotaro Horiguchi) > > > > > > Relations larger than wal_skip_threshold will have their files > > > fsync'ed rather than writing their WAL records. Previously this > > > was done only for COPY operations, but the implementation had a > > > bug that could cause data loss during crash recovery. > > > > I see it. It is giving weight on improvement. Looks good the overall > > structure of the description above. However, wal-skipping is always > > done regardless of table size. wal_skip_threshold is an optimization > > to choose which to use fsync or FPI records (that is, not WAL records > > in the common sense) at commit for speed. > > Well, as far as users are concerned, everything wrtiten to WAL is a WAL > record. I think that the significant point here is not that persistence is ensured by which of fsync or WAL , but whether WAL records are written for every insertion. The commit-time WA is just an alternative of fsync, which is faster than fsync'ing separate files for smaller files. > > So how about the following? > > > > All kinds of bulk-insertion are not WAL-logged then fsync'ed at > > commit. Using FPI WAL records instead of fsync for relations smaller > > than wal_skip_threshold. Previously this was done only for COPY > > operations and always using fsync, but the implementation had a bug > > that could cause data loss during crash recovery. > > That is too much detail for the release notes. We already will link to > the docs. Why put it here? It is just an more accurate (not an detailed) version of the previously proposed description. If we simplify that, I choose to remove explanation on wal_skip_threshold. How about this? WAL-logging is now skipped while all kinds of bulk-insertion, then relations are sync'ed to disk at commit. Previously this was done only for COPY operations, but the implementation had a bug that could cause data loss during crash recovery. regards. -- Kyotaro Horiguchi NTT Open Source Software Center
Re: PG 13 release notes, first draft
On Wed, May 13, 2020 at 07:18:38AM +0200, Fabien COELHO wrote: > > Hello Bruce, > > > > * e1ff780485 > > > > I was told in this email thread to not include that one. > > Ok. > > > > * 34a0a81bfb > > > > We already have: > > > > Reformat tables containing function information for better > > clarity (Tom Lane) > > > > so it seems it is covered as part of this. > > AFAICR this one is not by the same author, and although the point was about > better clarity, it was not about formating but rather about restructuring > text vs binary string function documentations. Then Tom reformatted the > result. Well, we were not even clear we should document changes in the functions section, so going into details of all the changes seems unwise. > > > * e829337d42 > > > > Uh, this is a doc link formatting addition. I think this falls into the > > error message logic, where it is nice when people want it, but they > > don't need to know about it ahead of time. > > Hmmm. ISTM that this is not really about "error message logic", it is about > navigating to libpq functions when one is reference in the description of > another to check what it does, which I had to do a lot while developing some > performance testing code for a project. I don't see it. > > > * "Document color support (Peter Eisentraut)" > > >THIS WAS NOT DOCUMENTED BEFORE? > > > > > > Not as such, AFAICR it was vaguely hinted about in the documentation of > > > command that would use it, but not even all of them. Now there is a new > > > specific section. > > > > Again, this is the first hash you gave. > > Possibly, but as the "THIS WAS NOT DOCUMENTED BEFORE?" question seemed to > still be in the release notes, I gathered that the information had not > reached its destination, hence the possible repetition. But maybe the issue > is that this answer is not satisfactory. Sorry for the inconvenience. I removed it already based on feedback from someone else. -- Bruce Momjian https://momjian.us EnterpriseDB https://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
Re: PG 13 release notes, first draft
On Wed, May 13, 2020 at 11:56:33AM +0900, Kyotaro Horiguchi wrote: > > > > Allow skipping of WAL for new tables and indexes if wal_level is > > 'minimal' (Kyotaro Horiguchi) > > > > Relations larger than wal_skip_threshold will have their files > > fsync'ed rather than writing their WAL records. Previously this > > was done only for COPY operations, but the implementation had a > > bug that could cause data loss during crash recovery. > > I see it. It is giving weight on improvement. Looks good the overall > structure of the description above. However, wal-skipping is always > done regardless of table size. wal_skip_threshold is an optimization > to choose which to use fsync or FPI records (that is, not WAL records > in the common sense) at commit for speed. Well, as far as users are concerned, everything wrtiten to WAL is a WAL record. > So how about the following? > > All kinds of bulk-insertion are not WAL-logged then fsync'ed at > commit. Using FPI WAL records instead of fsync for relations smaller > than wal_skip_threshold. Previously this was done only for COPY > operations and always using fsync, but the implementation had a bug > that could cause data loss during crash recovery. That is too much detail for the release notes. We already will link to the docs. Why put it here? -- Bruce Momjian https://momjian.us EnterpriseDB https://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
Re: PG 13 release notes, first draft
Hello Bruce, * e1ff780485 I was told in this email thread to not include that one. Ok. * 34a0a81bfb We already have: Reformat tables containing function information for better clarity (Tom Lane) so it seems it is covered as part of this. AFAICR this one is not by the same author, and although the point was about better clarity, it was not about formating but rather about restructuring text vs binary string function documentations. Then Tom reformatted the result. * e829337d42 Uh, this is a doc link formatting addition. I think this falls into the error message logic, where it is nice when people want it, but they don't need to know about it ahead of time. Hmmm. ISTM that this is not really about "error message logic", it is about navigating to libpq functions when one is reference in the description of another to check what it does, which I had to do a lot while developing some performance testing code for a project. * "Document color support (Peter Eisentraut)" THIS WAS NOT DOCUMENTED BEFORE? Not as such, AFAICR it was vaguely hinted about in the documentation of command that would use it, but not even all of them. Now there is a new specific section. Again, this is the first hash you gave. Possibly, but as the "THIS WAS NOT DOCUMENTED BEFORE?" question seemed to still be in the release notes, I gathered that the information had not reached its destination, hence the possible repetition. But maybe the issue is that this answer is not satisfactory. Sorry for the inconvenience. -- Fabien.
Re: PG 13 release notes, first draft
At Tue, 12 May 2020 16:38:09 -0400, Bruce Momjian wrote in > On Tue, May 12, 2020 at 01:09:08PM +0900, Kyotaro Horiguchi wrote: > > > > commit c6b9204 > > > > Author: Noah Misch > > > > AuthorDate: Sat Apr 4 12:25:34 2020 -0700 > > > > Commit: Noah Misch > > > > CommitDate: Sat Apr 4 12:25:34 2020 -0700 > > > > > > > > Skip WAL for new relfilenodes, under wal_level=minimal. > > > > > > > > Until now, only selected bulk operations (e.g. COPY) did this. If a > > > > given relfilenode received both a WAL-skipping COPY and a WAL-logged > > > > operation (e.g. INSERT), recovery could lose tuples from the COPY. > > > > See > > > > src/backend/access/transam/README section "Skipping WAL for New > > > > RelFileNode" for the new coding rules. Maintainers of table access > > > > methods should examine that section. > > > > > > OK, so how do we want to document this? Do I mention in the text below > > > the WAL skipping item that this fixes a bug where a mix of simultaneous > > > COPY and INSERT into a table could lose rows during crash recovery, or > > > create a new item? > > > > FWIW, as dicussed upthread, I suppose that the API change is not going > > to be in relnotes. > > > > something like this? > > > > - Fix bug of WAL-skipping optimiazation > > > > Previously a trasaction doing both of COPY and a WAL-logged operations > > like INSERT while wal_level=minimal can lead to loss of COPY'ed rows > > through crash recovery. Also this fix extends the WAL-skipping > > optimiazation to all kinds of bulk insert operations. > > Uh, that kind of mixes the bug fix and the feature in a way that it is > hard to understand. How about this? > > Allow skipping of WAL for new tables and indexes if wal_level is > 'minimal' (Kyotaro Horiguchi) > > Relations larger than wal_skip_threshold will have their files > fsync'ed rather than writing their WAL records. Previously this > was done only for COPY operations, but the implementation had a > bug that could cause data loss during crash recovery. I see it. It is giving weight on improvement. Looks good the overall structure of the description above. However, wal-skipping is always done regardless of table size. wal_skip_threshold is an optimization to choose which to use fsync or FPI records (that is, not WAL records in the common sense) at commit for speed. So how about the following? All kinds of bulk-insertion are not WAL-logged then fsync'ed at commit. Using FPI WAL records instead of fsync for relations smaller than wal_skip_threshold. Previously this was done only for COPY operations and always using fsync, but the implementation had a bug that could cause data loss during crash recovery. regards. -- Kyotaro Horiguchi NTT Open Source Software Center
Re: PG 13 release notes, first draft
On Tue, May 12, 2020 at 01:47:38PM -0400, Alvaro Herrera wrote: > On 2020-May-07, Bruce Momjian wrote: > > > OK, I have moved her name to be first. FYI, this commit was backpatched > > back through PG 11, though the commit message doesn't mention that. > > At some point I became an avid user of our src/tools/git_changelog, and > then it stopped making sense for me to highlight in the commit message > the fact that the commit is back-patched, since it's so obvious there. > Maybe that's wrong and I should get back in the habit of mentioning it. Uh, not sure. I don't need it since, as you said, src/tools/git_changelog covers it, but someone got confused since they looked at just the commit message without looking at src/tools/git_changelog. -- Bruce Momjian https://momjian.us EnterpriseDB https://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
Re: PG 13 release notes, first draft
On Tue, May 12, 2020 at 09:43:10AM +0200, Fabien COELHO wrote: > > Hello Bruce, > > > OK, section and item added, patch attached, > > Thanks! > > Some items that might be considered for the added documentation section: > > * e1ff780485 I was told in this email thread to not include that one. > * 34a0a81bfb We already have: Reformat tables containing function information for better clarity (Tom Lane) so it seems it is covered as part of this. > * e829337d42 Uh, this is a doc link formatting addition. I think this falls into the error message logic, where it is nice when people want it, but they don't need to know about it ahead of time. > * "Document color support (Peter Eisentraut)" >THIS WAS NOT DOCUMENTED BEFORE? > > Not as such, AFAICR it was vaguely hinted about in the documentation of > command that would use it, but not even all of them. Now there is a new > specific section. Again, this is the first hash you gave. -- Bruce Momjian https://momjian.us EnterpriseDB https://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
Re: PG 13 release notes, first draft
On Tue, May 12, 2020 at 01:09:08PM +0900, Kyotaro Horiguchi wrote: > > > commit c6b9204 > > > Author: Noah Misch > > > AuthorDate: Sat Apr 4 12:25:34 2020 -0700 > > > Commit: Noah Misch > > > CommitDate: Sat Apr 4 12:25:34 2020 -0700 > > > > > > Skip WAL for new relfilenodes, under wal_level=minimal. > > > > > > Until now, only selected bulk operations (e.g. COPY) did this. If a > > > given relfilenode received both a WAL-skipping COPY and a WAL-logged > > > operation (e.g. INSERT), recovery could lose tuples from the COPY. > > > See > > > src/backend/access/transam/README section "Skipping WAL for New > > > RelFileNode" for the new coding rules. Maintainers of table access > > > methods should examine that section. > > > > OK, so how do we want to document this? Do I mention in the text below > > the WAL skipping item that this fixes a bug where a mix of simultaneous > > COPY and INSERT into a table could lose rows during crash recovery, or > > create a new item? > > FWIW, as dicussed upthread, I suppose that the API change is not going > to be in relnotes. > > something like this? > > - Fix bug of WAL-skipping optimiazation > > Previously a trasaction doing both of COPY and a WAL-logged operations > like INSERT while wal_level=minimal can lead to loss of COPY'ed rows > through crash recovery. Also this fix extends the WAL-skipping > optimiazation to all kinds of bulk insert operations. Uh, that kind of mixes the bug fix and the feature in a way that it is hard to understand. How about this? Allow skipping of WAL for new tables and indexes if wal_level is 'minimal' (Kyotaro Horiguchi) Relations larger than wal_skip_threshold will have their files fsync'ed rather than writing their WAL records. Previously this was done only for COPY operations, but the implementation had a bug that could cause data loss during crash recovery. -- Bruce Momjian https://momjian.us EnterpriseDB https://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
Re: PG 13 release notes, first draft
On Mon, May 11, 2020 at 11:38:33PM -0400, Tom Lane wrote: > Bruce Momjian writes: > > On Mon, May 11, 2020 at 11:08:35PM -0400, Chapman Flack wrote: > >> 'specify' ? > > > I like that word if Tom prefers it. > > 'specify' works for me. Sure, done. -- Bruce Momjian https://momjian.us EnterpriseDB https://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
Re: PG 13 release notes, first draft
On 2020-May-07, Bruce Momjian wrote: > OK, I have moved her name to be first. FYI, this commit was backpatched > back through PG 11, though the commit message doesn't mention that. At some point I became an avid user of our src/tools/git_changelog, and then it stopped making sense for me to highlight in the commit message the fact that the commit is back-patched, since it's so obvious there. Maybe that's wrong and I should get back in the habit of mentioning it. -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Re: PG 13 release notes, first draft
Hello Bruce, OK, section and item added, patch attached, Thanks! Some items that might be considered for the added documentation section: * e1ff780485 * 34a0a81bfb * e829337d42 * "Document color support (Peter Eisentraut)" THIS WAS NOT DOCUMENTED BEFORE? Not as such, AFAICR it was vaguely hinted about in the documentation of command that would use it, but not even all of them. Now there is a new specific section. -- Fabien!
Re: PG 13 release notes, first draft
At Mon, 11 May 2020 20:12:04 -0400, Bruce Momjian wrote in > On Thu, May 7, 2020 at 09:22:02PM -0700, Noah Misch wrote: > > On Thu, May 07, 2020 at 09:38:34AM -0400, Bruce Momjian wrote: > > > > > > - Crash recovery was losing tuples written via COPY TO. This fixes > > > > > > the bug. > > > > > > > > > > This was not backpatched? > > > > > > > > Right. > > > > > > Oh. So you are saying we could lose COPY data on a crash, even after a > > > commit. That seems bad. Can you show me the commit info? I can't find > > > it. > > > > commit c6b9204 > > Author: Noah Misch > > AuthorDate: Sat Apr 4 12:25:34 2020 -0700 > > Commit: Noah Misch > > CommitDate: Sat Apr 4 12:25:34 2020 -0700 > > > > Skip WAL for new relfilenodes, under wal_level=minimal. > > > > Until now, only selected bulk operations (e.g. COPY) did this. If a > > given relfilenode received both a WAL-skipping COPY and a WAL-logged > > operation (e.g. INSERT), recovery could lose tuples from the COPY. See > > src/backend/access/transam/README section "Skipping WAL for New > > RelFileNode" for the new coding rules. Maintainers of table access > > methods should examine that section. > > OK, so how do we want to document this? Do I mention in the text below > the WAL skipping item that this fixes a bug where a mix of simultaneous > COPY and INSERT into a table could lose rows during crash recovery, or > create a new item? FWIW, as dicussed upthread, I suppose that the API change is not going to be in relnotes. something like this? - Fix bug of WAL-skipping optimiazation Previously a trasaction doing both of COPY and a WAL-logged operations like INSERT while wal_level=minimal can lead to loss of COPY'ed rows through crash recovery. Also this fix extends the WAL-skipping optimiazation to all kinds of bulk insert operations. regards. -- Kyotaro Horiguchi NTT Open Source Software Center
Re: PG 13 release notes, first draft
Bruce Momjian writes: > On Mon, May 11, 2020 at 11:08:35PM -0400, Chapman Flack wrote: >> 'specify' ? > I like that word if Tom prefers it. 'specify' works for me. regards, tom lane
Re: PG 13 release notes, first draft
On Mon, May 11, 2020 at 11:08:35PM -0400, Chapman Flack wrote: > On 05/11/20 22:48, Bruce Momjian wrote: > > On Mon, May 11, 2020 at 10:07:23PM -0400, Tom Lane wrote: > >> Bruce Momjian writes: > >>> Allow Unicode escapes, e.g., E'\u', U&'\', to represent any > >>> character available in the database encoding, even when the database > >>> encoding is not UTF-8 (Tom Lane) > >> > >> How about "to be used for" instead of "to represent"? "Represent" kind > >> of sounds like we're using these on output, which we aren't. > > > > Uh, I think "used for" is worse though, since we are not using it. I > > don't get the "output" feel of the word at all. > > 'specify' ? I like that word if Tom prefers it. -- Bruce Momjian https://momjian.us EnterpriseDB https://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
Re: PG 13 release notes, first draft
On 05/11/20 22:48, Bruce Momjian wrote: > On Mon, May 11, 2020 at 10:07:23PM -0400, Tom Lane wrote: >> Bruce Momjian writes: >>> Allow Unicode escapes, e.g., E'\u', U&'\', to represent any >>> character available in the database encoding, even when the database >>> encoding is not UTF-8 (Tom Lane) >> >> How about "to be used for" instead of "to represent"? "Represent" kind >> of sounds like we're using these on output, which we aren't. > > Uh, I think "used for" is worse though, since we are not using it. I > don't get the "output" feel of the word at all. 'specify' ? -Chap
Re: PG 13 release notes, first draft
On Tue, May 12, 2020 at 6:11 AM Justin Pryzby wrote: > > On Mon, May 11, 2020 at 10:52:41AM +0530, Amit Kapila wrote: > > > 1. We have allowed an (auto)vacuum to display additional information > > > about heap or index in case of an error in commit b61d161c14 [1]. > > > Now, in general, it might not be worth saying much about error > > > information but I think this one could help users in case they have > > > some corruption. For example, if one of the indexes on a relation has > > > some corrupted data (due to bad hardware or some bug), it will let the > > > user know the index information, and the user can take appropriate > > > action like either Reindex or maybe drop and recreate the index to > > > overcome the problem. > > I'm not opposed to including it, but I think it's still true that the user > doesn't need to know in advance that the error message will be additionally > helpful in the event of corruption. If we were to include more "error" items, > we might also include these: > > 71a8a4f6e36547bb060dbcc961ea9b57420f7190 Add backtrace support for error > reporting > 17a28b03645e27d73bf69a95d7569b61e58f06eb Improve the internal implementation > of ereport(). > 05f18c6b6b6e4b44302ee20a042cedc664532aa2 Added relation name in error > messages for constraint checks. > 33753ac9d7bc83dd9ccee9d5e678ed78a0725b4e Add object names to partition > integrity violations. > I think the first one (Add backtrace support for error reporting) seems to be quite useful as it can help to detect the problems faster. I think having a simple rule as Bruce has w.r.t "error messages" makes it easier to decide whether to take a particular commit or not but I feel some of these could help users to know the new functionality and might encourage them to upgrade to the new version. Sure, nobody is going to move due to only these features but along with other things, improved error handling is a good thing to know. -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com
Re: PG 13 release notes, first draft
On 2020/05/12 9:34, Bruce Momjian wrote: Could you add "Vinayak Pokale" as a co-author of the following feature since I sometimes read his old patch to create a patch [1] ? === E.1.3.1.6. System Views - Add system view pg_stat_progress_analyze to report analyze progress (Álvaro Herrera, Tatsuro Yamada) + Add system view pg_stat_progress_analyze to report analyze progress (Álvaro Herrera, Tatsuro Yamada, Vinayak Pokale) === Done. Hi Bruce, Thank you! Regards, Tatsuro Yamada
Re: PG 13 release notes, first draft
On Mon, May 11, 2020 at 10:23:53PM -0400, Tom Lane wrote: > Bruce Momjian writes: > > Well, are you suggesting a new section because the glossary shouldn't be > > listed under source code, or because you want the function reformatting > > added? We just need to understand what the purpose is. We already have > > the glossary listed, since that is new and user-visible. > > The implication of what you say here is that "is it user-visible?" > is a criterion for whether to release-note something. By that logic > we probably *should* relnote the function table layout changes, because > they sure as heck are user-visible. People might or might not notice > addition of a glossary, but I think just about every user consults > the function/operator tables regularly. > > I concur with Alvaro's position that if we are listing documentation > changes, pushing them under "Source Code" is not the way to do it. > That subsection has always been understood to be "stuff you don't > care about if you're not a hacker". > > So that sort of leads me to the conclusion that "major documentation > changes" might be a reasonable sub-heading for the release notes. OK, section and item added, patch attached, -- Bruce Momjian https://momjian.us EnterpriseDB https://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription + diff --git a/doc/src/sgml/release-13.sgml b/doc/src/sgml/release-13.sgml index 83470e9ba1..34211e2b68 100644 --- a/doc/src/sgml/release-13.sgml +++ b/doc/src/sgml/release-13.sgml @@ -2331,7 +2331,7 @@ Use the directory of the pg_upgrade binary as the default new 'bindir' location -Source Code +Documentation @@ -2347,6 +2347,27 @@ Add a glossary to the documentation (Corey Huinker, Jrgen Purtz, Roger Har + + + + +Reformat tables containing function information for better clarity (Tom Lane) + + + + + + + + + +Source Code + + +
Re: PG 13 release notes, first draft
On Tue, May 12, 2020 at 6:36 AM Bruce Momjian wrote: > > On Mon, May 11, 2020 at 07:41:55PM -0500, Justin Pryzby wrote: > > On Mon, May 11, 2020 at 10:52:41AM +0530, Amit Kapila wrote: > > > One more observation: > > > > > > Allow inserts to trigger autovacuum activity (Laurenz Albe, Darafei > > > Praliaskouski) > > > This new behavior allows pages to be set as all-visible, which then > > > allows index-only scans, ... > > > > > The above sentence sounds to mean that this feature allows index-only > > > scans in more number of cases after this feature. Is that what you > > > intend to say? If so, is that correct? Because I think this will > > > allow index-only scans to skip "Heap Fetches" in more cases. > > > > I think what you mean is that the autovacuum feature, in addition to > > encouraging the *planner* to choose an indexonly scan, will *also* allow (at > > execution time) fewer heap fetches for a plan which would have > > already/previously used IOS. Right ? So maybe it should say "allows OR > > IMPROVES index-only scans" or "allows plans which use IOS to run more > > efficiently". > > Yes, I see your point now. New text is: > > This new behavior reduces the work necessary when the table > needs to be frozen and allows pages to be set as all-visible. > All-visible pages allow index-only scans to access fewer heap rows. > The next text LGTM. Thanks. -- With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com
Re: PG 13 release notes, first draft
On Mon, May 11, 2020 at 10:07:23PM -0400, Tom Lane wrote: > Bruce Momjian writes: > > I like your wording, but the "that encoding" wasn't clear enough for me, > > so I reworded it to: > > > Allow Unicode escapes, e.g., E'\u', U&'\', to represent any > > character available in the database encoding, even when the database > > encoding is not UTF-8 (Tom Lane) > > How about "to be used for" instead of "to represent"? "Represent" kind > of sounds like we're using these on output, which we aren't. Uh, I think "used for" is worse though, since we are not using it. I don't get the "output" feel of the word at all. -- Bruce Momjian https://momjian.us EnterpriseDB https://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
Re: PG 13 release notes, first draft
On Tue, May 12, 2020 at 10:54:52AM +0900, Michael Paquier wrote: > On Mon, May 11, 2020 at 07:18:56PM -0400, Bruce Momjian wrote: > > On Fri, May 8, 2020 at 11:55:33AM +0900, Michael Paquier wrote: > >> Should e2e02191 be added to the notes? This commit means that we > >> actually dropped support for Windows 2000 (finally) at run-time. > > > > Oh, yes. This is much more important than the removal of support for > > non-ELF BSD systems, which I already listed. The new text is: > > > > Remove support for Windows 2000 (Michael Paquier) > > Sounds fine to me. > > >> At the same time I see no mention of 79dfa8af, which added better > >> error handling when backends the SSL context with incorrect bounds. > > > > I skipped that commit since people don't normally care about better > > error messages until they see the error message, and then they are happy > > it is there, unless this is some chronic error message problem we are > > fixing. > > Okay. > > > I thought this fell into the previous category about error messages, but > > coloring is different. Can we say these utilities now honor the color > > environment variables? > > Exactly, I actually became aware of that possibility after plugging > in the common logging APIs to oid2name and vacuumlo as of fc8cb94b so > this was not mentioned in the log message. And anything using > src/common/logging.c can make use of the colorized output with > PG_COLOR[S] set. > > > Are these the only new ones? > > I can recall an extra one in this case: pgbench as of 30a3e77. And I > don't see any new callers of pg_logging_init() in the stuff that > already existed in ~12. I am not sure we even mentioned this in 12. Should we document this somewhere? Maybe a blog posting? -- Bruce Momjian https://momjian.us EnterpriseDB https://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
Re: PG 13 release notes, first draft
Bruce Momjian writes: > Well, are you suggesting a new section because the glossary shouldn't be > listed under source code, or because you want the function reformatting > added? We just need to understand what the purpose is. We already have > the glossary listed, since that is new and user-visible. The implication of what you say here is that "is it user-visible?" is a criterion for whether to release-note something. By that logic we probably *should* relnote the function table layout changes, because they sure as heck are user-visible. People might or might not notice addition of a glossary, but I think just about every user consults the function/operator tables regularly. I concur with Alvaro's position that if we are listing documentation changes, pushing them under "Source Code" is not the way to do it. That subsection has always been understood to be "stuff you don't care about if you're not a hacker". So that sort of leads me to the conclusion that "major documentation changes" might be a reasonable sub-heading for the release notes. regards, tom lane
Re: PG 13 release notes, first draft
On 2020-May-11, Bruce Momjian wrote: > On Mon, May 11, 2020 at 08:34:56PM -0400, Alvaro Herrera wrote: > > Yes, we had to touch the source code in order to add documentation; but > > so what? Everything touches the source code, but that doesn't mean it > > should be listed there. I don't see what's the problem with having a > > new subsection in the relnotes entitled "Documentation" where these two > > items appears (glossary + new doc table format) are listed. It's not > > like it's going to cost us a lot of space or anything. > Well, are you suggesting a new section because the glossary shouldn't be > listed under source code, or because you want the function reformatting > added? We just need to understand what the purpose is. We already have > the glossary listed, since that is new and user-visible. IMO the table reformatting change is significant enough to be noteworthy. I'm suggesting that a new Documentation subsection would list both that and the glossary, separately from Source Code -- so it'd be E.1.3.10 and Source Code would be E.1.3.11. -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Re: PG 13 release notes, first draft
Bruce Momjian writes: > I like your wording, but the "that encoding" wasn't clear enough for me, > so I reworded it to: > Allow Unicode escapes, e.g., E'\u', U&'\', to represent any > character available in the database encoding, even when the database > encoding is not UTF-8 (Tom Lane) How about "to be used for" instead of "to represent"? "Represent" kind of sounds like we're using these on output, which we aren't. regards, tom lane
Re: PG 13 release notes, first draft
On Mon, May 11, 2020 at 07:18:56PM -0400, Bruce Momjian wrote: > On Fri, May 8, 2020 at 11:55:33AM +0900, Michael Paquier wrote: >> Should e2e02191 be added to the notes? This commit means that we >> actually dropped support for Windows 2000 (finally) at run-time. > > Oh, yes. This is much more important than the removal of support for > non-ELF BSD systems, which I already listed. The new text is: > > Remove support for Windows 2000 (Michael Paquier) Sounds fine to me. >> At the same time I see no mention of 79dfa8af, which added better >> error handling when backends the SSL context with incorrect bounds. > > I skipped that commit since people don't normally care about better > error messages until they see the error message, and then they are happy > it is there, unless this is some chronic error message problem we are > fixing. Okay. > I thought this fell into the previous category about error messages, but > coloring is different. Can we say these utilities now honor the color > environment variables? Exactly, I actually became aware of that possibility after plugging in the common logging APIs to oid2name and vacuumlo as of fc8cb94b so this was not mentioned in the log message. And anything using src/common/logging.c can make use of the colorized output with PG_COLOR[S] set. > Are these the only new ones? I can recall an extra one in this case: pgbench as of 30a3e77. And I don't see any new callers of pg_logging_init() in the stuff that already existed in ~12. -- Michael signature.asc Description: PGP signature
Re: PG 13 release notes, first draft
On Mon, May 11, 2020 at 6:23 PM Bruce Momjian wrote: > OK, I was able to add some of it cleanly: > > This allows efficient btree indexing of low cardinality columns by > storing duplicate keys only once. Users upgrading with pg_upgrade > will need to use REINDEX to make use of this feature. That seems like a reasonable compromise. Thanks! -- Peter Geoghegan
Re: PG 13 release notes, first draft
On Tue, May 12, 2020 at 8:51 AM Bruce Momjian wrote: > On Fri, May 8, 2020 at 12:07:09PM +0900, Amit Langote wrote: > > I have attached a patch with my suggestions above. > > OK, I slightly modified the wording of your first change, patch > attached. Thanks. I checked that what you committed looks fine. -- Amit Langote EnterpriseDB: http://www.enterprisedb.com
Re: PG 13 release notes, first draft
On Mon, May 11, 2020 at 09:15:43PM -0400, Bruce Momjian wrote: > On Mon, May 11, 2020 at 05:33:40PM -0700, Peter Geoghegan wrote: > > On Mon, May 11, 2020 at 5:14 PM Bruce Momjian wrote: > > > Agreed. How is this? > > > > > > This allows efficient btree indexing of low cardinality columns. > > > Users upgrading with pg_upgrade will need to use REINDEX to make > > > use of > > > this feature. > > > > I still think that the release notes should say that the key is only > > stored once, while TIDs that identify table rows are stored together > > as an array. Everything that's helpful (or harmful) about the feature > > happens as a consequence of that. This isn't hard to grasp > > intuitively, and is totally in line with how things like Oracle bitmap > > indexes are presented to ordinary users. > > I still don't think these details belong in the release notes. OK, I was able to add some of it cleanly: This allows efficient btree indexing of low cardinality columns by storing duplicate keys only once. Users upgrading with pg_upgrade will need to use REINDEX to make use of this feature. -- Bruce Momjian https://momjian.us EnterpriseDB https://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
Re: PG 13 release notes, first draft
On Mon, May 11, 2020 at 05:33:40PM -0700, Peter Geoghegan wrote: > On Mon, May 11, 2020 at 5:14 PM Bruce Momjian wrote: > > Agreed. How is this? > > > > This allows efficient btree indexing of low cardinality columns. > > Users upgrading with pg_upgrade will need to use REINDEX to make > > use of > > this feature. > > I still think that the release notes should say that the key is only > stored once, while TIDs that identify table rows are stored together > as an array. Everything that's helpful (or harmful) about the feature > happens as a consequence of that. This isn't hard to grasp > intuitively, and is totally in line with how things like Oracle bitmap > indexes are presented to ordinary users. I still don't think these details belong in the release notes. -- Bruce Momjian https://momjian.us EnterpriseDB https://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
Re: PG 13 release notes, first draft
On Mon, May 11, 2020 at 07:41:01PM -0500, Justin Pryzby wrote:
> |Allow function call backtraces of errors to be logged (Peter Eisentraut,
> Álvaro Herrera)
> |Server variable backtrace_functions specifies which C functions should
> generate backtraces on error.
>
> I think the details in the description are eclipsing the most important thing:
> backtraces on Assert(). I would say "Support for showing backtraces on
> error".
Uh, you mean this adds backtraces for errors and asserts? Are
non-developers running assert builds?
> Regarding this one:
> |Add system view pg_shmem_allocations to display shared memory usage (Andres
> Freund, Robert Haas)
> |WHAT IS THE ENTRY WITH NO NAME?
>
> There seems to be two special, "unnamed" cases:
> src/backend/storage/ipc/shmem.c-/* output shared memory allocated but
> not counted via the shmem index */
> src/backend/storage/ipc/shmem.c:values[0] =
> CStringGetTextDatum("");
> ...
> src/backend/storage/ipc/shmem.c-/* output as-of-yet unused shared
> memory */
> src/backend/storage/ipc/shmem.c-nulls[0] = true;
>
> That seems to be adequately documented:
> https://www.postgresql.org/docs/devel/view-pg-shmem-allocations.html
> |NULL for unused memory and for anonymous allocations.
OK, thanks. Comment removed.
> I would remove this part:
> "Previously, this could only be set at server start."
OK, you rae saying it is already clear, agreed, removed.
--
Bruce Momjian https://momjian.us
EnterpriseDB https://enterprisedb.com
+ As you are, so once was I. As I am, so you will be. +
+ Ancient Roman grave inscription +
Re: PG 13 release notes, first draft
On Mon, May 11, 2020 at 05:09:54PM -0400, Alvaro Herrera wrote: > On 2020-May-11, Alvaro Herrera wrote: > > > Hello > > > > > + > > > + > > > + > > > +Add psql commands to report operator classes and operator families > > > (Sergey Cherkashin, Nikita Glukhov, Alexander Korotkov) > > > + > > > > I think this item should list the commands in question: > > \dA, \dAc, \dAf, \dAo, \dAp > > (All the other psql entries in the relnotes do that). > > Sorry, it's the last four only -- \dA is an older command. OK, confirmed, thanks. -- Bruce Momjian https://momjian.us EnterpriseDB https://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
Re: PG 13 release notes, first draft
On Mon, May 11, 2020 at 07:41:55PM -0500, Justin Pryzby wrote: > On Mon, May 11, 2020 at 10:52:41AM +0530, Amit Kapila wrote: > > One more observation: > > > > Allow inserts to trigger autovacuum activity (Laurenz Albe, Darafei > > Praliaskouski) > > This new behavior allows pages to be set as all-visible, which then > > allows index-only scans, ... > > > The above sentence sounds to mean that this feature allows index-only > > scans in more number of cases after this feature. Is that what you > > intend to say? If so, is that correct? Because I think this will > > allow index-only scans to skip "Heap Fetches" in more cases. > > I think what you mean is that the autovacuum feature, in addition to > encouraging the *planner* to choose an indexonly scan, will *also* allow (at > execution time) fewer heap fetches for a plan which would have > already/previously used IOS. Right ? So maybe it should say "allows OR > IMPROVES index-only scans" or "allows plans which use IOS to run more > efficiently". Yes, I see your point now. New text is: This new behavior reduces the work necessary when the table needs to be frozen and allows pages to be set as all-visible. All-visible pages allow index-only scans to access fewer heap rows. -- Bruce Momjian https://momjian.us EnterpriseDB https://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
Re: PG 13 release notes, first draft
On Mon, May 11, 2020 at 08:34:56PM -0400, Alvaro Herrera wrote: > On 2020-May-11, Bruce Momjian wrote: > > > We have long discussed how much of the release notes is to reward > > behavior, and we have settled on having the names on the items, and the > > Acknowledgments section at the bottom. > > Yes, we had to touch the source code in order to add documentation; but > so what? Everything touches the source code, but that doesn't mean it > should be listed there. I don't see what's the problem with having a > new subsection in the relnotes entitled "Documentation" where these two > items appears (glossary + new doc table format) are listed. It's not > like it's going to cost us a lot of space or anything. > > I don't think there is any circularity argument BTW -- we're not going > to document that we added release notes. And changing the table format > is not entirely pointless, given that we've historically had trouble > with these tables (read: they're totally unusable in PDF). Well, are you suggesting a new section because the glossary shouldn't be listed under source code, or because you want the function reformatting added? We just need to understand what the purpose is. We already have the glossary listed, since that is new and user-visible. -- Bruce Momjian https://momjian.us EnterpriseDB https://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
Re: PG 13 release notes, first draft
On Mon, May 11, 2020 at 04:50:50PM -0400, Alvaro Herrera wrote: > Hello > > > + > > + > > + > > +Add psql commands to report operator classes and operator families (Sergey > > Cherkashin, Nikita Glukhov, Alexander Korotkov) > > + > > I think this item should list the commands in question: > \dA, \dAc, \dAf, \dAo, \dAp > (All the other psql entries in the relnotes do that). Good idea. I added this paragraph: The new commands are \dAc, \dAf, \dAo, and \dAp. I didn't see any changes to \dA except regression tests. -- Bruce Momjian https://momjian.us EnterpriseDB https://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
Re: PG 13 release notes, first draft
On Mon, May 11, 2020 at 10:52:41AM +0530, Amit Kapila wrote: > > 1. We have allowed an (auto)vacuum to display additional information > > about heap or index in case of an error in commit b61d161c14 [1]. > > Now, in general, it might not be worth saying much about error > > information but I think this one could help users in case they have > > some corruption. For example, if one of the indexes on a relation has > > some corrupted data (due to bad hardware or some bug), it will let the > > user know the index information, and the user can take appropriate > > action like either Reindex or maybe drop and recreate the index to > > overcome the problem. I'm not opposed to including it, but I think it's still true that the user doesn't need to know in advance that the error message will be additionally helpful in the event of corruption. If we were to include more "error" items, we might also include these: 71a8a4f6e36547bb060dbcc961ea9b57420f7190 Add backtrace support for error reporting 17a28b03645e27d73bf69a95d7569b61e58f06eb Improve the internal implementation of ereport(). 05f18c6b6b6e4b44302ee20a042cedc664532aa2 Added relation name in error messages for constraint checks. 33753ac9d7bc83dd9ccee9d5e678ed78a0725b4e Add object names to partition integrity violations. > One more observation: > > Allow inserts to trigger autovacuum activity (Laurenz Albe, Darafei > Praliaskouski) > This new behavior allows pages to be set as all-visible, which then > allows index-only scans, ... > The above sentence sounds to mean that this feature allows index-only > scans in more number of cases after this feature. Is that what you > intend to say? If so, is that correct? Because I think this will > allow index-only scans to skip "Heap Fetches" in more cases. I think what you mean is that the autovacuum feature, in addition to encouraging the *planner* to choose an indexonly scan, will *also* allow (at execution time) fewer heap fetches for a plan which would have already/previously used IOS. Right ? So maybe it should say "allows OR IMPROVES index-only scans" or "allows plans which use IOS to run more efficiently". Separate from Amit's comment, I suggest to say: | This new behavior allows autovacuum to set pages as all-visible, which then | allows index-only scans, ... ..otherwise it sounds like this feature implemented the concept of "all-visible". -- Justin
Re: PG 13 release notes, first draft
|Allow function call backtraces of errors to be logged (Peter Eisentraut,
Álvaro Herrera)
|Server variable backtrace_functions specifies which C functions should
generate backtraces on error.
I think the details in the description are eclipsing the most important thing:
backtraces on Assert(). I would say "Support for showing backtraces on error".
Regarding this one:
|Add system view pg_shmem_allocations to display shared memory usage (Andres
Freund, Robert Haas)
|WHAT IS THE ENTRY WITH NO NAME?
There seems to be two special, "unnamed" cases:
src/backend/storage/ipc/shmem.c-/* output shared memory allocated but
not counted via the shmem index */
src/backend/storage/ipc/shmem.c:values[0] =
CStringGetTextDatum("");
...
src/backend/storage/ipc/shmem.c-/* output as-of-yet unused shared
memory */
src/backend/storage/ipc/shmem.c-nulls[0] = true;
That seems to be adequately documented:
https://www.postgresql.org/docs/devel/view-pg-shmem-allocations.html
|NULL for unused memory and for anonymous allocations.
I would remove this part:
"Previously, this could only be set at server start."
--
Justin
Re: PG 13 release notes, first draft
On 2020-May-11, Bruce Momjian wrote: > We have long discussed how much of the release notes is to reward > behavior, and we have settled on having the names on the items, and the > Acknowledgments section at the bottom. Yes, we had to touch the source code in order to add documentation; but so what? Everything touches the source code, but that doesn't mean it should be listed there. I don't see what's the problem with having a new subsection in the relnotes entitled "Documentation" where these two items appears (glossary + new doc table format) are listed. It's not like it's going to cost us a lot of space or anything. I don't think there is any circularity argument BTW -- we're not going to document that we added release notes. And changing the table format is not entirely pointless, given that we've historically had trouble with these tables (read: they're totally unusable in PDF). -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Re: PG 13 release notes, first draft
On Mon, May 11, 2020 at 03:19:50PM +0900, Tatsuro Yamada wrote: > Hi Bruce, > > On 2020/05/05 12:16, Bruce Momjian wrote: > > I have committed the first draft of the PG 13 release notes. You can > > see them here: > > > > https://momjian.us/pgsql_docs/release-13.html > > > > It still needs markup, word wrap, and indenting. The community doc > > build should happen in a few hours. > > Thanks for working on this! :-D > > Could you add "Vinayak Pokale" as a co-author of the following feature since > I sometimes read his old patch to create a patch [1] ? > > === > E.1.3.1.6. System Views > > - Add system view pg_stat_progress_analyze to report analyze progress > (Álvaro Herrera, Tatsuro Yamada) > > + Add system view pg_stat_progress_analyze to report analyze progress > (Álvaro Herrera, Tatsuro Yamada, Vinayak Pokale) > === Done. -- Bruce Momjian https://momjian.us EnterpriseDB https://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
Re: PG 13 release notes, first draft
On Mon, May 11, 2020 at 5:14 PM Bruce Momjian wrote: > Agreed. How is this? > > This allows efficient btree indexing of low cardinality columns. > Users upgrading with pg_upgrade will need to use REINDEX to make use > of > this feature. I still think that the release notes should say that the key is only stored once, while TIDs that identify table rows are stored together as an array. Everything that's helpful (or harmful) about the feature happens as a consequence of that. This isn't hard to grasp intuitively, and is totally in line with how things like Oracle bitmap indexes are presented to ordinary users. -- Peter Geoghegan
Re: PG 13 release notes, first draft (ltree dot star)
On Sun, May 10, 2020 at 03:09:47PM -0500, Justin Pryzby wrote:
> > In ltree, when using adjacent asterisks with braces, e.g. "*{2}.*{3}",
> > properly interpret that as "*{5}" (Nikita Glukhov)
>
> I think that should say ".*" not "*", as in:
>
> > In ltree, when using adjacent asterisks with braces, e.g. ".*{2}.*{3}",
> > properly interpret that as "*{5}" (Nikita Glukhov)
>
> The existing text clearly came from the commit message, which (based on its
> regression tests) I think was the source of the missing dot.
>
> commit 9950c8aadf0edd31baec74a729d47d94af636c06
> Author: Tom Lane
> Date: Sat Mar 28 18:31:05 2020 -0400
>
> Fix lquery's behavior for consecutive '*' items.
>
> Something like "*{2}.*{3}" should presumably mean the same as
> "*{5}", but it didn't. Improve that.
> ...
OK, fixed to be:
In ltree, when using adjacent asterisks with braces, e.g. ".*{2}.*{3}",
properly interpret that as ".*{5}" (Nikita Glukhov)
I added two dots.
--
Bruce Momjian https://momjian.us
EnterpriseDB https://enterprisedb.com
+ As you are, so once was I. As I am, so you will be. +
+ Ancient Roman grave inscription +
Re: PG 13 release notes, first draft
On Mon, May 11, 2020 at 10:52:41AM +0530, Amit Kapila wrote: > One more observation: > > Allow inserts to trigger autovacuum activity (Laurenz Albe, Darafei > Praliaskouski) > This new behavior allows pages to be set as all-visible, which then > allows index-only scans, ... > > The above sentence sounds to mean that this feature allows index-only > scans in more number of cases after this feature. Is that what you > intend to say? If so, is that correct? Because I think this will Yes. > allow index-only scans to skip "Heap Fetches" in more cases. Uh, by definition an index-only scan only scans the index, not the heap, right? It is true there are fewer heap fetches, but fewer heap features I thought mean more index-only scans. -- Bruce Momjian https://momjian.us EnterpriseDB https://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
Re: PG 13 release notes, first draft
On Sat, May 9, 2020 at 11:16:27AM +0530, Amit Kapila wrote: > On Tue, May 5, 2020 at 8:46 AM Bruce Momjian wrote: > > > > I have committed the first draft of the PG 13 release notes. You can > > see them here: > > > > https://momjian.us/pgsql_docs/release-13.html > > > > Thanks for the work. I was today going through the release notes and > was wondering whether we should consider adding information about some > other work done for PG13. > 1. We have allowed an (auto)vacuum to display additional information > about heap or index in case of an error in commit b61d161c14 [1]. > Now, in general, it might not be worth saying much about error > information but I think this one could help users in case they have > some corruption. For example, if one of the indexes on a relation has > some corrupted data (due to bad hardware or some bug), it will let the > user know the index information, and the user can take appropriate > action like either Reindex or maybe drop and recreate the index to > overcome the problem. I mentioned my approach to error message changes in a previous email today: I skipped that commit since people don't normally care about better error messages until they see the error message, and then they are happy it is there, unless this is some chronic error message problem we are fixing. > 2. In the "Source Code" section, we can add information about > infrastructure enhancement for parallelism. Basically, "Allow > relation extension and page lock to conflict among parallel-group > members" [2][3]. This will allow improving the parallelism further in > many cases like (a) we can allow multiple workers to operate on a heap > and index in a parallel vacuum, (b) we can allow parallel Inserts, > etc. Uh, if there is no user-visible behavior change, this seems too low level for the release notes. -- Bruce Momjian https://momjian.us EnterpriseDB https://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
Re: PG 13 release notes, first draft
On Fri, May 8, 2020 at 09:14:01AM +0200, Fabien COELHO wrote: > > It seems (a) pointless > > I disagree, on the very principle of free software values as a social > movement. > > Documentation improvements should be encouraged, and recognizing these in > the release notes contributes to do that for what is a lot of unpaid work > given freely by many people. I do not see this as "pointless", on the > contrary, having something "free" in a mostly mercantile world is odd enough > to deserve some praise. > > How many hours have you spent on the function operator table improvements? > If someone else had contributed that and only that to a release, would it > not justify two lines of implicit thanks somewhere down in the release > notes? > > Moreover adding a documentation section costs next to nothing, so what is > the actual point of not doing it? Also, having some documentation > improvements listed under "source code" does not make sense: writing good, > precise and structured English is not "source code". We have long discussed how much of the release notes is to reward behavior, and we have settled on having the names on the items, and the Acknowledgments section at the bottom. If you want to revisit that decision, you should start a new thread because doing it for just this item doesn't make sense. -- Bruce Momjian https://momjian.us EnterpriseDB https://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
Re: PG 13 release notes, first draft
On Mon, May 11, 2020 at 05:05:29PM -0700, Peter Geoghegan wrote: > On Mon, May 11, 2020 at 4:10 PM Bruce Momjian wrote: > > > think that you should point out that deduplication works by storing > > > the duplicates in the obvious way: Only storing the key once per > > > distinct value (or once per distinct combination of values in the case > > > of multi-column indexes), followed by an array of TIDs (i.e. a posting > > > list). Each TID points to a separate row in the table. > > > > These are not details that should be in the release notes since the > > internal representation is not important for its use. > > I am not concerned about describing the specifics of the on-disk > representation, and I don't feel too strongly about the storage > parameter (leave it out). I only ask that the wording convey the fact > that the deduplication feature is not just a quantitative improvement > -- it's a qualitative behavioral change, that will help data > warehousing in particular. This wasn't the case with the v12 work on > B-Tree duplicates (as I said last year, I thought of the v12 stuff as > fixing a problem more than an enhancement). > > With the deduplication feature added to Postgres v13, the B-Tree code > can now gracefully deal with low cardinality data by compressing the > duplicates as needed. This is comparable to bitmap indexes in > proprietary database systems, but without most of their disadvantages > (in particular around heavyweight locking, deadlocks that abort > transactions, etc). It's easy to imagine this making a big difference > with analytics workloads. The v12 work made indexes with lots of > duplicates 15%-30% smaller (compared to v11), but the v13 work can > make them 60% - 80% smaller in many common cases (compared to v12). In > extreme cases indexes might even be ~12x smaller (though that will be > rare). Agreed. How is this? This allows efficient btree indexing of low cardinality columns. Users upgrading with pg_upgrade will need to use REINDEX to make use of this feature. -- Bruce Momjian https://momjian.us EnterpriseDB https://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
Re: PG 13 release notes, first draft
On Thu, May 7, 2020 at 09:22:02PM -0700, Noah Misch wrote: > On Thu, May 07, 2020 at 09:38:34AM -0400, Bruce Momjian wrote: > > > > > - Crash recovery was losing tuples written via COPY TO. This fixes > > > > > the bug. > > > > > > > > This was not backpatched? > > > > > > Right. > > > > Oh. So you are saying we could lose COPY data on a crash, even after a > > commit. That seems bad. Can you show me the commit info? I can't find > > it. > > commit c6b9204 > Author: Noah Misch > AuthorDate: Sat Apr 4 12:25:34 2020 -0700 > Commit: Noah Misch > CommitDate: Sat Apr 4 12:25:34 2020 -0700 > > Skip WAL for new relfilenodes, under wal_level=minimal. > > Until now, only selected bulk operations (e.g. COPY) did this. If a > given relfilenode received both a WAL-skipping COPY and a WAL-logged > operation (e.g. INSERT), recovery could lose tuples from the COPY. See > src/backend/access/transam/README section "Skipping WAL for New > RelFileNode" for the new coding rules. Maintainers of table access > methods should examine that section. OK, so how do we want to document this? Do I mention in the text below the WAL skipping item that this fixes a bug where a mix of simultaneous COPY and INSERT into a table could lose rows during crash recovery, or create a new item? -- Bruce Momjian https://momjian.us EnterpriseDB https://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
Re: PG 13 release notes, first draft
On Mon, May 11, 2020 at 4:10 PM Bruce Momjian wrote: > > think that you should point out that deduplication works by storing > > the duplicates in the obvious way: Only storing the key once per > > distinct value (or once per distinct combination of values in the case > > of multi-column indexes), followed by an array of TIDs (i.e. a posting > > list). Each TID points to a separate row in the table. > > These are not details that should be in the release notes since the > internal representation is not important for its use. I am not concerned about describing the specifics of the on-disk representation, and I don't feel too strongly about the storage parameter (leave it out). I only ask that the wording convey the fact that the deduplication feature is not just a quantitative improvement -- it's a qualitative behavioral change, that will help data warehousing in particular. This wasn't the case with the v12 work on B-Tree duplicates (as I said last year, I thought of the v12 stuff as fixing a problem more than an enhancement). With the deduplication feature added to Postgres v13, the B-Tree code can now gracefully deal with low cardinality data by compressing the duplicates as needed. This is comparable to bitmap indexes in proprietary database systems, but without most of their disadvantages (in particular around heavyweight locking, deadlocks that abort transactions, etc). It's easy to imagine this making a big difference with analytics workloads. The v12 work made indexes with lots of duplicates 15%-30% smaller (compared to v11), but the v13 work can make them 60% - 80% smaller in many common cases (compared to v12). In extreme cases indexes might even be ~12x smaller (though that will be rare). -- Peter Geoghegan
