Re: Parallel copy

2020-11-01 Thread Heikki Linnakangas
On 02/11/2020 09:10, Heikki Linnakangas wrote: On 02/11/2020 08:14, Amit Kapila wrote: We have discussed both these approaches (a) single producer multiple consumer, and (b) all workers doing the processing as you are saying in the beginning and concluded that (a) is better, see some of the

Re: making update/delete of inheritance trees scale better

2020-11-01 Thread Amit Langote
On Sat, Oct 31, 2020 at 7:26 AM Heikki Linnakangas wrote: > On 31/10/2020 00:12, Tom Lane wrote: > > Heikki Linnakangas writes: > >> But if you do: > > > >> postgres=# explain verbose update tab set a = 1, b = 2; > >> QUERY PLAN > >>

Reduce the number of special cases to build contrib modules on windows

2020-11-01 Thread David Rowley
Hi, At the moment we do very basic parsing of makefiles to build the visual studio project file in order to build our contrib modules on MSVC. This parsing is quite basic and still requires a number of special cases to get enough information into the project file in order for the build to

Re: Online checksums verification in the backend

2020-11-01 Thread Michael Paquier
On Sun, Nov 01, 2020 at 05:50:06PM -0800, Andres Freund wrote: > On 2020-11-02 10:45:00 +0900, Michael Paquier wrote: > > On 2020-11-02 10:05:25 +0900, Michael Paquier wrote: > > > > There is no place doing that on HEAD. > > > > > > Err? > > > How is this not doing IO while holding a buffer

Re: Parallel copy

2020-11-01 Thread Heikki Linnakangas
On 02/11/2020 08:14, Amit Kapila wrote: On Fri, Oct 30, 2020 at 10:11 PM Heikki Linnakangas wrote: Leader process: The leader process is simple. It picks the next FREE buffer, fills it with raw data from the file, and marks it as FILLED. If no buffers are FREE, wait. Worker process: 1.

reindex partitioned indexes: refactor ReindexRelationConcurrently ?

2020-11-01 Thread Justin Pryzby
This is a new thread about possible refactoring and commit a6642b3ae ("Add support for partitioned tables and indexes in REINDEX") Currently, ReindexPartitions() calls ReindexMultipleInternal() which loops around ReindexRelationConcurrently(), similar to what's done with ReindexMultipleTables()

Re: Collation versioning

2020-11-01 Thread Julien Rouhaud
On Sat, Oct 31, 2020 at 10:28 AM Thomas Munro wrote: > > On Fri, Oct 30, 2020 at 1:20 AM Thomas Munro wrote: > > 4. I didn't really like the use of '' for unknown. I figured out how > > to use NULL for that. > > Hrmph. That logic didn't work for the pattern ops case. I fixed > that, by

Re: should INSERT SELECT use a BulkInsertState?

2020-11-01 Thread Luc Vlaming
On 30.10.20 05:51, Justin Pryzby wrote: On Thu, Oct 22, 2020 at 01:29:53PM +0100, Simon Riggs wrote: On Fri, 16 Oct 2020 at 22:05, Justin Pryzby wrote: I made this conditional on BEGIN BULK/SET bulk, so I'll solicit comments on that. I think it would be better if this was self-tuning. So

Re: Parallel INSERT (INTO ... SELECT ...)

2020-11-01 Thread Greg Nancarrow
On Fri, Oct 30, 2020 at 5:00 PM Amit Kapila wrote: > > > So then to avoid > > errors from when parallel-mode is forced on and such unsafe INSERTs > > are run, the only real choice is to only allow parallelModeNeeded to > > be true for SELECT only (not INSERT), and this is kind of cheating and > >

Re: bulk typos

2020-11-01 Thread Michael Paquier
On Fri, Oct 30, 2020 at 09:08:01PM -0500, Justin Pryzby wrote: > On Sun, Oct 25, 2020 at 02:48:49PM -0500, Justin Pryzby wrote: >> Pavel, I can't understand this one. >> I guess s/producement/producing/ is too simple of a fix. >> Please check my proposed language. >>

Re: Additional Chapter for Tutorial

2020-11-01 Thread Erik Rijkers
On 2020-11-01 16:38, Jürgen Purtz wrote: On 30.10.20 17:45, Erik Rijkers wrote: And I wrote down some separate items: 1. 'Two Phase Locking' and 'TPL' should be, I think, 'Two-Phase Commit'. Please someone confirm. (no changes made) Erik Rijkers All suggestions so far are summarized in the

Re: Parallel copy

2020-11-01 Thread Amit Kapila
On Fri, Oct 30, 2020 at 10:11 PM Heikki Linnakangas wrote: > > Leader process: > > The leader process is simple. It picks the next FREE buffer, fills it > with raw data from the file, and marks it as FILLED. If no buffers are > FREE, wait. > > Worker process: > > 1. Claim next READY block from

Re: extension patch of CREATE OR REPLACE TRIGGER

2020-11-01 Thread Peter Smith
Hello Osumi-san. Below are my v14 patch review comments for your consideration. === (1) COMMENT File: NA Maybe next time consider using format-patch to make the patch. Then you can include a comment to give the background/motivation for this change. === (2) COMMENT File:

Re: Dereference before NULL check (src/backend/storage/ipc/latch.c)

2020-11-01 Thread Kyotaro Horiguchi
At Mon, 2 Nov 2020 16:22:09 +1300, Thomas Munro wrote in > On Mon, Nov 2, 2020 at 1:49 PM Kyotaro Horiguchi > wrote: > > At Sat, 31 Oct 2020 11:40:53 -0300, Ranier Vilela > > wrote in > > > Per Coverity. > > > > > > If test set->latch against NULL, is why it can be NULL. > > > ResetEvent can

Re: enable pg_stat_statements to track rows processed by REFRESH MATERIALIZED VIEW

2020-11-01 Thread Yuki Seino
The following review has been posted through the commitfest application: make installcheck-world: tested, passed Implements feature: tested, passed Spec compliant: tested, passed Documentation:tested, passed +1. I checked the patch and there were no problems. I hope

Re: Explicit NULL dereference (src/backend/utils/adt/ruleutils.c)

2020-11-01 Thread Kyotaro Horiguchi
At Sun, 01 Nov 2020 21:05:29 -0500, Tom Lane wrote in > Kyotaro Horiguchi writes: > > We cannot reach there with ev_action == NULL since it comes from a > > non-nullable column. Since most of the other columns has an assertion > > that !isnull, I think we should do the same thing for ev_action

Re: [PATCH] Support negative indexes in split_part

2020-11-01 Thread Nikhil Benesch
On 11/1/20 9:56 PM, Tom Lane wrote: Please add an entry to the upcoming commitfest, to make sure we don't lose track of this: https://commitfest.postgresql.org/30/ Done: https://commitfest.postgresql.org/30/2816/

Re: Dereference before NULL check (src/backend/storage/ipc/latch.c)

2020-11-01 Thread Thomas Munro
On Mon, Nov 2, 2020 at 1:49 PM Kyotaro Horiguchi wrote: > At Sat, 31 Oct 2020 11:40:53 -0300, Ranier Vilela wrote > in > > Per Coverity. > > > > If test set->latch against NULL, is why it can be NULL. > > ResetEvent can dereference NULL. > > If the returned event is WL_LATCH_SET, set->latch

Re: [PATCH] Support negative indexes in split_part

2020-11-01 Thread Tom Lane
Nikhil Benesch writes: > I posted the idea of allowing negative indexes in split_part to pgsql-general > last week, and it seemed there was some interest: > http://postgr.es/m/CAPWqQZR%2B-5pAZNSSrnmYczRaX-huemc%3DoO8URvDZvUA-M%3DMOBA%40mail.gmail.com > Attached is a patch, based on master, that

[PATCH] Support negative indexes in split_part

2020-11-01 Thread Nikhil Benesch
I posted the idea of allowing negative indexes in split_part to pgsql-general last week, and it seemed there was some interest: http://postgr.es/m/CAPWqQZR%2B-5pAZNSSrnmYczRaX-huemc%3DoO8URvDZvUA-M%3DMOBA%40mail.gmail.com Attached is a patch, based on master, that implements the approach as

Re: Explicit NULL dereference (src/backend/utils/adt/ruleutils.c)

2020-11-01 Thread Kyotaro Horiguchi
At Mon, 02 Nov 2020 10:36:10 +0900 (JST), Kyotaro Horiguchi wrote in > At Sat, 31 Oct 2020 11:49:07 -0300, Ranier Vilela wrote > in > > Per Coverity. > > > > make_ruledef function can dereference a NULL pointer (actions), > > if "ev_qual" is provided and "actions" does not exist. > > > >

Re: Explicit NULL dereference (src/backend/utils/adt/ruleutils.c)

2020-11-01 Thread Tom Lane
Kyotaro Horiguchi writes: > We cannot reach there with ev_action == NULL since it comes from a > non-nullable column. Since most of the other columns has an assertion > that !isnull, I think we should do the same thing for ev_action (and > ev_qual). SPI_getvalue() returns C-NULL for SQL-NULL (or

Re: Parallel Append can break run-time partition pruning

2020-11-01 Thread Amit Langote
On Mon, Nov 2, 2020 at 9:51 AM David Rowley wrote: > > On Tue, 27 Oct 2020 at 19:40, Amit Langote wrote: > > Some comments: > > Thanks for having a look at this. > > I've made some adjustments to those comments and pushed. Thank you. -- Amit Langote EDB: http://www.enterprisedb.com

Re: Online checksums verification in the backend

2020-11-01 Thread Andres Freund
Hi On 2020-11-02 10:45:00 +0900, Michael Paquier wrote: > On Sun, Nov 01, 2020 at 05:39:40PM -0800, Andres Freund wrote: > > I'm a bit limited writing - one handed for a while following an injury > > on Friday... > > Oops. Take care. Thanks! > > On 2020-11-02 10:05:25 +0900, Michael Paquier

Re: Online checksums verification in the backend

2020-11-01 Thread Michael Paquier
On Sun, Nov 01, 2020 at 05:39:40PM -0800, Andres Freund wrote: > I'm a bit limited writing - one handed for a while following an injury > on Friday... Oops. Take care. > On 2020-11-02 10:05:25 +0900, Michael Paquier wrote: > > There is no place doing that on HEAD. > > Err? > > /* see if

Re: Online checksums verification in the backend

2020-11-01 Thread Andres Freund
Hi, I'm a bit limited writing - one handed for a while following an injury on Friday... On 2020-11-02 10:05:25 +0900, Michael Paquier wrote: > On Thu, Oct 29, 2020 at 10:08:52PM -0700, Andres Freund wrote: > > I think its pretty much *never* OK to do IO while holding a buffer > > mapping lock.

Re: Explicit NULL dereference (src/backend/utils/adt/ruleutils.c)

2020-11-01 Thread Kyotaro Horiguchi
At Sat, 31 Oct 2020 11:49:07 -0300, Ranier Vilela wrote in > Per Coverity. > > make_ruledef function can dereference a NULL pointer (actions), > if "ev_qual" is provided and "actions" does not exist. > > The comment there is contradictory: " /* these could be nulls */ " > Because if "ev_qual"

Re: Online checksums verification in the backend

2020-11-01 Thread Michael Paquier
On Thu, Oct 29, 2020 at 10:08:52PM -0700, Andres Freund wrote: > I think its pretty much *never* OK to do IO while holding a buffer > mapping lock. You're locking a significant fraction of shared buffers > over IO. That's just not OK. Don't think there's any place doing so > currently either.

Re: Parallel Append can break run-time partition pruning

2020-11-01 Thread David Rowley
On Tue, 27 Oct 2020 at 19:40, Amit Langote wrote: > Some comments: Thanks for having a look at this. I've made some adjustments to those comments and pushed. David

Re: Dereference before NULL check (src/backend/storage/ipc/latch.c)

2020-11-01 Thread Kyotaro Horiguchi
At Sat, 31 Oct 2020 11:40:53 -0300, Ranier Vilela wrote in > Hi, > > Per Coverity. > > If test set->latch against NULL, is why it can be NULL. > ResetEvent can dereference NULL. If the returned event is WL_LATCH_SET, set->latch cannot be NULL. We shouldn't inadvertently ignore the unexpected

RE: Disable WAL logging to speed up data loading

2020-11-01 Thread osumi.takami...@fujitsu.com
Hello On Friday, October 30, 2020 1:32 PM Masahiko Sawada wrote: > > On 2020/10/29 19:21, Laurenz Albe wrote: > > > On Thu, 2020-10-29 at 11:42 +0900, Fujii Masao wrote: > > >>> But what if someone sets wal_level=none, performs some data > > >>> modifications, sets wal_level=archive and after

Re: how to replicate test results in cf-bot on travis

2020-11-01 Thread Tom Lane
Dave Cramer writes: > OK, checked and I definitely have the changes. I don't think the isolation > test is running. Is there some configuration that enables it? No, top-level "check-world" should invoke that ... but if you're unsure, you could cd to src/test/isolation and run check or

Re: how to replicate test results in cf-bot on travis

2020-11-01 Thread Dave Cramer
On Sun, 1 Nov 2020 at 13:46, Dave Cramer wrote: > > > On Sun., Nov. 1, 2020, 1:41 p.m. Thomas Munro, > wrote: > >> On Mon, Nov 2, 2020 at 1:58 AM Dave Cramer wrote: >> > For my patch https://commitfest.postgresql.org/30/2522/ >> > >> > When I run >> > >> > make -j4 all contrib && make

Getting rid of aggregate_dummy()

2020-11-01 Thread Tom Lane
While fooling with Gen_fmgrtab.pl for a nearby patch [1], I noticed that fmgrtab.c had a lot of entries pointing at aggregate_dummy, which seemed rather useless. So I experimented with removing them. It turns out that nodeWindowAgg.c is carelessly expecting them to be there, because it does

Getting Gen_fmgrtab.pl to generate macros for all pg_proc entries

2020-11-01 Thread Tom Lane
Over in [1] I noted a need for fmgroids.h macros for pg_proc entries that overload a particular C function. The F_XXX macros seem to be designed on the assumption that the only thing they're needed for is to call the function, so you don't really care which SQL alias you use to get there. That's

Re: how to replicate test results in cf-bot on travis

2020-11-01 Thread Dave Cramer
On Sun., Nov. 1, 2020, 1:41 p.m. Thomas Munro, wrote: > On Mon, Nov 2, 2020 at 1:58 AM Dave Cramer wrote: > > For my patch https://commitfest.postgresql.org/30/2522/ > > > > When I run > > > > make -j4 all contrib && make check-world > > locally > > > > I see 2 errors. > > > > When cf-bot runs

Re: how to replicate test results in cf-bot on travis

2020-11-01 Thread Thomas Munro
On Mon, Nov 2, 2020 at 1:58 AM Dave Cramer wrote: > For my patch https://commitfest.postgresql.org/30/2522/ > > When I run > > make -j4 all contrib && make check-world > locally > > I see 2 errors. > > When cf-bot runs this it sees > 35 out of 93 failed. > > How can I see the same errors? Hi

Re: psql \df choose functions by their arguments

2020-11-01 Thread David Christensen
> * Removed the tab-complete bit, it was too fragile and unhelpful I can’t speak for the specific patch, but tab completion of proc args for \df, \ef and friends has long been a desired feature of mine, particularly when you are dealing with functions with huge numbers of arguments and the

Re: Add important info about ANALYZE after create Functional Index

2020-11-01 Thread Fabrízio de Royes Mello
On Sun, 1 Nov 2020 at 09:29 Michael Paquier wrote: > On Sun, Nov 01, 2020 at 09:23:44AM +0900, Michael Paquier wrote: > > By doing so, there is no need to include pg_statistic.h in index.c. > > Except that, the logic looks fine at quick glance. In the long-term, > > I also think that it would

Re: psql \df choose functions by their arguments

2020-11-01 Thread Greg Sabino Mullane
Thanks for the feedback, attached is version two of the patch. Major changes: * Use booleans not generic "int x" * Build a quick list of abbreviations at the top of the function * Add array mapping for all types * Removed the tab-complete bit, it was too fragile and unhelpful Cheers, Greg diff

Re: Support for NSS as a libpq TLS backend

2020-11-01 Thread Andrew Dunstan
On 10/29/20 11:20 AM, Daniel Gustafsson wrote: >> On 28 Oct 2020, at 07:39, Andres Freund wrote: >> Have you done testing to ensure that NSS PG cooperates correctly with >> openssl PG? Is there a way we can make that easier to do? E.g. allowing >> to build frontend with NSS and backend with

how to replicate test results in cf-bot on travis

2020-11-01 Thread Dave Cramer
For my patch https://commitfest.postgresql.org/30/2522/ When I run make -j4 all contrib && make check-world locally I see 2 errors. When cf-bot runs this it sees 35 out of 93 failed. How can I see the same errors? Dave Cramer

Re: Add important info about ANALYZE after create Functional Index

2020-11-01 Thread Michael Paquier
On Sun, Nov 01, 2020 at 09:23:44AM +0900, Michael Paquier wrote: > By doing so, there is no need to include pg_statistic.h in index.c. > Except that, the logic looks fine at quick glance. In the long-term, > I also think that it would make sense to move both routnes out of > heap.c into a

Re: libpq compression

2020-11-01 Thread Daniil Zakhlystov
It looks there was a different version of pq_getbyte_if_available on my side, my bad. I’ve reapplied the patch and compiler is happy now, thanks! — Daniil Zakhlystov > On Nov 1, 2020, at 3:01 PM, Konstantin Knizhnik > wrote: > > Hi > > On 01.11.2020 12:37, Daniil Zakhlystov wrote: >> Hi,

Re: libpq compression

2020-11-01 Thread Daniil Zakhlystov
Hi, I have a couple of comments regarding the last patch, mostly these are minor issues. In src/backend/libpq/pqcomm.c, starting from the line 1114: int pq_getbyte_if_available(unsigned char *c) { int r; Assert(PqCommReadingMsg); if (PqRecvPointer

Re: Consistent error reporting for encryption/decryption in pgcrypto

2020-11-01 Thread Michael Paquier
On Sat, Oct 31, 2020 at 09:40:12PM +0100, Daniel Gustafsson wrote: > Ah yes, I accidentally fat-fingered the git add -p when splitting up the NSS > patch into bite size pieces. Sorry about that. The attached v2 has the error > declaration. Thanks for updatng the patch. Applied. -- Michael

Re: libpq compression

2020-11-01 Thread Konstantin Knizhnik
Hi On 01.11.2020 12:37, Daniil Zakhlystov wrote: Hi, I have a couple of comments regarding the last patch, mostly these are minor issues. In src/backend/libpq/pqcomm.c, starting from the line 1114: int pq_getbyte_if_available(unsigned char *c) { intr; Assert(PqCommReadingMsg); if

Re: Log message for GSS connection is missing once connection authorization is successful.

2020-11-01 Thread Bharath Rupireddy
On Sun, Nov 1, 2020 at 3:34 AM Euler Taveira wrote: > > No. The strings are specified in the appendStringInfo, hence you should add > _() > around the string to be translated. There is nothing to be translated if you > specify only the format identifier. You can always test if gettext extracts