Re: Pg14, pg_dumpall and "password_encryption=true"

2021-01-15 Thread Noah Misch
On Fri, Jan 15, 2021 at 01:35:50PM +0900, Ian Lawrence Barwick wrote: > $ tail -3 pg_upgrade_utility.log > ALTER ROLE "postgres" SET "password_encryption" TO 'true'; > psql:pg_upgrade_dump_globals.sql:75: ERROR: invalid value for > parameter "password_encryption": "true" > HINT:

Re: Logical Replication - behavior of ALTER PUBLICATION .. DROP TABLE and ALTER SUBSCRIPTION .. REFRESH PUBLICATION

2021-01-15 Thread Bharath Rupireddy
On Sat, Jan 16, 2021 at 12:02 PM Amit Kapila wrote: > > On Sat, Jan 16, 2021 at 11:52 AM Bharath Rupireddy > wrote: > > > > I made an entry in the commitfest[1], so that the patches will get a > > chance to run on all the platforms. > > > > Attaching v4 patch set i.e. 0001 - fix, 0002 - test

Re: Logical Replication - behavior of ALTER PUBLICATION .. DROP TABLE and ALTER SUBSCRIPTION .. REFRESH PUBLICATION

2021-01-15 Thread japin
On Sat, 16 Jan 2021 at 14:21, Bharath Rupireddy wrote: > I made an entry in the commitfest[1], so that the patches will get a > chance to run on all the platforms. > > Attaching v4 patch set i.e. 0001 - fix, 0002 - test case. > > [1] - https://commitfest.postgresql.org/32/2944/ > Thanks for

Re: Logical Replication - behavior of ALTER PUBLICATION .. DROP TABLE and ALTER SUBSCRIPTION .. REFRESH PUBLICATION

2021-01-15 Thread Amit Kapila
On Sat, Jan 16, 2021 at 11:52 AM Bharath Rupireddy wrote: > > I made an entry in the commitfest[1], so that the patches will get a > chance to run on all the platforms. > > Attaching v4 patch set i.e. 0001 - fix, 0002 - test case. > In the test, can we have multiple publications for the

Re: Logical Replication - behavior of ALTER PUBLICATION .. DROP TABLE and ALTER SUBSCRIPTION .. REFRESH PUBLICATION

2021-01-15 Thread Bharath Rupireddy
I made an entry in the commitfest[1], so that the patches will get a chance to run on all the platforms. Attaching v4 patch set i.e. 0001 - fix, 0002 - test case. [1] - https://commitfest.postgresql.org/32/2944/ With Regards, Bharath Rupireddy. EnterpriseDB: http://www.enterprisedb.com

Re: Determine parallel-safety of partition relations for Inserts

2021-01-15 Thread Amit Kapila
On Fri, Jan 15, 2021 at 7:35 PM tsunakawa.ta...@fujitsu.com wrote: > > From: Amit Kapila > > This will surely increase planning time but the execution is reduced > > to an extent due to parallelism that it won't matter for either of the > > cases if we see just total time. For example, see the

new release pspg

2021-01-15 Thread Pavel Stehule
Hi I released pspg 4.0.0 https://github.com/okbob/pspg/releases/tag/4.0.0 Now with the possibility to export content to file or clipboard in CSV, TSVC, text or INSERT formats. pspg is a pager like "less" or "more" designed specially for usage in TUI database clients like "psql". It can work

Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit

2021-01-15 Thread Bharath Rupireddy
On Thu, Jan 14, 2021 at 3:52 PM Fujii Masao wrote: > On 2021/01/09 10:12, Bharath Rupireddy wrote: > > On Fri, Jan 8, 2021 at 9:55 AM Bharath Rupireddy > > wrote: > >> I will make the changes and post a new patch set soon. > > > > Attaching v9 patch set that has addressed the review comments on

Re: remove unneeded pstrdup in fetch_table_list

2021-01-15 Thread Amit Kapila
On Thu, Jan 14, 2021 at 3:05 PM Amit Kapila wrote: > > On Thu, Jan 14, 2021 at 10:51 AM Tom Lane wrote: > > > > Michael Paquier writes: > > > On Thu, Jan 14, 2021 at 01:17:57AM +, Hou, Zhijie wrote: > > Thanks. I am thinking to backpatch this even though there is no > > problem

Re: Determine parallel-safety of partition relations for Inserts

2021-01-15 Thread Amit Kapila
On Fri, Jan 15, 2021 at 6:45 PM Amit Langote wrote: > > On Fri, Jan 15, 2021 at 9:59 PM Amit Kapila wrote: > > We want to do this for Inserts where only Select can be parallel and > > Inserts will always be done by the leader backend. This is actually > > the case we first want to implement. > >

Re: Occasional tablespace.sql failures in check-world -jnn

2021-01-15 Thread Michael Paquier
On Fri, Jan 15, 2021 at 09:59:02AM +0100, Peter Eisentraut wrote: > I vaguely recall that this had something to do with SELinux (or something > similar?), where it matters in what context you create a file or directory > and then certain properties attach to it that are relevant to subsequent >

Re: Key management with tests

2021-01-15 Thread Michael Paquier
On Fri, Jan 15, 2021 at 08:20:36PM -0800, Andres Freund wrote: > On 2021-01-15 20:49:10 -0500, Bruce Momjian wrote: >> What Perl tap tests run initdb and manage the cluster? I didn't find >> any. > > find . -name '*.pl'|xargs grep 'use PostgresNode;' > > should give you a nearly complete list.

Re: Wrong usage of RelationNeedsWAL

2021-01-15 Thread Noah Misch
On Wed, Jan 13, 2021 at 04:07:05PM +0900, Kyotaro Horiguchi wrote: > The definition of the macro RelationNeedsWAL has been changed by > c6b92041d3 to include conditions related to the WAL-skip optimization > but some uses of the macro are not relevant to the optimization. That > misuses are

Re: Add table access method as an option to pgbench

2021-01-15 Thread Michael Paquier
On Fri, Jan 15, 2021 at 01:22:45PM -0800, Andres Freund wrote: > I think that objection is right. All that's needed to change this from > the client side is to do something like > PGOPTIONS='-c default_table_access_method=foo' pgbench ... > > I don't think adding pgbench options for individual

Re: Key management with tests

2021-01-15 Thread Andres Freund
Hi, On 2021-01-15 20:49:10 -0500, Bruce Momjian wrote: > On Fri, Jan 15, 2021 at 04:56:24PM -0800, Andres Freund wrote: > > On 2021-01-15 19:21:32 -0500, Bruce Momjian wrote: > > > You have to understand cryptography and Postgres internals to understand > > > the design, and I don't think it is

Re: Is Recovery actually paused?

2021-01-15 Thread Masahiko Sawada
On Wed, Jan 13, 2021 at 9:20 PM Dilip Kumar wrote: > > On Wed, Jan 13, 2021 at 3:35 PM Dilip Kumar wrote: > > > > On Wed, Jan 13, 2021 at 3:27 PM Yugo NAGATA wrote: > > > > > > On Thu, 10 Dec 2020 11:25:23 +0530 > > > Dilip Kumar wrote: > > > > > > > > > However, I wonder users don't expect

Re: Key management with tests

2021-01-15 Thread Bruce Momjian
On Fri, Jan 15, 2021 at 04:56:24PM -0800, Andres Freund wrote: > On 2021-01-15 19:21:32 -0500, Bruce Momjian wrote: > > You have to understand cryptography and Postgres internals to understand > > the design, and I don't think it is realistic to explain that all to the > > community. We did much

Re: Key management with tests

2021-01-15 Thread Andres Freund
Hi, On 2021-01-15 19:21:32 -0500, Bruce Momjian wrote: > On Fri, Jan 15, 2021 at 02:37:56PM -0800, Andres Freund wrote: > > On 2021-01-15 16:47:19 -0500, Bruce Momjian wrote: > > > > I am not even sure there is a consensus on the design, without which > > > > any commit is always premature. > > >

Re: poc - possibility to write window function in PL languages

2021-01-15 Thread Tom Lane
Pavel Stehule writes: > [ plpgsql-window-functions-20210104.patch.gz ] I spent some time looking at this patch. It would certainly be appealing to have some ability to write custom window functions without descending into C; but I'm not very happy about the details. I'm okay with the idea of

Re: Change default of checkpoint_completion_target

2021-01-15 Thread Andres Freund
Hi, On 2021-01-15 23:05:02 +0100, Tomas Vondra wrote: > Yeah. The flushing probably makes that mostly unnecessary, but we still > allow disabling that. I'm not really convinced replacing it with a > compile-time #define is a good idea, exactly because it can't be changed > if needed. It's also

Re: Key management with tests

2021-01-15 Thread Andres Freund
Hi, On 2021-01-15 16:47:19 -0500, Bruce Momjian wrote: > On Fri, Jan 15, 2021 at 04:23:22PM -0500, Robert Haas wrote: > > On Fri, Jan 15, 2021 at 3:49 PM Bruce Momjian wrote: > > I don't think that's appropriate. Several prominent community members > > have told you that the patch, as committed

Re: Rename of triggers for partitioned tables

2021-01-15 Thread Alvaro Herrera
On 2020-Nov-27, Arne Roland wrote: > I got too annoyed at building queries for gexec all the time. So wrote > a patch to fix the issue that the rename of partitioned trigger > doesn't affect children. As you say, triggers on children don't necessarily have to have the same name as on parent;

increase size of pg_commit_ts buffers

2021-01-15 Thread Alvaro Herrera
I wrote this patch last year in response to a customer issue and I thought I had submitted it here, but evidently I didn't. So here it is. The short story is: in commit 5364b357fb11 we increased the size of pg_commit (née pg_clog) but we didn't increase the size of pg_commit_ts to match. When

Re: Key management with tests

2021-01-15 Thread Bruce Momjian
On Fri, Jan 15, 2021 at 04:59:17PM -0500, Robert Haas wrote: > On Fri, Jan 15, 2021 at 4:47 PM Bruce Momjian wrote: > > If people want changes, I need to hear about it here. I have address > > everything people have mentioned in these threads so far. > > That does not match my perception of the

Re: Key management with tests

2021-01-15 Thread David G. Johnston
On Fri, Jan 15, 2021 at 2:59 PM Robert Haas wrote: > On Fri, Jan 15, 2021 at 4:47 PM Bruce Momjian wrote: > > If people want changes, I need to hear about it here. I have address > > everything people have mentioned in these threads so far. > > That does not match my perception of the

Re: Change default of checkpoint_completion_target

2021-01-15 Thread Tomas Vondra
On 1/15/21 10:51 PM, Andres Freund wrote: > Hi, > > On 2020-12-08 12:41:35 -0500, Tom Lane wrote: >> FWIW, I kind of like the idea of getting rid of it completely. >> Is there really ever a good reason to set it to something different >> than that? If not, well, we have too many GUCs already,

Re: Key management with tests

2021-01-15 Thread Robert Haas
On Fri, Jan 15, 2021 at 4:47 PM Bruce Momjian wrote: > If people want changes, I need to hear about it here. I have address > everything people have mentioned in these threads so far. That does not match my perception of the situation. -- Robert Haas EDB: http://www.enterprisedb.com

Re: Outdated replication protocol error?

2021-01-15 Thread Andres Freund
Hi, On 2021-01-14 16:40:26 +0900, Fujii Masao wrote: > On 2021/01/12 9:06, Jeff Davis wrote: > > Commit 5ee2197767 (about 4 years old) introduced the error: > > > >"IDENTIFY_SYSTEM has not been run before START_REPLICATION" > > > > But it seems like running START_REPLICATION first works (at

Re: Change default of checkpoint_completion_target

2021-01-15 Thread Andres Freund
Hi, On 2020-12-08 12:41:35 -0500, Tom Lane wrote: > FWIW, I kind of like the idea of getting rid of it completely. > Is there really ever a good reason to set it to something different > than that? If not, well, we have too many GUCs already, and each > of them carries nonzero performance,

Re: Key management with tests

2021-01-15 Thread Bruce Momjian
On Fri, Jan 15, 2021 at 04:23:22PM -0500, Robert Haas wrote: > On Fri, Jan 15, 2021 at 3:49 PM Bruce Momjian wrote: > > I am planning to apply this next week. > > I don't think that's appropriate. Several prominent community members > have told you that the patch, as committed the first time,

Re: patch: reduce overhead of execution of CALL statement in no atomic mode from PL/pgSQL

2021-01-15 Thread Tom Lane
Pavel Stehule writes: > [ plpgsql-using-local-resowner-for-call-plans-20200108.patch ] I took a quick look through this patch, just reading it without any testing. A few thoughts: * Instead of adding an argument to GetCachedPlan and ReleaseCachedPlan, I think it'd be better to *replace* the

Re: WIP: document the hook system

2021-01-15 Thread David G. Johnston
On Fri, Jan 15, 2021 at 12:28 AM Peter Eisentraut < peter.eisentr...@enterprisedb.com> wrote: > On 2020-12-31 04:28, David Fetter wrote: > > This could probably use a lot of filling in, but having it in the > > actual documentation beats needing to know folklore even to know > > that the

Re: Wrong usage of RelationNeedsWAL

2021-01-15 Thread Andres Freund
Hi, On 2021-01-13 16:07:05 +0900, Kyotaro Horiguchi wrote: > Commit c6b92041d3 changed the definition of RelationNeedsWAL(). > > -#define RelationNeedsWAL(relation) \ > - ((relation)->rd_rel->relpersistence == RELPERSISTENCE_PERMANENT) > +#define RelationNeedsWAL(relation)

Re: Key management with tests

2021-01-15 Thread Robert Haas
On Fri, Jan 15, 2021 at 3:49 PM Bruce Momjian wrote: > I am planning to apply this next week. I don't think that's appropriate. Several prominent community members have told you that the patch, as committed the first time, needed a lot more work. There hasn't been enough time between then and

Re: Add table access method as an option to pgbench

2021-01-15 Thread Andres Freund
Hi, On 2020-11-25 12:41:25 +0900, Michael Paquier wrote: > On Tue, Nov 24, 2020 at 03:32:38PM -0800, David Zhang wrote: > > But, providing another option for the end user may not be a bad idea, and it > > might make the tests easier at some points. > > My first thought is that we have no need to

Re: jit and explain nontext

2021-01-15 Thread Justin Pryzby
On Fri, Jan 15, 2021 at 02:53:49PM -0500, Tom Lane wrote: > On balance I agree with Peter's opinion that this isn't worth > changing. I would be for the patch if the executor had a little > more freedom of action, but as things stand there's not much > freedom there. Thanks for looking CF:

Re: Improve pg_dump dumping publication tables

2021-01-15 Thread Tom Lane
"Hsu, John" writes: > I was wondering if there's a good reason in pg_dump getPublicationTables() > to iterate through all tables one by one and querying to see if it has a > corresponding publication other than memory concerns? I just came across this entry in the CommitFest, and I see that

Re: Printing backtrace of postgres processes

2021-01-15 Thread Andres Freund
On 2021-01-15 09:53:05 +0100, Peter Eisentraut wrote: > On 2020-12-08 10:38, vignesh C wrote: > > I have implemented printing of backtrace based on handling it in > > CHECK_FOR_INTERRUPTS. This patch also includes the change to allow > > getting backtrace of any particular process based on the

Re: Add table access method as an option to pgbench

2021-01-15 Thread David Zhang
Hi, `v6-0001-add-table-access-method-as-an-option-to-pgbench` fixed the wording problems for pgbench document and help as they were pointed out by Justin and Youichi. `0001-update-tablespace-to-keep-document-consistency` is trying to make the *tablespace* to be more consistent in pgbench

Re: jit and explain nontext

2021-01-15 Thread Tom Lane
Justin Pryzby writes: > On Sat, Nov 21, 2020 at 10:26:00AM -0600, Justin Pryzby wrote: >> On Sat, Nov 21, 2020 at 08:39:11AM +0100, Peter Eisentraut wrote: >>> In this context, I don't see the point of this change. If you set jit=off >>> explicitly, then there is no need to clutter the EXPLAIN

Re: [patch] [doc] Further note required activity aspect of automatic checkpoint and archving

2021-01-15 Thread David G. Johnston
On Fri, Jan 15, 2021 at 12:16 AM Peter Eisentraut < peter.eisentr...@enterprisedb.com> wrote: > On 2020-10-12 23:54, David G. Johnston wrote: > > --- a/doc/src/sgml/backup.sgml > > +++ b/doc/src/sgml/backup.sgml > > @@ -722,6 +722,8 @@ test ! -f > > /mnt/server/archivedir/000100A90065

Re: WIP: System Versioned Temporal Table

2021-01-15 Thread legrand legrand
Hello, it seems that Oracle (11R2) doesn't add the Start and End timestamp columns and permit statement like select * from tt union select * from tt AS OF TIMESTAMP (SYSTIMESTAMP - INTERVAL '6' SECOND) minus select * from tt VERSIONS BETWEEN TIMESTAMP (SYSTIMESTAMP - INTERVAL '6' second) and

Re: O(n^2) system calls in RemoveOldXlogFiles()

2021-01-15 Thread Andres Freund
Hi, Thanks for pushing! Sorry to not get around to a review before... On 2021-01-15 11:42:50 +0900, Michael Paquier wrote: > On Fri, Jan 15, 2021 at 03:25:24PM +1300, Thomas Munro wrote: > > Thanks Michael! Another notch for the unnecessary system call > > hitlist:

Re: fdatasync(2) on macOS

2021-01-15 Thread Bruce Momjian
On Fri, Jan 15, 2021 at 12:55:52PM -0500, Tom Lane wrote: > > So... does this unreleased function flush drive caches? We know that > > fsync(2) doesn't, based on Apple's advice[1] for databases hackers to > > call fcntl(fd, F_FULLSYNC, 0) instead. We do that. > > pg_test_fsync results make it

Git, diffs, and patches

2021-01-15 Thread Bruce Momjian
I learned a few things when working on the key management patch that I want to share here in case it helps anyone: * git diff effectively creates a squashed diff of all commits/changes * git format-patch wants to retain each commit (no squash) * git format-patch has information about file name

Re: fdatasync(2) on macOS

2021-01-15 Thread Tom Lane
Thomas Munro writes: > While following along with the nearby investigation into weird > cross-version Apple toolchain issues that confuse configure, I noticed > that the newer buildfarm Macs say: > checking for fdatasync... (cached) yes > That's a bit strange because there's no man page and no

Re: PG vs LLVM 12 on seawasp, next round

2021-01-15 Thread Tom Lane
Alvaro Herrera writes: > On 2020-Dec-11, Fabien COELHO wrote: >> I'll look into it when I have time, which make take some time. Hopefully >> over the holidays. > This is still happening ... Any chance you can have a look at it? If you don't have time to debug it, perhaps you could just disable

Re: PG vs LLVM 12 on seawasp, next round

2021-01-15 Thread Alvaro Herrera
On 2020-Dec-11, Fabien COELHO wrote: > > I hadn't checked that before, but for the last few days there's been a > > different failure than the one I saw earlier: > > > > +ERROR: could not load library > >

Re: Yet another fast GiST build

2021-01-15 Thread Andrey Borodin
> 15 янв. 2021 г., в 10:24, Peter Eisentraut > написал(а): > > I noticed this patch while working on another patch for pageinspect [0], and > this one appears to introduce a problem similar to the one the other patch > attempts to fix: The "itemlen" output parameters are declared to be of

Re: WIP: System Versioned Temporal Table

2021-01-15 Thread Simon Riggs
On Fri, Jan 15, 2021 at 4:56 PM Surafel Temesgen wrote: > > > > On Fri, Jan 15, 2021 at 12:22 AM Simon Riggs > wrote: >> >> SELECT * FROM foo FOR SYSTEM_TIME AS OF ... >> should NOT include the Start and End timestamp columns >> because this acts like a normal query just with a different

Re: WIP: System Versioned Temporal Table

2021-01-15 Thread Surafel Temesgen
On Fri, Jan 15, 2021 at 12:22 AM Simon Riggs wrote: > SELECT * FROM foo FOR SYSTEM_TIME AS OF ... > should NOT include the Start and End timestamp columns > because this acts like a normal query just with a different snapshot > timestamp > > SELECT * FROM foo FOR SYSTEM_TIME BETWEEN x AND y >

Re: WIP: System Versioned Temporal Table

2021-01-15 Thread Simon Riggs
On Fri, Jan 15, 2021 at 4:46 PM Surafel Temesgen wrote: > > > > On Fri, Jan 15, 2021 at 12:27 AM Simon Riggs > wrote: >> >> >> Yes, I think it can. The current situation is that the Start or End is >> set to the Transaction Start Timestamp. >> So if t2 starts before t1, then if t1 creates a row

Re: WIP: System Versioned Temporal Table

2021-01-15 Thread Surafel Temesgen
On Fri, Jan 15, 2021 at 12:27 AM Simon Riggs wrote: > > Yes, I think it can. The current situation is that the Start or End is > set to the Transaction Start Timestamp. > So if t2 starts before t1, then if t1 creates a row and t2 deletes it > then we will have start=t1 end=t2, but t2 Your tests

Re: Transactions involving multiple postgres foreign servers, take 2

2021-01-15 Thread Zhihong Yu
Hi, For v32-0004-Add-PrepareForeignTransaction-API.patch : + * Whenever a foreign transaction is processed, the corresponding FdwXact + * entry is update. To avoid holding the lock during transaction processing + * which may take an unpredicatable time the in-memory data of foreign entry is

Re: pg_preadv() and pg_pwritev()

2021-01-15 Thread Tom Lane
Sergey Shinderuk writes: > On 15.01.2021 04:45, Tom Lane wrote: >> Hence, I propose the attached. This works as far as I can tell >> to fix the problem you're seeing. > Yes, it works fine. Thank you very much. Great. Pushed with a little more polishing. regards, tom

Re: list of extended statistics on psql

2021-01-15 Thread Tomas Vondra
On 1/15/21 9:47 AM, Julien Rouhaud wrote: > On Wed, Jan 13, 2021 at 10:22:05AM +0900, Tatsuro Yamada wrote: >> Hi Tomas, >> >> On 2021/01/13 7:48, Tatsuro Yamada wrote: >>> On 2021/01/12 20:08, Tomas Vondra wrote: On 1/12/21 2:57 AM, Tatsuro Yamada wrote: > On 2021/01/09 9:01, Tomas

Re: POC: postgres_fdw insert batching

2021-01-15 Thread Amit Langote
On Sat, Jan 16, 2021 at 12:00 AM tsunakawa.ta...@fujitsu.com wrote: > From: Amit Langote > > Okay, so maybe not moving the whole logic into the FDW's > > BeginForeignModify(), but at least if we move this... > > > > @@ -441,6 +449,72 @@ ExecInsert(ModifyTableState *mtstate, > > + /* > > +

RE: POC: postgres_fdw insert batching

2021-01-15 Thread tsunakawa.ta...@fujitsu.com
From: Amit Langote > Okay, so maybe not moving the whole logic into the FDW's > BeginForeignModify(), but at least if we move this... > > @@ -441,6 +449,72 @@ ExecInsert(ModifyTableState *mtstate, > + /* > +* Determine if the FDW supports batch insert and determine the > batch > +

Re: {CREATE INDEX, REINDEX} CONCURRENTLY improvements

2021-01-15 Thread Álvaro Herrera
So one last remaining improvement was to have VACUUM ignore processes doing CIC and RC to compute the Xid horizon of tuples to remove. I think we can do something simple like the attached patch. -- Álvaro Herrera Valdivia, Chile "Aprender sin pensar es inútil; pensar sin aprender,

RE: Disable WAL logging to speed up data loading

2021-01-15 Thread osumi.takami...@fujitsu.com
Hi, Movead Thanks for your comments. > I read the patch and have two points: > > 1. I do basebackup for database then switch wal level from logical to none to > logical and of cause I archive the wal segments. Next I do PITR base on the > basebackup, as a result it success startup with a waring

RE: Determine parallel-safety of partition relations for Inserts

2021-01-15 Thread tsunakawa.ta...@fujitsu.com
From: Amit Langote > Sorry, I haven't looked at the linked threads and the latest patches > there closely enough yet, so I may be misreading this, but if the > inserts will always be done by the leader backend as you say, then why > does the planner need to be checking the parallel safety of the

RE: Disable WAL logging to speed up data loading

2021-01-15 Thread osumi.takami...@fujitsu.com
Hi Thank you everyone On Thursday, January 14, 2021 9:27 AM Tsunakawa, Takayuki/綱川 貴之 wrote: > From: Kyotaro Horiguchi > > > XLogSetRecordFlags(XLOG_MARK_UNIMPORTANT | > > XLOG_MARK_ESSENTIAL); > > > XLogRegisterData((char *) , sizeof(dummy)); > > > > > > (Here's a word play -

RE: Determine parallel-safety of partition relations for Inserts

2021-01-15 Thread tsunakawa.ta...@fujitsu.com
From: Amit Kapila > This will surely increase planning time but the execution is reduced > to an extent due to parallelism that it won't matter for either of the > cases if we see just total time. For example, see the latest results > for parallel inserts posted by Haiying Tang [3]. There might

Re: POC: postgres_fdw insert batching

2021-01-15 Thread Amit Langote
On Fri, Jan 15, 2021 at 12:05 AM Tomas Vondra wrote: > Attached is v9 with all of those tweaks, Thanks. > except for moving the BatchSize > call to BeginForeignModify - I tried that, but it did not seem like an > improvement, because we'd still need the checks for API callbacks in > ExecInsert

Re: {CREATE INDEX, REINDEX} CONCURRENTLY improvements

2021-01-15 Thread Álvaro Herrera
On 2021-Jan-12, Álvaro Herrera wrote: > On 2021-Jan-12, Álvaro Herrera wrote: > > > > For the 0001 patch, since ReindexIndexInfo is used only within > > > ReindexRelationConcurrently() I think it’s a function-local structure > > > type. So we can declare it within the function. What do you

Re: Determine parallel-safety of partition relations for Inserts

2021-01-15 Thread Amit Langote
On Fri, Jan 15, 2021 at 9:59 PM Amit Kapila wrote: > We want to do this for Inserts where only Select can be parallel and > Inserts will always be done by the leader backend. This is actually > the case we first want to implement. Sorry, I haven't looked at the linked threads and the latest

Re: Wrong HINT during database recovery when occur a minimal wal.

2021-01-15 Thread lchch1990
>I think it's also important to suggest to the users how they can turn >on hot_standby on their standby. So, perhaps-a-bit-verbose hint would >be like this. >"Either start this standby from base backup taken after setting >wal_level to \"replica\" on the primary, or turn off hot_standby >here."

Re: Determine parallel-safety of partition relations for Inserts

2021-01-15 Thread Amit Kapila
On Fri, Jan 15, 2021 at 5:53 PM Ashutosh Bapat wrote: > > On Fri, Jan 15, 2021 at 3:48 PM Amit Kapila wrote: > > > > While reviewing parallel insert [1] (Insert into Select) and > > parallel copy patches [2], it came to my notice that both the patches > > traverse the entire partition

Re: Commitfest 2021-01 Now in Progress

2021-01-15 Thread Masahiko Sawada
Hi, On Fri, Jan 1, 2021 at 9:22 PM Masahiko Sawada wrote: > > Hi, > > Happy new year to all! > > The January Commitfest for PG14 is now in progress! I'm happy to > volunteer to manage it. > Now that a half month has passed, the current state is: Needs review: 163 (-25) Waiting on Author: 35

Re: Determine parallel-safety of partition relations for Inserts

2021-01-15 Thread Ashutosh Bapat
On Fri, Jan 15, 2021 at 3:48 PM Amit Kapila wrote: > > While reviewing parallel insert [1] (Insert into Select) and > parallel copy patches [2], it came to my notice that both the patches > traverse the entire partition hierarchy to determine parallel-safety > of partitioned relations. This

Re: Wrong HINT during database recovery when occur a minimal wal.

2021-01-15 Thread Kyotaro Horiguchi
At Fri, 15 Jan 2021 17:04:19 +0800, "lchch1...@sina.cn" wrote in > > >Mmm. Maybe something's missing. If you took the base-backup using > >pg_basebackup, that means max_wal_senders > 0 on the primary. If you > >lowered wal_level in the backup (or replica) then started it, You > >would get

Re: Logical Replication - behavior of ALTER PUBLICATION .. DROP TABLE and ALTER SUBSCRIPTION .. REFRESH PUBLICATION

2021-01-15 Thread Bharath Rupireddy
On Fri, Jan 15, 2021 at 4:54 PM japin wrote: > On Fri, 15 Jan 2021 at 18:57, Bharath Rupireddy > wrote: > > On Fri, Jan 15, 2021 at 4:03 PM Amit Kapila wrote: > >> That sounds like a better way to fix and in fact, I was about to > >> suggest the same after reading your previous email. I'll

Re: Improve handling of parameter differences in physical replication

2021-01-15 Thread Sergei Kornilov
The following review has been posted through the commitfest application: make installcheck-world: tested, passed Implements feature: tested, passed Spec compliant: not tested Documentation:tested, passed Hello Look good for me. I think the patch is ready for commiter.

Re: Logical Replication - behavior of ALTER PUBLICATION .. DROP TABLE and ALTER SUBSCRIPTION .. REFRESH PUBLICATION

2021-01-15 Thread japin
On Fri, 15 Jan 2021 at 18:57, Bharath Rupireddy wrote: > On Fri, Jan 15, 2021 at 4:03 PM Amit Kapila wrote: >> That sounds like a better way to fix and in fact, I was about to >> suggest the same after reading your previous email. I'll think more on >> this but in the meantime, can you add

Re: cost_sort vs cost_agg

2021-01-15 Thread Ashutosh Bapat
On Thu, Jan 14, 2021 at 7:12 PM Andy Fan wrote: > > Currently the cost_sort doesn't consider the number of columns to sort, which > means the cost of SELECT * FROM t ORDER BY a; equals with the SELECT * > FROM t ORDER BY a, b; which is obviously wrong. The impact of this is when we > choose the

Re: Logical Replication - behavior of ALTER PUBLICATION .. DROP TABLE and ALTER SUBSCRIPTION .. REFRESH PUBLICATION

2021-01-15 Thread Bharath Rupireddy
On Fri, Jan 15, 2021 at 4:03 PM Amit Kapila wrote: > That sounds like a better way to fix and in fact, I was about to > suggest the same after reading your previous email. I'll think more on > this but in the meantime, can you add the test case in the patch as > requested earlier as well. @Li

Re: Logical Replication - behavior of ALTER PUBLICATION .. DROP TABLE and ALTER SUBSCRIPTION .. REFRESH PUBLICATION

2021-01-15 Thread Bharath Rupireddy
On Fri, Jan 15, 2021 at 4:20 PM Amit Kapila wrote: > I feel it is better to not do anything for this because neither we > have a test to reproduce the problem nor is it clear from theory if > there is any problem to solve here. +1 to ignore 0002 patch. Thanks Amit. With Regards, Bharath

Re: Logical Replication - behavior of ALTER PUBLICATION .. DROP TABLE and ALTER SUBSCRIPTION .. REFRESH PUBLICATION

2021-01-15 Thread Amit Kapila
On Fri, Jan 15, 2021 at 11:48 AM Bharath Rupireddy wrote: > > On Thu, Jan 14, 2021 at 10:53 AM Amit Kapila wrote: > > > 0002 - Invalidates the relation map cache in subscriber syscache > > > invalidation callbacks. Currently, I'm setting entry->state to > > > SUBREL_STATE_UNKNOWN in the new

Re: Logical Replication - behavior of ALTER PUBLICATION .. DROP TABLE and ALTER SUBSCRIPTION .. REFRESH PUBLICATION

2021-01-15 Thread Amit Kapila
On Fri, Jan 15, 2021 at 2:46 PM japin wrote: > > > > > I'm sorry, the 0001 patch is totally wrong. If we have only one > > publication, it works, > > however, this is a special case. If we have multiple publication in one > > subscription, > > it doesn't work. Here is a usecase. > > > > Step

Determine parallel-safety of partition relations for Inserts

2021-01-15 Thread Amit Kapila
While reviewing parallel insert [1] (Insert into Select) and parallel copy patches [2], it came to my notice that both the patches traverse the entire partition hierarchy to determine parallel-safety of partitioned relations. This is required because before considering the Insert or Copy can

Re: Added schema level support for publication.

2021-01-15 Thread vignesh C
Thanks Rahila for your comments, please find my thoughts below. On Tue, Jan 12, 2021 at 5:16 PM Rahila Syed wrote: > > Hi Vignesh, > > I had a look at the patch, please consider following comments. > > On Thu, Jan 7, 2021 at 10:03 PM vignesh C wrote: >> >> Hi, >> >> This feature adds schema

Re: Parallel INSERT (INTO ... SELECT ...)

2021-01-15 Thread Amit Kapila
On Thu, Jan 14, 2021 at 2:37 PM Tang, Haiying wrote: > > Hi Greg, Amit > Cc:hackers > > > > > 4. Have you checked the overhead of this on the planner for > > > > different kinds of statements like inserts into tables having 100 > > > > or 500 partitions? Similarly, it is good to check the

Re: Logical Replication - behavior of ALTER PUBLICATION .. DROP TABLE and ALTER SUBSCRIPTION .. REFRESH PUBLICATION

2021-01-15 Thread japin
On Fri, 15 Jan 2021 at 15:49, japin wrote: > On Fri, 15 Jan 2021 at 14:50, Bharath Rupireddy > wrote: >> On Fri, Jan 15, 2021 at 11:33 AM Hou, Zhijie >> wrote: >>> >>> > On Thu, Jan 14, 2021 at 5:36 PM Li Japin wrote >>> > > Do we really need to access PUBLICATIONRELMAP in this patch? What

RE: ResourceOwner refactoring

2021-01-15 Thread kuroda.hay...@fujitsu.com
Dear Heikki, > Hmm. ResOwnerReleaseTupleDesc() does exist, those functions are needed > for the callbacks. I think you meant the wrappers around > ResourceOwnerRemember and ResourceOwnerForget, like > ResourceOwnerRememberCatCacheRef(). I admit those are not fully > consistent: I didn't

Re: fdatasync(2) on macOS

2021-01-15 Thread Thomas Munro
On Fri, Jan 15, 2021 at 7:53 PM Thomas Munro wrote: > That was fun, but now I'm asking myself: do we really want to use an > IO synchronisation facility that's not declared by the vendor? I should add, the default wal_sync_method is open_datasync, not fdatasync. I'm pretty suspicious of that

Re: Wrong HINT during database recovery when occur a minimal wal.

2021-01-15 Thread lchch1...@sina.cn
>Mmm. Maybe something's missing. If you took the base-backup using >pg_basebackup, that means max_wal_senders > 0 on the primary. If you >lowered wal_level in the backup (or replica) then started it, You >would get something like this. >| FATAL: WAL streaming (max_wal_senders > 0) requires

Re: Occasional tablespace.sql failures in check-world -jnn

2021-01-15 Thread Peter Eisentraut
On 2020-12-09 02:29, Andres Freund wrote: I suspect this is related to the pg_upgrade test and the main regression test running at the same time. We have the following in src/test/regress/GNUMakefile # Tablespace setup .PHONY: tablespace-setup tablespace-setup: echo $(realpath

Re: Printing backtrace of postgres processes

2021-01-15 Thread Peter Eisentraut
On 2020-12-08 10:38, vignesh C wrote: I have implemented printing of backtrace based on handling it in CHECK_FOR_INTERRUPTS. This patch also includes the change to allow getting backtrace of any particular process based on the suggestions. Attached patch has the implementation for the same.

Re: list of extended statistics on psql

2021-01-15 Thread Julien Rouhaud
On Wed, Jan 13, 2021 at 10:22:05AM +0900, Tatsuro Yamada wrote: > Hi Tomas, > > On 2021/01/13 7:48, Tatsuro Yamada wrote: > > On 2021/01/12 20:08, Tomas Vondra wrote: > > > On 1/12/21 2:57 AM, Tatsuro Yamada wrote: > > > > On 2021/01/09 9:01, Tomas Vondra wrote: > > > ...> > > > > > While working

Re: Parallel INSERT (INTO ... SELECT ...)

2021-01-15 Thread Amit Kapila
On Fri, Dec 11, 2020 at 4:30 PM Greg Nancarrow wrote: > > Posting an updated set of patches to address recent feedback: > Here is an additional review of v11-0001-Enable-parallel-SELECT-for-INSERT-INTO-.-SELECT. There are quite a few comments raised on the V11-0001* patch. I suggest first post a

Re: Wrong HINT during database recovery when occur a minimal wal.

2021-01-15 Thread Kyotaro Horiguchi
At Fri, 15 Jan 2021 15:32:58 +0800, "lchch1...@sina.cn" wrote in > > Sorry, I don't known why it showed in wrong format, and try to correct it. > - > > When I do PITR in a strange step, I get this FATAL: > > 2021-01-15 15:02:52.364 CST [14958] FATAL: hot standby is not possible >

RE: libpq debug log

2021-01-15 Thread iwata....@fujitsu.com
Hi, Thank you for your review. I modified the code in response to those reviews. This patch includes these items: - Fix the code according to reviews - Fix COPY output issue - Change to not support Protocol v2.0 It is rarely used anymore and de-support it makes code more simpler.

Re: pg_upgrade test for binary compatibility of core data types

2021-01-15 Thread Peter Eisentraut
On 2021-01-12 22:44, Andrew Dunstan wrote: Cross version pg_upgrade is tested regularly in the buildfarm, but not using test.sh. Instead it uses the saved data repository from a previous run of the buildfarm client for the source branch, and tries to upgrade that to the target branch. Does it