Re: adding partitioned tables to publications

2021-01-11 Thread Mark Zhao
Thanks for your reply. The patch is exactly what I want. My English name is Mark Zhao, which should be the current email name. Thanks, Mark Zhao --Original-- From: "Amit Kapila";

Re: Added schema level support for publication.

2021-01-11 Thread Bharath Rupireddy
On Mon, Jan 11, 2021 at 5:28 PM Bharath Rupireddy wrote: > > On Mon, Jan 11, 2021 at 5:25 PM Amit Kapila wrote: > > > > On Mon, Jan 11, 2021 at 4:29 PM Li Japin wrote: > > > > > > > > > On Jan 11, 2021, at 5:06 PM, Bharath Rupireddy > > > wrote: > > > > > > On Mon, Jan 11, 2021 at 1:29 PM

Re: Add Nullif case for eval_const_expressions_mutator

2021-01-11 Thread Tom Lane
"Hou, Zhijie" writes: > I notice that there are no Nullif case in eval_const_expression. > Since Nullif is similar to Opexpr and is easy to implement, > I try add this case in eval_const_expressions_mutator. I think this patch should be about a tenth the size. Try modeling it on the

Re: Proposal: Global Index

2021-01-11 Thread 曾文旌
> 2021年1月8日 16:26,Julien Rouhaud 写道: > > On Fri, Jan 8, 2021 at 4:02 PM 曾文旌 wrote: >> >>> 2021年1月7日 22:16,Bruce Momjian 写道: >>> >>> On Thu, Jan 7, 2021 at 05:44:01PM +0800, 曾文旌 wrote: I've been following this topic for a long time. It's been a year since the last response.

Re: adding partitioned tables to publications

2021-01-11 Thread Amit Kapila
On Wed, Dec 30, 2020 at 8:03 PM 赵锐 <875941...@qq.com> wrote: > > The first file of Amit's patch can not only re-range the code, but also fix a > hidden bug. > To make it easy to see, I attach another patch. > "RelationIdGetRelation" will increase ref on owner->relrefarr, without >

Re: Added schema level support for publication.

2021-01-11 Thread Amit Kapila
On Mon, Jan 11, 2021 at 4:29 PM Li Japin wrote: > > > On Jan 11, 2021, at 5:06 PM, Bharath Rupireddy > wrote: > > On Mon, Jan 11, 2021 at 1:29 PM japin wrote: > > Sorry, I misunderstood. After the test (ce6a71fa530). I found that if we do > not insert data > between step (5) and (6), it will

Re: pg_upgrade test for binary compatibility of core data types

2021-01-11 Thread Peter Eisentraut
On 2020-12-27 20:07, Justin Pryzby wrote: On Wed, Dec 16, 2020 at 11:22:23AM -0600, Justin Pryzby wrote: On Sun, Dec 06, 2020 at 12:02:48PM -0600, Justin Pryzby wrote: I meant to notice if the binary format is accidentally changed again, which was what happened here: 7c15cef86 Base

Re: Multiple hosts in connection string failed to failover in non-hot standby mode

2021-01-11 Thread Tom Lane
Hubert Zhang writes: > As for patch 0004, find ':' after "could not connect to" may failed when > error message like: > "could not connect to host "localhost" (::1), port 12345: Connection > refused", where p2 will point to "::1" instead of ": Connection refused". But > since it's only used

Re: pg_upgrade test for binary compatibility of core data types

2021-01-11 Thread Bruce Momjian
On Mon, Jan 11, 2021 at 03:28:08PM +0100, Peter Eisentraut wrote: > On 2020-12-27 20:07, Justin Pryzby wrote: > > rebased on 6df7a9698bb036610c1e8c6d375e1be38cb26d5f > > I think these patches could use some in-place documentation of what they are > trying to achieve and how they do it. The

Re: Added schema level support for publication.

2021-01-11 Thread Bharath Rupireddy
On Mon, Jan 11, 2021 at 5:25 PM Amit Kapila wrote: > > On Mon, Jan 11, 2021 at 4:29 PM Li Japin wrote: > > > > > > On Jan 11, 2021, at 5:06 PM, Bharath Rupireddy > > wrote: > > > > On Mon, Jan 11, 2021 at 1:29 PM japin wrote: > > > > Sorry, I misunderstood. After the test (ce6a71fa530). I

RE: [POC] Fast COPY FROM command for the table with foreign partitions

2021-01-11 Thread Tang, Haiying
Hi Andrey, I had a general look at this extension feature, I think it's beneficial for some application scenarios of PostgreSQL. So I did 7 performance cases test on your patch(v13). The results are really good. As you can see below we can get 7-10 times improvement with this patch. PSA

Re: [PATCH] Feature improvement for CLOSE, FETCH, MOVE tab completion

2021-01-11 Thread Peter Eisentraut
On 2021-01-05 10:56, Masahiko Sawada wrote: BTW according to the documentation, the options of DECLARE statement (BINARY, INSENSITIVE, SCROLL, and NO SCROLL) are order-sensitive. DECLARE name [ BINARY ] [ INSENSITIVE ] [ [ NO ] SCROLL ] CURSOR [ { WITH | WITHOUT } HOLD ] FOR query But I

Re: libpq compression

2021-01-11 Thread Konstantin Knizhnik
On 09.01.2021 23:31, Justin Pryzby wrote: On Thu, Dec 17, 2020 at 05:54:28PM +0300, Konstantin Knizhnik wrote: I am maintaining this code in g...@github.com:postgrespro/libpq_compression.git repository. I will be pleased if anybody, who wants to suggest any bug fixes/improvements of libpq

Re: Proposal: Global Index

2021-01-11 Thread 曾文旌
> 2021年1月7日 23:04,Robert Haas 写道: > > On Thu, Jan 7, 2021 at 4:44 AM 曾文旌 wrote: >> I've been following this topic for a long time. It's been a year since the >> last response. >> It was clear that our customers wanted this feature as well, and a large >> number of them mentioned it. >> >>

Re: [Bug Fix] Logical replication on partition table is very slow and CPU is 99%

2021-01-11 Thread Ashutosh Bapat
On Mon, Jan 4, 2021 at 7:07 PM 赵锐 <875941...@qq.com> wrote: > > > This patch adds the missed decrement to resolve the problem. > > > Previous discussion is here: > https://www.postgresql.org/message-id/flat/CA+HiwqH=Y85vRK3mOdjEkqFK+E=ST=eQiHdpj43L=_ejmoo...@mail.gmail.com > And I believe patch

Re: multi-install PostgresNode

2021-01-11 Thread Peter Eisentraut
On 2020-12-20 18:09, Andrew Dunstan wrote: On 12/19/20 11:19 AM, Andrew Dunstan wrote: This turns out to be remarkably short, with the use of a little eval magic. Give the attached, this test program works just fine: #!/bin/perl use PostgresNodePath; $ENV{PG_REGRESS} =

Re: popcount

2021-01-11 Thread Peter Eisentraut
On 2020-12-30 17:41, David Fetter wrote: The input may have more than 2 billion bits set to 1. The biggest possible result should be 8 billion for bytea (1 GB with all bits set to 1). So shouldn't this function return an int8? It does now, and thanks for looking at this. The documentation

Logical Replication - behavior of ALTER PUBLICATION .. DROP TABLE and ALTER SUBSCRIPTION .. REFRESH PUBLICATION

2021-01-11 Thread Bharath Rupireddy
Hi, While providing thoughts on the design in [1], I found a strange behaviour with the $subject. The use case is shown below as a sequence of steps that need to be run on publisher and subscriber to arrive at the strange behaviour. In step 5, the table is dropped from the publication and in

Re: Multiple hosts in connection string failed to failover in non-hot standby mode

2021-01-11 Thread Hubert Zhang
Hi Tom, I agree to get detailed error message for each failed host as your patch 0001. As for patch 0004, find ':' after "could not connect to" may failed when error message like: "could not connect to host "localhost" (::1), port 12345: Connection refused", where p2 will point to "::1"

Re: popcount

2021-01-11 Thread David Fetter
On Mon, Jan 11, 2021 at 03:50:54PM +0100, Peter Eisentraut wrote: > On 2020-12-30 17:41, David Fetter wrote: > > > The input may have more than 2 billion bits set to 1. The biggest possible > > > result should be 8 billion for bytea (1 GB with all bits set to 1). > > > So shouldn't this function

Re: Moving other hex functions to /common

2021-01-11 Thread Bruce Momjian
On Mon, Jan 11, 2021 at 04:45:14PM +0900, Michael Paquier wrote: > On Wed, Jan 06, 2021 at 08:58:23AM -0500, Bruce Momjian wrote: > > Fine. Do you want to add the overflow to the patch I posted, for all > > encoding types? > > Yeah. It looks that it would be good to be consistent as well for >

Re: Proposal: Global Index

2021-01-11 Thread Bruce Momjian
On Mon, Jan 11, 2021 at 07:40:18PM +0800, 曾文旌 wrote: > >> In addition you mentioned: "It is still unclear if these use-cases justify > >> the architectural changes needed to enable global indexes." > >> Please also describe the problems you see, I will confirm each specific > >> issue one by

Re: Key management with tests

2021-01-11 Thread Stephen Frost
Greetings, * Bruce Momjian (br...@momjian.us) wrote: > On Mon, Jan 11, 2021 at 08:12:00PM +0900, Masahiko Sawada wrote: > > Looking at the patch, it supports three algorithms but only > > PG_CIPHER_AES_KWP is used in the core for now: > > > > +/* > > + * Supported symmetric encryption algorithm.

Re: Proposal: Global Index

2021-01-11 Thread Peter Geoghegan
On Mon, Jan 11, 2021 at 10:37 AM Robert Haas wrote: > I actually think the idea of lazily deleting the index entries is > pretty good, but it won't work if the way the global index is > implemented is by adding a tableoid column. Perhaps there is an opportunity to apply some of the

Re: Proposal: Global Index

2021-01-11 Thread Bruce Momjian
On Mon, Jan 11, 2021 at 01:37:02PM -0500, Robert Haas wrote: > However, there is a VACUUM amplification effect to worry about here ... > That's not necessarily a death sentence for every use case, but it's > going to be pretty bad for tables that are big and heavily updated. Yeah, I had not

Re: Proposal: Global Index

2021-01-11 Thread Tom Lane
Bruce Momjian writes: > On Mon, Jan 11, 2021 at 07:40:18PM +0800, 曾文旌 wrote: >> This is indeed a typical scenario for a partitioned table. >> there are two basic operations >> 1) Monthly DETACH old child table >> 2) Monthly ATTACH new child table >> >> For 1) The DETACH old child table can be

Re: Proposal: Global Index

2021-01-11 Thread Bruce Momjian
On Mon, Jan 11, 2021 at 11:01:20AM -0800, Peter Geoghegan wrote: > However, it probably would be okay if a global index feature performed > poorly in scenarios where partitions get lots of UPDATEs that produce > lots of index bloat and cause lots of LP_DEAD line pointers to > accumulate in heap

Re: Key management with tests

2021-01-11 Thread Stephen Frost
Greetings, * Bruce Momjian (br...@momjian.us) wrote: > On Mon, Jan 11, 2021 at 12:54:49PM -0500, Stephen Frost wrote: > > Although, another approach and one that I've discussed a bit with Bruce, > > is to have more keys- such as a key for temporary files, and perhaps > > even a key for logged

Re: Key management with tests

2021-01-11 Thread Stephen Frost
Greetings, * Bruce Momjian (br...@momjian.us) wrote: > On Mon, Jan 11, 2021 at 01:23:27PM -0500, Stephen Frost wrote: > > Yes, and it avoids the issue of using a single key for too much, which > > is also a concern. The remaining larger issues are to figure out a > > place to put the tag for

Re: Key management with tests

2021-01-11 Thread Bruce Momjian
On Mon, Jan 11, 2021 at 12:54:49PM -0500, Stephen Frost wrote: > Although, another approach and one that I've discussed a bit with Bruce, > is to have more keys- such as a key for temporary files, and perhaps > even a key for logged relations and a different for unlogged.. Or Yes, we have to

Re: [POC] Fast COPY FROM command for the table with foreign partitions

2021-01-11 Thread Tomas Vondra
Hi Andrey, Unfortunately, this no longer applies :-( I tried to apply this on top of c532d15ddd (from 2020/12/30) but even that has non-trivial conflicts. Can you send a rebased version? regards -- Tomas Vondra EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: Proposal: Global Index

2021-01-11 Thread Robert Haas
On Mon, Jan 11, 2021 at 12:46 PM Bruce Momjian wrote: > > For 1) The DETACH old child table can be finished immediately, global index > > can be kept valid after DETACH is completed, and the cleanup of garbage > > data in global index can be deferred to VACUUM. > > This is similar to the global

Re: Key management with tests

2021-01-11 Thread Bruce Momjian
On Mon, Jan 11, 2021 at 01:23:27PM -0500, Stephen Frost wrote: > Greetings, > > * Bruce Momjian (br...@momjian.us) wrote: > > On Mon, Jan 11, 2021 at 12:54:49PM -0500, Stephen Frost wrote: > > > Although, another approach and one that I've discussed a bit with Bruce, > > > is to have more keys-

Re: Key management with tests

2021-01-11 Thread Bruce Momjian
On Mon, Jan 11, 2021 at 08:12:00PM +0900, Masahiko Sawada wrote: > On Sun, Jan 10, 2021 at 11:51 PM Bruce Momjian wrote: > > OK, here they are with numeric prefixes. It was actually tricky to > > figure out how to create a squashed format-patch based on another branch. > > Thank you for

Re: libpq compression

2021-01-11 Thread Tomas Vondra
On 1/11/21 2:53 PM, Konstantin Knizhnik wrote: > > ... > > New version of libpq compression patch is attached. > It can be also be found at g...@github.com:postgrespro/libpq_compression.git > Seems it bit-rotted already, so here's a slightly fixed version. 1) Fixes the MSVC makefile. The list

Re: Key management with tests

2021-01-11 Thread Bruce Momjian
On Mon, Jan 11, 2021 at 02:19:22PM -0500, Stephen Frost wrote: > Greetings, > > * Bruce Momjian (br...@momjian.us) wrote: > > On Mon, Jan 11, 2021 at 01:23:27PM -0500, Stephen Frost wrote: > > > Yes, and it avoids the issue of using a single key for too much, which > > > is also a concern. The

Re: Key management with tests

2021-01-11 Thread Stephen Frost
Greetings, * Bruce Momjian (br...@momjian.us) wrote: > On Mon, Jan 11, 2021 at 02:19:22PM -0500, Stephen Frost wrote: > > outputs from the GCM encryption and is what provides the integrity / > > authentication of the encrypted data to be able to detect if it's been > > modified. Unfortunately,

Re: Proposal: Global Index

2021-01-11 Thread Bruce Momjian
On Mon, Jan 11, 2021 at 12:05:43PM -0800, Peter Geoghegan wrote: > On Mon, Jan 11, 2021 at 11:25 AM Bruce Momjian wrote: > > Once you layer on all the places a global index will be worse than just > > creating a single large table, or a partitioned table with an index per > > child, there might

Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits

2021-01-11 Thread Tomas Vondra
On 1/11/21 10:00 PM, Anastasia Lubennikova wrote: On 11.01.2021 01:35, Tomas Vondra wrote: Hi, I started looking at this patch again, hoping to get it committed in this CF, but I think there's a regression in handling TOAST tables (compared to the v3 patch submitted by Pavan in February

Re: Deleting older versions in unique indexes to avoid page splits

2021-01-11 Thread Victor Yegorov
this. Therefore I didn't perform any tests here, will wait for the next version. -- Victor Yegorov 20210111-v13-regression.diffs Description: Binary data

Re: Implement for window functions

2021-01-11 Thread Tom Lane
I started to look through this patch, and the first thing I'm wondering about is why bother with a new pg_proc column, ie why not just apply the behavior to any window function? The fact that the SQL committee restricts this syntax to a few window functions is just their usual design tic of

Re: Deleting older versions in unique indexes to avoid page splits

2021-01-11 Thread Peter Geoghegan
On Mon, Jan 11, 2021 at 12:19 PM Victor Yegorov wrote: > (see attached diff). It doesn't look like the fault of this patch, though. > > I suppose you plan to send another revision before committing this. > Therefore I didn't perform any tests here, will wait for the next version. I imagine that

Re: Proposal: Global Index

2021-01-11 Thread Peter Geoghegan
On Mon, Jan 11, 2021 at 11:25 AM Bruce Momjian wrote: > Once you layer on all the places a global index will be worse than just > creating a single large table, or a partitioned table with an index per > child, there might not be much usefulness left. A POC patch might tell > us that, and might

Re: COPY FREEZE and setting PD_ALL_VISIBLE/visibility map bits

2021-01-11 Thread Anastasia Lubennikova
On 11.01.2021 01:35, Tomas Vondra wrote: Hi, I started looking at this patch again, hoping to get it committed in this CF, but I think there's a regression in handling TOAST tables (compared to the v3 patch submitted by Pavan in February 2019). The test I'm running a very simple test (see

Re: Deleting older versions in unique indexes to avoid page splits

2021-01-11 Thread Victor Yegorov
пн, 11 янв. 2021 г. в 22:10, Peter Geoghegan : > I imagine that this happened because you have track_io_timing=on in > postgresql.conf. Doesn't the same failure take place with the current > master branch? > > I have my own way of running the locally installed Postgres when I > want "make

Re: Logical Replication - behavior of ALTER PUBLICATION .. DROP TABLE and ALTER SUBSCRIPTION .. REFRESH PUBLICATION

2021-01-11 Thread japin
On Tue, 12 Jan 2021 at 11:37, Amit Kapila wrote: > On Mon, Jan 11, 2021 at 6:51 PM Bharath Rupireddy > wrote: >> >> Hi, >> >> While providing thoughts on the design in [1], I found a strange >> behaviour with the $subject. The use case is shown below as a sequence >> of steps that need to be

Re: Deleting older versions in unique indexes to avoid page splits

2021-01-11 Thread Peter Geoghegan
On Sun, Jan 10, 2021 at 4:06 PM Peter Geoghegan wrote: > Attached is v13, which has this tweak, and other miscellaneous cleanup > based on review from both Victor and Heikki. I consider this version > of the patch to be committable. I intend to commit something close to > it in the next week,

Re: Logical Replication - behavior of ALTER PUBLICATION .. DROP TABLE and ALTER SUBSCRIPTION .. REFRESH PUBLICATION

2021-01-11 Thread japin
On Tue, 12 Jan 2021 at 13:39, Amit Kapila wrote: > On Tue, Jan 12, 2021 at 9:58 AM japin wrote: >> >> >> On Tue, 12 Jan 2021 at 11:37, Amit Kapila wrote: >> > On Mon, Jan 11, 2021 at 6:51 PM Bharath Rupireddy >> > wrote: >> >> >> >> Hi, >> >> >> >> While providing thoughts on the design in

Fix a typo in SearchCatCache function comment

2021-01-11 Thread Bharath Rupireddy
Hi, While reviewing the patch in [1], I found that SearchCatCache function uses SearchCatCacheInternal as it's function name in the comment. It looks like a typo, attaching a very small patch to correct it. Thoughts? [1] -

RE: Disable WAL logging to speed up data loading

2021-01-11 Thread osumi.takami...@fujitsu.com
On Tuesday, January 12, 2021 12:52 PM Takayuki/綱川 貴之 wrote: > From: Osumi, Takamichi/大墨 昂道 > > I updated the patch following this discussion, and fixed the > > documentation as well. > > > + (rmid == RM_GIST_ID && info == RM_GIST_ID) || > + (rmid ==

Re: Reduce the number of special cases to build contrib modules on windows

2021-01-11 Thread Michael Paquier
On Wed, Dec 30, 2020 at 10:07:29AM +1300, David Rowley wrote: > -#ifdef LOWER_NODE > +/* > + * Below we ignore the fact that LOWER_NODE is defined when compiling with > + * MSVC. The reason for this is that earlier versions of the MSVC build > + * scripts failed to define LOWER_NODE. More recent

Re: O(n^2) system calls in RemoveOldXlogFiles()

2021-01-11 Thread Michael Paquier
On Tue, Jan 12, 2021 at 07:15:21PM +1300, Thomas Munro wrote: > Hah, I even knew that, apparently, but forgot. Adding Michael who > wrote a patch. It'd be nice to fix this, at least in 14. Yeah, this rings a bell. I never went back to it even if the thing looks rather clean at quick glance

RE: Add Nullif case for eval_const_expressions_mutator

2021-01-11 Thread Hou, Zhijie
> > I notice that there are no Nullif case in eval_const_expression. > > Since Nullif is similar to Opexpr and is easy to implement, I try add > > this case in eval_const_expressions_mutator. > > I think this patch should be about a tenth the size. Try modeling it on > the T_SubscriptingRef-etc

"has_column_privilege()" issue with attnums and non-existent columns

2021-01-11 Thread Ian Lawrence Barwick
Greetings Consider a table set up as follows: CREATE TABLE foo (id INT, val TEXT); ALTER TABLE foo DROP COLUMN val; When specifying the name of a dropped or non-existent column, the function "has_column_privilege()" works as expected: postgres=# SELECT has_column_privilege('foo',

Re: Logical Replication - behavior of ALTER PUBLICATION .. DROP TABLE and ALTER SUBSCRIPTION .. REFRESH PUBLICATION

2021-01-11 Thread Bharath Rupireddy
On Tue, Jan 12, 2021 at 9:05 AM Amit Kapila wrote: > > On Mon, Jan 11, 2021 at 6:51 PM Bharath Rupireddy > wrote: > > > > Hi, > > > > While providing thoughts on the design in [1], I found a strange > > behaviour with the $subject. The use case is shown below as a sequence > > of steps that need

Re: Logical Replication - behavior of ALTER PUBLICATION .. DROP TABLE and ALTER SUBSCRIPTION .. REFRESH PUBLICATION

2021-01-11 Thread Amit Kapila
On Tue, Jan 12, 2021 at 11:39 AM Bharath Rupireddy wrote: > > On Tue, Jan 12, 2021 at 9:05 AM Amit Kapila wrote: > > > > On Mon, Jan 11, 2021 at 6:51 PM Bharath Rupireddy > > wrote: > > > > > > Hi, > > > > > > While providing thoughts on the design in [1], I found a strange > > > behaviour with

Re: Logical Replication - behavior of ALTER PUBLICATION .. DROP TABLE and ALTER SUBSCRIPTION .. REFRESH PUBLICATION

2021-01-11 Thread Amit Kapila
On Tue, Jan 12, 2021 at 9:58 AM japin wrote: > > > On Tue, 12 Jan 2021 at 11:37, Amit Kapila wrote: > > On Mon, Jan 11, 2021 at 6:51 PM Bharath Rupireddy > > wrote: > >> > >> Hi, > >> > >> While providing thoughts on the design in [1], I found a strange > >> behaviour with the $subject. The use

Re: O(n^2) system calls in RemoveOldXlogFiles()

2021-01-11 Thread Andres Freund
Hi, On 2021-01-11 16:35:56 +1300, Thomas Munro wrote: > I noticed that RemoveXlogFile() has this code: > > /* > * Before deleting the file, see if it can be recycled as a future log > * segment. Only recycle normal files, pg_standby for example can > create >

Re: O(n^2) system calls in RemoveOldXlogFiles()

2021-01-11 Thread Thomas Munro
On Tue, Jan 12, 2021 at 6:55 PM Andres Freund wrote: > I found this before as well: > https://postgr.es/m/CAB7nPqTB3VcKSSrW2Qj59tYYR2H4+n=5pzbdwou+x9iqvnm...@mail.gmail.com Hah, I even knew that, apparently, but forgot. Adding Michael who wrote a patch. It'd be nice to fix this, at least in

Re: Fix a typo in SearchCatCache function comment

2021-01-11 Thread Michael Paquier
On Tue, Jan 12, 2021 at 12:17:20PM +0530, Bharath Rupireddy wrote: > While reviewing the patch in [1], I found that SearchCatCache function > uses SearchCatCacheInternal as it's function name in the comment. It > looks like a typo, attaching a very small patch to correct it. Good catch. I'll go

Re: libpq compression

2021-01-11 Thread Konstantin Knizhnik
On 11.01.2021 20:38, Tomas Vondra wrote: On 1/11/21 2:53 PM, Konstantin Knizhnik wrote: ... New version of libpq compression patch is attached. It can be also be found at g...@github.com:postgrespro/libpq_compression.git Seems it bit-rotted already, so here's a slightly fixed version. 1)

RE: Add Nullif case for eval_const_expressions_mutator

2021-01-11 Thread Hou, Zhijie
> > I think this patch should be about a tenth the size. Try modeling it > > on the T_SubscriptingRef-etc case, ie, use ece_generic_processing and > > then ece_evaluate_expr to cover the generic cases. OpExpr is common > > enough to deserve specially optimized code, but NullIf isn't, so shorter

RE: Disable WAL logging to speed up data loading

2021-01-11 Thread tsunakawa.ta...@fujitsu.com
From: Osumi, Takamichi/大墨 昂道 > Oops, sorry for this careless mistake. > Fixed. And again, regression test produces no failure. Now correct. (Remains ready for committer.) Regards Takayuki Tsunakawa

Re: [PATCH] Feature improvement for CLOSE, FETCH, MOVE tab completion

2021-01-11 Thread Fujii Masao
On Tue, Jan 12, 2021 at 10:00 AM Masahiko Sawada wrote: > > On Mon, Jan 11, 2021 at 11:00 PM Peter Eisentraut > wrote: > > > > On 2021-01-05 10:56, Masahiko Sawada wrote: > > > BTW according to the documentation, the options of DECLARE statement > > > (BINARY, INSENSITIVE, SCROLL, and NO SCROLL)

RE: POC: postgres_fdw insert batching

2021-01-11 Thread tsunakawa.ta...@fujitsu.com
From: Tomas Vondra > Attached is a v6 of this patch, rebased to current master and with some minor > improvements (mostly comments and renaming the "end" struct field to > "values_end" which I think is more descriptive). Thank you very much. In fact, my initial patches used values_end, and I

Re: [Patch] Optimize dropping of relation buffers using dlist

2021-01-11 Thread Amit Kapila
On Fri, Jan 8, 2021 at 7:03 AM Kyotaro Horiguchi wrote: > > At Thu, 7 Jan 2021 09:25:22 +, "k.jami...@fujitsu.com" > wrote in: > > > Thanks for the detailed tests. NBuffers/32 seems like an appropriate > > > value for the threshold based on these results. I would like to > > > slightly

Re: list of extended statistics on psql

2021-01-11 Thread Tatsuro Yamada
Hi Tomas, On 2021/01/09 9:01, Tomas Vondra wrote: On 1/8/21 1:14 AM, Tomas Vondra wrote: On 1/8/21 12:52 AM, Tatsuro Yamada wrote: On 2021/01/08 0:56, Tomas Vondra wrote: On 1/7/21 3:47 PM, Alvaro Herrera wrote: On 2021-Jan-07, Tomas Vondra wrote: On 1/7/21 1:46 AM, Tatsuro Yamada wrote:

Re: POC: postgres_fdw insert batching

2021-01-11 Thread Tomas Vondra
Hi, Attached is a v6 of this patch, rebased to current master and with some minor improvements (mostly comments and renaming the "end" struct field to "values_end" which I think is more descriptive). The one thing that keeps bugging me is convert_prep_stmt_params - it dies the right thing,

Re: [POC] Fast COPY FROM command for the table with foreign partitions

2021-01-11 Thread Andrey V. Lepikhov
On 1/11/21 11:16 PM, Tomas Vondra wrote: Hi Andrey, Unfortunately, this no longer applies :-( I tried to apply this on top of c532d15ddd (from 2020/12/30) but even that has non-trivial conflicts. Can you send a rebased version? regards Applied on 044aa9e70e. -- regards, Andrey Lepikhov

Re: logical replication worker accesses catalogs in error context callback

2021-01-11 Thread Masahiko Sawada
On Mon, Jan 11, 2021 at 4:46 PM Bharath Rupireddy wrote: > > On Mon, Jan 11, 2021 at 10:56 AM Masahiko Sawada > wrote: > > Agreed. Attached the updated patch. > > Thanks for the updated patch. Looks like the comment crosses the 80 > char limit, I have corrected it. And also changed the variable

Outdated replication protocol error?

2021-01-11 Thread Jeff Davis
Commit 5ee2197767 (about 4 years old) introduced the error: "IDENTIFY_SYSTEM has not been run before START_REPLICATION" But it seems like running START_REPLICATION first works (at least in the simple case). We should either: 1. Document that IDENTIFY_SYSTEM must always be run before

Re: Key management with tests

2021-01-11 Thread Masahiko Sawada
On Tue, Jan 12, 2021 at 3:23 AM Stephen Frost wrote: > > Greetings, > > * Bruce Momjian (br...@momjian.us) wrote: > > On Mon, Jan 11, 2021 at 12:54:49PM -0500, Stephen Frost wrote: > > > Although, another approach and one that I've discussed a bit with Bruce, > > > is to have more keys- such as a

Re: Huge memory consumption on partitioned table with FKs

2021-01-11 Thread Keisuke Kuroda
Hi Amit-san, > Thanks for checking. Indeed, it should have been added to the January > commit-fest. I've added it to the March one: > > https://commitfest.postgresql.org/32/2930/ Thank you for your quick response! -- Keisuke Kuroda NTT Software Innovation Center keisuke.kuroda.3...@gmail.com

Re: pg_dump, ATTACH, and independently restorable child partitions

2021-01-11 Thread Tom Lane
Justin Pryzby writes: > [ v5-0001-pg_dump-output-separate-object-for-ALTER-TABLE.AT.patch ] Pushed with mostly cosmetic edits. One thing I changed that isn't cosmetic is that I set the ArchiveEntry's owner to be the owner of the child table. Although we aren't going to do any sort of ALTER

RE: Disable WAL logging to speed up data loading

2021-01-11 Thread osumi.takami...@fujitsu.com
Hi On Mon, Jan 11, 2021 9:14 AM Tsunakawa, Takayuki wrote: > From: Masahiko Sawada > > I think it's better to have index AM (and perhaps table AM) control it > > instead of filtering in XLogInsert(). Because otherwise third-party > > access methods that use LSN like gist indexes cannot write

Re: Logical Replication - behavior of ALTER PUBLICATION .. DROP TABLE and ALTER SUBSCRIPTION .. REFRESH PUBLICATION

2021-01-11 Thread Amit Kapila
On Mon, Jan 11, 2021 at 6:51 PM Bharath Rupireddy wrote: > > Hi, > > While providing thoughts on the design in [1], I found a strange > behaviour with the $subject. The use case is shown below as a sequence > of steps that need to be run on publisher and subscriber to arrive at > the strange

Re: [PATCH] Feature improvement for CLOSE, FETCH, MOVE tab completion

2021-01-11 Thread Masahiko Sawada
On Mon, Jan 11, 2021 at 11:00 PM Peter Eisentraut wrote: > > On 2021-01-05 10:56, Masahiko Sawada wrote: > > BTW according to the documentation, the options of DECLARE statement > > (BINARY, INSENSITIVE, SCROLL, and NO SCROLL) are order-sensitive. > > > > DECLARE name [ BINARY ] [ INSENSITIVE ] [

Re: Key management with tests

2021-01-11 Thread Bruce Momjian
On Tue, Jan 12, 2021 at 09:32:54AM +0900, Masahiko Sawada wrote: > On Tue, Jan 12, 2021 at 3:23 AM Stephen Frost wrote: > > Right, or ensure that the actual IV used is distinct (such as by using > > another bit in the IV to distinguish logged-vs-unlogged), but it seems > > saner to just use a

Re: Moving other hex functions to /common

2021-01-11 Thread Michael Paquier
On Mon, Jan 11, 2021 at 11:27:30AM -0500, Bruce Momjian wrote: > Sure, I realize the elog/pg_log is odd, but the alternatives seem worse. I guess that it depends on the use cases. If there is no need to worry about shared libraries, elog/pg_log would do just fine. > You can take ownership of my

Re: [POC] Fast COPY FROM command for the table with foreign partitions

2021-01-11 Thread Andrey V. Lepikhov
On 1/11/21 4:59 PM, Tang, Haiying wrote: Hi Andrey, I had a general look at this extension feature, I think it's beneficial for some application scenarios of PostgreSQL. So I did 7 performance cases test on your patch(v13). The results are really good. As you can see below we can get 7-10

Re: Movement of restart_lsn position movement of logical replication slots is very slow

2021-01-11 Thread Jammie
Hi Amit, Thanks for the response . Can you please let me know what pg_current_wal_lsn returns ? is this position the LSN of the next log record to be created, or is it the LSN of the last log record already created and inserted in the log? The document says - it returns current WAL write

RE: Disable WAL logging to speed up data loading

2021-01-11 Thread tsunakawa.ta...@fujitsu.com
From: Osumi, Takamichi/大墨 昂道 > I updated the patch following this discussion, and fixed the documentation as > well. + (rmid == RM_GIST_ID && info == RM_GIST_ID) || + (rmid == RM_GENERIC_ID))) info is wrong for GiST, and one pair of parenthesis is unnecessary.

Re: pg_upgrade test for binary compatibility of core data types

2021-01-11 Thread Justin Pryzby
On Mon, Jan 11, 2021 at 03:28:08PM +0100, Peter Eisentraut wrote: > On 2020-12-27 20:07, Justin Pryzby wrote: > > On Wed, Dec 16, 2020 at 11:22:23AM -0600, Justin Pryzby wrote: > > > On Sun, Dec 06, 2020 at 12:02:48PM -0600, Justin Pryzby wrote: > > > > I meant to notice if the binary format is

Re: libpq compression

2021-01-11 Thread Justin Pryzby
On Mon, Jan 11, 2021 at 04:53:51PM +0300, Konstantin Knizhnik wrote: > On 09.01.2021 23:31, Justin Pryzby wrote: > > I suggest that there should be an enum of algorithms, which is constant > > across > > all servers. They would be unconditionally included and not #ifdef > > depending > > on

Re: A failure of standby to follow timeline switch

2021-01-11 Thread Fujii Masao
On Sat, Jan 9, 2021 at 5:08 AM Alvaro Herrera wrote: > > Masao-san: Are you intending to act as committer for these? Since the > bug is mine I can look into it, but since you already did all the > reviewing work, I'm good with you giving it the final push. Thanks! I'm thinking to push the

Re: Key management with tests

2021-01-11 Thread Neil Chen
Hi Stephen, On Tue, Jan 12, 2021 at 10:47 AM Stephen Frost wrote: > > This is an interesting question but ultimately I don't think we should > be looking at this from the perspective of allowing arbitrary changes to > the page format. The challenge is that much of the page format, today, > is

Re: Single transaction in the tablesync worker?

2021-01-11 Thread Amit Kapila
On Mon, Jan 11, 2021 at 3:53 PM Ajin Cherian wrote: > > On Thu, Jan 7, 2021 at 3:20 PM Amit Kapila wrote: > > > > BTW, I have analyzed whether we need any modifications to > > > pg_dump/restore for this patch as this changes the state of one of the > > > fields in the system table and concluded

Re: Added schema level support for publication.

2021-01-11 Thread Li Japin
On Jan 11, 2021, at 5:06 PM, Bharath Rupireddy mailto:bharath.rupireddyforpostg...@gmail.com>> wrote: On Mon, Jan 11, 2021 at 1:29 PM japin mailto:japi...@hotmail.com>> wrote: Say a user has created a publication for a schema with hundreds of tables in it, at some point later, can he stop

Re: Key management with tests

2021-01-11 Thread Masahiko Sawada
On Sun, Jan 10, 2021 at 11:51 PM Bruce Momjian wrote: > > On Sun, Jan 10, 2021 at 06:04:12PM +1300, Thomas Munro wrote: > > On Sun, Jan 10, 2021 at 3:45 PM Bruce Momjian wrote: > > > Does anyone know why the cfbot applied the patch listed second first > > > here? > > > > > >

Re: [HACKERS] Custom compression methods

2021-01-11 Thread Dilip Kumar
On Mon, Jan 11, 2021 at 12:21 PM Justin Pryzby wrote: > > On Mon, Jan 11, 2021 at 12:11:54PM +0530, Dilip Kumar wrote: > > On Mon, Jan 11, 2021 at 11:00 AM Dilip Kumar wrote: > > > On Sun, Jan 10, 2021 at 10:59 PM Justin Pryzby > > > wrote: > > > > > > > > On Mon, Jan 04, 2021 at 04:57:16PM

Re: Single transaction in the tablesync worker?

2021-01-11 Thread Ajin Cherian
On Thu, Jan 7, 2021 at 3:20 PM Amit Kapila wrote: > > BTW, I have analyzed whether we need any modifications to > > pg_dump/restore for this patch as this changes the state of one of the > > fields in the system table and concluded that we don't need any > > change. For subscriptions, we don't

Re: Added schema level support for publication.

2021-01-11 Thread japin
On Mon, 11 Jan 2021 at 14:15, Bharath Rupireddy wrote: > On Sun, Jan 10, 2021 at 11:21 PM vignesh C wrote: >> On Sat, Jan 9, 2021 at 8:08 PM Bharath Rupireddy >> wrote: >> > I think this feature can be useful, in case a user has a lot of tables >> > to publish inside a schema. Having said

Re: Improper use about DatumGetInt32

2021-01-11 Thread Michael Paquier
On Sat, Jan 09, 2021 at 01:41:39PM +0100, Peter Eisentraut wrote: > If we had a way to do such testing then we wouldn't need these markers. But > AFAICT, we don't. Not sure I am following your point here. Taking your patch, it is possible to trigger the version of get_raw_page() <= 1.8 just with

Re: Added schema level support for publication.

2021-01-11 Thread Bharath Rupireddy
On Mon, Jan 11, 2021 at 1:29 PM japin wrote: > >> > Say a user has created a publication for a schema with hundreds of > >> > tables in it, at some point later, can he stop replicating a single or > >> > some tables from that schema? > >> > > >> > >> There is no provision for this currently. > >