Re: [Patch] ALTER SYSTEM READ ONLY

2021-03-02 Thread Dilip Kumar
in point [2], because of which you have to postpone RECOVERY_STATE_DONE untill system is set back to read-write. Can you explain this? -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: [Patch] ALTER SYSTEM READ ONLY

2021-03-02 Thread Dilip Kumar
y is the best way to solve this but as of now I don't have anything better to suggest, and immediately I couldn’t think of any problem with this solution. But I will think about this again. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: [Patch] ALTER SYSTEM READ ONLY

2021-03-02 Thread Dilip Kumar
On Tue, Mar 2, 2021 at 9:01 PM Dilip Kumar wrote: > > > > We don't want that to happen in cases where previous > > recovery-end-checkpoint is > > skipped in startup. We want Checkpointer first to convey the barrier to all > > backends but, the backend shouldn

Re: Parallel INSERT (INTO ... SELECT ...)

2021-03-03 Thread Dilip Kumar
ld not add that cost. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Parallel INSERT (INTO ... SELECT ...)

2021-03-03 Thread Dilip Kumar
tiply the rows from the GatherPath. I can not see the rows from GatherPath is ever set to 0. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: [Patch] ALTER SYSTEM READ ONLY

2021-03-03 Thread Dilip Kumar
On Wed, Mar 3, 2021 at 4:50 PM Amul Sul wrote: > > On Wed, Mar 3, 2021 at 12:08 PM Dilip Kumar wrote: > > > Yes, it is possible to allow wal temporarily for itself by setting > LocalXLogInsertAllowed, but when we request Checkpointer for the > end-of-recovery > checkp

Re: Parallel INSERT (INTO ... SELECT ...)

2021-03-04 Thread Dilip Kumar
On Thu, Mar 4, 2021 at 7:16 AM Greg Nancarrow wrote: > > On Thu, Mar 4, 2021 at 1:07 AM Dilip Kumar wrote: > > > > On Wed, Mar 3, 2021 at 5:50 PM Greg Nancarrow wrote: > > > > > I agree that assert is only for debug build, but once we add and > > asser

Re: Parallel INSERT (INTO ... SELECT ...)

2021-03-04 Thread Dilip Kumar
those checks for Updates as well. They will need > some adjustment if we remove that check but the intent is clear. +1 -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] Custom compression methods

2021-03-04 Thread Dilip Kumar
veryone. Another advantage is that > getting those first two patches committed or even just applied locally > on a branch would, at least IMHO, make it a lot simpler to see what > potential problem spots remain - and by "problem" I mean mostly from a > performance point of view. Okay, I will work on this. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] Custom compression methods

2021-03-06 Thread Dilip Kumar
On Sun, Mar 7, 2021 at 1:27 AM Justin Pryzby wrote: > > On Sat, Mar 06, 2021 at 08:59:16PM +0530, Dilip Kumar wrote: > > - Alter table set compression, will not rewrite the old data, so only > > the new tuple will be compressed with the new compression method. > > - N

Re: [HACKERS] Custom compression methods

2021-03-07 Thread Dilip Kumar
t do that. > You'd have a table defined as pglz but with lz4 in the data files. > In the best case, it would give errors about corrupt lz4 data. Yeah, we can not do that. Just missed that part :) -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] Custom compression methods

2021-03-07 Thread Dilip Kumar
ort alternate compression in the dump itself > (in addition to zlib), so the name will be confusing. I suggest > --no-toast-compression, like the GUC. And the same for pg_restore. Ok. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: [Patch] ALTER SYSTEM READ ONLY

2021-03-08 Thread Dilip Kumar
On Wed, Mar 3, 2021 at 8:56 PM Robert Haas wrote: > > On Tue, Mar 2, 2021 at 7:22 AM Dilip Kumar wrote: > > Why do we need to move promote related code in XLogAcceptWrites? > > IMHO, this promote related handling should be in StartupXLOG only. > > That will look cle

Re: New Table Access Methods for Multi and Single Inserts

2021-03-08 Thread Dilip Kumar
o invent a new version table_insert_v2? And also why it is named table_insert* instead of table_tuple_insert*? -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] Custom compression methods

2021-03-08 Thread Dilip Kumar
REWRITE is given and the compression method is the same as the existing then also we can not skip the rewrite because we don't know the history, the user might alter the compression method multiple times without rewrite. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] Custom compression methods

2021-03-09 Thread Dilip Kumar
On Tue, Mar 9, 2021 at 1:13 PM Justin Pryzby wrote: > > On Tue, Mar 09, 2021 at 01:04:10PM +0530, Dilip Kumar wrote: > > On Tue, Mar 9, 2021 at 2:45 AM Robert Haas wrote: > > > > > Yeah, vacuum full or cluster will not re-compress the data. How about > > pr

Re: [HACKERS] Custom compression methods

2021-03-09 Thread Dilip Kumar
ess the data stored inside an already formed composite type but I will analyze this. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Parallel INSERT (INTO ... SELECT ...)

2021-03-10 Thread Dilip Kumar
here a table has a lot of partition then it can be turned off using reloption. And, if for some users most of the tables are like that where they always have a lot of partition then the user can turn it off using guc. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] Custom compression methods

2021-03-10 Thread Dilip Kumar
On Thu, Mar 11, 2021 at 2:21 AM Robert Haas wrote: > > On Wed, Mar 10, 2021 at 6:52 AM Dilip Kumar wrote: > > The pending comment is providing a way to rewrite a table and > > re-compress the data with the current compression method. > > I spent some time poking a

Re: [HACKERS] Custom compression methods

2021-03-10 Thread Dilip Kumar
m because it might exist in the WAL and if you do the crash recovery then might get those lz4 compressed data back. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Is Recovery actually paused?

2021-03-11 Thread Dilip Kumar
On Fri, 12 Mar 2021 at 2:04 AM, Robert Haas wrote: > On Mon, Mar 1, 2021 at 12:08 AM Dilip Kumar wrote: > > > > > I'll wait for a day before marking this RfC in case anyone have > > > > > further comments. > > > > > > > >

Re: [HACKERS] Custom compression methods

2021-03-11 Thread Dilip Kumar
a mac user who wants to take > on a hopefully-quick project. Thanks for your help. I did not understand the reason for removal of lz4.h header check? -if test "$with_lz4" = yes; then - AC_CHECK_HEADERS(lz4/lz4.h, [], - [AC_CHECK_HEADERS(lz4.h, [], [AC_MSG_ERROR([lz4.h header

Re: [HACKERS] Custom compression methods

2021-03-11 Thread Dilip Kumar
ade can enable the slice > behavior. Yeah, that makes sense, I will change in the next version. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] Custom compression methods

2021-03-11 Thread Dilip Kumar
On Fri, Mar 12, 2021 at 9:03 AM Justin Pryzby wrote: > > On Fri, Mar 12, 2021 at 08:38:41AM +0530, Dilip Kumar wrote: > > On Thu, Mar 11, 2021 at 11:55 PM Justin Pryzby wrote: > > > On Wed, Mar 10, 2021 at 08:28:58PM -0600, Justin Pryzby wrote: > > > > This incl

Re: [HACKERS] Custom compression methods

2021-03-11 Thread Dilip Kumar
On Fri, Mar 12, 2021 at 2:54 AM Justin Pryzby wrote: > > On Thu, Mar 11, 2021 at 10:07:30AM +0530, Dilip Kumar wrote: > > On Thu, Mar 11, 2021 at 8:50 AM Justin Pryzby wrote: > > > > > > Looking at v23-0002-alter-table-set-compression, ATRew

Re: [HACKERS] Custom compression methods

2021-03-11 Thread Dilip Kumar
installed > pkg-config), but I eventually realized that LZ4 was effectively disabled, > because we set HAVE_LZ4, but the code tested instead WITH_LIBLZ4. So is it working on the Mac with your latest changes in 0005 or we need to wait for the result? Also is there any reason for changing HAVE_LIBLZ4 to USE_LZ4? -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] Custom compression methods

2021-03-11 Thread Dilip Kumar
in the next version. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] Custom compression methods

2021-03-12 Thread Dilip Kumar
installed > pkg-config), but I eventually realized that LZ4 was effectively disabled, > because we set HAVE_LZ4, but the code tested instead WITH_LIBLZ4. With this patch, I see USE_LZ4 is never defined in my centos machine(even --with-lz4), however it was working fine without the 0005 patch. I

Re: [HACKERS] Custom compression methods

2021-03-12 Thread Dilip Kumar
On Fri, Mar 12, 2021 at 2:12 PM Dilip Kumar wrote: > > On Fri, Mar 12, 2021 at 10:45 AM Justin Pryzby wrote: > > > > On Thu, Mar 11, 2021 at 12:25:26PM -0600, Justin Pryzby wrote: > > > On Wed, Mar 10, 2021 at 08:28:58PM -0600, Justin Pryzby wrote: > > > >

Re: [HACKERS] Custom compression methods

2021-03-15 Thread Dilip Kumar
coming from the source table then we will be inserting those compressed data as it is in the target attribute without externalizing. Maybe it is done to avoid such performance impacts? Well, we can do the same for the compression also and just provide some mechanism to recompress maybe in vacuum full/cluster. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] Custom compression methods

2021-03-15 Thread Dilip Kumar
nse to me in case we want to support different compression methods for different data types. Currently, we have only pglz and lz4 but if we are not planning for custom compression in the future then we can support 2 more built-in compression methods so I still feel having an attribute level option makes more sense. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] Custom compression methods

2021-03-16 Thread Dilip Kumar
On Tue, Mar 16, 2021 at 12:59 AM Robert Haas wrote: > > On Mon, Mar 15, 2021 at 8:14 AM Dilip Kumar wrote: > > In the attached patches I have changed this, ... > > OK, so just looking over this patch series, here's what I think: > > Regarding 0003: > > The

Re: [HACKERS] Custom compression methods

2021-03-16 Thread Dilip Kumar
On Tue, Mar 16, 2021 at 4:07 PM Dilip Kumar wrote: > > INSERT TIME > Head: 17418.299 ms Patch: 20956.231 ms > > CTAS TIME: > Head: 12837.872 ms Patch: 16775.739 ms On quick analysis with perf it appeared that the performance is degrading because of deforming - 16.19%

Re: [HACKERS] Custom compression methods

2021-03-16 Thread Dilip Kumar
ho would be unhappy. If that is only the argument then it's possible today as well. I mean you can INSERT INTO .. SELECT FROM where source attribute as compressed data but the target attribute as external storage then also we will move the compressed data as it is to the target table. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Re: parallel distinct union and aggregate support patch

2020-10-27 Thread Dilip Kumar
tuplesort_end(state->batches[i]); + state->batches[i] = NULL; + } + } I did not understand this part, that once each worker has performed their local batch-wise sort why we are clearing the baches? I mean individual workers have their on batches so eventually they supposed to get merged. Can you explain this part and also it will be better if you can add the comments. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Re: parallel distinct union and aggregate support patch

2020-10-27 Thread Dilip Kumar
On Tue, Oct 27, 2020 at 3:27 PM Dilip Kumar wrote: > > On Fri, Oct 23, 2020 at 11:58 AM bu...@sohu.com wrote: > > > > > Interesting idea. So IIUC, whenever a worker is scanning the tuple it > > > will directly put it into the respective batch(shared tuple stor

Re: parallel distinct union and aggregate support patch

2020-10-27 Thread Dilip Kumar
On Tue, Oct 27, 2020 at 5:43 PM Robert Haas wrote: > > On Thu, Oct 22, 2020 at 5:08 AM Dilip Kumar wrote: > > Interesting idea. So IIUC, whenever a worker is scanning the tuple it > > will directly put it into the respective batch(shared tuple store), > > based on the

Re: Re: parallel distinct union and aggregate support patch

2020-11-03 Thread Dilip Kumar
uery is doing the grouping on the aggregated value of the subquery, due to this we are not selecting the parallelism in the head because in the inner aggregation the number of groups is huge and if we select the parallelism we need to transfer a lot of tuple through the tuple queue and we will also hav

Re: extension patch of CREATE OR REPLACE TRIGGER

2020-11-06 Thread Dilip Kumar
1 and then test2 can try it with OR REPLACE syntax. This way we can reduce the extra statement execution which is not necessary. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Re: parallel distinct union and aggregate support patch

2020-11-07 Thread Dilip Kumar
On Tue, Nov 3, 2020 at 6:06 PM Dilip Kumar wrote: > > On Thu, Oct 29, 2020 at 12:53 PM bu...@sohu.com wrote: > > > > > 1) It's better to always include the whole patch series - including the > > > parts that have not changed. Otherwise people have to scavenge

Re: logical streaming of xacts via test_decoding is broken

2020-11-08 Thread Dilip Kumar
t; an empty transaction before the commit of transaction which we are > going to stream changes for (before s1_commit). > > Thoughts? The analysis seems correct to me, I will work on it. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: logical streaming of xacts via test_decoding is broken

2020-11-08 Thread Dilip Kumar
On Mon, Nov 9, 2020 at 11:04 AM Dilip Kumar wrote: > > On Mon, Nov 9, 2020 at 11:00 AM Amit Kapila wrote: > > > > Michael reported a BF failure [1] related to one of the logical > > streaming test case and I've analyzed the issue. As responded on > > pgsql-com

Re: logical streaming of xacts via test_decoding is broken

2020-11-09 Thread Dilip Kumar
On Mon, Nov 9, 2020 at 11:31 AM Amit Kapila wrote: > > On Mon, Nov 9, 2020 at 11:21 AM Dilip Kumar wrote: > > > > On Mon, Nov 9, 2020 at 11:04 AM Dilip Kumar wrote: > > > > > > On Mon, Nov 9, 2020 at 11:00 AM Amit Kapila > > > wrote: > > &g

Re: logical streaming of xacts via test_decoding is broken

2020-11-09 Thread Dilip Kumar
On Mon, Nov 9, 2020 at 1:34 PM Dilip Kumar wrote: > > On Mon, Nov 9, 2020 at 11:31 AM Amit Kapila wrote: > > > > On Mon, Nov 9, 2020 at 11:21 AM Dilip Kumar wrote: > > > > > > On Mon, Nov 9, 2020 at 11:04 AM Dilip Kumar wrote: > > > > >

Re: logical streaming of xacts via test_decoding is broken

2020-11-09 Thread Dilip Kumar
On Mon, Nov 9, 2020 at 4:21 PM Amit Kapila wrote: > > On Mon, Nov 9, 2020 at 3:01 PM Dilip Kumar wrote: > > > > On Mon, Nov 9, 2020 at 1:34 PM Dilip Kumar wrote: > > > > > > On Mon, Nov 9, 2020 at 11:31 AM Amit Kapila > > > wrote: > > >

Re: logical streaming of xacts via test_decoding is broken

2020-11-09 Thread Dilip Kumar
On Mon, Nov 9, 2020 at 5:37 PM Dilip Kumar wrote: > > On Mon, Nov 9, 2020 at 4:21 PM Amit Kapila wrote: > > > > On Mon, Nov 9, 2020 at 3:01 PM Dilip Kumar wrote: > > > > > > On Mon, Nov 9, 2020 at 1:34 PM Dilip Kumar wrote: > > > > >

Re: logical streaming of xacts via test_decoding is broken

2020-11-09 Thread Dilip Kumar
On Tue, Nov 10, 2020 at 8:14 AM Amit Kapila wrote: > > On Mon, Nov 9, 2020 at 6:00 PM Dilip Kumar wrote: > > > > On Mon, Nov 9, 2020 at 5:37 PM Dilip Kumar wrote: > > > > > > On Mon, Nov 9, 2020 at 4:21 PM Amit Kapila > > > wrote: > > >

Re: logical streaming of xacts via test_decoding is broken

2020-11-09 Thread Dilip Kumar
On Tue, Nov 10, 2020 at 10:52 AM Amit Kapila wrote: > > On Tue, Nov 10, 2020 at 10:26 AM Dilip Kumar wrote: > > > > On Tue, Nov 10, 2020 at 8:14 AM Amit Kapila wrote: > > > > > > On Mon, Nov 9, 2020 at 6:00 PM Dilip Kumar wrote: > > > > &g

Re: logical streaming of xacts via test_decoding is broken

2020-11-10 Thread Dilip Kumar
On Tue, Nov 10, 2020 at 11:18 AM Dilip Kumar wrote: > > On Tue, Nov 10, 2020 at 10:52 AM Amit Kapila wrote: > > > > On Tue, Nov 10, 2020 at 10:26 AM Dilip Kumar wrote: > > > > > > On Tue, Nov 10, 2020 at 8:14 AM Amit Kapila > > > wrote: > &g

Re: logical streaming of xacts via test_decoding is broken

2020-11-10 Thread Dilip Kumar
On Tue, Nov 10, 2020 at 7:20 PM Amit Kapila wrote: > > On Tue, Nov 10, 2020 at 2:25 PM Dilip Kumar wrote: > > > > On Tue, Nov 10, 2020 at 11:18 AM Dilip Kumar wrote: > > > > > > On Tue, Nov 10, 2020 at 10:52 AM Amit Kapila > > > wrote: > >

Re: logical streaming of xacts via test_decoding is broken

2020-11-11 Thread Dilip Kumar
On Wed, Nov 11, 2020 at 6:59 PM Amit Kapila wrote: > > On Wed, Nov 11, 2020 at 10:00 AM Dilip Kumar wrote: > > > > On Tue, Nov 10, 2020 at 7:20 PM Amit Kapila wrote: > > > > > > On Tue, Nov 10, 2020 at 2:25 PM Dilip Kumar wrote: > > > > > &g

Re: logical streaming of xacts via test_decoding is broken

2020-11-11 Thread Dilip Kumar
On Thu, Nov 12, 2020 at 8:45 AM Amit Kapila wrote: > > On Wed, Nov 11, 2020 at 7:05 PM Dilip Kumar wrote: > > > > On Wed, Nov 11, 2020 at 6:59 PM Amit Kapila wrote: > > > > > > On Wed, Nov 11, 2020 at 10:00 AM Dilip Kumar > > > wrote: > &g

Re: logical streaming of xacts via test_decoding is broken

2020-11-11 Thread Dilip Kumar
On Thu, 12 Nov 2020 at 11:37 AM, Amit Kapila wrote: > On Thu, Nov 12, 2020 at 11:29 AM Dilip Kumar > wrote: > > > > On Thu, Nov 12, 2020 at 8:45 AM Amit Kapila > wrote: > > > > > > > > Another thing I am thinking let's just not expose sk

Re: logical streaming of xacts via test_decoding is broken

2020-11-12 Thread Dilip Kumar
On Tue, Nov 10, 2020 at 7:20 PM Amit Kapila wrote: > > On Tue, Nov 10, 2020 at 2:25 PM Dilip Kumar wrote: > > > > On Tue, Nov 10, 2020 at 11:18 AM Dilip Kumar wrote: > > > > > > On Tue, Nov 10, 2020 at 10:52 AM Amit Kapila > > > wrote: > >

Remove unused variable from SharedSort

2020-11-12 Thread Dilip Kumar
While going through the code I noticed that the nTapes member in SharedSort is unused. This is just initialized with nworkers but never used. The attached patch removes this variable. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com v1-0001-Remove-unused-structure-member

Re: logical streaming of xacts via test_decoding is broken

2020-11-14 Thread Dilip Kumar
On Fri, Nov 13, 2020 at 3:18 PM Amit Kapila wrote: > > On Thu, Nov 12, 2020 at 3:10 PM Dilip Kumar wrote: > > > > On Tue, Nov 10, 2020 at 7:20 PM Amit Kapila wrote: > > > > > > On Tue, Nov 10, 2020 at 2:25 PM Dilip Kumar wrote: > > > > > 3. Ca

Re: Remove unused variable from SharedSort

2020-11-15 Thread Dilip Kumar
On Sun, Nov 15, 2020 at 12:50 PM Bharath Rupireddy wrote: > > On Thu, Nov 12, 2020 at 5:29 PM Dilip Kumar wrote: > > > > While going through the code I noticed that the nTapes member in > > SharedSort is unused. This is just initialized with nworkers but > > n

Re: Re: parallel distinct union and aggregate support patch

2020-11-17 Thread Dilip Kumar
On Sun, Nov 8, 2020 at 11:54 AM Dilip Kumar wrote: > > On Tue, Nov 3, 2020 at 6:06 PM Dilip Kumar wrote: > > > > On Thu, Oct 29, 2020 at 12:53 PM bu...@sohu.com wrote: > > > > > > > 1) It's better to always include the whole patch series - inclu

Re: [HACKERS] Custom compression methods

2020-11-23 Thread Dilip Kumar
On Sat, Nov 21, 2020 at 3:50 AM Robert Haas wrote: > > On Wed, Nov 11, 2020 at 9:39 AM Dilip Kumar wrote: > > There were a few problems in this rebased version, basically, the > > compression options were not passed while compressing values from the > > brin_form_tup

Re: [HACKERS] Custom compression methods

2020-11-24 Thread Dilip Kumar
uilt-in type, we can directly call the corresponding handler function and get the routine. The only thing is to avoid duplicating in decompression routine we need to convert CompressionId to Oid before calling GetCompressionRoutine(), but now we can avoid sys cache lookup for the built-in type. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] Custom compression methods

2020-11-24 Thread Dilip Kumar
On Tue, Nov 24, 2020 at 7:14 PM Robert Haas wrote: > > On Tue, Nov 24, 2020 at 7:11 AM Dilip Kumar wrote: > > About (4), one option is that we directly call the correct handler > > function for the built-in type directly from > > toast_(de)compress(_slice) functions

Re: parallel distinct union and aggregate support patch

2020-11-28 Thread Dilip Kumar
ommon operator then first the Parallel Redistribute operator will use the tuplestore for redistributing the data as per the worker and then each worker might use the disk again to sort their respective data. Instead of that while redistributing the data itself we can use the parallel sort so that each worker gets their respective batch in form of sorted tapes. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Is Recovery actually paused?

2020-11-30 Thread Dilip Kumar
On Mon, Nov 30, 2020 at 12:17 PM Yugo NAGATA wrote: Thanks for looking into this. > On Thu, 22 Oct 2020 20:36:48 +0530 > Dilip Kumar wrote: > > > On Thu, Oct 22, 2020 at 7:50 PM Dilip Kumar wrote: > > > > > > On Thu, Oct 22, 2020 at 6:59 AM Kyotaro Horiguchi

Re: [HACKERS] Custom compression methods

2021-01-10 Thread Dilip Kumar
On Sun, Jan 10, 2021 at 10:59 PM Justin Pryzby wrote: > > On Mon, Jan 04, 2021 at 04:57:16PM +0530, Dilip Kumar wrote: > > On Mon, Jan 4, 2021 at 6:52 AM Justin Pryzby wrote: > > > And fails pg_upgrade check, apparently losing track of the compression (?) > > > >

Re: [HACKERS] Custom compression methods

2021-01-10 Thread Dilip Kumar
On Mon, Jan 11, 2021 at 11:00 AM Dilip Kumar wrote: > > On Sun, Jan 10, 2021 at 10:59 PM Justin Pryzby wrote: > > > > On Mon, Jan 04, 2021 at 04:57:16PM +0530, Dilip Kumar wrote: > > > On Mon, Jan 4, 2021 at 6:52 AM Justin Pryzby wrote: > > > > And fa

Re: [HACKERS] Custom compression methods

2021-01-11 Thread Dilip Kumar
On Mon, Jan 11, 2021 at 12:21 PM Justin Pryzby wrote: > > On Mon, Jan 11, 2021 at 12:11:54PM +0530, Dilip Kumar wrote: > > On Mon, Jan 11, 2021 at 11:00 AM Dilip Kumar wrote: > > > On Sun, Jan 10, 2021 at 10:59 PM Justin Pryzby > > > wrote: > > > >

Re: Logical Replication - behavior of ALTER PUBLICATION .. DROP TABLE and ALTER SUBSCRIPTION .. REFRESH PUBLICATION

2021-01-13 Thread Dilip Kumar
ication, I think > when the tables that aren't in publication should not be replicated. > > Attached the patch that fixes it. Thought? > Instead of doing this, I would expect that the RelationSyncCache entry should be removed when the relation is dropped from the publication. So if that is done then it will reload the publication and then it will not find that that relation as published and it will ignore the changes. But the patch doesn't seem to be exactly on that line. Am I missing something here? -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Logical Replication - behavior of ALTER PUBLICATION .. DROP TABLE and ALTER SUBSCRIPTION .. REFRESH PUBLICATION

2021-01-13 Thread Dilip Kumar
On Wed, Jan 13, 2021 at 3:06 PM japin wrote: > > > On Wed, 13 Jan 2021 at 17:23, Dilip Kumar wrote: > > On Tue, Jan 12, 2021 at 4:47 PM Li Japin wrote: > >> > >> > >> On Jan 12, 2021, at 5:47 PM, japin wrote: > >> > >> > >> O

Re: Logical Replication - behavior of ALTER PUBLICATION .. DROP TABLE and ALTER SUBSCRIPTION .. REFRESH PUBLICATION

2021-01-13 Thread Dilip Kumar
On Wed, Jan 13, 2021 at 3:08 PM Bharath Rupireddy wrote: > > On Wed, Jan 13, 2021 at 2:53 PM Dilip Kumar wrote: > > > IIUC the logical replication only replicate the tables in publication, I > > > think > > > when the tables that aren't i

Re: Is Recovery actually paused?

2021-01-13 Thread Dilip Kumar
On Wed, Jan 13, 2021 at 3:27 PM Yugo NAGATA wrote: > > On Thu, 10 Dec 2020 11:25:23 +0530 > Dilip Kumar wrote: > > > > > However, I wonder users don't expect pg_is_wal_replay_paused to wait. > > > > Especially, if max_standby_streaming_delay is

Re: Logical Replication - behavior of ALTER PUBLICATION .. DROP TABLE and ALTER SUBSCRIPTION .. REFRESH PUBLICATION

2021-01-13 Thread Dilip Kumar
On Wed, Jan 13, 2021 at 3:08 PM Bharath Rupireddy wrote: > > On Wed, Jan 13, 2021 at 2:53 PM Dilip Kumar wrote: > > > IIUC the logical replication only replicate the tables in publication, I > > > think > > > when the tables that aren't i

Re: Is Recovery actually paused?

2021-01-13 Thread Dilip Kumar
On Wed, Jan 13, 2021 at 3:35 PM Dilip Kumar wrote: > > On Wed, Jan 13, 2021 at 3:27 PM Yugo NAGATA wrote: > > > > On Thu, 10 Dec 2020 11:25:23 +0530 > > Dilip Kumar wrote: > > > > > > > However, I wonder users don't expect pg_is_w

Re: Is Recovery actually paused?

2021-01-16 Thread Dilip Kumar
ot good. I > > think we want the recovery to be paused but want the wal receiver to > > continue receiving WAL. > > I had misunderstood the code and the patch, please ignore this > comment. Pausing the recovery here is fine with me. Thanks for the review Sawada-S

Re: Is Recovery actually paused?

2021-01-16 Thread Dilip Kumar
On Thu, Jan 14, 2021 at 6:49 PM Yugo NAGATA wrote: > > On Wed, 13 Jan 2021 17:49:43 +0530 > Dilip Kumar wrote: > > > On Wed, Jan 13, 2021 at 3:35 PM Dilip Kumar wrote: > > > > > > On Wed, Jan 13, 2021 at 3:27 PM Yugo NAGATA wrote: > > >

Re: Is Recovery actually paused?

2021-01-17 Thread Dilip Kumar
On Sat, Jan 16, 2021 at 8:59 AM Masahiko Sawada wrote: > > On Wed, Jan 13, 2021 at 9:20 PM Dilip Kumar wrote: > > > > On Wed, Jan 13, 2021 at 3:35 PM Dilip Kumar wrote: > > > > > > On Wed, Jan 13, 2021 at 3:27 PM Yugo NAGATA wrote: > > >

Re: Is Recovery actually paused?

2021-01-17 Thread Dilip Kumar
On Sun, Jan 17, 2021 at 1:48 PM Dilip Kumar wrote: > > On Sat, Jan 16, 2021 at 8:59 AM Masahiko Sawada wrote: > > > > On Wed, Jan 13, 2021 at 9:20 PM Dilip Kumar wrote: > > > > > > On Wed, Jan 13, 2021 at 3:35 PM Dilip Kumar wrote: > > > > >

Re: Is Recovery actually paused?

2021-01-18 Thread Dilip Kumar
On Tue, 19 Jan 2021 at 8:12 AM, Yugo NAGATA wrote: > On Sun, 17 Jan 2021 11:33:52 +0530 > Dilip Kumar wrote: > > > On Thu, Jan 14, 2021 at 6:49 PM Yugo NAGATA wrote: > > > > > > On Wed, 13 Jan 2021 17:49:43 +0530 > > > Dilip Kumar wrote: > >

Re: Is Recovery actually paused?

2021-01-18 Thread Dilip Kumar
On Tue, Jan 19, 2021 at 10:30 AM Kyotaro Horiguchi wrote: > > At Tue, 19 Jan 2021 11:41:18 +0900, Yugo NAGATA wrote in > > On Sun, 17 Jan 2021 11:33:52 +0530 > > Dilip Kumar wrote: > > > > > > > > > > > > > > I thi

Re: [patch] Help information for pg_dump

2021-01-19 Thread Dilip Kumar
are mentioned in capital letters in both > pg_dump and pg_dumpall cases, so changing "do NOT" to "do not" seems > okay to me. > > Patch LGTM. Also "do NOT" is inconsistent with the other message where we are saying "do not" so +1 -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: TOAST condition for column size

2021-01-19 Thread Dilip Kumar
gt; > I understand that TOAST is not effective for small data and it's > not recommended to create a table containing hundreds of columns, > but I think cases that can be successful should be successful. > > Any thoughts? How this can be correct? because while

Re: TOAST condition for column size

2021-01-19 Thread Dilip Kumar
On Tue, 19 Jan 2021 at 6:28 PM, Amit Kapila wrote: > On Tue, Jan 19, 2021 at 5:18 PM Dilip Kumar wrote: > > > > On Mon, Jan 18, 2021 at 7:53 PM torikoshia > wrote: > > > > > > Hi, > > > > > > When I created a table consisting of 400 VARCHAR

Re: Is Recovery actually paused?

2021-01-19 Thread Dilip Kumar
On Tue, Jan 19, 2021 at 8:34 AM Dilip Kumar wrote: > > On Tue, 19 Jan 2021 at 8:12 AM, Yugo NAGATA wrote: >> >> On Sun, 17 Jan 2021 11:33:52 +0530 >> Dilip Kumar wrote: >> >> > On Thu, Jan 14, 2021 at 6:49 PM Yugo NAGATA wrote: >> > > >

Re: [HACKERS] Custom compression methods

2021-01-19 Thread Dilip Kumar
On Wed, Jan 20, 2021 at 12:37 AM Justin Pryzby wrote: > > Thanks for updating the patch. Thanks for the review > On Mon, Jan 4, 2021 at 6:52 AM Justin Pryzby wrote: > > The most recent patch doesn't compile --without-lz4: > On Tue, Jan 05, 2021 at 11:19:33AM +0530, Di

Race condition in recovery?

2021-01-21 Thread Dilip Kumar
00098. rsync: link_stat "/wal_archive/ins30wfm02dbs/000E.history" failed: No such file or directory (2) rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1179) [sender=3.1.2] rsync: link_stat "/wal_archive/ins30wfm02dbs/000C00F8" failed: No such file or directory (2) -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Is Recovery actually paused?

2021-01-21 Thread Dilip Kumar
On Thu, Jan 21, 2021 at 3:29 PM Bharath Rupireddy wrote: Thanks for reviewing Bharat. > On Tue, Jan 19, 2021 at 9:32 PM Dilip Kumar wrote: > > In the last patch there were some local changes which I did not add to > > the patch and it was giving compilation warning so fixed that

Re: Is Recovery actually paused?

2021-01-22 Thread Dilip Kumar
keep doubling, max of 5s, or something of > that sort), so I'm not sure there's actually any real problem in > practice. So to me it seems more likely that an interface that is > based on waiting will cause difficulty for tool-writers than one that > does not. > > Other p

Re: Race condition in recovery?

2021-01-22 Thread Dilip Kumar
On Fri, Jan 22, 2021 at 2:05 AM Robert Haas wrote: > > On Thu, Jan 21, 2021 at 4:00 AM Dilip Kumar wrote: > > 8. Node3, get it because walsender of Node2 read it from TL13 and send > > it and Node2 write in the new WAL file but with TL12. > &g

Re: Is Recovery actually paused?

2021-01-22 Thread Dilip Kumar
On Thu, Jan 21, 2021 at 6:20 PM Yugo NAGATA wrote: > > On Tue, 19 Jan 2021 21:32:31 +0530 > Dilip Kumar wrote: > > > On Tue, Jan 19, 2021 at 8:34 AM Dilip Kumar wrote: > > > > > > On Tue, 19 Jan 2021 at 8:12 AM, Yugo NAGATA wrote: > > >> > &

Re: Is Recovery actually paused?

2021-01-23 Thread Dilip Kumar
On Sat, Jan 23, 2021 at 9:56 AM Dilip Kumar wrote: > > On Fri, Jan 22, 2021 at 2:18 AM Robert Haas wrote: > > > > On Mon, Jan 18, 2021 at 9:42 PM Yugo NAGATA wrote: > > > If it is acceptable that pg_is_wal_replay_paused() makes users wait, > > > I'm o

Re: Is Recovery actually paused?

2021-01-23 Thread Dilip Kumar
On Sat, 23 Jan 2021 at 4:40 PM, Bharath Rupireddy < bharath.rupireddyforpostg...@gmail.com> wrote: > On Sat, Jan 23, 2021 at 1:36 PM Dilip Kumar wrote: > > Please find the patch for the same. I haven't added a test case for > > this yet. I mean we can write a test

Re: Is Recovery actually paused?

2021-01-23 Thread Dilip Kumar
On Sat, Jan 23, 2021 at 4:40 PM Bharath Rupireddy wrote: > > On Sat, Jan 23, 2021 at 1:36 PM Dilip Kumar wrote: > > Please find the patch for the same. I haven't added a test case for > > this yet. I mean we can write a test case to pause the recovery and > > get t

Re: Is Recovery actually paused?

2021-01-24 Thread Dilip Kumar
On Sun, Jan 24, 2021 at 12:16 PM Bharath Rupireddy wrote: > > On Sun, Jan 24, 2021 at 7:17 AM Dilip Kumar wrote: > > On Sat, 23 Jan 2021 at 4:40 PM, Bharath Rupireddy > > wrote: > >> > >> On Sat, Jan 23, 2021 at 1:36 PM Dilip Kumar wrote: > >> >

Re: Is Recovery actually paused?

2021-01-24 Thread Dilip Kumar
On Mon, Jan 25, 2021 at 6:12 AM Masahiko Sawada wrote: > > On Sun, Jan 17, 2021 at 5:19 PM Dilip Kumar wrote: > > > > On Sat, Jan 16, 2021 at 8:59 AM Masahiko Sawada > > wrote: > > > > > > On Wed, Jan 13, 2021 at 9:20 PM Dilip Kumar wrote: > > &

Re: Is Recovery actually paused?

2021-01-24 Thread Dilip Kumar
On Mon, Jan 25, 2021 at 8:42 AM Kyotaro Horiguchi wrote: > > At Sun, 24 Jan 2021 14:26:08 +0530, Dilip Kumar wrote > in > > On Sun, Jan 24, 2021 at 12:16 PM Bharath Rupireddy > > wrote: > > > > > > On Sun, Jan 24, 2021 at 7:17 AM Dilip Kumar wrote:

Re: Identify missing publications from publisher while create/alter subscription.

2021-01-25 Thread Dilip Kumar
bscription set publication. > Thoughts? > So basically, the create subscription will throw an error if the publication does not exist. So will you throw an error if we try to drop the publication which is subscribed by some subscription? I mean basically, you are creating a dependency that if you are creating a subscription then there must be a publication that is not completely insane but then we will have to disallow dropping the publication as well. Am I missing something? -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Is Recovery actually paused?

2021-01-25 Thread Dilip Kumar
On Sun, Jan 24, 2021 at 12:17 PM Bharath Rupireddy wrote: > > On Sun, Jan 24, 2021 at 11:29 AM Dilip Kumar wrote: > > > Some comments on the v6 patch: > > > > [2] Typo - it's "requested" + * 'paused requested' - if pause is > > > reqe

Re: Identify missing publications from publisher while create/alter subscription.

2021-01-25 Thread Dilip Kumar
On Mon, Jan 25, 2021 at 2:48 PM Bharath Rupireddy wrote: > > On Mon, Jan 25, 2021 at 2:42 PM Dilip Kumar wrote: > > > > On Mon, Jan 25, 2021 at 1:10 PM vignesh C wrote: > > > > > > On Thu, Jan 21, 2021 at 10:21 PM Bharath Rupireddy > > > wrote: &g

Re: Identify missing publications from publisher while create/alter subscription.

2021-01-25 Thread Dilip Kumar
On Mon, Jan 25, 2021 at 3:38 PM Bharath Rupireddy wrote: > > On Mon, Jan 25, 2021 at 3:07 PM Dilip Kumar wrote: > > > > So basically, the create subscription will throw an error if the > > > > publication does not exist. So will you throw an error if we try to >

Re: Is Recovery actually paused?

2021-01-26 Thread Dilip Kumar
better > > > thoughts. > > > > Maybe we should leave the existing function pg_is_wal_replay_paused() > > alone and add a new one with the name you suggest that returns text. > > That would create less burden for tool authors. > > +1 > Yeah, we can do that, I will send an updated patch soon. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Support ALTER SUBSCRIPTION ... ADD/DROP PUBLICATION ... syntax

2021-01-27 Thread Dilip Kumar
ax, > for the above > two cases, we can use the following: > > ALTER SUBSCRIPTION mysub1 DROP PUBLICATION mypub4; > > ALTER SUBSCRIPTION mysub1 DROP PUBLICATION mypub7, mypub8; > > I think it's more convenient. Any thoughts? +1 for the idea -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

<    1   2   3   4   5   6   7   8   9   10   >