Re: [HACKERS] Custom compression methods

2021-03-11 Thread Dilip Kumar
wants to take > on a hopefully-quick project. Thanks for your help. I did not understand the reason for removal of lz4.h header check? -if test "$with_lz4" = yes; then - AC_CHECK_HEADERS(lz4/lz4.h, [], - [AC_CHECK_HEADERS(lz4.h, [], [AC_MSG_ERROR([lz4.h header file is require

Re: Is Recovery actually paused?

2021-03-11 Thread Dilip Kumar
On Fri, 12 Mar 2021 at 2:04 AM, Robert Haas wrote: > On Mon, Mar 1, 2021 at 12:08 AM Dilip Kumar wrote: > > > > > I'll wait for a day before marking this RfC in case anyone have > > > > > further comments. > > > > > > > > Oka

Re: [HACKERS] Custom compression methods

2021-03-10 Thread Dilip Kumar
and if you do the crash recovery then might get those lz4 compressed data back. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] Custom compression methods

2021-03-10 Thread Dilip Kumar
On Thu, Mar 11, 2021 at 2:21 AM Robert Haas wrote: > > On Wed, Mar 10, 2021 at 6:52 AM Dilip Kumar wrote: > > The pending comment is providing a way to rewrite a table and > > re-compress the data with the current compression method. > > I spent some time poking a

Re: Parallel INSERT (INTO ... SELECT ...)

2021-03-10 Thread Dilip Kumar
s a lot of partition then it can be turned off using reloption. And, if for some users most of the tables are like that where they always have a lot of partition then the user can turn it off using guc. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] Custom compression methods

2021-03-09 Thread Dilip Kumar
inside an already formed composite type but I will analyze this. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] Custom compression methods

2021-03-09 Thread Dilip Kumar
On Tue, Mar 9, 2021 at 1:13 PM Justin Pryzby wrote: > > On Tue, Mar 09, 2021 at 01:04:10PM +0530, Dilip Kumar wrote: > > On Tue, Mar 9, 2021 at 2:45 AM Robert Haas wrote: > > > > > Yeah, vacuum full or cluster will not re-compress the data. How about > > pr

Re: [HACKERS] Custom compression methods

2021-03-08 Thread Dilip Kumar
compression method is the same as the existing then also we can not skip the rewrite because we don't know the history, the user might alter the compression method multiple times without rewrite. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: New Table Access Methods for Multi and Single Inserts

2021-03-08 Thread Dilip Kumar
d to invent a new version table_insert_v2? And also why it is named table_insert* instead of table_tuple_insert*? -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: [Patch] ALTER SYSTEM READ ONLY

2021-03-08 Thread Dilip Kumar
On Wed, Mar 3, 2021 at 8:56 PM Robert Haas wrote: > > On Tue, Mar 2, 2021 at 7:22 AM Dilip Kumar wrote: > > Why do we need to move promote related code in XLogAcceptWrites? > > IMHO, this promote related handling should be in StartupXLOG only. > > That will look cle

Re: [HACKERS] Custom compression methods

2021-03-07 Thread Dilip Kumar
ssion in the dump itself > (in addition to zlib), so the name will be confusing. I suggest > --no-toast-compression, like the GUC. And the same for pg_restore. Ok. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] Custom compression methods

2021-03-07 Thread Dilip Kumar
do that. > You'd have a table defined as pglz but with lz4 in the data files. > In the best case, it would give errors about corrupt lz4 data. Yeah, we can not do that. Just missed that part :) -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] Custom compression methods

2021-03-06 Thread Dilip Kumar
On Sun, Mar 7, 2021 at 1:27 AM Justin Pryzby wrote: > > On Sat, Mar 06, 2021 at 08:59:16PM +0530, Dilip Kumar wrote: > > - Alter table set compression, will not rewrite the old data, so only > > the new tuple will be compressed with the new compression method. > &g

Re: [HACKERS] Custom compression methods

2021-03-04 Thread Dilip Kumar
> getting those first two patches committed or even just applied locally > on a branch would, at least IMHO, make it a lot simpler to see what > potential problem spots remain - and by "problem" I mean mostly from a > performance point of view. Okay, I will work on this. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Parallel INSERT (INTO ... SELECT ...)

2021-03-04 Thread Dilip Kumar
checks for Updates as well. They will need > some adjustment if we remove that check but the intent is clear. +1 -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Parallel INSERT (INTO ... SELECT ...)

2021-03-04 Thread Dilip Kumar
On Thu, Mar 4, 2021 at 7:16 AM Greg Nancarrow wrote: > > On Thu, Mar 4, 2021 at 1:07 AM Dilip Kumar wrote: > > > > On Wed, Mar 3, 2021 at 5:50 PM Greg Nancarrow wrote: > > > > > I agree that assert is only for debug build, but once we add and > > asser

Re: [Patch] ALTER SYSTEM READ ONLY

2021-03-03 Thread Dilip Kumar
On Wed, Mar 3, 2021 at 4:50 PM Amul Sul wrote: > > On Wed, Mar 3, 2021 at 12:08 PM Dilip Kumar wrote: > > > Yes, it is possible to allow wal temporarily for itself by setting > LocalXLogInsertAllowed, but when we request Checkpointer for the > end-of-recovery > che

Re: Parallel INSERT (INTO ... SELECT ...)

2021-03-03 Thread Dilip Kumar
from the GatherPath. I can not see the rows from GatherPath is ever set to 0. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Parallel INSERT (INTO ... SELECT ...)

2021-03-03 Thread Dilip Kumar
t add that cost. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: [Patch] ALTER SYSTEM READ ONLY

2021-03-02 Thread Dilip Kumar
On Tue, Mar 2, 2021 at 9:01 PM Dilip Kumar wrote: > > > > We don't want that to happen in cases where previous > > recovery-end-checkpoint is > > skipped in startup. We want Checkpointer first to convey the barrier to all > > backends but, the backend shouldn't wr

Re: [Patch] ALTER SYSTEM READ ONLY

2021-03-02 Thread Dilip Kumar
est way to solve this but as of now I don't have anything better to suggest, and immediately I couldn’t think of any problem with this solution. But I will think about this again. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: [Patch] ALTER SYSTEM READ ONLY

2021-03-02 Thread Dilip Kumar
in point [2], because of which you have to postpone RECOVERY_STATE_DONE untill system is set back to read-write. Can you explain this? -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Race condition in recovery?

2021-03-02 Thread Dilip Kumar
On Sat, Jan 23, 2021 at 10:06 AM Dilip Kumar wrote: > > But I am afraid that whether this adjustment (setting based on > receiveTLI) is done based on some analysis or part of some bug fix. I > will try to find the history of this and maybe based on that we can > make a better de

Re: [HACKERS] Custom compression methods

2021-03-02 Thread Dilip Kumar
On Mon, Mar 1, 2021 at 8:53 PM Dilip Kumar wrote: > > Now, I think the only pending thing is related to the expandedrecord, > basically, currently, we have detoasted the compressed filed only in > expanded_record_set_field_internal function. I am still not > completely sure tha

Re: [HACKERS] Custom compression methods

2021-03-01 Thread Dilip Kumar
his release the slicing issue was fixed) then we can call the full decompression routine from the slicing function. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] Custom compression methods

2021-02-28 Thread Dilip Kumar
= false; > The false assignment was copied from namespace.c: baseSearchPathValid. I will fix this. So as of now, we can make this patch such that it is enough to work with the built-in method and later we can add another enhancement patch that can work with the custom compression methods. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Is Recovery actually paused?

2021-02-28 Thread Dilip Kumar
On Fri, Feb 26, 2021 at 1:33 PM Kyotaro Horiguchi wrote: > > At Thu, 25 Feb 2021 13:22:53 +0530, Dilip Kumar wrote > in > > On Thu, Feb 25, 2021 at 12:42 PM Kyotaro Horiguchi > > wrote: > > > The latest version applies (almost) cleanly to the current master and &

Re: [HACKERS] Custom compression methods

2021-02-28 Thread Dilip Kumar
HIDE_COMPRESSAM false > src/test/regress/sql/compression.sql:\set HIDE_COMPRESSAM false > > Whereas the patch I sent had at the end: > > +\set HIDE_COMPRESSAM on > > ("on" is the default when run under pg_regress) I will fix this. [1] https://docs.unrealengine.com/en-US/API/Runtime/Core/Compression/LZ4_decompress_safe_partial/index.html -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] Custom compression methods

2021-02-26 Thread Dilip Kumar
On Fri, Feb 26, 2021 at 8:10 PM Dilip Kumar wrote: > > On Sun, Feb 21, 2021 at 5:33 PM Dilip Kumar wrote: > > > > Based on offlist discussion with Robert, I have done further analysis > of the composite type data. So the Idea is that I have analyzed all > the caller

Re: Is Recovery actually paused?

2021-02-24 Thread Dilip Kumar
further comment on this. > > I'll wait for a day before marking this RfC in case anyone have > further comments. Okay. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Is Recovery actually paused?

2021-02-24 Thread Dilip Kumar
didn't seem > stable I push that a bit strongly but it actually consists of only > three values and not likely to get further values. So I don't insist > on the style so strongly here. > Changed as per the suggestion. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.c

Re: Is Recovery actually paused?

2021-02-24 Thread Dilip Kumar
On Wed, Feb 24, 2021 at 3:25 PM Dilip Kumar wrote: > > > > > The reason for the checkpoint is to move to "paused" state in a > > > > reasonable time. I think we need to mention that reason rather than > > > > what is don

Re: Is Recovery actually paused?

2021-02-24 Thread Dilip Kumar
On Wed, Feb 24, 2021 at 2:26 PM Kyotaro Horiguchi wrote: > > At Wed, 24 Feb 2021 13:15:27 +0530, Dilip Kumar wrote > in > > On Wed, Feb 24, 2021 at 12:39 PM Kyotaro Horiguchi > > wrote: > > > > > > At Tue, 23 Feb 2021 12:03:32 +0530, Dilip Kumar >

Re: Is Recovery actually paused?

2021-02-23 Thread Dilip Kumar
On Wed, Feb 24, 2021 at 12:39 PM Kyotaro Horiguchi wrote: > > At Tue, 23 Feb 2021 12:03:32 +0530, Dilip Kumar wrote > in > > On Fri, Feb 12, 2021 at 3:26 AM Robert Haas wrote: > > > There might be some more to say here, but those are things I notice on > > > a

Re: Is Recovery actually paused?

2021-02-22 Thread Dilip Kumar
On Fri, Feb 12, 2021 at 3:26 AM Robert Haas wrote: > > On Thu, Feb 11, 2021 at 6:07 AM Dilip Kumar wrote: > > > Thanks for the patch. I tested the new function and it works as > > > expected. I have no further comments on the v13 patch. > > > > Thanks for the

Re: [HACKERS] Custom compression methods

2021-02-21 Thread Dilip Kumar
On Sat, Feb 20, 2021 at 11:04 AM Dilip Kumar wrote: > > On Sat, Feb 20, 2021 at 2:51 AM Robert Haas wrote: > > > > On Fri, Feb 19, 2021 at 11:12 AM Dilip Kumar wrote: > > I think that these performance tests aren't really exercising the > > expanded-record stuff

Re: [HACKERS] Custom compression methods

2021-02-20 Thread Dilip Kumar
On Sat, Feb 20, 2021 at 11:04 AM Dilip Kumar wrote: > Even after the above case, we might say it is still not a problem for > this patch because even though t2 doesn't have a direct relationship > with lz4 but it has an indirect relationship with lz4 via t1. So I > think this par

Re: [HACKERS] Custom compression methods

2021-02-19 Thread Dilip Kumar
On Sat, Feb 20, 2021 at 2:51 AM Robert Haas wrote: > > On Fri, Feb 19, 2021 at 11:12 AM Dilip Kumar wrote: > I think that these performance tests aren't really exercising the > expanded-record stuff, just the ExecEvalRow changes. We need to test > that test case, and I tend to

Re: [HACKERS] Custom compression methods

2021-02-19 Thread Dilip Kumar
hould make it correct. Fixed > TOAST_COMPRESS_SET_SIZE_AND_METHOD() could Assert something about cm_method. While replying to the comments, I realised that I have missed it. I will fix it in the next version. > Small delta patch with a few other suggested changes attached. Merged -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] Custom compression methods

2021-02-19 Thread Dilip Kumar
On Thu, Feb 11, 2021 at 1:37 AM Robert Haas wrote: > > On Wed, Feb 10, 2021 at 9:52 AM Dilip Kumar wrote: > > [ new patches ] > > I think that in both varattrib_4b and toast_internals.h it would be > better to pick a less generic field name. In toast_internals.h it's >

Re: [HACKERS] Custom compression methods

2021-02-16 Thread Dilip Kumar
On Sat, Feb 13, 2021 at 8:14 PM Dilip Kumar wrote: > > On Thu, Feb 11, 2021 at 8:17 PM Robert Haas wrote: > > > > On Thu, Feb 11, 2021 at 7:36 AM Dilip Kumar wrote: > > > W.R.T the attached patch, In HeapTupleHeaderGetDatum, we don't even > > > attempt to d

Re: [HACKERS] Custom compression methods

2021-02-16 Thread Dilip Kumar
lz with (level 1) ); Thanks for the review and patch for HIDE_COMPRESSAM, I will merge this into the main patch. And work on other comments after fixing the issue related to compressed data in composite types. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] Custom compression methods

2021-02-13 Thread Dilip Kumar
On Thu, Feb 11, 2021 at 8:17 PM Robert Haas wrote: > > On Thu, Feb 11, 2021 at 7:36 AM Dilip Kumar wrote: > > W.R.T the attached patch, In HeapTupleHeaderGetDatum, we don't even > > attempt to detoast if there is no external field in the tuple, in POC > > I have got

Re: [HACKERS] Custom compression methods

2021-02-11 Thread Dilip Kumar
whether the tuple has any compressed data or not as we have for detecting the external data (HEAP_HASEXTERNAL). So I will do some more analysis in this area and try to come up with a clean solution. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com POC_fix_compression_in_rowtype.patch Description: Binary data

Re: Is Recovery actually paused?

2021-02-11 Thread Dilip Kumar
On Thu, Feb 11, 2021 at 3:20 PM Bharath Rupireddy wrote: > > On Wed, Feb 10, 2021 at 8:39 PM Dilip Kumar wrote: > > > > On Wed, Feb 10, 2021 at 10:02 AM Dilip Kumar wrote: > > > > > > I don't find any problem with this approach as well, but I personally >

Re: Is Recovery actually paused?

2021-02-10 Thread Dilip Kumar
On Wed, Feb 10, 2021 at 10:02 AM Dilip Kumar wrote: > > I don't find any problem with this approach as well, but I personally > feel that the other approach where we don't wait in any API and just > return the recovery pause state is much simpler and more flexible. So > I will m

Re: [HACKERS] Custom compression methods

2021-02-09 Thread Dilip Kumar
On Wed, Feb 10, 2021 at 1:42 AM Robert Haas wrote: > > Please remember to trim unnecessary quoted material. Okay, I will. > On Sun, Feb 7, 2021 at 6:45 AM Dilip Kumar wrote: > > [ a whole lot of quoted stuff ] > > > > Conclusion: > > 1. In most cases lz4 is fa

Re: Is Recovery actually paused?

2021-02-09 Thread Dilip Kumar
ait in any API and just return the recovery pause state is much simpler and more flexible. So I will make the pending changes in that patch and let's see what are the other opinion and based on that we can conclude. Thanks for the patch. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] Custom compression methods

2021-02-09 Thread Dilip Kumar
On Tue, Feb 9, 2021 at 2:08 PM Dilip Kumar wrote: > > On Sun, Feb 7, 2021 at 5:15 PM Dilip Kumar wrote: > > > > On Fri, Feb 5, 2021 at 8:11 PM Dilip Kumar wrote: > > > > > > On Wed, Feb 3, 2021 at 2:07 AM Robert Haas wrote: > > > > > &g

Re: [HACKERS] Custom compression methods

2021-02-09 Thread Dilip Kumar
On Sun, Feb 7, 2021 at 5:15 PM Dilip Kumar wrote: > > On Fri, Feb 5, 2021 at 8:11 PM Dilip Kumar wrote: > > > > On Wed, Feb 3, 2021 at 2:07 AM Robert Haas wrote: > > > > > > Even more review comments, still looking mostly at 0001: > > > &g

Re: Is Recovery actually paused?

2021-02-08 Thread Dilip Kumar
On Tue, Feb 9, 2021 at 8:54 AM Yugo NAGATA wrote: > > On Tue, 09 Feb 2021 10:58:04 +0900 (JST) > Kyotaro Horiguchi wrote: > > > At Mon, 8 Feb 2021 17:05:52 +0530, Dilip Kumar > > wrote in > > > On Mon, Feb 8, 2021 at 2:19 PM Yugo NAGATA wrote: > > &

Re: Is Recovery actually paused?

2021-02-08 Thread Dilip Kumar
On Tue, Feb 9, 2021 at 7:28 AM Kyotaro Horiguchi wrote: > > At Mon, 8 Feb 2021 17:05:52 +0530, Dilip Kumar wrote > in > > On Mon, Feb 8, 2021 at 2:19 PM Yugo NAGATA wrote: > > > > > > On Mon, 08 Feb 2021 17:32:46 +0900 (JST) > > > Kyotaro Horiguchi

Re: Is Recovery actually paused?

2021-02-08 Thread Dilip Kumar
s be as below NOT PAUSED -> PAUSE REQUESTED or PAUSED (maybe we should always go to request and then paused but there is nothing wrong with going to paused) PAUSE REQUESTED -> NOT PAUSE or PAUSED (either cancel the request or get paused) PAUSED -> NOT PAUSED (from PAUSED we should not go to the PAUSE_REQUESTED without going to NOT PAUSED) -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Is Recovery actually paused?

2021-02-07 Thread Dilip Kumar
On Mon, Feb 8, 2021 at 9:49 AM Bharath Rupireddy wrote: > > On Mon, Feb 8, 2021 at 9:35 AM Dilip Kumar wrote: > > > > > If a user call pg_wal_replay_pause while waiting in > > > > > RecoveryRequiresIntParameter, > > > > > the state become

Re: Is Recovery actually paused?

2021-02-07 Thread Dilip Kumar
On Mon, Feb 8, 2021 at 8:18 AM Yugo NAGATA wrote: > > On Mon, 8 Feb 2021 07:51:22 +0530 > Dilip Kumar wrote: > > > On Mon, 8 Feb 2021 at 6:38 AM, Yugo NAGATA wrote: > > > > > Hi, > > > > > > On Sun, 7 Feb 2021 19:27:02 +0530 > > > Dili

Re: Is Recovery actually paused?

2021-02-07 Thread Dilip Kumar
On Mon, 8 Feb 2021 at 6:38 AM, Yugo NAGATA wrote: > Hi, > > On Sun, 7 Feb 2021 19:27:02 +0530 > Dilip Kumar wrote: > > > On Sun, Feb 7, 2021 at 6:44 PM Bharath Rupireddy > > wrote: > > > > > > On Fri, Feb 5, 2021 at 10:14 AM Bharath Ru

Re: Is Recovery actually paused?

2021-02-07 Thread Dilip Kumar
r the variable now is used within the for loop in > WaitForWALToBecomeAvailable, it's re-calculated anyways. It's being > used within case XLOG_FROM_STREAM: > > Am I missing something? Yeah, I don't see any reason for doing this, maybe it got copy pasted by

Re: [HACKERS] Custom compression methods

2021-02-05 Thread Dilip Kumar
ng place. Done > varlena.c shouldn't need toast_internals.h either, and if it did, it > should be in alphabetical order. > It was the wrong usage, fixed now. Please refer to the latest patch at https://www.postgresql.org/message-id/CAFiTN-v9Cs1MORnp-3bGZ5QBwr5v3VarSvfaDizHi1acXES5xQ%40mail.gmail.com -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] Custom compression methods

2021-02-04 Thread Dilip Kumar
On Fri, Feb 5, 2021 at 3:51 AM Robert Haas wrote: > > On Thu, Feb 4, 2021 at 11:39 AM Dilip Kumar wrote: > > Yeah, actually, I thought I would avoid calling slot_getallattrs if > > none of the attributes got decompress. I agree if we call this before > > we can av

Re: Is Recovery actually paused?

2021-02-04 Thread Dilip Kumar
On Fri, Feb 5, 2021 at 6:22 AM Bharath Rupireddy wrote: > > On Thu, Feb 4, 2021 at 7:20 PM Dilip Kumar wrote: > > > How can we do that this is not a 1 byte flag this is enum so I don't > > > think we can read any atomic state without a spin lock here. > > >

Re: Is Recovery actually paused?

2021-02-04 Thread Dilip Kumar
On Thu, Feb 4, 2021 at 10:19 PM Robert Haas wrote: > > On Thu, Feb 4, 2021 at 7:46 AM Dilip Kumar wrote: > > How can we do that this is not a 1 byte flag this is enum so I don't > > think we can read any atomic state without a spin lock here. > > I think this discussio

Re: [HACKERS] Custom compression methods

2021-02-04 Thread Dilip Kumar
we need to set this slot reference in the ScanState also otherwise that might point to the old slot. I haven't yet analyzed where all we might be keeping the reference to that old slot. Or I am missing something. Anyway, I will get a better idea once I try to implement this. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Is Recovery actually paused?

2021-02-04 Thread Dilip Kumar
On Thu, Feb 4, 2021 at 6:16 PM Dilip Kumar wrote: > > On Thu, Feb 4, 2021 at 4:58 PM Bharath Rupireddy > wrote: > > > > On Thu, Feb 4, 2021 at 10:28 AM Dilip Kumar wrote: > > > Please find an updated patch which addresses these comments. > > > > Thank

Re: Is Recovery actually paused?

2021-02-04 Thread Dilip Kumar
On Thu, Feb 4, 2021 at 4:58 PM Bharath Rupireddy wrote: > > On Thu, Feb 4, 2021 at 10:28 AM Dilip Kumar wrote: > > Please find an updated patch which addresses these comments. > > Thanks for the patch. I tested the new function pg_get_wal_replay_pause_state: >

Re: Is Recovery actually paused?

2021-02-04 Thread Dilip Kumar
On Thu, Feb 4, 2021 at 4:58 PM Bharath Rupireddy wrote: > > On Thu, Feb 4, 2021 at 10:28 AM Dilip Kumar wrote: > > Please find an updated patch which addresses these comments. > > Thanks for the patch. I tested the new function pg_get_wal_replay_pause_state: >

Re: Correct comment in StartupXLOG().

2021-02-03 Thread Dilip Kumar
On Thu, Feb 4, 2021 at 9:39 AM Amul Sul wrote: > > On Thu, Feb 4, 2021 at 6:18 AM Kyotaro Horiguchi > wrote: > > > > At Wed, 3 Feb 2021 16:36:13 +0530, Amul Sul wrote in > > > On Wed, Feb 3, 2021 at 2:48 PM Dilip Kumar wrote: > > > > > > &g

Re: Is Recovery actually paused?

2021-02-03 Thread Dilip Kumar
On Mon, Feb 1, 2021 at 1:41 PM Dilip Kumar wrote: > > On Mon, Feb 1, 2021 at 11:59 AM Kyotaro Horiguchi > wrote: > > > > At Sun, 31 Jan 2021 11:24:30 +0530, Dilip Kumar > > wrote in > > > On Fri, Jan 29, 2021 at 4:33 PM Dilip Kumar wrote: > > > &

Re: Correct comment in StartupXLOG().

2021-02-03 Thread Dilip Kumar
n't done anything about it, only replaced the "boolean flag" to > "recovery > state" in the attached patch. I don't think the atomic is correct, it's no more boolean so it is better we get rid of this part of the comment -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] Custom compression methods

2021-02-02 Thread Dilip Kumar
ere are a bunch of things to fix > here still, I don't think this is that far from being committable. I > don't at this point see too much in terms of big design problems. > Probably the CompareCompressionMethodAndDecompress() is the closest to > a design-level problem, and certainly something needs to be done about > it, but even that is a fairly localized problem in the context of the > entire patch. Thanks, Robert for the detailed review. I will work on these comments and post the updated patch. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Faulty HEAP_XMAX_LOCK_ONLY & HEAP_KEYS_UPDATED hintbit combination

2021-02-01 Thread Dilip Kumar
> UPDATE or a DELETE; as opposed to SELECT FOR NO KEY UPDATE which stands > for a future UPDATE that will only change columns that aren't part of > any keys. Yeah, that makes sense. > So I think that I misspoke earlier in this thread when I said this is a > bug, and that the right fix here is to remove the Assert() and change > amcheck to match. +1 -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Faulty HEAP_XMAX_LOCK_ONLY & HEAP_KEYS_UPDATED hintbit combination

2021-02-01 Thread Dilip Kumar
On Mon, Feb 1, 2021 at 4:05 PM Julien Rouhaud wrote: > > On Mon, Feb 1, 2021 at 2:05 PM Dilip Kumar wrote: > > > > On Sun, Jan 24, 2021 at 9:31 PM Alvaro Herrera > > wrote: > > > > > > On 2021-Jan-24, Julien Rouhaud wrote: > > > &g

Re: Printing backtrace of postgres processes

2021-02-01 Thread Dilip Kumar
d, if logging is disabled backtrace will be logged to the > console where the postmaster was started. > > > 13) Not sure, if it's an overkill, but how about pg_print_callstack > > returning a warning/notice along with true, which just says, "See > > <<>>". Thoughts? > &g

Re: Is Recovery actually paused?

2021-02-01 Thread Dilip Kumar
On Mon, Feb 1, 2021 at 11:59 AM Kyotaro Horiguchi wrote: > > At Sun, 31 Jan 2021 11:24:30 +0530, Dilip Kumar wrote > in > > On Fri, Jan 29, 2021 at 4:33 PM Dilip Kumar wrote: > > > > > > On Fri, Jan 29, 2021 at 3:25 PM Yugo NAGATA wrote: > > > &g

Re: Faulty HEAP_XMAX_LOCK_ONLY & HEAP_KEYS_UPDATED hintbit combination

2021-01-31 Thread Dilip Kumar
s_update) bits |= HEAP_XMAX_LOCK_ONLY; Basically, if it is "select for update" then we will mark infomask2 as HEAP_KEYS_UPDATED and the informask as HEAP_XMAX_LOCK_ONLY. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Is Recovery actually paused?

2021-01-30 Thread Dilip Kumar
On Fri, Jan 29, 2021 at 4:33 PM Dilip Kumar wrote: > > On Fri, Jan 29, 2021 at 3:25 PM Yugo NAGATA wrote: > > > > On Thu, 28 Jan 2021 09:55:42 +0530 > > Dilip Kumar wrote: > > > > > On Wed, Jan 27, 2021 at 2:28 PM Dilip Kumar wrote: > > > >

Re: Is Recovery actually paused?

2021-01-29 Thread Dilip Kumar
On Fri, Jan 29, 2021 at 3:25 PM Yugo NAGATA wrote: > > On Thu, 28 Jan 2021 09:55:42 +0530 > Dilip Kumar wrote: > > > On Wed, Jan 27, 2021 at 2:28 PM Dilip Kumar wrote: > > > > > > On Wed, Jan 27, 2021 at 2:06 PM Yugo NAGATA wrote: > > >

Re: [HACKERS] Custom compression methods

2021-01-28 Thread Dilip Kumar
0.857.750.857.7 > 39src/bin/psql/describe.c60.755.160.654.2 > 40contrib/cmzlib/cmzlib.cNANA74.787.5 > Thanks, Neha for testing this, overall coverage looks good to me except compress_pglz.c, compressamapi.c and cmzlib.c. I will analyze this and see if we can improve coverage for these files or not. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Is Recovery actually paused?

2021-01-27 Thread Dilip Kumar
On Wed, Jan 27, 2021 at 2:28 PM Dilip Kumar wrote: > > On Wed, Jan 27, 2021 at 2:06 PM Yugo NAGATA wrote: > > > > On Wed, 27 Jan 2021 13:29:23 +0530 > > Dilip Kumar wrote: > > > > > On Wed, Jan 27, 2021 at 12:50 PM Masahiko Sawada > > > wrote:

Re: Support ALTER SUBSCRIPTION ... ADD/DROP PUBLICATION ... syntax

2021-01-27 Thread Dilip Kumar
t; syntax user has to write three separate commands. > > > > If we want add and drop some publications, we can use SET PUBLICATION, it > is more convenient than ADD and DROP PUBLICATION, however if we just want to > add (or drop) publication into (or from) subcription whi

Re: Is Recovery actually paused?

2021-01-27 Thread Dilip Kumar
On Wed, Jan 27, 2021 at 2:06 PM Yugo NAGATA wrote: > > On Wed, 27 Jan 2021 13:29:23 +0530 > Dilip Kumar wrote: > > > On Wed, Jan 27, 2021 at 12:50 PM Masahiko Sawada > > wrote: > > > > > > On Tue, Jan 26, 2021 at 2:00 AM Robert Haas wrote: >

Re: Support ALTER SUBSCRIPTION ... ADD/DROP PUBLICATION ... syntax

2021-01-27 Thread Dilip Kumar
he above > two cases, we can use the following: > > ALTER SUBSCRIPTION mysub1 DROP PUBLICATION mypub4; > > ALTER SUBSCRIPTION mysub1 DROP PUBLICATION mypub7, mypub8; > > I think it's more convenient. Any thoughts? +1 for the idea -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Is Recovery actually paused?

2021-01-26 Thread Dilip Kumar
better > > > thoughts. > > > > Maybe we should leave the existing function pg_is_wal_replay_paused() > > alone and add a new one with the name you suggest that returns text. > > That would create less burden for tool authors. > > +1 > Yeah, we can do that, I will send an updated patch soon. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Identify missing publications from publisher while create/alter subscription.

2021-01-25 Thread Dilip Kumar
On Mon, Jan 25, 2021 at 3:38 PM Bharath Rupireddy wrote: > > On Mon, Jan 25, 2021 at 3:07 PM Dilip Kumar wrote: > > > > So basically, the create subscription will throw an error if the > > > > publication does not exist. So will you throw an error if we try to >

Re: Identify missing publications from publisher while create/alter subscription.

2021-01-25 Thread Dilip Kumar
On Mon, Jan 25, 2021 at 2:48 PM Bharath Rupireddy wrote: > > On Mon, Jan 25, 2021 at 2:42 PM Dilip Kumar wrote: > > > > On Mon, Jan 25, 2021 at 1:10 PM vignesh C wrote: > > > > > > On Thu, Jan 21, 2021 at 10:21 PM Bharath Rupireddy > > > wrote: &g

Re: Is Recovery actually paused?

2021-01-25 Thread Dilip Kumar
On Sun, Jan 24, 2021 at 12:17 PM Bharath Rupireddy wrote: > > On Sun, Jan 24, 2021 at 11:29 AM Dilip Kumar wrote: > > > Some comments on the v6 patch: > > > > [2] Typo - it's "requested" + * 'paused requested' - if pause is > > > reqested but recove

Re: Identify missing publications from publisher while create/alter subscription.

2021-01-25 Thread Dilip Kumar
publication. > Thoughts? > So basically, the create subscription will throw an error if the publication does not exist. So will you throw an error if we try to drop the publication which is subscribed by some subscription? I mean basically, you are creating a dependency that if you are creating a subscription then there must be a publication that is not completely insane but then we will have to disallow dropping the publication as well. Am I missing something? -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Is Recovery actually paused?

2021-01-24 Thread Dilip Kumar
On Mon, Jan 25, 2021 at 8:42 AM Kyotaro Horiguchi wrote: > > At Sun, 24 Jan 2021 14:26:08 +0530, Dilip Kumar wrote > in > > On Sun, Jan 24, 2021 at 12:16 PM Bharath Rupireddy > > wrote: > > > > > > On Sun, Jan 24, 2021 at 7:17 AM Dilip Kumar wrote:

Re: Is Recovery actually paused?

2021-01-24 Thread Dilip Kumar
On Mon, Jan 25, 2021 at 6:12 AM Masahiko Sawada wrote: > > On Sun, Jan 17, 2021 at 5:19 PM Dilip Kumar wrote: > > > > On Sat, Jan 16, 2021 at 8:59 AM Masahiko Sawada > > wrote: > > > > > > On Wed, Jan 13, 2021 at 9:20 PM Dilip Kumar wrote: > > &

Re: Is Recovery actually paused?

2021-01-24 Thread Dilip Kumar
On Sun, Jan 24, 2021 at 12:16 PM Bharath Rupireddy wrote: > > On Sun, Jan 24, 2021 at 7:17 AM Dilip Kumar wrote: > > On Sat, 23 Jan 2021 at 4:40 PM, Bharath Rupireddy > > wrote: > >> > >> On Sat, Jan 23, 2021 at 1:36 PM Dilip Kumar wrote: > >> >

Re: Is Recovery actually paused?

2021-01-23 Thread Dilip Kumar
On Sat, Jan 23, 2021 at 4:40 PM Bharath Rupireddy wrote: > > On Sat, Jan 23, 2021 at 1:36 PM Dilip Kumar wrote: > > Please find the patch for the same. I haven't added a test case for > > this yet. I mean we can write a test case to pause the recovery and > > get the st

Re: Is Recovery actually paused?

2021-01-23 Thread Dilip Kumar
On Sat, 23 Jan 2021 at 4:40 PM, Bharath Rupireddy < bharath.rupireddyforpostg...@gmail.com> wrote: > On Sat, Jan 23, 2021 at 1:36 PM Dilip Kumar wrote: > > Please find the patch for the same. I haven't added a test case for > > this yet. I mean we can write a test case

Re: Is Recovery actually paused?

2021-01-23 Thread Dilip Kumar
On Sat, Jan 23, 2021 at 9:56 AM Dilip Kumar wrote: > > On Fri, Jan 22, 2021 at 2:18 AM Robert Haas wrote: > > > > On Mon, Jan 18, 2021 at 9:42 PM Yugo NAGATA wrote: > > > If it is acceptable that pg_is_wal_replay_paused() makes users wait, > > > I'm ok for

Re: Is Recovery actually paused?

2021-01-22 Thread Dilip Kumar
On Thu, Jan 21, 2021 at 6:20 PM Yugo NAGATA wrote: > > On Tue, 19 Jan 2021 21:32:31 +0530 > Dilip Kumar wrote: > > > On Tue, Jan 19, 2021 at 8:34 AM Dilip Kumar wrote: > > > > > > On Tue, 19 Jan 2021 at 8:12 AM, Yugo NAGATA wrote: > > >> > &

Re: Race condition in recovery?

2021-01-22 Thread Dilip Kumar
On Fri, Jan 22, 2021 at 2:05 AM Robert Haas wrote: > > On Thu, Jan 21, 2021 at 4:00 AM Dilip Kumar wrote: > > 8. Node3, get it because walsender of Node2 read it from TL13 and send > > it and Node2 write in the new WAL file but with TL12. > &g

Re: Is Recovery actually paused?

2021-01-22 Thread Dilip Kumar
re there's actually any real problem in > practice. So to me it seems more likely that an interface that is > based on waiting will cause difficulty for tool-writers than one that > does not. > > Other people may feel differently, of course... I think this is the better way of handling this.

Re: Is Recovery actually paused?

2021-01-21 Thread Dilip Kumar
On Thu, Jan 21, 2021 at 3:29 PM Bharath Rupireddy wrote: Thanks for reviewing Bharat. > On Tue, Jan 19, 2021 at 9:32 PM Dilip Kumar wrote: > > In the last patch there were some local changes which I did not add to > > the patch and it was giving compilation warning so f

Race condition in recovery?

2021-01-21 Thread Dilip Kumar
nc: link_stat "/wal_archive/ins30wfm02dbs/000E.history" failed: No such file or directory (2) rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1179) [sender=3.1.2] rsync: link_stat "/wal_archive/ins30wfm02dbs/000C00F8" failed: No such file or directory (2) -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] Custom compression methods

2021-01-19 Thread Dilip Kumar
On Wed, Jan 20, 2021 at 12:37 AM Justin Pryzby wrote: > > Thanks for updating the patch. Thanks for the review > On Mon, Jan 4, 2021 at 6:52 AM Justin Pryzby wrote: > > The most recent patch doesn't compile --without-lz4: > On Tue, Jan 05, 2021 at 11:19:33AM +0530,

Re: Is Recovery actually paused?

2021-01-19 Thread Dilip Kumar
On Tue, Jan 19, 2021 at 8:34 AM Dilip Kumar wrote: > > On Tue, 19 Jan 2021 at 8:12 AM, Yugo NAGATA wrote: >> >> On Sun, 17 Jan 2021 11:33:52 +0530 >> Dilip Kumar wrote: >> >> > On Thu, Jan 14, 2021 at 6:49 PM Yugo NAGATA wrote: >> > > >

Re: TOAST condition for column size

2021-01-19 Thread Dilip Kumar
On Tue, 19 Jan 2021 at 6:28 PM, Amit Kapila wrote: > On Tue, Jan 19, 2021 at 5:18 PM Dilip Kumar wrote: > > > > On Mon, Jan 18, 2021 at 7:53 PM torikoshia > wrote: > > > > > > Hi, > > > > > > When I created a table consisting of 400 VARCHAR

<    5   6   7   8   9   10   11   12   13   14   >