Re: [HACKERS] Custom compression methods

2020-11-23 Thread Dilip Kumar
On Sat, Nov 21, 2020 at 3:50 AM Robert Haas wrote: > > On Wed, Nov 11, 2020 at 9:39 AM Dilip Kumar wrote: > > There were a few problems in this rebased version, basically, the > > compression options were not passed while compressing values from the > > brin_for

Re: logical streaming of xacts via test_decoding is broken

2020-11-12 Thread Dilip Kumar
On Tue, Nov 10, 2020 at 7:20 PM Amit Kapila wrote: > > On Tue, Nov 10, 2020 at 2:25 PM Dilip Kumar wrote: > > > > On Tue, Nov 10, 2020 at 11:18 AM Dilip Kumar wrote: > > > > > > On Tue, Nov 10, 2020 at 10:52 AM Amit Kapila > > > wrote: > >

Re: logical streaming of xacts via test_decoding is broken

2020-11-10 Thread Dilip Kumar
On Tue, Nov 10, 2020 at 7:20 PM Amit Kapila wrote: > > On Tue, Nov 10, 2020 at 2:25 PM Dilip Kumar wrote: > > > > On Tue, Nov 10, 2020 at 11:18 AM Dilip Kumar wrote: > > > > > > On Tue, Nov 10, 2020 at 10:52 AM Amit Kapila > > > wrote: > >

Remove unused variable from SharedSort

2020-11-12 Thread Dilip Kumar
While going through the code I noticed that the nTapes member in SharedSort is unused. This is just initialized with nworkers but never used. The attached patch removes this variable. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com v1-0001-Remove-unused-structure-member

Re: Remove unused variable from SharedSort

2020-11-15 Thread Dilip Kumar
On Sun, Nov 15, 2020 at 12:50 PM Bharath Rupireddy wrote: > > On Thu, Nov 12, 2020 at 5:29 PM Dilip Kumar wrote: > > > > While going through the code I noticed that the nTapes member in > > SharedSort is unused. This is just initialized with nworkers but > > n

Re: Re: parallel distinct union and aggregate support patch

2020-11-03 Thread Dilip Kumar
ouping on the aggregated value of the subquery, due to this we are not selecting the parallelism in the head because in the inner aggregation the number of groups is huge and if we select the parallelism we need to transfer a lot of tuple through the tuple queue and we will also have to serialize/deser

Re: Is Recovery actually paused?

2020-10-22 Thread Dilip Kumar
On Thu, Oct 22, 2020 at 7:50 PM Dilip Kumar wrote: > > On Thu, Oct 22, 2020 at 6:59 AM Kyotaro Horiguchi > wrote: > > > > At Wed, 21 Oct 2020 11:14:24 -0400, Robert Haas > > wrote in > > > On Wed, Oct 21, 2020 at 7:16 AM Dilip Kumar wrote: > > >

Re: Is Recovery actually paused?

2020-10-22 Thread Dilip Kumar
On Thu, Oct 22, 2020 at 6:59 AM Kyotaro Horiguchi wrote: > > At Wed, 21 Oct 2020 11:14:24 -0400, Robert Haas wrote > in > > On Wed, Oct 21, 2020 at 7:16 AM Dilip Kumar wrote: > > > One idea could be, if the recovery process is waiting for WAL and a > > > recove

Re: Is Recovery actually paused?

2020-10-21 Thread Dilip Kumar
On Tue, Oct 20, 2020 at 5:59 PM Dilip Kumar wrote: > > On Tue, Oct 20, 2020 at 3:00 PM Simon Riggs wrote: > > > > On Tue, 20 Oct 2020 at 09:50, Dilip Kumar wrote: > > > > > > > Why would we want this? What problem are you trying to solve? > > &g

Re: Parallel Inserts in CREATE TABLE AS

2020-12-29 Thread Dilip Kumar
On Mon, Dec 28, 2020 at 10:45 AM Dilip Kumar wrote: > > On Sun, Dec 27, 2020 at 2:20 PM Bharath Rupireddy > wrote: > > > > On Sat, Dec 26, 2020 at 11:11 AM Dilip Kumar wrote: > > > I have reviewed part of v15-0001 patch, I have a few comments, I

Re: Parallel Inserts in CREATE TABLE AS

2020-12-29 Thread Dilip Kumar
On Wed, 30 Dec 2020 at 10:47 AM, Bharath Rupireddy < bharath.rupireddyforpostg...@gmail.com> wrote: > On Wed, Dec 30, 2020 at 10:32 AM Dilip Kumar > wrote: > > I have completed reviewing 0001, I don't have more comments, just one > > question. Soon I will revi

Re: Parallel Inserts in CREATE TABLE AS

2020-12-30 Thread Dilip Kumar
On Wed, Dec 30, 2020 at 10:49 AM Dilip Kumar wrote: > > On Wed, 30 Dec 2020 at 10:47 AM, Bharath Rupireddy > wrote: >> >> On Wed, Dec 30, 2020 at 10:32 AM Dilip Kumar wrote: >> > I have completed reviewing 0001, I don't have more comments, just one >

Re: Parallel Inserts in CREATE TABLE AS

2020-12-30 Thread Dilip Kumar
arallelInsertState to > +void > +SetParallelInsertState(QueryDesc *queryDesc) > > Change IsParallelInsertionAllowedInCTAS to > > +bool > +IsParallelInsertionAllowed(ParallelInsertKind pinskind, IntoClause *into) > +{ > > Thoughts? > I haven’t thought about these structures yet but yeah making them generic will be good. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Parallel Inserts in CREATE TABLE AS

2021-01-05 Thread Dilip Kumar
dd_u64_impl, > > pg_atomic_compare_exchange_u64_impl, pg_atomic_init_u64_impl and their > > u32 counterparts use pass the parameter as volatile pg_atomic_uint64 *ptr. But in your case, I do not understand the intention that where do you think that the compiler can optimize it and read the old value? -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Added schema level support for publication.

2021-01-10 Thread Dilip Kumar
> CREATE PUBLICATION production_publication FOR ALL TABLES SCHEMA > > production WITH skip = marketing, accounts, sales; --> meaning is, > > replicate all the tables in the schema production except marketing, > > accounts, sales tables. > > One more point

Re: [HACKERS] Custom compression methods

2021-01-10 Thread Dilip Kumar
On Mon, Jan 11, 2021 at 11:00 AM Dilip Kumar wrote: > > On Sun, Jan 10, 2021 at 10:59 PM Justin Pryzby wrote: > > > > On Mon, Jan 04, 2021 at 04:57:16PM +0530, Dilip Kumar wrote: > > > On Mon, Jan 4, 2021 at 6:52 AM Justin Pryzby wrote: > > > > And fa

Re: [HACKERS] Custom compression methods

2021-01-10 Thread Dilip Kumar
On Sun, Jan 10, 2021 at 10:59 PM Justin Pryzby wrote: > > On Mon, Jan 04, 2021 at 04:57:16PM +0530, Dilip Kumar wrote: > > On Mon, Jan 4, 2021 at 6:52 AM Justin Pryzby wrote: > > > And fails pg_upgrade check, apparently losing track of the compression (?) > > > >

Re: [HACKERS] Custom compression methods

2021-01-11 Thread Dilip Kumar
On Mon, Jan 11, 2021 at 12:21 PM Justin Pryzby wrote: > > On Mon, Jan 11, 2021 at 12:11:54PM +0530, Dilip Kumar wrote: > > On Mon, Jan 11, 2021 at 11:00 AM Dilip Kumar wrote: > > > On Sun, Jan 10, 2021 at 10:59 PM Justin Pryzby > > > wrote: > > > >

Re: TOAST condition for column size

2021-01-19 Thread Dilip Kumar
On Tue, 19 Jan 2021 at 6:28 PM, Amit Kapila wrote: > On Tue, Jan 19, 2021 at 5:18 PM Dilip Kumar wrote: > > > > On Mon, Jan 18, 2021 at 7:53 PM torikoshia > wrote: > > > > > > Hi, > > > > > > When I created a table consisting of 400 VARCHAR

Re: Is Recovery actually paused?

2021-01-19 Thread Dilip Kumar
On Tue, Jan 19, 2021 at 8:34 AM Dilip Kumar wrote: > > On Tue, 19 Jan 2021 at 8:12 AM, Yugo NAGATA wrote: >> >> On Sun, 17 Jan 2021 11:33:52 +0530 >> Dilip Kumar wrote: >> >> > On Thu, Jan 14, 2021 at 6:49 PM Yugo NAGATA wrote: >> > > >

Race condition in recovery?

2021-01-21 Thread Dilip Kumar
nc: link_stat "/wal_archive/ins30wfm02dbs/000E.history" failed: No such file or directory (2) rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1179) [sender=3.1.2] rsync: link_stat "/wal_archive/ins30wfm02dbs/000C00F8" failed: No such file or directory (2) -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Is Recovery actually paused?

2021-01-21 Thread Dilip Kumar
On Thu, Jan 21, 2021 at 3:29 PM Bharath Rupireddy wrote: Thanks for reviewing Bharat. > On Tue, Jan 19, 2021 at 9:32 PM Dilip Kumar wrote: > > In the last patch there were some local changes which I did not add to > > the patch and it was giving compilation warning so f

Re: Is Recovery actually paused?

2021-01-16 Thread Dilip Kumar
On Thu, Jan 14, 2021 at 6:49 PM Yugo NAGATA wrote: > > On Wed, 13 Jan 2021 17:49:43 +0530 > Dilip Kumar wrote: > > > On Wed, Jan 13, 2021 at 3:35 PM Dilip Kumar wrote: > > > > > > On Wed, Jan 13, 2021 at 3:27 PM Yugo NAGATA wrote: > > >

Re: Is Recovery actually paused?

2021-01-17 Thread Dilip Kumar
On Sat, Jan 16, 2021 at 8:59 AM Masahiko Sawada wrote: > > On Wed, Jan 13, 2021 at 9:20 PM Dilip Kumar wrote: > > > > On Wed, Jan 13, 2021 at 3:35 PM Dilip Kumar wrote: > > > > > > On Wed, Jan 13, 2021 at 3:27 PM Yugo NAGATA wrote: > > >

Re: Is Recovery actually paused?

2021-01-16 Thread Dilip Kumar
good. I > > think we want the recovery to be paused but want the wal receiver to > > continue receiving WAL. > > I had misunderstood the code and the patch, please ignore this > comment. Pausing the recovery here is fine with me. Thanks for the review Sawada-San,

Re: Is Recovery actually paused?

2021-01-17 Thread Dilip Kumar
On Sun, Jan 17, 2021 at 1:48 PM Dilip Kumar wrote: > > On Sat, Jan 16, 2021 at 8:59 AM Masahiko Sawada wrote: > > > > On Wed, Jan 13, 2021 at 9:20 PM Dilip Kumar wrote: > > > > > > On Wed, Jan 13, 2021 at 3:35 PM Dilip Kumar wrote: > > > > >

Re: Is Recovery actually paused?

2021-01-22 Thread Dilip Kumar
re there's actually any real problem in > practice. So to me it seems more likely that an interface that is > based on waiting will cause difficulty for tool-writers than one that > does not. > > Other people may feel differently, of course... I think this is the better way of handling this.

Re: Race condition in recovery?

2021-01-22 Thread Dilip Kumar
On Fri, Jan 22, 2021 at 2:05 AM Robert Haas wrote: > > On Thu, Jan 21, 2021 at 4:00 AM Dilip Kumar wrote: > > 8. Node3, get it because walsender of Node2 read it from TL13 and send > > it and Node2 write in the new WAL file but with TL12. > &g

Re: Is Recovery actually paused?

2021-01-22 Thread Dilip Kumar
On Thu, Jan 21, 2021 at 6:20 PM Yugo NAGATA wrote: > > On Tue, 19 Jan 2021 21:32:31 +0530 > Dilip Kumar wrote: > > > On Tue, Jan 19, 2021 at 8:34 AM Dilip Kumar wrote: > > > > > > On Tue, 19 Jan 2021 at 8:12 AM, Yugo NAGATA wrote: > > >> > &

Re: Is Recovery actually paused?

2021-01-23 Thread Dilip Kumar
On Sat, Jan 23, 2021 at 9:56 AM Dilip Kumar wrote: > > On Fri, Jan 22, 2021 at 2:18 AM Robert Haas wrote: > > > > On Mon, Jan 18, 2021 at 9:42 PM Yugo NAGATA wrote: > > > If it is acceptable that pg_is_wal_replay_paused() makes users wait, > > > I'm ok for

Re: [HACKERS] Custom compression methods

2021-01-19 Thread Dilip Kumar
On Wed, Jan 20, 2021 at 12:37 AM Justin Pryzby wrote: > > Thanks for updating the patch. Thanks for the review > On Mon, Jan 4, 2021 at 6:52 AM Justin Pryzby wrote: > > The most recent patch doesn't compile --without-lz4: > On Tue, Jan 05, 2021 at 11:19:33AM +0530,

Re: Is Recovery actually paused?

2021-01-18 Thread Dilip Kumar
On Tue, 19 Jan 2021 at 8:12 AM, Yugo NAGATA wrote: > On Sun, 17 Jan 2021 11:33:52 +0530 > Dilip Kumar wrote: > > > On Thu, Jan 14, 2021 at 6:49 PM Yugo NAGATA wrote: > > > > > > On Wed, 13 Jan 2021 17:49:43 +0530 > > > Dilip Kumar wrote: > >

Re: TOAST condition for column size

2021-01-19 Thread Dilip Kumar
tive for small data and it's > not recommended to create a table containing hundreds of columns, > but I think cases that can be successful should be successful. > > Any thoughts? How this can be correct? because while forming the tuple you might need the alignment.

Re: Is Recovery actually paused?

2021-01-23 Thread Dilip Kumar
On Sat, 23 Jan 2021 at 4:40 PM, Bharath Rupireddy < bharath.rupireddyforpostg...@gmail.com> wrote: > On Sat, Jan 23, 2021 at 1:36 PM Dilip Kumar wrote: > > Please find the patch for the same. I haven't added a test case for > > this yet. I mean we can write a test case

Re: Is Recovery actually paused?

2021-01-23 Thread Dilip Kumar
On Sat, Jan 23, 2021 at 4:40 PM Bharath Rupireddy wrote: > > On Sat, Jan 23, 2021 at 1:36 PM Dilip Kumar wrote: > > Please find the patch for the same. I haven't added a test case for > > this yet. I mean we can write a test case to pause the recovery and > > get the st

Re: Is Recovery actually paused?

2021-01-24 Thread Dilip Kumar
On Sun, Jan 24, 2021 at 12:16 PM Bharath Rupireddy wrote: > > On Sun, Jan 24, 2021 at 7:17 AM Dilip Kumar wrote: > > On Sat, 23 Jan 2021 at 4:40 PM, Bharath Rupireddy > > wrote: > >> > >> On Sat, Jan 23, 2021 at 1:36 PM Dilip Kumar wrote: > >> >

Re: Is Recovery actually paused?

2021-01-24 Thread Dilip Kumar
On Mon, Jan 25, 2021 at 6:12 AM Masahiko Sawada wrote: > > On Sun, Jan 17, 2021 at 5:19 PM Dilip Kumar wrote: > > > > On Sat, Jan 16, 2021 at 8:59 AM Masahiko Sawada > > wrote: > > > > > > On Wed, Jan 13, 2021 at 9:20 PM Dilip Kumar wrote: > > &

Re: Is Recovery actually paused?

2021-01-24 Thread Dilip Kumar
On Mon, Jan 25, 2021 at 8:42 AM Kyotaro Horiguchi wrote: > > At Sun, 24 Jan 2021 14:26:08 +0530, Dilip Kumar wrote > in > > On Sun, Jan 24, 2021 at 12:16 PM Bharath Rupireddy > > wrote: > > > > > > On Sun, Jan 24, 2021 at 7:17 AM Dilip Kumar wrote:

Re: Identify missing publications from publisher while create/alter subscription.

2021-01-25 Thread Dilip Kumar
On Mon, Jan 25, 2021 at 2:48 PM Bharath Rupireddy wrote: > > On Mon, Jan 25, 2021 at 2:42 PM Dilip Kumar wrote: > > > > On Mon, Jan 25, 2021 at 1:10 PM vignesh C wrote: > > > > > > On Thu, Jan 21, 2021 at 10:21 PM Bharath Rupireddy > > > wrote: &g

Re: Identify missing publications from publisher while create/alter subscription.

2021-01-25 Thread Dilip Kumar
publication. > Thoughts? > So basically, the create subscription will throw an error if the publication does not exist. So will you throw an error if we try to drop the publication which is subscribed by some subscription? I mean basically, you are creating a dependency that if you are creating a subscription then there must be a publication that is not completely insane but then we will have to disallow dropping the publication as well. Am I missing something? -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Is Recovery actually paused?

2021-01-25 Thread Dilip Kumar
On Sun, Jan 24, 2021 at 12:17 PM Bharath Rupireddy wrote: > > On Sun, Jan 24, 2021 at 11:29 AM Dilip Kumar wrote: > > > Some comments on the v6 patch: > > > > [2] Typo - it's "requested" + * 'paused requested' - if pause is > > > reqested but recove

Re: Identify missing publications from publisher while create/alter subscription.

2021-01-25 Thread Dilip Kumar
On Mon, Jan 25, 2021 at 3:38 PM Bharath Rupireddy wrote: > > On Mon, Jan 25, 2021 at 3:07 PM Dilip Kumar wrote: > > > > So basically, the create subscription will throw an error if the > > > > publication does not exist. So will you throw an error if we try to >

Re: Is Recovery actually paused?

2021-01-18 Thread Dilip Kumar
On Tue, Jan 19, 2021 at 10:30 AM Kyotaro Horiguchi wrote: > > At Tue, 19 Jan 2021 11:41:18 +0900, Yugo NAGATA wrote in > > On Sun, 17 Jan 2021 11:33:52 +0530 > > Dilip Kumar wrote: > > > > > > > > > > > > > > I thi

Re: [patch] Help information for pg_dump

2021-01-19 Thread Dilip Kumar
are mentioned in capital letters in both > pg_dump and pg_dumpall cases, so changing "do NOT" to "do not" seems > okay to me. > > Patch LGTM. Also "do NOT" is inconsistent with the other message where we are saying "do not" so +1 -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Logical Replication - behavior of ALTER PUBLICATION .. DROP TABLE and ALTER SUBSCRIPTION .. REFRESH PUBLICATION

2021-01-13 Thread Dilip Kumar
On Wed, Jan 13, 2021 at 3:08 PM Bharath Rupireddy wrote: > > On Wed, Jan 13, 2021 at 2:53 PM Dilip Kumar wrote: > > > IIUC the logical replication only replicate the tables in publication, I > > > think > > > when the tables that aren't in pub

Re: Logical Replication - behavior of ALTER PUBLICATION .. DROP TABLE and ALTER SUBSCRIPTION .. REFRESH PUBLICATION

2021-01-13 Thread Dilip Kumar
On Wed, Jan 13, 2021 at 3:06 PM japin wrote: > > > On Wed, 13 Jan 2021 at 17:23, Dilip Kumar wrote: > > On Tue, Jan 12, 2021 at 4:47 PM Li Japin wrote: > >> > >> > >> On Jan 12, 2021, at 5:47 PM, japin wrote: > >> > >> > >> O

Re: Logical Replication - behavior of ALTER PUBLICATION .. DROP TABLE and ALTER SUBSCRIPTION .. REFRESH PUBLICATION

2021-01-13 Thread Dilip Kumar
es that aren't in publication should not be replicated. > > Attached the patch that fixes it. Thought? > Instead of doing this, I would expect that the RelationSyncCache entry should be removed when the relation is dropped from the publication. So if that is done then it will reload the publication and then it will not find that that relation as published and it will ignore the changes. But the patch doesn't seem to be exactly on that line. Am I missing something here? -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Logical Replication - behavior of ALTER PUBLICATION .. DROP TABLE and ALTER SUBSCRIPTION .. REFRESH PUBLICATION

2021-01-13 Thread Dilip Kumar
On Wed, Jan 13, 2021 at 3:08 PM Bharath Rupireddy wrote: > > On Wed, Jan 13, 2021 at 2:53 PM Dilip Kumar wrote: > > > IIUC the logical replication only replicate the tables in publication, I > > > think > > > when the tables that aren't in pub

Re: Is Recovery actually paused?

2021-01-13 Thread Dilip Kumar
On Wed, Jan 13, 2021 at 3:35 PM Dilip Kumar wrote: > > On Wed, Jan 13, 2021 at 3:27 PM Yugo NAGATA wrote: > > > > On Thu, 10 Dec 2020 11:25:23 +0530 > > Dilip Kumar wrote: > > > > > > > However, I wonder users don't expect pg_is_wa

Re: Is Recovery actually paused?

2021-01-13 Thread Dilip Kumar
On Wed, Jan 13, 2021 at 3:27 PM Yugo NAGATA wrote: > > On Thu, 10 Dec 2020 11:25:23 +0530 > Dilip Kumar wrote: > > > > > However, I wonder users don't expect pg_is_wal_replay_paused to wait. > > > > Especially, if max_standby_streaming_delay is -1, t

Re: [HACKERS] Custom compression methods

2021-01-03 Thread Dilip Kumar
palloc(sizeof(int32)); > + > + /* initialize with the default acceleration */ > + *acceleration = 1; > + > + if (list_length(options) > 0) > + { > + ListCell*lc; > + > + foreach(lc, options) > + { > + DefElem*def = (DefElem *) lfirst(lc); > + > + if (strcmp(def->defname, "acceleration") == 0) > + *acceleration = pg_atoi(defGetString(def), > sizeof(int32), 0); > > Don't you need to say "else: error: unknown compression option" ? > > + /* > +* Compression option must be only valid if we are updating the > compression > +* method. > +*/ > + Assert(DatumGetPointer(acoptions) == NULL || > OidIsValid(newcompression)); > + > > should say "need be valid only if .." > Thanks for the review, I will work on these and respond along with the updated patches. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Parallel Inserts in CREATE TABLE AS

2021-01-05 Thread Dilip Kumar
ome code that is generic for CTAS and INSERT INTO SELECT *, So is it possible to take out that common code to a separate base patch? Later both CTAS and INSERT INTO SELECT * can expand that for their usage. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Parallel Inserts in CREATE TABLE AS

2021-01-05 Thread Dilip Kumar
On Wed, Jan 6, 2021 at 11:26 AM Bharath Rupireddy wrote: > > On Wed, Jan 6, 2021 at 10:17 AM Dilip Kumar wrote: > > > > On Wed, Jan 6, 2021 at 9:23 AM Bharath Rupireddy > > wrote: > > > > > > > +/* > > + * List the commands here for which para

Re: Faulty HEAP_XMAX_LOCK_ONLY & HEAP_KEYS_UPDATED hintbit combination

2021-01-31 Thread Dilip Kumar
s_update) bits |= HEAP_XMAX_LOCK_ONLY; Basically, if it is "select for update" then we will mark infomask2 as HEAP_KEYS_UPDATED and the informask as HEAP_XMAX_LOCK_ONLY. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Is Recovery actually paused?

2021-02-01 Thread Dilip Kumar
On Mon, Feb 1, 2021 at 11:59 AM Kyotaro Horiguchi wrote: > > At Sun, 31 Jan 2021 11:24:30 +0530, Dilip Kumar wrote > in > > On Fri, Jan 29, 2021 at 4:33 PM Dilip Kumar wrote: > > > > > > On Fri, Jan 29, 2021 at 3:25 PM Yugo NAGATA wrote: > > > &g

Re: Faulty HEAP_XMAX_LOCK_ONLY & HEAP_KEYS_UPDATED hintbit combination

2021-02-01 Thread Dilip Kumar
On Mon, Feb 1, 2021 at 4:05 PM Julien Rouhaud wrote: > > On Mon, Feb 1, 2021 at 2:05 PM Dilip Kumar wrote: > > > > On Sun, Jan 24, 2021 at 9:31 PM Alvaro Herrera > > wrote: > > > > > > On 2021-Jan-24, Julien Rouhaud wrote: > > > &g

Re: Printing backtrace of postgres processes

2021-02-01 Thread Dilip Kumar
d, if logging is disabled backtrace will be logged to the > console where the postmaster was started. > > > 13) Not sure, if it's an overkill, but how about pg_print_callstack > > returning a warning/notice along with true, which just says, "See > > <<>>". Thoughts? > &g

Re: Faulty HEAP_XMAX_LOCK_ONLY & HEAP_KEYS_UPDATED hintbit combination

2021-02-01 Thread Dilip Kumar
> UPDATE or a DELETE; as opposed to SELECT FOR NO KEY UPDATE which stands > for a future UPDATE that will only change columns that aren't part of > any keys. Yeah, that makes sense. > So I think that I misspoke earlier in this thread when I said this is a > bug, and that the right fix here is to remove the Assert() and change > amcheck to match. +1 -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Is Recovery actually paused?

2021-02-03 Thread Dilip Kumar
On Mon, Feb 1, 2021 at 1:41 PM Dilip Kumar wrote: > > On Mon, Feb 1, 2021 at 11:59 AM Kyotaro Horiguchi > wrote: > > > > At Sun, 31 Jan 2021 11:24:30 +0530, Dilip Kumar > > wrote in > > > On Fri, Jan 29, 2021 at 4:33 PM Dilip Kumar wrote: > > > &

Re: Correct comment in StartupXLOG().

2021-02-03 Thread Dilip Kumar
On Thu, Feb 4, 2021 at 9:39 AM Amul Sul wrote: > > On Thu, Feb 4, 2021 at 6:18 AM Kyotaro Horiguchi > wrote: > > > > At Wed, 3 Feb 2021 16:36:13 +0530, Amul Sul wrote in > > > On Wed, Feb 3, 2021 at 2:48 PM Dilip Kumar wrote: > > > > > > &g

Re: Is Recovery actually paused?

2021-02-04 Thread Dilip Kumar
On Thu, Feb 4, 2021 at 4:58 PM Bharath Rupireddy wrote: > > On Thu, Feb 4, 2021 at 10:28 AM Dilip Kumar wrote: > > Please find an updated patch which addresses these comments. > > Thanks for the patch. I tested the new function pg_get_wal_replay_pause_state: >

Re: Is Recovery actually paused?

2021-02-04 Thread Dilip Kumar
On Thu, Feb 4, 2021 at 4:58 PM Bharath Rupireddy wrote: > > On Thu, Feb 4, 2021 at 10:28 AM Dilip Kumar wrote: > > Please find an updated patch which addresses these comments. > > Thanks for the patch. I tested the new function pg_get_wal_replay_pause_state: >

Re: Is Recovery actually paused?

2021-02-04 Thread Dilip Kumar
On Thu, Feb 4, 2021 at 6:16 PM Dilip Kumar wrote: > > On Thu, Feb 4, 2021 at 4:58 PM Bharath Rupireddy > wrote: > > > > On Thu, Feb 4, 2021 at 10:28 AM Dilip Kumar wrote: > > > Please find an updated patch which addresses these comments. > > > > Thank

Re: [HACKERS] Custom compression methods

2021-02-04 Thread Dilip Kumar
we need to set this slot reference in the ScanState also otherwise that might point to the old slot. I haven't yet analyzed where all we might be keeping the reference to that old slot. Or I am missing something. Anyway, I will get a better idea once I try to implement this. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Is Recovery actually paused?

2021-01-27 Thread Dilip Kumar
On Wed, Jan 27, 2021 at 2:28 PM Dilip Kumar wrote: > > On Wed, Jan 27, 2021 at 2:06 PM Yugo NAGATA wrote: > > > > On Wed, 27 Jan 2021 13:29:23 +0530 > > Dilip Kumar wrote: > > > > > On Wed, Jan 27, 2021 at 12:50 PM Masahiko Sawada > > > wrote:

Re: [HACKERS] Custom compression methods

2021-01-28 Thread Dilip Kumar
0.857.750.857.7 > 39src/bin/psql/describe.c60.755.160.654.2 > 40contrib/cmzlib/cmzlib.cNANA74.787.5 > Thanks, Neha for testing this, overall coverage looks good to me except compress_pglz.c, compressamapi.c and cmzlib.c. I will analyze this and see if we can improve coverage for these files or not. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Is Recovery actually paused?

2021-01-30 Thread Dilip Kumar
On Fri, Jan 29, 2021 at 4:33 PM Dilip Kumar wrote: > > On Fri, Jan 29, 2021 at 3:25 PM Yugo NAGATA wrote: > > > > On Thu, 28 Jan 2021 09:55:42 +0530 > > Dilip Kumar wrote: > > > > > On Wed, Jan 27, 2021 at 2:28 PM Dilip Kumar wrote: > > > >

Re: Is Recovery actually paused?

2021-01-29 Thread Dilip Kumar
On Fri, Jan 29, 2021 at 3:25 PM Yugo NAGATA wrote: > > On Thu, 28 Jan 2021 09:55:42 +0530 > Dilip Kumar wrote: > > > On Wed, Jan 27, 2021 at 2:28 PM Dilip Kumar wrote: > > > > > > On Wed, Jan 27, 2021 at 2:06 PM Yugo NAGATA wrote: > > >

Re: [HACKERS] Custom compression methods

2021-02-02 Thread Dilip Kumar
ere are a bunch of things to fix > here still, I don't think this is that far from being committable. I > don't at this point see too much in terms of big design problems. > Probably the CompareCompressionMethodAndDecompress() is the closest to > a design-level problem, and certainly something needs to be done about > it, but even that is a fairly localized problem in the context of the > entire patch. Thanks, Robert for the detailed review. I will work on these comments and post the updated patch. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Correct comment in StartupXLOG().

2021-02-03 Thread Dilip Kumar
n't done anything about it, only replaced the "boolean flag" to > "recovery > state" in the attached patch. I don't think the atomic is correct, it's no more boolean so it is better we get rid of this part of the comment -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2021-06-16 Thread Dilip Kumar
d to provide control to the user by providing two different commands, e.g. COPY DATABASE for the existing method (copydir) and CREATE DATABASE for the new method (fully wal logged)? -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Decoding speculative insert with toast leaks memory

2021-06-07 Thread Dilip Kumar
On Mon, Jun 7, 2021 at 6:34 PM Amit Kapila wrote: > On Mon, Jun 7, 2021 at 6:04 PM Dilip Kumar wrote: > > > > I have fixed all pending review comments and also added a test case > which is working fine. > > > > Few observations and questions on testcase: >

Re: Race condition in recovery?

2021-06-09 Thread Dilip Kumar
On Wed, Jun 9, 2021 at 12:14 PM Dilip Kumar wrote: > > On Wed, Jun 9, 2021 at 2:07 AM Robert Haas wrote: > 2021-06-09 12:11:08.618 IST [122456] LOG: entering standby mode > 2021-06-09 12:11:08.622 IST [122456] LOG: restored log file > "0002.history" from

Re: Race condition in recovery?

2021-06-09 Thread Dilip Kumar
On Wed, Jun 9, 2021 at 1:37 PM Dilip Kumar wrote: > > On Wed, Jun 9, 2021 at 12:14 PM Dilip Kumar wrote: > > > > On Wed, Jun 9, 2021 at 2:07 AM Robert Haas wrote: > > 2021-06-09 12:11:08.618 IST [122456] LOG: entering standby mode > > 2021-06-09 12:11:08.622 I

Re: Race condition in recovery?

2021-06-09 Thread Dilip Kumar
09 12:11:08.627 IST [122456] LOG: redo starts at 0/228 2021-06-09 12:11:08.627 IST [122456] LOG: consistent recovery state reached at 0/300 Next, I will investigate, without a fix on v11 (maybe v12, v10..) why it is not hitting the defect location at all. And after that, I will check t

Re: Decoding speculative insert with toast leaks memory

2021-06-09 Thread Dilip Kumar
On Wed, Jun 9, 2021 at 11:03 AM Amit Kapila wrote: > On Tue, Jun 8, 2021 at 5:16 PM Dilip Kumar wrote: > > > > Based on the off list discussion, I have modified the test based on > > the idea showed in > > "isolation/specs/insert-conflict-specconflict.spec&quo

Re: Decoding speculative insert with toast leaks memory

2021-06-07 Thread Dilip Kumar
On Mon, Jun 7, 2021 at 8:46 AM Dilip Kumar wrote: > On Mon, 7 Jun 2021 at 8:30 AM, Amit Kapila > wrote: > >> On Wed, Jun 2, 2021 at 11:52 AM Amit Kapila >> wrote: >> > >> > On Wed, Jun 2, 2021 at 11:38 AM Dilip Kumar >> wrote: >> >

Re: Race condition in recovery?

2021-06-07 Thread Dilip Kumar
. $standby->poll_query_until('postgres', "SELECT pg_is_in_recovery() = 'f';") or die "Timed out while waiting for promotion"; I will try to generate a version for 9.6 based on this idea and see how it goes -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Decoding speculative insert with toast leaks memory

2021-06-09 Thread Dilip Kumar
On Wed, Jun 9, 2021 at 4:22 PM Amit Kapila wrote: > On Wed, Jun 9, 2021 at 4:12 PM Dilip Kumar wrote: > >> Few comments: > >> 1. The test has a lot of similarities and test duplication with what > >> we are doing in insert-conflict-specconflict.spec. Can we m

Re: Race condition in recovery?

2021-06-08 Thread Dilip Kumar
On Tue, Jun 8, 2021 at 11:13 AM Dilip Kumar wrote: > > # Wait until the node exits recovery. > $standby->poll_query_until('postgres', "SELECT pg_is_in_recovery() = 'f';") > or die "Timed out while waiting for promotion"; > > I will try to generate a versi

Re: Decoding speculative insert with toast leaks memory

2021-06-10 Thread Dilip Kumar
her comments and also prepared patches for the back branches. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com From dcea4c36267ad2dc58dd0a57733a6f6276e2d754 Mon Sep 17 00:00:00 2001 From: Dilip Kumar Date: Tue, 8 Jun 2021 17:06:39 +0530 Subject: [PATCH v5 1/2] Bug fix for specu

Re: Decoding speculative insert with toast leaks memory

2021-06-13 Thread Dilip Kumar
hat we have used base for our test so let's not push this test until it becomes stable. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2021-06-17 Thread Dilip Kumar
On Thu, Jun 17, 2021 at 2:50 PM Heikki Linnakangas wrote: > > On 17/06/2021 08:45, Dilip Kumar wrote: > > Another problem with directly scanning the directory is, how we > > are supposed to get the "relpersistence" which is stored in pg_class > > tuple right

Re: Toast compression method options

2021-06-21 Thread Dilip Kumar
On Fri, Jun 18, 2021 at 12:13 PM Michael Paquier wrote: > > On Thu, May 06, 2021 at 07:23:48PM +0530, Dilip Kumar wrote: > > I have fixed some comments offlist reported by Justin. Apart from > > that, I have also improved documentation and test case. Stil it has a > > l

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2021-06-15 Thread Dilip Kumar
On Tue, Jun 15, 2021 at 5:34 PM Heikki Linnakangas wrote: > > On 15/06/2021 14:20, Dilip Kumar wrote: > > Design Idea: . Then > > we can get the relfilenode of every file we need to copy, and prepare > > a list of all such relfilenode. > > I guess that would

[Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2021-06-15 Thread Dilip Kumar
refactoring/cleanup and testing. - Might need to relook into the SMGR level API usage. Credits: --- Thanks to Robert Haas, for suggesting this idea and the high-level design. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com From e472d3cb744dc45641d36e919098f9570f80a8fd Mon

Re: Decoding speculative insert with toast leaks memory

2021-06-08 Thread Dilip Kumar
On Mon, Jun 7, 2021 at 6:45 PM Dilip Kumar wrote: >> >> >> 2. In the test, there seems to be an assumption that we can unlock s2 >> and s3 one after another, and then both will start waiting on s-1 but >> isn't it possible that before s2 start waiting on s1,

Re: Decoding speculative insert with toast leaks memory

2021-06-14 Thread Dilip Kumar
On Mon, Jun 14, 2021 at 9:44 AM Dilip Kumar wrote: > > On Mon, Jun 14, 2021 at 8:34 AM Amit Kapila wrote: > > > > I think the test in this patch is quite similar to what Noah has > > pointed in the nearby thread [1] to be failing at some intervals. Can > > you al

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2021-06-16 Thread Dilip Kumar
()log_newpage()On Thu, Jun 17, 2021 at 3:28 AM Andres Freund wrote: > > Hi, > > On 2021-06-15 16:50:24 +0530, Dilip Kumar wrote: > > The patch modifies both CREATE DATABASE and ALTER DATABASE..SET > > TABLESPACE to be fully WAL-logged. > > Generally quite a bit

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2021-06-17 Thread Dilip Kumar
> no? Right now cloning a 1TB database costs a few hundred bytes of WAL and > about > 1TB of write IO. With the proposed approach, the write volume approximately > doubles, because there'll also be about 1TB in WAL. Make sense. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Decoding speculative insert with toast leaks memory

2021-06-17 Thread Dilip Kumar
replicate_valid) + if (!found) + { + /* + * Make the new entry valid enough for the callbacks to look at, in + * case any of them get invoked during the more complicated + * initialization steps below. + */ On head: if (!found) { /* immediately make a new entry valid enough to satisfy callbacks */ -- Regar

Re: Race condition in recovery?

2021-06-11 Thread Dilip Kumar
recovery/tmp_check/t_002_archiving_primary_data/archives\\0003.history" 2021-06-10 22:44:36.113 EDT [60c2ce0c.283c:5] LOG: received immediate shutdown request 2021-06-10 22:44:36.129 EDT [60c2ce0c.283c:6] LOG: database system is shut down I am not able to figure out why the archive command is failing. -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: RFC: Logging plan of the running query

2021-05-13 Thread Dilip Kumar
On Thu, May 13, 2021 at 5:18 PM Dilip Kumar wrote: > > On Thu, May 13, 2021 at 5:15 PM Bharath Rupireddy > wrote: > > > > On Thu, May 13, 2021 at 5:14 PM Dilip Kumar wrote: > > > > > > On Thu, May 13, 2021 at 4:16 PM Bharath Rupireddy > > > wro

Re: RFC: Logging plan of the running query

2021-05-13 Thread Dilip Kumar
On Thu, May 13, 2021 at 5:15 PM Bharath Rupireddy wrote: > > On Thu, May 13, 2021 at 5:14 PM Dilip Kumar wrote: > > > > On Thu, May 13, 2021 at 4:16 PM Bharath Rupireddy > > wrote: > > > > > > I'm saying that - currently, queries are logged with

Re: [Patch] ALTER SYSTEM READ ONLY

2021-05-11 Thread Dilip Kumar
e "if checks" whether the XLogAcceptWrites() is called or not, instead we can just rely on the state, if it is #4 then we have to call XLogAcceptWrites(). If so then I think it's okay to have an additional state, just wanted to know what idea you had in mind? -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: OOM in spgist insert

2021-05-12 Thread Dilip Kumar
On Wed, 12 May 2021 at 1:43 PM, Pavel Borisov wrote: > ср, 12 мая 2021 г. в 11:09, Dilip Kumar : > >> While testing something on spgist I found that at certain point while >> inserting in spgist it is going for doPickSplit, but even after split >> is is not able to find a

Re: Race condition in recovery?

2021-05-11 Thread Dilip Kumar
On Tue, May 11, 2021 at 1:42 PM Kyotaro Horiguchi wrote: > > At Mon, 10 May 2021 14:27:21 +0530, Dilip Kumar wrote > in > > On Mon, May 10, 2021 at 2:05 PM Kyotaro Horiguchi > > wrote: > > > > > I thought that the reason using receiveTLI instead of > >

Re: Remove "FROM" in "DELETE FROM" when using tab-completion

2021-05-11 Thread Dilip Kumar
DELETE ... USING" > is not as good as it is now.(I guess that's why the comment was added). And > for now, IMHO, we can remove the comment directly. But your patch is doing nothing to add the implementation for DELETE.. USING. Basically, the tab completion support for DELETEUSING is still pending right? -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

Re: Remove "FROM" in "DELETE FROM" when using tab-completion

2021-05-11 Thread Dilip Kumar
On Tue, May 11, 2021 at 3:03 PM tanghy.f...@fujitsu.com wrote: > > On Tuesday, May 11, 2021 5:44 PM, Dilip Kumar wrote: > >But your patch is doing nothing to add the implementation for DELETE.. > >USING. Basically, the tab completion support for DELETEUSING is > >st

Re: [Patch] ALTER SYSTEM READ ONLY

2021-05-11 Thread Dilip Kumar
y state is changed to WALPROHIBIT_STATE_READ_WRITE? -- Regards, Dilip Kumar EnterpriseDB: http://www.enterprisedb.com

<    2   3   4   5   6   7   8   9   10   11   >