On Thu, Jan 30, 2020 at 4:11 PM Amit Kapila wrote:
>
> On Fri, Jan 10, 2020 at 10:14 AM Dilip Kumar wrote:
> >
> > On Mon, Jan 6, 2020 at 2:11 PM Amit Kapila wrote:
> > >
> >
> > >
> > > Few more comments:
> > >
> > > v4-0007-Implement-streaming-mode-in-ReorderBu
On Fri, Jan 10, 2020 at 10:14 AM Dilip Kumar wrote:
>
> On Mon, Jan 6, 2020 at 2:11 PM Amit Kapila wrote:
> >
>
> >
> > Few more comments:
> >
> > v4-0007-Implement-streaming-mode-in-ReorderBuffer
> > 1.
> > +ReorderBufferStreamTXN(ReorderBuffer *rb, ReorderBuffer
On Tue, Jan 28, 2020 at 1:55 PM Dilip Kumar wrote:
>
> On Tue, Jan 28, 2020 at 1:30 PM Amit Kapila wrote:
> >
> > On Tue, Jan 28, 2020 at 11:58 AM Dilip Kumar wrote:
> > >
> > > On Tue, Jan 28, 2020 at 11:43 AM Amit Kapila
> > > wrote:
> > > >
> > > > > > It seems to me that we need to add all
On Tue, Jan 28, 2020 at 1:30 PM Amit Kapila wrote:
>
> On Tue, Jan 28, 2020 at 11:58 AM Dilip Kumar wrote:
> >
> > On Tue, Jan 28, 2020 at 11:43 AM Amit Kapila
> > wrote:
> > >
> > > > > It seems to me that we need to add all of this new handling because
> > > > > while taking the decision whet
On Tue, Jan 28, 2020 at 11:58 AM Dilip Kumar wrote:
>
> On Tue, Jan 28, 2020 at 11:43 AM Amit Kapila wrote:
> >
> > > > It seems to me that we need to add all of this new handling because
> > > > while taking the decision whether to stream or not we don't know
> > > > whether the txn has changes
On Tue, Jan 28, 2020 at 11:43 AM Amit Kapila wrote:
>
> On Tue, Jan 28, 2020 at 11:34 AM Dilip Kumar wrote:
> >
> > On Tue, Jan 28, 2020 at 11:28 AM Amit Kapila
> > wrote:
> > >
> > > On Wed, Jan 22, 2020 at 10:30 AM Dilip Kumar
> > > wrote:
> > > >
> > > > On Tue, Jan 14, 2020 at 10:44 AM Am
On Tue, Jan 28, 2020 at 11:34 AM Dilip Kumar wrote:
>
> On Tue, Jan 28, 2020 at 11:28 AM Amit Kapila wrote:
> >
> > On Wed, Jan 22, 2020 at 10:30 AM Dilip Kumar wrote:
> > >
> > > On Tue, Jan 14, 2020 at 10:44 AM Amit Kapila
> > > wrote:
> > > >
> > > >
> > > > Hmm, I think this can turn out t
On Tue, Jan 28, 2020 at 11:28 AM Amit Kapila wrote:
>
> On Wed, Jan 22, 2020 at 10:30 AM Dilip Kumar wrote:
> >
> > On Tue, Jan 14, 2020 at 10:44 AM Amit Kapila
> > wrote:
> > >
> > >
> > > Hmm, I think this can turn out to be inefficient because we can easily
> > > end up spilling the data eve
On Wed, Jan 22, 2020 at 10:30 AM Dilip Kumar wrote:
>
> On Tue, Jan 14, 2020 at 10:44 AM Amit Kapila wrote:
> >
> >
> > Hmm, I think this can turn out to be inefficient because we can easily
> > end up spilling the data even when we don't need to so. Consider
> > cases, where part of the streame
On Wed, Jan 22, 2020 at 10:07 PM Alvaro Herrera
wrote:
>
> I looked at this patchset and it seemed natural to apply 0008 next
> (adding work_mem to subscriptions).
>
I am not so sure whether we need this patch as the exact scenario
where it can help is not very clear to me and neither did anyone
I looked at this patchset and it seemed natural to apply 0008 next
(adding work_mem to subscriptions). Attached is Dilip's latest version,
plus my review changes. This will break the patch tester's logic; sorry
about that.
What part of this change is what sets the process's
logical_decoding_work
On Sat, Jan 4, 2020 at 4:07 PM Amit Kapila wrote:
>
Update on the open items
> As per my understanding apart from the above comments, the known
> pending work for this patchset is as follows:
> a. The two open items agreed to you in the email [3]. -> The first part is
> done and the second part
On Tue, Jan 14, 2020 at 10:56:37AM +0530, Dilip Kumar wrote:
On Sat, Jan 11, 2020 at 3:07 AM Alvaro Herrera wrote:
On 2020-Jan-10, Alvaro Herrera wrote:
> Here's a rebase of this patch series. I didn't change anything except
... this time with attachments ...
The patch set fails to apply o
On Mon, Jan 13, 2020 at 3:18 PM Dilip Kumar wrote:
>
> On Thu, Jan 9, 2020 at 12:09 PM Amit Kapila wrote:
> >
> > On Thu, Jan 9, 2020 at 10:30 AM Dilip Kumar wrote:
> > >
> > > On Thu, Jan 9, 2020 at 9:35 AM Amit Kapila
> > > wrote:
> > > >
> > > > > The problem is that when we
> > > > > get
On Thu, Jan 9, 2020 at 12:09 PM Amit Kapila wrote:
>
> On Thu, Jan 9, 2020 at 10:30 AM Dilip Kumar wrote:
> >
> > On Thu, Jan 9, 2020 at 9:35 AM Amit Kapila wrote:
> > >
> > > On Wed, Jan 8, 2020 at 1:12 PM Dilip Kumar wrote:
> > > >
> > > > I have observed one more design issue.
> > > >
> > >
On 2020-Jan-10, Alvaro Herrera wrote:
> From 7d671806584fff71067c8bde38b2f642ba1331a9 Mon Sep 17 00:00:00 2001
> From: Dilip Kumar
> Date: Wed, 20 Nov 2019 16:41:13 +0530
> Subject: [PATCH v6 10/12] Enable streaming for all subscription TAP tests
This patch turns a lot of test into the streamed
Here's a rebase of this patch series. I didn't change anything except
1. disregard what was 0005, since I already pushed it.
2. roll 0003 into 0002.
3. rebase 0007 (now 0005) to account for the reorderbuffer changes.
(I did notice that 0005 adds a new boolean any_data_sent, which is
silly -- it
I pushed 0005 (the rbtxn flags thing) after some light editing.
It's been around for long enough ...
--
Álvaro Herrerahttps://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
On Mon, Dec 30, 2019 at 3:43 PM Amit Kapila wrote:
>
> On Sun, Dec 29, 2019 at 1:34 PM Dilip Kumar wrote:
> >
> > I have observed some more issues
> >
> > 1. Currently, In ReorderBufferCommit, it is always expected that
> > whenever we get REORDER_BUFFER_CHANGE_INTERNAL_SPEC_CONFIRM, we must
> >
On Sat, Jan 4, 2020 at 4:07 PM Amit Kapila wrote:
>
> On Mon, Dec 30, 2019 at 3:11 PM Dilip Kumar wrote:
> >
> > On Thu, Dec 12, 2019 at 9:44 AM Dilip Kumar wrote:
> > > > 0002-Issue-individual-invalidations-with-wal_level-log
> > > >
On Thu, Jan 9, 2020 at 12:09 PM Amit Kapila wrote:
>
> On Thu, Jan 9, 2020 at 10:30 AM Dilip Kumar wrote:
> >
> > On Thu, Jan 9, 2020 at 9:35 AM Amit Kapila wrote:
> > >
> > > On Wed, Jan 8, 2020 at 1:12 PM Dilip Kumar wrote:
> > > >
> > > > I have observed one more design issue.
> > > >
> > >
On Thu, Jan 9, 2020 at 10:30 AM Dilip Kumar wrote:
>
> On Thu, Jan 9, 2020 at 9:35 AM Amit Kapila wrote:
> >
> > On Wed, Jan 8, 2020 at 1:12 PM Dilip Kumar wrote:
> > >
> > > I have observed one more design issue.
> > >
> >
> > Good observation.
> >
> > > The problem is that when we
> > > get a
On Thu, Jan 9, 2020 at 9:35 AM Amit Kapila wrote:
>
> On Wed, Jan 8, 2020 at 1:12 PM Dilip Kumar wrote:
> >
> > I have observed one more design issue.
> >
>
> Good observation.
>
> > The problem is that when we
> > get a toasted chunks we remember the changes in the memory(hash table)
> > but do
On Wed, Jan 8, 2020 at 1:12 PM Dilip Kumar wrote:
>
> I have observed one more design issue.
>
Good observation.
> The problem is that when we
> get a toasted chunks we remember the changes in the memory(hash table)
> but don't stream until we get the actual change on the main table.
> Now, the
On Mon, Jan 6, 2020 at 4:44 PM Dilip Kumar wrote:
>
> On Mon, Jan 6, 2020 at 4:36 PM Amit Kapila wrote:
> >
> > On Mon, Jan 6, 2020 at 3:56 PM Dilip Kumar wrote:
> > >
> > > On Mon, Jan 6, 2020 at 2:11 PM Amit Kapila
> > > wrote:
> > > >
> > > > 3.
> > > > +static void
> > > > +ReorderBufferSt
On Mon, Jan 6, 2020 at 4:36 PM Amit Kapila wrote:
>
> On Mon, Jan 6, 2020 at 3:56 PM Dilip Kumar wrote:
> >
> > On Mon, Jan 6, 2020 at 2:11 PM Amit Kapila wrote:
> > >
> > > 3.
> > > +static void
> > > +ReorderBufferStreamTXN(ReorderBuffer *rb, ReorderBufferTXN *txn)
> > > {
> > > ..
> > > + /*
On Mon, Jan 6, 2020 at 3:56 PM Dilip Kumar wrote:
>
> On Mon, Jan 6, 2020 at 2:11 PM Amit Kapila wrote:
> >
> > 3.
> > +static void
> > +ReorderBufferStreamTXN(ReorderBuffer *rb, ReorderBufferTXN *txn)
> > {
> > ..
> > + /*
> > + * If this is a subxact, we need to stream the top-level transaction
On Mon, Jan 6, 2020 at 2:11 PM Amit Kapila wrote:
>
> On Mon, Jan 6, 2020 at 9:21 AM Dilip Kumar wrote:
> >
> > On Sat, Jan 4, 2020 at 4:07 PM Amit Kapila wrote:
> > >
> > >
> > > It is better to merge it with the main patch for
> > > "Implement-streaming-mode-in-ReorderBuffer", otherwise, it is
On Mon, Jan 6, 2020 at 9:21 AM Dilip Kumar wrote:
>
> On Sat, Jan 4, 2020 at 4:07 PM Amit Kapila wrote:
> >
> >
> > It is better to merge it with the main patch for
> > "Implement-streaming-mode-in-ReorderBuffer", otherwise, it is a bit
> > difficult to review.
> Actually, we can merge 0008, 0009
On Sat, Jan 4, 2020 at 4:07 PM Amit Kapila wrote:
>
> On Mon, Dec 30, 2019 at 3:11 PM Dilip Kumar wrote:
> >
> > On Thu, Dec 12, 2019 at 9:44 AM Dilip Kumar wrote:
> > >
> > Yesterday, Tomas has posted the latest version of the patch set which
> > contain the fix for schema send part. Meanwhile
On Mon, Dec 30, 2019 at 3:11 PM Dilip Kumar wrote:
>
> On Thu, Dec 12, 2019 at 9:44 AM Dilip Kumar wrote:
> >
> Yesterday, Tomas has posted the latest version of the patch set which
> contain the fix for schema send part. Meanwhile, I was working on few
> review comments/bugfixes and refactoring
On Sat, Jan 4, 2020 at 10:00 AM Amit Kapila wrote:
>
> On Sun, Dec 29, 2019 at 1:34 PM Dilip Kumar wrote:
> > On Sat, Dec 28, 2019 at 9:33 PM Tomas Vondra
> > wrote:
> > +static void
> > +set_schema_sent_in_streamed_txn(RelationSyncEntry *entry, TransactionId
> > xid)
> > +{
> > + MemoryContext
On Sun, Dec 29, 2019 at 1:34 PM Dilip Kumar wrote:
> On Sat, Dec 28, 2019 at 9:33 PM Tomas Vondra
> wrote:
> >
> >
> > Yeah, the "is_schema_sent" flag in ReorderBufferTXN does not work - it
> > needs to be in the RelationSyncEntry. In fact, I already have code for
> > that in my private repositor
On Tue, Dec 24, 2019 at 10:58 AM Robert Haas wrote:
>
> On Thu, Dec 12, 2019 at 3:41 AM Amit Kapila wrote:
>
> > I think the way invalidations work for logical replication is that
> > normally, we always start a new transaction before decoding each
> > commit which allows us to accept the invalid
On Mon, Dec 30, 2019 at 3:43 PM Amit Kapila wrote:
>
> On Sun, Dec 29, 2019 at 1:34 PM Dilip Kumar wrote:
> >
> > I have observed some more issues
> >
> > 1. Currently, In ReorderBufferCommit, it is always expected that
> > whenever we get REORDER_BUFFER_CHANGE_INTERNAL_SPEC_CONFIRM, we must
> >
On Thu, Dec 26, 2019 at 12:36 PM Masahiko Sawada
wrote:
>
> On Tue, 24 Dec 2019 at 17:21, Amit Kapila wrote:
> >
> > >
> > > Thank you for explanation. The plan makes sense. But I think in the
> > > current design it's a problem that logical replication worker doesn't
> > > receive changes (and d
On Sun, Dec 29, 2019 at 1:34 PM Dilip Kumar wrote:
>
> I have observed some more issues
>
> 1. Currently, In ReorderBufferCommit, it is always expected that
> whenever we get REORDER_BUFFER_CHANGE_INTERNAL_SPEC_CONFIRM, we must
> have already got REORDER_BUFFER_CHANGE_INTERNAL_SPEC_INSERT and in
>
On Sat, Dec 28, 2019 at 9:33 PM Tomas Vondra
wrote:
>
> On Tue, Dec 10, 2019 at 10:23:19AM +0530, Dilip Kumar wrote:
> >On Tue, Dec 10, 2019 at 9:52 AM Amit Kapila wrote:
> >>
> >> On Mon, Dec 2, 2019 at 2:02 PM Dilip Kumar wrote:
> >> >
> >> > On Sun, Dec 1, 2019 at 7:58 AM Michael Paquier
>
On Tue, 24 Dec 2019 at 17:21, Amit Kapila wrote:
>
> On Tue, Dec 24, 2019 at 11:17 AM Masahiko Sawada
> wrote:
> >
> > On Fri, 20 Dec 2019 at 22:30, Amit Kapila wrote:
> > >
> > >
> > > The main aim of this feature is to reduce apply lag. Because if we
> > > send all the changes together it can
On Tue, Dec 24, 2019 at 11:17 AM Masahiko Sawada
wrote:
>
> On Fri, 20 Dec 2019 at 22:30, Amit Kapila wrote:
> >
> >
> > The main aim of this feature is to reduce apply lag. Because if we
> > send all the changes together it can delay there apply because of
> > network delay, whereas if most of
On Fri, 20 Dec 2019 at 22:30, Amit Kapila wrote:
>
> On Fri, Dec 20, 2019 at 11:47 AM Masahiko Sawada
> wrote:
> >
> > On Mon, 2 Dec 2019 at 17:32, Dilip Kumar wrote:
> > >
> > > On Sun, Dec 1, 2019 at 7:58 AM Michael Paquier
> > > wrote:
> > > >
> > > > On Fri, Nov 22, 2019 at 01:18:11PM +053
On Thu, Dec 12, 2019 at 3:41 AM Amit Kapila wrote:
> I don't think we have evaluated it yet, but we should do it. The
> point to note is that it is only for the case when wal_level is
> 'logical' (see IsSubTransactionAssignmentPending) in which case we
> already log more WAL, so this might not im
On Sun, Dec 22, 2019 at 5:04 PM vignesh C wrote:
>
> Few comments:
> assert variable should be within #ifdef USE_ASSERT_CHECKING in patch
> v2-0008-Add-support-for-streaming-to-built-in-replication.patch:
> + int64 subidx;
> + boolfound = false;
>
On Mon, Dec 2, 2019 at 2:02 PM Dilip Kumar wrote:
>
> On Sun, Dec 1, 2019 at 7:58 AM Michael Paquier wrote:
> >
> > On Fri, Nov 22, 2019 at 01:18:11PM +0530, Dilip Kumar wrote:
> > > I have rebased the patch on the latest head and also fix the issue of
> > > "concurrent abort handling of the (sub
On Fri, Dec 20, 2019 at 2:00 PM Kyotaro Horiguchi
wrote:
>
> Hello.
>
> At Fri, 13 Dec 2019 14:46:20 +0530, Amit Kapila
> wrote in
> > On Wed, Dec 11, 2019 at 11:46 PM Robert Haas wrote:
> > >
> > > On Mon, Dec 2, 2019 at 3:32 AM Dilip Kumar wrote:
> > > > I have rebased the patch set on the l
On Fri, Dec 20, 2019 at 11:47 AM Masahiko Sawada
wrote:
>
> On Mon, 2 Dec 2019 at 17:32, Dilip Kumar wrote:
> >
> > On Sun, Dec 1, 2019 at 7:58 AM Michael Paquier wrote:
> > >
> > > On Fri, Nov 22, 2019 at 01:18:11PM +0530, Dilip Kumar wrote:
> > > > I have rebased the patch on the latest head a
Hello.
At Fri, 13 Dec 2019 14:46:20 +0530, Amit Kapila wrote
in
> On Wed, Dec 11, 2019 at 11:46 PM Robert Haas wrote:
> >
> > On Mon, Dec 2, 2019 at 3:32 AM Dilip Kumar wrote:
> > > I have rebased the patch set on the latest head.
> >
> > 0001 looks like a clever approach, but are you sure it
On Mon, 2 Dec 2019 at 17:32, Dilip Kumar wrote:
>
> On Sun, Dec 1, 2019 at 7:58 AM Michael Paquier wrote:
> >
> > On Fri, Nov 22, 2019 at 01:18:11PM +0530, Dilip Kumar wrote:
> > > I have rebased the patch on the latest head and also fix the issue of
> > > "concurrent abort handling of the (sub)t
On Thu, Dec 12, 2019 at 9:45 AM Dilip Kumar wrote:
>
> On Wed, Dec 11, 2019 at 5:22 PM Amit Kapila wrote:
> >
> > On Mon, Dec 9, 2019 at 1:27 PM Dilip Kumar wrote:
> > >
> > > I have review the patch set and here are few comments/questions
> > >
> > > 1.
> > > +static void
> > > +pg_decode_strea
On Wed, Dec 11, 2019 at 11:46 PM Robert Haas wrote:
>
> On Mon, Dec 2, 2019 at 3:32 AM Dilip Kumar wrote:
> > I have rebased the patch set on the latest head.
>
> 0001 looks like a clever approach, but are you sure it doesn't hurt
> performance when many small XLOG records are being inserted? I t
On Wed, Dec 11, 2019 at 11:46 PM Robert Haas wrote:
>
> On Mon, Dec 2, 2019 at 3:32 AM Dilip Kumar wrote:
> > I have rebased the patch set on the latest head.
>
> 0001 looks like a clever approach, but are you sure it doesn't hurt
> performance when many small XLOG records are being inserted? I t
On Wed, Dec 11, 2019 at 5:22 PM Amit Kapila wrote:
>
> On Mon, Dec 9, 2019 at 1:27 PM Dilip Kumar wrote:
> >
> > I have review the patch set and here are few comments/questions
> >
> > 1.
> > +static void
> > +pg_decode_stream_change(LogicalDecodingContext *ctx,
> > + ReorderBufferTXN *txn,
> > +
On Mon, Dec 2, 2019 at 3:32 AM Dilip Kumar wrote:
> I have rebased the patch set on the latest head.
0001 looks like a clever approach, but are you sure it doesn't hurt
performance when many small XLOG records are being inserted? I think
XLogRecordAssemble() can get pretty hot in some workloads.
On Mon, Dec 9, 2019 at 1:27 PM Dilip Kumar wrote:
>
> I have review the patch set and here are few comments/questions
>
> 1.
> +static void
> +pg_decode_stream_change(LogicalDecodingContext *ctx,
> + ReorderBufferTXN *txn,
> + Relation relation,
> + ReorderBufferChange *change)
> +{
> + OutputPlug
On Tue, Dec 10, 2019 at 9:52 AM Amit Kapila wrote:
>
> On Mon, Dec 2, 2019 at 2:02 PM Dilip Kumar wrote:
> >
> > On Sun, Dec 1, 2019 at 7:58 AM Michael Paquier wrote:
> > >
> > > On Fri, Nov 22, 2019 at 01:18:11PM +0530, Dilip Kumar wrote:
> > > > I have rebased the patch on the latest head and
On Mon, Dec 2, 2019 at 2:02 PM Dilip Kumar wrote:
>
> On Sun, Dec 1, 2019 at 7:58 AM Michael Paquier wrote:
> >
> > On Fri, Nov 22, 2019 at 01:18:11PM +0530, Dilip Kumar wrote:
> > > I have rebased the patch on the latest head and also fix the issue of
> > > "concurrent abort handling of the (sub
On Mon, Dec 2, 2019 at 2:01 PM Dilip Kumar wrote:
>
> On Sun, Dec 1, 2019 at 7:58 AM Michael Paquier wrote:
> >
> > On Fri, Nov 22, 2019 at 01:18:11PM +0530, Dilip Kumar wrote:
> > > I have rebased the patch on the latest head and also fix the issue of
> > > "concurrent abort handling of the (sub
On Fri, Nov 22, 2019 at 01:18:11PM +0530, Dilip Kumar wrote:
> I have rebased the patch on the latest head and also fix the issue of
> "concurrent abort handling of the (sub)transaction." and attached as
> (v1-0013-Extend-handling-of-concurrent-aborts-for-streamin) along with
> the complete patch s
On Tue, Nov 19, 2019 at 5:25 PM Amit Kapila wrote:
>
> On Sat, Nov 16, 2019 at 6:44 PM Amit Kapila wrote:
> >
> > On Thu, Nov 7, 2019 at 5:13 PM Amit Kapila wrote:
> > >
> > > Some notes before commit:
> > > --
> > > 1.
> > > Commit message need to be changed
On Wed, Nov 20, 2019 at 8:22 PM Dilip Kumar wrote:
>
> On Wed, Nov 20, 2019 at 11:15 AM Dilip Kumar wrote:
> >
> > On Tue, Nov 19, 2019 at 5:23 PM Amit Kapila wrote:
> > >
> > > On Mon, Nov 18, 2019 at 5:02 PM Dilip Kumar wrote:
> > > >
> > > > On Fri, Nov 15, 2019 at 4:19 PM Amit Kapila
> >
On Tue, Nov 19, 2019 at 5:23 PM Amit Kapila wrote:
>
> On Mon, Nov 18, 2019 at 5:02 PM Dilip Kumar wrote:
> >
> > On Fri, Nov 15, 2019 at 4:19 PM Amit Kapila wrote:
> > >
> > > On Fri, Nov 15, 2019 at 4:01 PM Dilip Kumar wrote:
> > > >
> > > > On Fri, Nov 15, 2019 at 3:50 PM Amit Kapila
> > >
On Sat, Nov 16, 2019 at 6:44 PM Amit Kapila wrote:
>
> On Thu, Nov 7, 2019 at 5:13 PM Amit Kapila wrote:
> >
> > Some notes before commit:
> > --
> > 1.
> > Commit message need to be changed for the first patch
> > --
On Mon, Nov 18, 2019 at 5:02 PM Dilip Kumar wrote:
>
> On Fri, Nov 15, 2019 at 4:19 PM Amit Kapila wrote:
> >
> > On Fri, Nov 15, 2019 at 4:01 PM Dilip Kumar wrote:
> > >
> > > On Fri, Nov 15, 2019 at 3:50 PM Amit Kapila
> > > wrote:
> > > >
> > > >
> > > > Few other comments on this patch:
>
On Fri, Nov 15, 2019 at 4:19 PM Amit Kapila wrote:
>
> On Fri, Nov 15, 2019 at 4:01 PM Dilip Kumar wrote:
> >
> > On Fri, Nov 15, 2019 at 3:50 PM Amit Kapila wrote:
> > >
> > >
> > > Few other comments on this patch:
> > > 1.
> > > + case REORDER_BUFFER_CHANGE_INVALIDATION:
> > > +
> > > + /*
>
On Thu, Nov 7, 2019 at 5:13 PM Amit Kapila wrote:
>
> Some notes before commit:
> --
> 1.
> Commit message need to be changed for the first patch
> -
> A.
> > The memory limit is defined by
On Fri, Nov 15, 2019 at 4:01 PM Dilip Kumar wrote:
>
> On Fri, Nov 15, 2019 at 3:50 PM Amit Kapila wrote:
> >
> >
> > Few other comments on this patch:
> > 1.
> > + case REORDER_BUFFER_CHANGE_INVALIDATION:
> > +
> > + /*
> > + * Execute the invalidation message locally.
> > + *
> > + * XXX Do we
On Fri, Nov 15, 2019 at 3:50 PM Amit Kapila wrote:
>
> On Thu, Nov 14, 2019 at 3:40 PM Dilip Kumar wrote:
> >
> >
> > Apart from this, I have another question in
> > 0003-Issue-individual-invalidations-with-wal_level-logical.patch
> >
> > @@ -543,6 +588,18 @@ RegisterSnapshotInvalidation(Oid dbId
On Thu, Nov 14, 2019 at 3:40 PM Dilip Kumar wrote:
>
>
> Apart from this, I have another question in
> 0003-Issue-individual-invalidations-with-wal_level-logical.patch
>
> @@ -543,6 +588,18 @@ RegisterSnapshotInvalidation(Oid dbId, Oid relId)
> {
> AddSnapshotInvalidationMessage(&transInvalInfo
On Thu, Nov 14, 2019 at 12:10 PM Amit Kapila wrote:
>
> On Thu, Nov 14, 2019 at 9:37 AM Dilip Kumar wrote:
> >
> > On Wed, Nov 13, 2019 at 5:55 PM Amit Kapila wrote:
> > >
> > > On Thu, Oct 3, 2019 at 1:18 PM Dilip Kumar wrote:
> > > >
> > >
> > > As mentioned by me a few days back that the fir
On Thu, Nov 14, 2019 at 9:37 AM Dilip Kumar wrote:
>
> On Wed, Nov 13, 2019 at 5:55 PM Amit Kapila wrote:
> >
> > On Thu, Oct 3, 2019 at 1:18 PM Dilip Kumar wrote:
> > >
> >
> > As mentioned by me a few days back that the first patch in this series
> > is ready to go [1] (I am hoping Tomas will
On Wed, Nov 13, 2019 at 5:55 PM Amit Kapila wrote:
>
> On Thu, Oct 3, 2019 at 1:18 PM Dilip Kumar wrote:
> >
>
> As mentioned by me a few days back that the first patch in this series
> is ready to go [1] (I am hoping Tomas will pick it up), so I have
> started the review of other patches
>
> Rev
On Thu, Oct 3, 2019 at 1:18 PM Dilip Kumar wrote:
>
As mentioned by me a few days back that the first patch in this series
is ready to go [1] (I am hoping Tomas will pick it up), so I have
started the review of other patches
Review/Questions on 0002-Immediately-WAL-log-assignments.patch
On Tue, Nov 12, 2019 at 4:12 PM Alexey Kondratov
wrote:
>
> On 04.11.2019 13:05, Kuntal Ghosh wrote:
> > On Mon, Nov 4, 2019 at 3:32 PM Dilip Kumar wrote:
> >> So your result shows that with "streaming on", performance is
> >> degrading? By any chance did you try to see where is the bottleneck?
On 04.11.2019 13:05, Kuntal Ghosh wrote:
On Mon, Nov 4, 2019 at 3:32 PM Dilip Kumar wrote:
So your result shows that with "streaming on", performance is
degrading? By any chance did you try to see where is the bottleneck?
Right. But, as we increase the logical_decoding_work_mem, the
performa
On Thu, Nov 7, 2019 at 3:50 PM Dilip Kumar wrote:
>
> On Thu, Nov 7, 2019 at 3:19 PM Amit Kapila wrote:
>
> > What do you think?
> I have reviewed your changes and looks fine to me.
>
Okay, thanks. I am also happy with the two patches I have posted in
my last email [1].
Tomas, would you like t
On Thu, Nov 7, 2019 at 3:19 PM Amit Kapila wrote:
>
> On Wed, Nov 6, 2019 at 11:33 AM vignesh C wrote:
> >
> > I have made one change to the configuration file in
> > contrib/test_decoding directory, with that the coverage seems to be
> > fine. I have seen that the coverage is almost like the cod
On Wed, Nov 6, 2019 at 11:33 AM vignesh C wrote:
>
> I have made one change to the configuration file in
> contrib/test_decoding directory, with that the coverage seems to be
> fine. I have seen that the coverage is almost like the code before
> applying the patch. I have attached the test change
On Mon, Nov 4, 2019 at 5:22 PM Amit Kapila wrote:
>
> On Wed, Oct 30, 2019 at 9:38 AM vignesh C wrote:
> >
> > On Tue, Oct 22, 2019 at 10:52 PM Tomas Vondra
> > wrote:
> > >
> > > I think the patch should do the simplest thing possible, i.e. what it
> > > does today. Otherwise we'll never get it
On Mon, Nov 4, 2019 at 5:22 PM Amit Kapila wrote:
>
> On Wed, Oct 30, 2019 at 9:38 AM vignesh C wrote:
> >
> > On Tue, Oct 22, 2019 at 10:52 PM Tomas Vondra
> > wrote:
> > >
> > > I think the patch should do the simplest thing possible, i.e. what it
> > > does today. Otherwise we'll never get it
On Wed, Oct 30, 2019 at 9:38 AM vignesh C wrote:
>
> On Tue, Oct 22, 2019 at 10:52 PM Tomas Vondra
> wrote:
> >
> > I think the patch should do the simplest thing possible, i.e. what it
> > does today. Otherwise we'll never get it committed.
> >
> I found a couple of crashes while reviewing and t
On Thu, Oct 24, 2019 at 7:07 PM Amit Kapila wrote:
>
> On Tue, Oct 22, 2019 at 10:30 AM Dilip Kumar wrote:
> >
> > I have merged bugs_and_review_comments_fix.patch changes to 0001 and 0002.
> >
>
> I was wondering whether we have checked the code coverage after this
> patch? Previously, the exis
On Mon, Nov 4, 2019 at 3:32 PM Dilip Kumar wrote:
>
> So your result shows that with "streaming on", performance is
> degrading? By any chance did you try to see where is the bottleneck?
>
Right. But, as we increase the logical_decoding_work_mem, the
performance improves. I've not analyzed the bo
On Mon, Nov 4, 2019 at 2:43 PM Kuntal Ghosh wrote:
>
> Hello hackers,
>
> I've done some performance testing of this feature. Following is my
> test case (taken from an earlier thread):
>
> postgres=# CREATE TABLE large_test (num1 bigint, num2 double
> precision, num3 double precision);
> postgres
Hello hackers,
I've done some performance testing of this feature. Following is my
test case (taken from an earlier thread):
postgres=# CREATE TABLE large_test (num1 bigint, num2 double
precision, num3 double precision);
postgres=# \timing on
postgres=# EXPLAIN (ANALYZE, BUFFERS) INSERT INTO larg
On Wed, Oct 30, 2019 at 9:38 AM vignesh C wrote:
>
I have noticed one more problem in the logic of setting the logical
decoding work mem from the create subscription command. Suppose in
subscription command we don't give the work mem then it sends the
garbage value to the walsender and the walsen
On Tue, Oct 22, 2019 at 10:52 PM Tomas Vondra
wrote:
>
> I think the patch should do the simplest thing possible, i.e. what it
> does today. Otherwise we'll never get it committed.
>
I found a couple of crashes while reviewing and testing flushing of
open transaction data:
Issue 1:
#0 0x7f22c
On Tue, Oct 22, 2019 at 10:30 AM Dilip Kumar wrote:
>
> I have merged bugs_and_review_comments_fix.patch changes to 0001 and 0002.
>
I was wondering whether we have checked the code coverage after this
patch? Previously, the existing tests seem to be covering most parts
of the function ReorderBu
On Wed, Oct 23, 2019 at 12:32 AM Alexey Kondratov
wrote:
>
> On 22.10.2019 20:22, Tomas Vondra wrote:
> >
> > I think the patch should do the simplest thing possible, i.e. what it
> > does today. Otherwise we'll never get it committed.
> >
>
> I have to agree with Tomas, that keeping things as sim
On Tue, Oct 22, 2019 at 10:42 PM Tomas Vondra
wrote:
>
> On Tue, Oct 22, 2019 at 10:30:16AM +0530, Dilip Kumar wrote:
> >
> >I have moved it out as a separate patch (0003) so that if we need that
> >we need this for the streaming transaction then we can keep this.
> >>
>
> I'm OK with moving it to
On 22.10.2019 20:22, Tomas Vondra wrote:
On Tue, Oct 22, 2019 at 11:01:48AM +0530, Dilip Kumar wrote:
On Tue, Oct 22, 2019 at 10:46 AM Amit Kapila
wrote:
In general, yours and Alexy's test results
show that there is merit by having workers applying such transactions.
OTOH, as noted above
On Tue, Oct 22, 2019 at 11:01:48AM +0530, Dilip Kumar wrote:
On Tue, Oct 22, 2019 at 10:46 AM Amit Kapila wrote:
On Thu, Oct 3, 2019 at 1:18 PM Dilip Kumar wrote:
>
> I have attempted to test the performance of (Stream + Spill) vs
> (Stream + BGW pool) and I can see the similar gain what Alex
On Tue, Oct 22, 2019 at 10:30:16AM +0530, Dilip Kumar wrote:
On Fri, Oct 18, 2019 at 5:32 PM Amit Kapila wrote:
On Mon, Oct 14, 2019 at 3:09 PM Dilip Kumar wrote:
>
> On Thu, Oct 3, 2019 at 4:03 AM Tomas Vondra
> wrote:
> >
> >
> > Sure, I wasn't really proposing to adding all stats from tha
On Tue, Oct 22, 2019 at 10:46 AM Amit Kapila wrote:
>
> On Thu, Oct 3, 2019 at 1:18 PM Dilip Kumar wrote:
> >
> > I have attempted to test the performance of (Stream + Spill) vs
> > (Stream + BGW pool) and I can see the similar gain what Alexey had
> > shown[1].
> >
> > In addition to this, I hav
On Thu, Oct 3, 2019 at 1:18 PM Dilip Kumar wrote:
>
> I have attempted to test the performance of (Stream + Spill) vs
> (Stream + BGW pool) and I can see the similar gain what Alexey had
> shown[1].
>
> In addition to this, I have rebased the latest patchset [2] without
> the two-phase logical dec
On Fri, Oct 18, 2019 at 5:32 PM Amit Kapila wrote:
>
> On Mon, Oct 14, 2019 at 3:09 PM Dilip Kumar wrote:
> >
> > On Thu, Oct 3, 2019 at 4:03 AM Tomas Vondra
> > wrote:
> > >
> > >
> > > Sure, I wasn't really proposing to adding all stats from that patch,
> > > including those related to streami
On Mon, Oct 21, 2019 at 2:50 PM Amit Kapila wrote:
>
> On Mon, Oct 21, 2019 at 10:48 AM Dilip Kumar wrote:
> >
> > On Fri, Oct 18, 2019 at 5:32 PM Amit Kapila wrote:
> >
> > > 3.
> > > @@ -2479,7 +2480,7 @@ ReorderBufferSerializeTXN(ReorderBuffer *rb,
> > > ReorderBufferTXN *txn)
> > >
> > > /
On Mon, Oct 21, 2019 at 10:48 AM Dilip Kumar wrote:
>
> On Fri, Oct 18, 2019 at 5:32 PM Amit Kapila wrote:
>
> > 3.
> > @@ -2479,7 +2480,7 @@ ReorderBufferSerializeTXN(ReorderBuffer *rb,
> > ReorderBufferTXN *txn)
> >
> > /* update the statistics */
> > rb->spillCount += 1;
> > - rb->spillTxn
On Fri, Oct 18, 2019 at 5:32 PM Amit Kapila wrote:
I have replied to some of your questions inline. I will work on the
remaining comments and post the patch for the same.
> > >
> > > Sure, I wasn't really proposing to adding all stats from that patch,
> > > including those related to streaming.
On Mon, Oct 14, 2019 at 3:09 PM Dilip Kumar wrote:
>
> On Thu, Oct 3, 2019 at 4:03 AM Tomas Vondra
> wrote:
> >
> >
> > Sure, I wasn't really proposing to adding all stats from that patch,
> > including those related to streaming. We need to extract just those
> > related to spilling. And yes, i
On Thu, Oct 3, 2019 at 4:03 AM Tomas Vondra
wrote:
>
> On Wed, Oct 02, 2019 at 04:27:30AM +0530, Amit Kapila wrote:
> >On Tue, Oct 1, 2019 at 7:21 PM Tomas Vondra
> >wrote:
> >
> >> On Tue, Oct 01, 2019 at 06:55:52PM +0530, Amit Kapila wrote:
> >> >
> >> >On further testing, I found that the patc
301 - 400 of 497 matches
Mail list logo
