stat_progress_update_param(PROGRESS_VACUUM_INDEX_COMPLETED,
> >+
> > pg_atomic_add_fetch_u32(&(pvs->shared->nindexes_completed), 1));
> >+}
>
> >Since parallel vacuum always sets the arg, I think we don't need to
> > check it.
>
> The arg is only set during parallel vacuum. During non-parallel vacuum,
> It's NULL. This check can be removed, but I realize now that we do need
> an Assert(pvs). Do you agree?
Agreed to add the assertion in this function.
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
On Wed, Feb 1, 2023 at 10:25 AM Michael Paquier wrote:
>
> On Fri, Jan 06, 2023 at 04:28:42PM +0900, Masahiko Sawada wrote:
> > I've attached the simple patch to add the progress reporting option to
> > pg_verifybackup. The progress information is displayed with --progress
>
On Wed, Feb 1, 2023 at 6:28 PM Amit Kapila wrote:
>
> On Wed, Feb 1, 2023 at 2:09 PM Masahiko Sawada wrote:
> >
> > On Fri, Jan 20, 2023 at 7:45 PM Amit Kapila wrote:
> > >
> > > On Tue, Jan 17, 2023 at 2:41 PM Amit Kapila
> > > wrote:
> >
long time, (e.g., the network socket buffer got full due to the
apply worker waiting on a lock), I think users should unblock it by
themselves. Or it might be practically better to shutdown the
subscriber first in the logical replication case, unlike the physical
replication case. I've not studied the time-delayed logical
replication patch yet, though.
Regards,
[1]
https://www.postgresql.org/message-id/CAA4eK1%2BpD654%2BXnrPugYueh7Oh22EBGTr6dA_fS0%2BgPiHayG9A%40mail.gmail.com
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
On Tue, Jan 31, 2023 at 3:59 PM Masahiko Sawada wrote:
>
> On Tue, Jan 31, 2023 at 3:56 PM Amit Kapila wrote:
> >
> > On Tue, Jan 31, 2023 at 11:12 AM Masahiko Sawada
> > wrote:
> > >
> > > On Mon, Jan 30, 2023 at 9:41 PM Masahiko Sawada
>
On Tue, Jan 31, 2023 at 3:56 PM Amit Kapila wrote:
>
> On Tue, Jan 31, 2023 at 11:12 AM Masahiko Sawada
> wrote:
> >
> > On Mon, Jan 30, 2023 at 9:41 PM Masahiko Sawada
> > wrote:
> >
> > I've attached patches for HEAD and backbranches. Please review th
On Mon, Jan 30, 2023 at 9:41 PM Masahiko Sawada wrote:
>
> On Mon, Jan 30, 2023 at 8:30 PM Amit Kapila wrote:
> >
> > On Fri, Jan 27, 2023 at 4:31 PM Hayato Kuroda (Fujitsu)
> > wrote:
> > >
> > > Thank you for making the patch! I'm stil
On Mon, Jan 30, 2023 at 8:24 PM Amit Kapila wrote:
>
> On Thu, Dec 8, 2022 at 8:17 AM Masahiko Sawada wrote:
> >
> > On Mon, Nov 21, 2022 at 4:31 PM Amit Kapila wrote:
> >
> > One idea to fix this issue is that in
> > ReplicationSlotsComputeRequiredXmin(), w
On Mon, Jan 30, 2023 at 1:31 PM John Naylor
wrote:
>
> On Sun, Jan 29, 2023 at 9:50 PM Masahiko Sawada wrote:
> >
> > On Sat, Jan 28, 2023 at 8:33 PM John Naylor
> > wrote:
>
> > > The first implementation should be simple, easy to test/verify, easy to
&
> > On the subscriber, if the streaming option is set to parallel, it directs
> > the leader
> > apply worker to send changes to the shared memory queue or to serialize
> > changes and apply them at the end of the transaction.
> >
>
> Changed.
>
> Attach the new version patch which addressed all comments so far (the v88-0001
> has been committed, so we only have one remaining patch this time).
>
I have one comment on v89 patch:
+ /*
+* Using 'immediate' mode returns false to cause a switch to
+* PARTIAL_SERIALIZE mode so that the remaining changes will
be serialized.
+*/
+ if (logical_replication_mode == LOGICAL_REP_MODE_IMMEDIATE)
+ return false;
+
Probably we might want to add unlikely() here since we could pass
through this path very frequently?
The rest looks good to me.
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
pecially in back branches as this is an exposed API.
Yes, we should not remove the already_locked parameter in
backbranches. So I was thinking of keeping it on back branches.
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
rammatically correct to me.
>
> "It also changes to the column names max_dead_tuples and
> num_dead_tuples and to show the progress information in bytes."
>
I've changed the commit message in the v23 patch. Please check it.
Other comments are also incorporated in the v23 patch. Thank you for
the comments!
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
>
> Normal walsenders which are not for tablesync create a replication slot with
> NOEXPORT_SNAPSHOT option. I think in this case, CreateInitDecodingContext() is
> called with need_full_snapshot = false, and slot->effective_xmin is not
> updated.
Right. This is how we create a slot used by an apply worker.
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
On Sat, Jan 28, 2023 at 8:33 PM John Naylor
wrote:
>
> On Thu, Jan 26, 2023 at 3:33 PM Masahiko Sawada wrote:
> >
> > On Thu, Jan 26, 2023 at 3:54 PM John Naylor
> > wrote:
>
> > I think that we need to prevent concurrent updates (RT_SET() and
> > R
I felt the description seems not to be suitable for current behavior.
> > > > A short description should be like "Sets a behavior of logical
> > > > replication", and
> > > > further descriptions can be added in lond description.
> > > I adjuste
On Fri, Jan 27, 2023 at 3:17 PM Andres Freund wrote:
>
> Hi,
>
> On 2023-01-27 14:24:51 +0900, Masahiko Sawada wrote:
> > If I'm understanding this result correctly, it seems to me that your
> > patch works well with the WAL DIO patch (WALDIO vs. WAL DIO & WAL
> &
11605
> 1024 107554 105541105942109370
> 204888552790248069990555
> 409661323548145870461743
If I'm understanding this result correctly, it seems to me that your
patch works well with the WAL DIO patch (WALDIO vs. WAL DIO & WAL
BUFFERS READ), but there seems no visible performance gain with only
your patch (HEAD vs. WAL BUFFERS READ). So it seems to me that your
patch should be included in the WAL DIO patch rather than applying it
alone. Am I missing something?
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
On Thu, Jan 26, 2023 at 3:54 PM John Naylor
wrote:
>
> On Wed, Jan 25, 2023 at 8:42 AM Masahiko Sawada wrote:
> >
> > On Mon, Jan 23, 2023 at 8:20 PM John Naylor
> > wrote:
> > >
> > > On Mon, Jan 16, 2023 at 3:18 PM Masahiko Sawada
> > > wro
On Mon, Jan 23, 2023 at 8:20 PM John Naylor
wrote:
>
> On Mon, Jan 16, 2023 at 3:18 PM Masahiko Sawada wrote:
> >
> > On Mon, Jan 16, 2023 at 2:02 PM John Naylor
> > wrote:
>
> In v21, all of your v20 improvements to the radix tree template and test have
>
LETED,
+
pg_atomic_add_fetch_u32(&(pvs->shared->nindexes_completed), 1));
+}
Since parallel vacuum always sets the arg, I think we don't need to check it.
Regards,
[1] https://www.postgresql.org/docs/devel/source-format.html
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
ublisher and the maximum
> number of origins on subscribers. Similarly,
> wal_retrieve_retry_interval' is used for different purposes on
> subscriber and standby nodes.
Using the existing parameter makes sense to me. But if we use
logical_decoding_mode also on the subscriber, as Shveta Malik also
suggested, probably it's better to rename it so as not to confuse. For
example, logical_replication_mode or something.
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
On Tue, Jan 17, 2023 at 8:06 PM John Naylor
wrote:
>
> On Mon, Jan 16, 2023 at 3:18 PM Masahiko Sawada wrote:
> >
> > On Mon, Jan 16, 2023 at 2:02 PM John Naylor
> > wrote:
>
> Attached is an update that mostly has the modest goal of getting CI green
> again.
On Tue, Jan 17, 2023 at 6:14 PM houzj.f...@fujitsu.com
wrote:
>
> On Tuesday, January 17, 2023 2:46 PM Masahiko Sawada
> wrote:
> >
> > On Tue, Jan 17, 2023 at 12:37 PM houzj.f...@fujitsu.com
> > wrote:
> > > Attach the new version 0001 patc
e leader of parallel apply workers.
*/
if (leader && leader->pid !=
beentry->st_procpid)
{
values[28] = Int32GetDatum(leader->pid);
nulls[28] = false;
}
+ else
+ {
+ int
leader_pid = GetLeaderApplyWorkerPid(beentry->st_procpid);
+
+ if (leader_pid != InvalidPid)
+ {
+ values[28] =
Int32GetDatum(leader_pid);
+ nulls[28] = false;
+ }
+ }
}
I'm slightly concerned that there could be overhead of executing
GetLeaderApplyWorkerPid () for every backend process except for
parallel query workers. The number of such backends could be large and
GetLeaderApplyWorkerPid() acquires the lwlock. For example, does it
make sense to check (st_backendType == B_BG_WORKER) before calling
GetLeaderApplyWorkerPid()? Or it might not be a problem since it's
LogicalRepWorkerLock which is not likely to be contended.
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
On Tue, Jan 17, 2023 at 1:55 PM Amit Kapila wrote:
>
> On Tue, Jan 17, 2023 at 8:59 AM Amit Kapila wrote:
> >
> > On Tue, Jan 17, 2023 at 8:35 AM Masahiko Sawada
> > wrote:
> > >
> > > On Mon, Jan 16, 2023 at 3:19 PM Amit Kapila
> > > wro
+1712,7 @@ apply_handle_stream_stop(StringInfo s)
}
in_streamed_transaction = false;
+ stream_xid = InvalidTransactionId;
We reset stream_xid also in stream_close_file() but probably it's no
longer necessary?
How about adding an assertion in apply_handle_stream_start() to make
su
On Mon, Jan 16, 2023 at 2:02 PM John Naylor
wrote:
>
> On Thu, Jan 12, 2023 at 9:51 PM Masahiko Sawada wrote:
> >
> > On Thu, Jan 12, 2023 at 5:21 PM John Naylor
> > wrote:
> > >
> > > Okay, I'll squash the previous patch and work on cleaning up the
&g
On Fri, Dec 23, 2022 at 4:33 PM John Naylor
wrote:
>
>
>
> On Thu, Dec 22, 2022 at 10:00 PM Masahiko Sawada
> wrote:
>
> > If the value is a power of 2, it seems to work perfectly fine. But for
> > example if it's 700MB, the total memory exceeds the limit:
&
ync workers then
> > we have a separate column leader_sync_pid? I think that doesn't sound like a
> > good idea and moreover one can refer to docs for clarification.
>
> I agree that leader_pid would be better not only for future parallel copy
> sync feature,
> but also it
ions like a parallel
> vacuum, so I don't see a reason to not do the same for parallel apply
> workers.
+1
The parallel apply workers have different properties than the parallel
query workers since they execute different transactions and don't use
group locking but it would be a good h
On Thu, Jan 12, 2023 at 5:21 PM John Naylor
wrote:
>
> On Thu, Jan 12, 2023 at 12:44 PM Masahiko Sawada
> wrote:
> >
> > On Wed, Jan 11, 2023 at 12:13 PM John Naylor
> > wrote:
> > >
> > > On Tue, Jan 10, 2023 at 7:08 PM Masahiko Sawada
> >
On Wed, Jan 11, 2023 at 12:13 PM John Naylor
wrote:
>
> On Tue, Jan 10, 2023 at 7:08 PM Masahiko Sawada wrote:
>
> > It looks no problem in terms of vacuum integration, although I've not
> > fully tested yet. TID store uses the radix tree as the main storage,
> > and w
ReplicationSlot. I have been looking around the slot code to see if
> > there are other inconsistencies, and did not notice anything standing
> > out. Will fix..
>
> And done, thanks!
Thank you!
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
t;data.two_phase_at = ctx->reader->EndRecPtr;
SpinLockRelease(>mutex);
It seems to me an oversight of commit a8fd13cab0b. I've attached the
small patch to fix it.
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
fix_spinlock.patch
Description: Binary data
ation/simplification for growing into node125
> (my version needs an assertion or fallback, but works well now), found by
> another reading of Andres' prototype There is a lot of good engineering
> there, we should try to preserve it.
Agreed.
Regards,
[1] https://db.in.tum.de/~leis/papers/artsync.pdf
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
On Fri, Jan 6, 2023 at 9:00 PM Amit Kapila wrote:
>
> On Thu, Jan 5, 2023 at 11:46 AM Masahiko Sawada wrote:
> >
> > Agreed. Attached the updated patch.
> >
>
> Thanks, the patch looks good to me. I think it would be probably good
> to backpatch this but i
les, but not the beginning of the analyze.
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
is very welcome.
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
v1-0001-Add-progress-reporting-to-pg_verifybackup.patch
Description: Binary data
nup and
reset them at the end. That is, these values are valid only in index
vacuum phase and index cleanup phase. Otherwise, 0.
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
On Wed, Jan 4, 2023 at 6:42 PM Amit Kapila wrote:
>
> On Wed, Jan 4, 2023 at 12:16 PM Masahiko Sawada wrote:
> >
> > It seems to be confusing and the user won't get the result even if
> > they search it by transactionid = 741. So I've attached the patch to
&g
---+--+--+--+---++---+-+---+--+++---+-+--+---
spectoken | | | | ||
746 | | 1 | | 3/4| 535618 |
ExclusiveLock | t | f|
(1 row)
Regards,
--
Masahiko
On Wed, Jan 4, 2023 at 2:31 PM Masahiko Sawada wrote:
>
> On Tue, Jan 3, 2023 at 2:40 PM wangw.f...@fujitsu.com
> wrote:
> >
> > On Mon, Jan 2, 2023 at 18:54 PM Amit Kapila wrote:
> > > On Fri, Dec 30, 2022 at 3:55 PM wangw.f...@fujitsu.com
> > &g
* worker.
+ */
+static void
+HandleParallelApplyMessage(ParallelApplyWorkerInfo *winfo, StringInfo msg)
In addition, the same is true for 'winfo'.
The rest looks good to me.
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
On Tue, Dec 27, 2022 at 2:24 PM John Naylor
wrote:
>
> On Tue, Dec 27, 2022 at 12:14 AM Masahiko Sawada
> wrote:
> >
> > On Fri, Dec 23, 2022 at 8:47 PM John Naylor
> > wrote:
>
> > These 4 patches make sense to me.We can merge them into 0002 patch
>
&g
On Mon, Dec 26, 2022 at 10:29 PM Amit Kapila wrote:
>
> On Mon, Dec 26, 2022 at 6:33 PM Masahiko Sawada wrote:
> >
> > ---
> > +if (!pa_can_start(xid))
> > +return;
> > +
> > +/* First time through, initialize
you
think about how we can expand this template method to deal with DSA
memory? I imagined that we load say radixtree_template.h with some
macros to use the radix tree like we do for simplehash.h. And
radixtree_template.h further loads xxx_impl.h files for some internal
functions.
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
rker));
+}
How about using !am_tablesync_worker() instead of
!OidIsValid(MyLogicalRepWorker->relid) for better readability?
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
below part of the code in ReorderBufferStreamTXN:
> > if (txn->snapshot_now == NULL)
> > {
> > ...
> > dlist_foreach(subxact_i, >subtxns)
> > {
> > ReorderBufferTXN *subtxn;
> >
> > subtxn = dlist_container(ReorderBufferTXN, node, subxact_i.c
any changes? Is there any reason we
want to change 017 in a separate patch?
> And 017* is not modified because streaming transactions and
> non-streaming transactions are tested alternately in this test.
How about 029_on_error.pl? It also sets logical_decoding_work_mem to
64kb to test the STREAM COMMIT case.
>
> I collected the time to run these tests before and after applying the patch
> set
> on my machine. In debug version, it saves about 5.3 s; and in release version,
> it saves about 1.8 s. The time of each test is attached.
Nice improvements.
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
On Thu, Dec 22, 2022 at 6:19 PM Amit Kapila wrote:
>
> On Thu, Dec 22, 2022 at 12:47 PM Masahiko Sawada
> wrote:
> >
> > On Thu, Dec 22, 2022 at 4:05 PM Amit Kapila wrote:
> > >
> >
> > > I think
> > > instead of adding new tests
On Fri, Dec 23, 2022 at 12:21 PM Amit Kapila wrote:
>
> On Thu, Dec 22, 2022 at 6:18 PM Masahiko Sawada wrote:
> >
> > On Thu, Dec 22, 2022 at 7:04 PM Amit Kapila wrote:
> > >
> > > On Thu, Dec 22, 2022 at 11:39 AM Masahiko Sawada
> > > wrote:
>
On Thu, Dec 22, 2022 at 7:24 PM John Naylor
wrote:
>
>
> On Wed, Dec 21, 2022 at 3:09 PM Masahiko Sawada wrote:
> >
> > On Tue, Dec 20, 2022 at 3:09 PM John Naylor
> > wrote:
>
> > > https://www.postgresql.org/message-id/20220704211822.kfxtzpcdmslzm2
On Thu, Dec 22, 2022 at 7:04 PM Amit Kapila wrote:
>
> On Thu, Dec 22, 2022 at 11:39 AM Masahiko Sawada
> wrote:
> >
> > Thank you for updating the patch. Here are some comments on v64 patches:
> >
> > While testing the patch, I realized that if all stre
On Wed, Dec 21, 2022 at 2:44 AM Imseih (AWS), Sami wrote:
>
> Attached is a patch to check scanned pages rather
> than blockno.
Thank you for the patch. It looks good to me.
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
not break this design. For example, we would need to have a
callback for index scan loop so that the caller (i.e. lazy vacuum) can
do its work.
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
Or considering the name of
logical_decoding_mode, we might want to have a non-empty string, say
'normal' as Kuroda-san proposed, as the default value.
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
On Thu, Dec 22, 2022 at 4:05 PM Amit Kapila wrote:
>
> On Wed, Dec 21, 2022 at 7:25 PM Masahiko Sawada wrote:
> >
> > On Wed, Dec 21, 2022 at 10:14 PM shiy.f...@fujitsu.com
> > wrote:
> >
> > The patch looks good to me. Some minor comments are:
> >
>
return;
+
+elog(ERROR, "invalid pending streaming block number");
+}
I think it's helpful if the error message shows the invalid block number.
---
On Wed, Dec 7, 2022 at 10:13 PM houzj.f...@fujitsu.com
wrote
On Wed, Dec 21, 2022 at 10:14 PM shiy.f...@fujitsu.com
wrote:
>
> On Wed, Dec 21, 2022 4:54 PM Amit Kapila wrote:
> >
> > On Wed, Dec 21, 2022 at 1:55 PM Peter Smith
> > wrote:
> > >
> > > On Wed, Dec 21, 2022 at 6:22 PM Masahiko Sawada
> > wro
On Tue, Dec 20, 2022 at 3:09 PM John Naylor
wrote:
>
>
> On Mon, Dec 19, 2022 at 2:14 PM Masahiko Sawada wrote:
> >
> > On Tue, Dec 13, 2022 at 1:04 AM Masahiko Sawada
> > wrote:
>
> > > Looking at other code using DSA such as tidbitmap.c and nodeHash.c
r the minimum value of
logical_decoding_work_mem to 0. I agree it's helpful for testing and
debugging but setting logical_decoding_work_mem = 0 doesn't benefit
users at all, rather brings risks.
I prefer the idea Kuroda-san previously proposed; setting
logical_decoding_mode = 'immediate' means setting
logical_decoding_work_mem = 0. We might not need to have it as an enum
parameter since it has only two values, though.
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
On Fri, Dec 16, 2022 at 12:49 AM Tom Lane wrote:
>
> Masahiko Sawada writes:
> > I don't think we need additional PG_TRY() for that since exec_stmts()
> > is already called in PG_TRY() if there is an exception block. I meant
> > to call stmt_end() in PG_CATCH() in exec
On Thu, Dec 15, 2022 at 12:28 PM houzj.f...@fujitsu.com
wrote:
>
> On Wednesday, December 14, 2022 2:49 PM Amit Kapila
> wrote:
>
> >
> > On Wed, Dec 14, 2022 at 9:50 AM houzj.f...@fujitsu.com
> > wrote:
> > >
> > > On Tuesday, Decem
On Thu, Dec 15, 2022 at 4:53 PM Kyotaro Horiguchi
wrote:
>
> At Thu, 15 Dec 2022 08:41:21 +0100, Pavel Stehule
> wrote in
> > čt 15. 12. 2022 v 8:25 odesílatel Masahiko Sawada
> > napsal:
> > > Is this a bug in plpgsql?
> > >
> >
> >
/master/pg_hint_plan.c#L4870
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
test_plpgsql.patch
Description: Binary data
On Wed, Dec 14, 2022 at 3:48 PM Amit Kapila wrote:
>
> On Wed, Dec 14, 2022 at 9:50 AM houzj.f...@fujitsu.com
> wrote:
> >
> > On Tuesday, December 13, 2022 11:25 PM Masahiko Sawada
> > wrote:
> > >
> > > Here are comments on v59 0001,
On Wed, Dec 14, 2022 at 1:20 PM houzj.f...@fujitsu.com
wrote:
>
> On Tuesday, December 13, 2022 11:25 PM Masahiko Sawada
> wrote:
> >
> > On Sun, Dec 11, 2022 at 8:45 PM houzj.f...@fujitsu.com
> > wrote:
> > >
> > > On Friday, De
umcleanup() or a common index scan routine like
btvacuumscan() checks the progress. I don't think index AM needs to
pass the value down to many of its functions. So it makes sense to me
to pass it via IndexVacuumInfo.
Having said that, I'd like to hear opinions also from other hackers, I
might be wrong and
rker() immediately after sending
STREAM_ABORT?
---
PA acquires the transaction lock in AccessShare mode whereas LA
acquires it in AccessExclusiveMode. Is it better to do the opposite?
Like a backend process acquires a lock on its XID in Exclusive mode,
we can have PA acquire the lock on its XID in Exclusive mode whereas
other attempts to acquire it in Share mode to wait.
---
void
pa_lock_stream(TransactionId xid, LOCKMODE lockmode)
{
LockApplyTransactionForSession(MyLogicalRepWorker->subid, xid,
PARALLEL_APPLY_LOCK_STREAM, lockmode);
}
I think since we don't need to let the caller to specify the lock mode
but need only shared and exclusive modes, we can make it simple by
having a boolean argument say shared instead of lockmode.
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
On Mon, Dec 12, 2022 at 7:14 PM John Naylor
wrote:
>
>
> On Fri, Dec 9, 2022 at 8:33 PM Masahiko Sawada wrote:
> >
> > On Fri, Dec 9, 2022 at 5:53 PM John Naylor
> > wrote:
> > >
>
> > > I don't think that'd be very controversial, but I'm a
On Fri, Dec 9, 2022 at 5:53 PM John Naylor wrote:
>
>
> On Fri, Dec 9, 2022 at 8:20 AM Masahiko Sawada wrote:
>
> > In the meanwhile, I've been working on vacuum integration. There are
> > two things I'd like to discuss some time:
> >
> > The first is the m
On Fri, Dec 9, 2022 at 3:05 PM Amit Kapila wrote:
>
> On Fri, Dec 9, 2022 at 7:45 AM Peter Smith wrote:
> >
> > On Thu, Dec 8, 2022 at 7:43 PM Masahiko Sawada
> > wrote:
> > >
> > > On Thu, Dec 8, 2022 at 4:42 PM Amit Kapila
> > > wro
On Tue, Dec 6, 2022 at 7:32 PM John Naylor wrote:
>
> On Fri, Dec 2, 2022 at 11:42 PM Masahiko Sawada wrote:
> >
> > > On Mon, Nov 14, 2022 at 7:59 PM John Naylor
> > > wrote:
> > > >
> > > > - Optimize node128 insert.
> > >
> &
On Thu, Dec 8, 2022 at 4:42 PM Amit Kapila wrote:
>
> On Thu, Dec 8, 2022 at 12:42 PM Masahiko Sawada wrote:
> >
> > On Wed, Dec 7, 2022 at 10:03 PM houzj.f...@fujitsu.com
> > wrote:
> > >
> > >
> > > > +st
On Wed, Dec 7, 2022 at 10:03 PM houzj.f...@fujitsu.com
wrote:
>
> On Wednesday, December 7, 2022 7:51 PM Masahiko Sawada
> wrote:
> >
> > On Mon, Dec 5, 2022 at 1:29 PM houzj.f...@fujitsu.com
> > wrote:
> > >
> > > On Sunday, De
On Thu, Dec 8, 2022 at 1:52 PM Amit Kapila wrote:
>
> On Wed, Dec 7, 2022 at 6:33 PM houzj.f...@fujitsu.com
> wrote:
> >
> > On Wednesday, December 7, 2022 7:51 PM Masahiko Sawada
> > wrote:
> > >
> >
> > > ---
> > > When max_p
ssion.
Regards,
[1]
https://www.postgresql.org/message-id/tencent_7EB71DA5D7BA00EB0B429DCE45D0452B6406%40qq.com
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
fix_slot_xmin_race_condition.patch
Description: Binary data
On Wed, Dec 7, 2022 at 4:31 PM Amit Kapila wrote:
>
> On Wed, Dec 7, 2022 at 10:10 AM Masahiko Sawada wrote:
> >
> > On Wed, Dec 7, 2022 at 1:29 PM Amit Kapila wrote:
> > >
> > > Right, but the leader will anyway exit at some point either due to
Command(); /* Completes the preceding Begin command. */
+ if (!IsTransactionBlock())
+ {
+ BeginTransactionBlock();
+ CommitTransactionCommand(); /* Completes the preceding
Begin command. */
+ }
Do we need this change? In my environment, 'make check-world' passes
without this change.
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
On Wed, Dec 7, 2022 at 12:55 PM Amit Kapila wrote:
>
> On Wed, Dec 7, 2022 at 7:31 AM Masahiko Sawada wrote:
> >
> > On Wed, Dec 7, 2022 at 8:46 AM Peter Smith wrote:
> > >
> > > >
> > > > Yeah, I think this can also help in reducing the tim
On Wed, Dec 7, 2022 at 1:29 PM Amit Kapila wrote:
>
> On Wed, Dec 7, 2022 at 9:00 AM Masahiko Sawada wrote:
> >
> > On Thu, Dec 1, 2022 at 7:17 PM houzj.f...@fujitsu.com
> > wrote:
> > >
> > > > ---
> > >
On Thu, Dec 1, 2022 at 7:17 PM houzj.f...@fujitsu.com
wrote:
>
> On Thursday, December 1, 2022 3:58 PM Masahiko Sawada
> wrote:
> >
> > On Wed, Nov 30, 2022 at 10:51 PM houzj.f...@fujitsu.com
> > wrote:
> > >
> > > On Wednesday, November 30, 2
but probably the developer
option for testing the parallel apply feature would be PGC_SIGHUP.
Also, since streaming changes is not specific to logical replication
but to logical decoding, I'm not sure logical_replication_XXX is a
good name. IMO having force_stream_mode and a different GUC for
testing the parallel apply feature makes sense to me.
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
tend it for subscriber-side testing
> then we can introduce new options like client_serialize for the case
> being discussed in the email [1].
Setting logical_replication_mode = 'client_serialize' implies that the
publisher behaves as server_stream? or do you mean we can set like
logical_replication_mode = 'server_stream, client_serialize'?
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
EATH, PARALLEL_VACUUM_PROGRESS_TIMEOUT,
+
WAIT_EVENT_PARALLEL_VACUUM_FINISH);
+ResetLatch(MyLatch);
I think we don't necessarily need to use
PARALLEL_VACUUM_PROGRESS_TIMEOUT here. Probably we can use 1000L
instead. If we want to use PARALLEL_VACUUM_PROGRESS_TIMEOUT, we need
comments for th
On Thu, Dec 1, 2022 at 4:00 PM John Naylor wrote:
>
>
> On Wed, Nov 30, 2022 at 11:09 PM Masahiko Sawada
> wrote:
> >
> > I've investigated this issue and have a question about using atomic
> > variables on palloc'ed memory. In non-parallel vacuum cases,
> >
ransactionCommand();
+ReplicationOriginNameForLogicalRep(MySubscription->oid, InvalidOid,
+
originname, sizeof(originname));
+originid = replorigin_by_name(originname, true);
+if (!OidIsValid(originid))
+originid = replorigin_create(originname);
This code looks to allow parallel workers to use different origins in
cases where the origin doesn't exist, but is that okay? Shouldn't we
pass miassing_ok = false in this case?
---
cfbot seems to fails:
https://cirrus-ci.com/task/6264595342426112
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
d only after commit the parallel apply worker would be
> allowed to apply it?
+1
The code coverage report shows that we don't cover the partial
serialization codes. This GUC would improve the code coverage.
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
On Wed, Nov 23, 2022 at 2:10 AM Andres Freund wrote:
>
> On 2022-11-21 17:06:56 +0900, Masahiko Sawada wrote:
> > Sure. I've attached the v10 patches. 0004 is the pure refactoring
> > patch and 0005 patch introduces the pointer tagging.
>
> This failed on cfbot, with som
intend to replace this code with something faster, but before I do so
> the tests should probably exercise the deletion paths more. Since VACUUM
Indeed, there are some tests for deletion but all of them delete all
keys in the node so we end up deleting the node. I've added tests of
repeating deletion and insertion as well as additional assertions.
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
On Fri, Nov 25, 2022 at 6:47 PM John Naylor
wrote:
>
>
>
> On Thu, Nov 24, 2022 at 9:54 PM Masahiko Sawada wrote:
> >
> > [v11]
>
> There is one more thing that just now occurred to me: In expanding the use of
> size classes, that makes rebasing and rewor
On Fri, Nov 25, 2022 at 5:00 PM John Naylor
wrote:
>
> On Thu, Nov 24, 2022 at 9:54 PM Masahiko Sawada wrote:
> >
> > So it seems that there are two candidates of rt_node structure: (1)
> > all nodes except for node256 are variable-size nodes and use pointer
>
> > objections.
>
> And done.
Thank you!
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
04237.edtahvrwb3uwd5rs%40alap3.anarazel.de
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
fix_comment_in_SnapBuildFindSnapshot.patch
Description: Binary data
On Fri, Nov 25, 2022 at 5:58 PM Maxim Orlov wrote:
>
>
>
> On Fri, 25 Nov 2022 at 09:40, Amit Kapila wrote:
>>
>> On Thu, Nov 24, 2022 at 4:43 PM Amit Kapila wrote:
>> >
>> > On Thu, Nov 24, 2022 at 1:48 PM Masahiko Sawada
>> > wrote:
>>
on here.
> > Let's add tests in a separate commit and let the actual committer to decide
> > what to do, should we?
> >
>
> +1 to not have a test for this as the scenario can already be tested
> by the existing set of tests.
Agreed not to have a test case for this.
I've attac
ts, the patch includes a new scenario to
reproduce this issue. However, since the issue can be reproduced also
by the existing scenario (with low probability, though), I'm not sure
it's worth adding the new scenario.
I've not checked if the patch works for version 14 or older yet.
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
reset_initial_running_xacts.patch
Description: Binary data
nly in variable-sized nodes,
rt_node has only count, shift, and chunk, so 4 bytes in total. If so,
the size of node3 (ie. fixed-sized node) is (4 + 3 + (1) + 3*8)? The
size doesn't change but there is 1 byte padding space.
Also, even if we have the node3 a variable-sized node, size class 1
for node3 could be a good choice since it also doesn't need padding
space and could be a good alternative to path compression.
node3 : 5 + 3 + 3*8 = 32 bytes
size class 1 : 5 + 3 + 1*8 = 16 bytes
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
On Thu, Nov 17, 2022 at 12:24 AM Masahiko Sawada wrote:
>
> On Wed, Nov 16, 2022 at 4:39 PM John Naylor
> wrote:
> >
> >
> > On Wed, Nov 16, 2022 at 12:33 PM Masahiko Sawada
> > wrote:
> > >
> > > On Wed, Nov 16, 2022 at 1:46 PM John Naylor
&
AM
call this function.
---
+if (!IsParallelWorker())
+ivinfo.report_parallel_progress = true;
+else
+ivinfo.report_parallel_progress = false;
We can do like:
ivinfo.report_parallel_progress = !IsParallelWorker();
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
501 - 600 of 2753 matches
Mail list logo
