Hi,
On 11/13/23 5:24 AM, shveta malik wrote:
On Thu, Nov 9, 2023 at 8:56 AM Amit Kapila <amit.kapil...@gmail.com> wrote:
Apart from the above, I would like to discuss the slot sync work
distribution strategy of this patch. The current implementation as
explained in the commit message [1] works well if the slots belong to
multiple databases. It is clear from the data in emails [2][3][4] that
having more workers really helps if the slots belong to multiple
databases. But I think if all the slots belong to one or very few
databases then such a strategy won't be as good. Now, on one hand, we
get very good numbers for a particular workload with the strategy used
in the patch but OTOH it may not be adaptable to various different
kinds of workloads. So, I have a question whether we should try to
optimize this strategy for various kinds of workloads or for the first
version let's use a single-slot sync-worker and then we can enhance
the functionality in later patches either in PG17 itself or in PG18 or
later versions.
I can work on separating the patch. We can first focus on single
worker design and then we can work on multi-worker design either
immediately (if needed) or we can target it in the second draft of the
patch. I would like to know the thoughts of others on this.
If we need to put more thoughts on the workers distribution strategy
then I also think it's better to focus on a single worker and then
improve/discuss a distribution design later on.
One thing to note is that a lot of the complexity of
the patch is attributed to the multi-worker strategy which may still
not be efficient, so there is an argument to go with a simpler
single-slot sync-worker strategy and then enhance it in future
versions as we learn more about various workloads. It will also help
to develop this feature incrementally instead of doing all the things
in one go and taking a much longer time than it should.
Agreed. With multi-workers, a lot of complexity (dsa, locks etc) have
come into play. We can decide better on our workload distribution
strategy among workers once we have more clarity on different types of
workloads.
Agreed.
Regards,
--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com
