> On 29 Mar 2021, at 15:36, David Steele wrote:
> A rebase is also required so marked Waiting for Author.
Many months on and this patch still needs a rebase to apply, and the thread has
stalled. I'm marking this Returned with Feedback. Please feel free to open a
new entry if you return to
naka; robertmhaas; pgsql
Subject: Re: Re: parallel distinct union and aggregate support patch
On Tue, 30 Mar 2021 at 22:33, bu...@sohu.com wrote:
> I have written a plan with similar functions, It is known that the following
> two situations do not work well.
I read through this thread an
On Tue, 30 Mar 2021 at 22:33, bu...@sohu.com wrote:
> I have written a plan with similar functions, It is known that the following
> two situations do not work well.
I read through this thread and also wondered about a Parallel
Partition type operator. It also seems to me that if it could be
> This patch has not gotten any review in the last two CFs and is unlikely
> to be committed for PG14 so I have moved it to the 2021-07 CF. A rebase
> is also required so marked Waiting for Author.
>
> I can see this is a work in progress, but you may want to consider the
> several suggestions
On 1/25/21 9:14 AM, bu...@sohu.com wrote:
Now, I rewrite batch hashagg and sort, add some comment and combin too
patches. base on master 2ad78a87f018260d4474eee63187e1cc73c9b976.
They are support rescan and change GUC
enable_batch_hashagg/enable_batch_sort to
> 1.
> +#define BATCH_SORT_MAX_BATCHES 512
>
> Did you decide this number based on some experiment or is there some
> analysis behind selecting this number?
When there are too few batches, if a certain process works too slowly, it will
cause unbalanced load.
When there are too many batches, FD
On Fri, Nov 27, 2020 at 9:25 PM Heikki Linnakangas wrote:
>
> I also had a quick look at the patch and the comments made so far. Summary:
>
> 1. The performance results are promising.
>
> 2. The code needs comments.
>
> Regarding the design:
>
> Thomas Munro mentioned the idea of a "Parallel
On Fri, Nov 27, 2020 at 10:55 AM Heikki Linnakangas wrote:
> I think a non-buffering Reparttion node would be simpler, and thus
> better. In these patches, you have a BatchSort node, and batchstore, but
> a simple Parallel Repartition node could do both. For example, to
> implement distinct:
>
>
I also had a quick look at the patch and the comments made so far. Summary:
1. The performance results are promising.
2. The code needs comments.
Regarding the design:
Thomas Munro mentioned the idea of a "Parallel Repartition" node that
would redistribute tuples like this. As I understand
On Sun, Nov 8, 2020 at 11:54 AM Dilip Kumar wrote:
>
> On Tue, Nov 3, 2020 at 6:06 PM Dilip Kumar wrote:
> >
> > On Thu, Oct 29, 2020 at 12:53 PM bu...@sohu.com wrote:
> > >
> > > > 1) It's better to always include the whole patch series - including the
> > > > parts that have not changed.
On Tue, Nov 3, 2020 at 6:06 PM Dilip Kumar wrote:
>
> On Thu, Oct 29, 2020 at 12:53 PM bu...@sohu.com wrote:
> >
> > > 1) It's better to always include the whole patch series - including the
> > > parts that have not changed. Otherwise people have to scavenge the
> > > thread and search for all
On Thu, Oct 29, 2020 at 12:53 PM bu...@sohu.com wrote:
>
> > 1) It's better to always include the whole patch series - including the
> > parts that have not changed. Otherwise people have to scavenge the
> > thread and search for all the pieces, which may be a source of issues.
> > Also, it
> 1) It's better to always include the whole patch series - including the
> parts that have not changed. Otherwise people have to scavenge the
> thread and search for all the pieces, which may be a source of issues.
> Also, it confuses the patch tester [1] which tries to apply patches from
> a
Hi,
On Wed, Oct 28, 2020 at 05:37:40PM +0800, bu...@sohu.com wrote:
Hi
Here is patch for parallel distinct union aggregate and grouping sets support
using batch hash agg.
Please review.
how to use:
set enable_batch_hashagg = on
how to work:
like batch sort, but not sort each batch, just save
batch store module.
By the way, use atomic operations instead LWLock for shared tuple store get
next read page.
patch 4:
using batch hash agg support parallels
发件人： bu...@sohu.com
发送时间： 2020-10-19 22:42
收件人： pgsql-hackers
主题： parallel distinct union and aggregate support patch
Hi hackers,
I
> On Tue, Oct 27, 2020 at 3:27 PM Dilip Kumar wrote:
> >
> > On Fri, Oct 23, 2020 at 11:58 AM bu...@sohu.com wrote:
> > >
> > > > Interesting idea. So IIUC, whenever a worker is scanning the tuple it
> > > > will directly put it into the respective batch(shared tuple store),
> > > > based on
On Tue, Oct 27, 2020 at 5:43 PM Robert Haas wrote:
>
> On Thu, Oct 22, 2020 at 5:08 AM Dilip Kumar wrote:
> > Interesting idea. So IIUC, whenever a worker is scanning the tuple it
> > will directly put it into the respective batch(shared tuple store),
> > based on the hash on grouping column
On Tue, Oct 27, 2020 at 3:27 PM Dilip Kumar wrote:
>
> On Fri, Oct 23, 2020 at 11:58 AM bu...@sohu.com wrote:
> >
> > > Interesting idea. So IIUC, whenever a worker is scanning the tuple it
> > > will directly put it into the respective batch(shared tuple store),
> > > based on the hash on
On Thu, Oct 22, 2020 at 5:08 AM Dilip Kumar wrote:
> Interesting idea. So IIUC, whenever a worker is scanning the tuple it
> will directly put it into the respective batch(shared tuple store),
> based on the hash on grouping column and once all the workers are
> doing preparing the batch then
On Fri, Oct 23, 2020 at 11:58 AM bu...@sohu.com wrote:
>
> > Interesting idea. So IIUC, whenever a worker is scanning the tuple it
> > will directly put it into the respective batch(shared tuple store),
> > based on the hash on grouping column and once all the workers are
> > doing preparing the
> Interesting idea. So IIUC, whenever a worker is scanning the tuple it
> will directly put it into the respective batch(shared tuple store),
> based on the hash on grouping column and once all the workers are
> doing preparing the batch then each worker will pick those baches one
> by one,
plan could break", mean some on path
using this path? no, BathSortPath on for some special path(Unique, GroupAgg
...).
bu...@sohu.com
From: Thomas Munro
Date: 2020-10-21 12:27
To: bu...@sohu.com
CC: pgsql-hackers
Subject: Re: parallel distinct union and aggregate support patch
On Tue
On Mon, Oct 19, 2020 at 8:19 PM bu...@sohu.com wrote:
>
> Hi hackers,
> I write a path for soupport parallel distinct, union and aggregate using
> batch sort.
> steps:
> 1. generate hash value for group clauses values, and using mod hash value
> save to batch
> 2. end of outer plan, wait all
On Tue, Oct 20, 2020 at 3:49 AM bu...@sohu.com wrote:
> I write a path for soupport parallel distinct, union and aggregate using
> batch sort.
> steps:
> 1. generate hash value for group clauses values, and using mod hash value
> save to batch
> 2. end of outer plan, wait all other workers
Hi hackers,
I write a path for soupport parallel distinct, union and aggregate using batch
sort.
steps:
1. generate hash value for group clauses values, and using mod hash value save
to batch
2. end of outer plan, wait all other workers finish write to batch
3. echo worker get a unique batch
25 matches
Mail list logo
