Re: 回复: An implementation of multi-key sort

2024-08-10 Thread John Naylor
On Fri, Jul 26, 2024 at 6:18 PM Yao Wang wrote: > 2. Should we update optimizer to take in account statistic > (pg_statistic->stadistinct) and choose mksort/qsort accordingly? > According to the result, the choosing mechanism in optimizer > almost eliminated all regressions. Please note that eve

Re: 回复: An implementation of multi-key sort

2024-07-09 Thread Robert Haas
On Sun, Jul 7, 2024 at 2:32 AM Konstantin Knizhnik wrote: > If mksort really provides advantage only when there are a lot of > duplicates (for prefix keys?) and of small fraction of duplicates there > is even some (small) regression > then IMHO taking in account in planner information about estima

Re: 回复: An implementation of multi-key sort

2024-07-09 Thread Tomas Vondra
BTW I forgot to report that I intended to test this on 32-bit ARM too, because that sometimes triggers "funny" behavior, but the build fails like this: In file included from tuplesort.c:630: mk_qsort_tuple.c: In function ‘mkqs_compare_datum_by_shortcut’: mk_qsort_tuple.c:167:23: warning: implicit

Re: 回复: An implementation of multi-key sort

2024-07-08 Thread Tomas Vondra
On 7/7/24 08:32, Konstantin Knizhnik wrote: > > On 04/07/2024 3:45 pm, Yao Wang wrote: >> Generally, the benefit of mksort is mainly from duplicated values and >> sort >> keys: the more duplicated values and sort keys are, the bigger benefit it >> gets. > ... >> 1. Use distinct stats info of ta

Re: 回复: An implementation of multi-key sort

2024-07-08 Thread Tomas Vondra
On 7/4/24 14:45, Yao Wang wrote: > Hi John, > > Thanks for your kind message. I talked to Heikki before getting Tomas's > response, and he said "no promise but I will take a look". That's why I > added his email. I have updated the CF entry and added Tomas as reviewer. > > Hi Tomas, > > Again

Re: 回复: An implementation of multi-key sort

2024-07-06 Thread Konstantin Knizhnik
On 04/07/2024 3:45 pm, Yao Wang wrote: Generally, the benefit of mksort is mainly from duplicated values and sort keys: the more duplicated values and sort keys are, the bigger benefit it gets. ... 1. Use distinct stats info of table to enable mksort It's kind of heuristics: in optimizer, ch

Re: 回复: An implementation of multi-key sort

2024-06-20 Thread John Naylor
On Fri, Jun 14, 2024 at 6:20 PM Yao Wang wrote: > > hi Tomas, > > So many thanks for your kind response and detailed report. I am working > on locating issues based on your report/script and optimizing code, and > will update later. Hi, This is an interesting proof-of-concept! Given the above, I

Re: 回复: An implementation of multi-key sort

2024-06-14 Thread Tomas Vondra
On 6/14/24 13:20, Yao Wang wrote: > hi Tomas, > > So many thanks for your kind response and detailed report. I am working > on locating issues based on your report/script and optimizing code, and > will update later. > > Could you please also send me the script to generate report pdf > from th

Re: 回复: An implementation of multi-key sort

2024-06-14 Thread Yao Wang
hi Tomas, So many thanks for your kind response and detailed report. I am working on locating issues based on your report/script and optimizing code, and will update later. Could you please also send me the script to generate report pdf from the test results (explain*.log)? I can try to make one

Re: 回复: An implementation of multi-key sort

2024-06-07 Thread Yao Wang
ect c1 from t2 order by c6, c5, c4, c3, c2, c1; > > > > Results: > > > > MKsort: > > 12374.427 ms > > 12528.068 ms > > 12554.718 ms > > > > qsort: > > 12251.422 ms > > 12279.938 ms > > 12280.254 ms > > > > MKsort is a

Re: 回复: An implementation of multi-key sort

2024-05-31 Thread Yao Wang
> > > > 获取Outlook for Android > > ________ > > From: Heikki Linnakangas > > Sent: Thursday, May 23, 2024 8:47:29 PM > > To: Wang Yao ; PostgreSQL Hackers > > > > Cc: inte...@outlook.com > > Subject: Re: 回复: An imple

Re: 回复: An implementation of multi-key sort

2024-05-24 Thread Yao Wang
6 PM Wang Yao wrote: > > > > 获取Outlook for Android > > From: Heikki Linnakangas > Sent: Thursday, May 23, 2024 8:47:29 PM > To: Wang Yao ; PostgreSQL Hackers > > Cc: inte...@outlook.com > Subject: Re: 回复: An implementation of m

Re: 回复: An implementation of multi-key sort

2024-05-23 Thread Heikki Linnakangas
On 23/05/2024 15:39, Wang Yao wrote: No obvious perf regression is expected because PG will follow original qsort code path when mksort is disabled. For the case, the only extra cost is the check in tuplesort_sort_memtuples() to enter mksort code path. And what about the case the mksort is enab

回复: An implementation of multi-key sort

2024-05-23 Thread Wang Yao
g Yao ; PostgreSQL Hackers 抄送: inte...@outlook.com 主题: Re: An implementation of multi-key sort On 22/05/2024 15:48, Wang Yao wrote: > Comparing to classic quick sort, it can get significant performance > improvement once multiple keys are available. A rough test shows it got > ~129% improvement

Re: An implementation of multi-key sort

2024-05-22 Thread Heikki Linnakangas
On 22/05/2024 15:48, Wang Yao wrote: Comparing to classic quick sort, it can get significant performance improvement once multiple keys are available. A rough test shows it got ~129% improvement than qsort for ORDER BY on 6 keys, and ~52% for CREATE INDEX on the same data set. (See more details i

An implementation of multi-key sort

2024-05-22 Thread Wang Yao
Hi hackers, I'd submit an implementation of multi-key sort for review. Please see the code as attachment. Thanks for your reponse in advance. Overview MKsort (multi-key sort) is an alternative of standard qsort algorithm, which has better performance for particular sort scenarios