Re: Some performance degradation in REL_16 vs REL_15

2023-11-15 Thread Andres Freund
Hi, On 2023-11-15 10:09:06 -0500, Tom Lane wrote: > "Anton A. Melnikov" writes: > > I can't understand why i get the opposite results on my pc and on the > > server. It is clear that the absolute > > TPS values will be different for various configurations. This is normal. > > But differences?

Re: Some performance degradation in REL_16 vs REL_15

2023-11-15 Thread Andres Freund
Hi, On 2023-11-15 11:33:44 +0300, Anton A. Melnikov wrote: > The configure options and test scripts on my pc and server were the same: > export CFLAGS="-O2" > ./configure --enable-debug --with-perl --with-icu --enable-depend > --enable-tap-tests > #reinstall > #reinitdb > #create database bench

Re: Some performance degradation in REL_16 vs REL_15

2023-11-15 Thread Tom Lane
"Anton A. Melnikov" writes: > I can't understand why i get the opposite results on my pc and on the server. > It is clear that the absolute > TPS values will be different for various configurations. This is normal. But > differences? > Is it unlikely that some kind of reference configuration is

Re: Some performance degradation in REL_16 vs REL_15

2023-11-15 Thread Anton A. Melnikov
On 30.10.2023 22:51, Andres Freund wrote: There's really no point in comparing peformance with assertions enabled (leaving aside assertions that cause extreme performance difference, making development harder). We very well might have added assertions making things more expensive, without

Re: Some performance degradation in REL_16 vs REL_15

2023-10-30 Thread Andres Freund
Hi, On 2023-10-30 15:28:53 +0300, Anton A. Melnikov wrote: > For REL_16_STABLE at 7cc2f59dd the average TPS was: 2020+-70, > for REL_10_STABLE at c18c12c98 - 2260+-70 > > The percentage difference was approximately 11%. > Please see the 16vs10.png picture with the graphical representation of the

Re: Some performance degradation in REL_16 vs REL_15

2023-10-17 Thread Tom Lane
=?utf-8?B?6YKx5a6H6Iiq?= writes: > I wrote a script and test on branch REL_[10-16]_STABLE, and do see > performance drop in REL_13_STABLE, which is about 1~2%. I'm really skeptical that we should pay much attention to these numbers. You've made several of the mistakes that we typically tell

Re: Some performance degradation in REL_16 vs REL_15

2023-10-17 Thread Andres Freund
Hi, On 2023-10-16 11:04:25 +0300, Anton A. Melnikov wrote: > On 13.10.2023 05:05, Andres Freund wrote: > > Could you provide a bit more details about how you ran the benchmark? The > > reason I am asking is that ~330 TPS is pretty slow for -c20. Even on > > spinning > > rust and using the

Re: Some performance degradation in REL_16 vs REL_15

2023-10-17 Thread 邱宇航
I wrote a script and test on branch REL_[10-16]_STABLE, and do see performance drop in REL_13_STABLE, which is about 1~2%. scale round 10 11 12 13 14 15 16 1 1 7922.2 8018.3 8102.8 7838.3 7829.2 7870.0 7846.1 2 7922.4 7923.5

Re: Some performance degradation in REL_16 vs REL_15

2023-10-16 Thread Anton A. Melnikov
On 13.10.2023 05:05, Andres Freund wrote: Could you provide a bit more details about how you ran the benchmark? The reason I am asking is that ~330 TPS is pretty slow for -c20. Even on spinning rust and using the default settings, I get considerably higher results. Oh - I do get results

Re: Some performance degradation in REL_16 vs REL_15

2023-10-12 Thread Andres Freund
Hi, On 2023-10-12 11:00:22 +0300, Anton A. Melnikov wrote: > Found that simple test pgbench -c20 -T20 -j8 gives approximately > for REL_15_STABLE at 5143f76: 336+-1 TPS > and > for REL_16_STABLE at 4ac7635f: 324+-1 TPS > > The performance drop is approximately 3,5% while the corrected standard

Re: Some performance degradation in REL_16 vs REL_15

2023-10-12 Thread Michael Paquier
On Thu, Oct 12, 2023 at 09:20:36PM +1300, David Rowley wrote: > It would be interesting to know what's to blame here and if you can > attribute it to a certain commit. +1. -- Michael signature.asc Description: PGP signature

Re: Some performance degradation in REL_16 vs REL_15

2023-10-12 Thread David Rowley
On Thu, 12 Oct 2023 at 21:01, Anton A. Melnikov wrote: > > Greetengs! > > Found that simple test pgbench -c20 -T20 -j8 gives approximately > for REL_15_STABLE at 5143f76: 336+-1 TPS > and > for REL_16_STABLE at 4ac7635f: 324+-1 TPS > > And is it worth spending time bisecting for the commit where

Some performance degradation in REL_16 vs REL_15

2023-10-12 Thread Anton A. Melnikov
Greetengs! Found that simple test pgbench -c20 -T20 -j8 gives approximately for REL_15_STABLE at 5143f76: 336+-1 TPS and for REL_16_STABLE at 4ac7635f: 324+-1 TPS The performance drop is approximately 3,5% while the corrected standard deviation is only 0.3%. See the raw_data.txt attached.