On 30/09/2021 07:16, Anton Ivanov wrote:
Results on a Ryzen 5 3600 - 6 cores 12 threads
I will also have a look into the "maximum" measurement for multi-thread.
It does not tie up with the drop in average across the board.
A.
Without
1: ovn-northd basic scale test -- 200 Hypervisors, 200 Logical
Ports/Hypervisor -- ovn-northd -- dp-groups=yes
---
Maximum (NB in msec): 1256
Average (NB in msec): 679.463785
Maximum (SB in msec): 25
Average (SB in msec): 22.489798
Maximum (northd-loop in msec): 1347
Average (northd-loop in msec): 799.944878
2: ovn-northd basic scale test -- 200 Hypervisors, 200 Logical
Ports/Hypervisor -- ovn-northd -- dp-groups=no
---
Maximum (NB in msec): 1956
Average (NB in msec): 809.387285
Maximum (SB in msec): 24
Average (SB in msec): 21.649258
Maximum (northd-loop in msec): 2011
Average (northd-loop in msec): 961.718686
5: ovn-northd basic scale test -- 500 Hypervisors, 50 Logical
Ports/Hypervisor -- ovn-northd -- dp-groups=yes
---
Maximum (NB in msec): 557
Average (NB in msec): 474.010337
Maximum (SB in msec): 15
Average (SB in msec): 13.927192
Maximum (northd-loop in msec): 1261
Average (northd-loop in msec): 580.999122
6: ovn-northd basic scale test -- 500 Hypervisors, 50 Logical
Ports/Hypervisor -- ovn-northd -- dp-groups=no
---
Maximum (NB in msec): 756
Average (NB in msec): 625.614724
Maximum (SB in msec): 15
Average (SB in msec): 14.181048
Maximum (northd-loop in msec): 1649
Average (northd-loop in msec): 746.208332
With
1: ovn-northd basic scale test -- 200 Hypervisors, 200 Logical
Ports/Hypervisor -- ovn-northd -- dp-groups=yes
---
Maximum (NB in msec): 1140
Average (NB in msec): 631.125000
Maximum (SB in msec): 24
Average (SB in msec): 21.453609
Maximum (northd-loop in msec): 6080
Average (northd-loop in msec): 759.718815
2: ovn-northd basic scale test -- 200 Hypervisors, 200 Logical
Ports/Hypervisor -- ovn-northd -- dp-groups=no
---
Maximum (NB in msec): 1210
Average (NB in msec): 673.000000
Maximum (SB in msec): 27
Average (SB in msec): 22.453125
Maximum (northd-loop in msec): 6514
Average (northd-loop in msec): 808.596842
5: ovn-northd basic scale test -- 500 Hypervisors, 50 Logical
Ports/Hypervisor -- ovn-northd -- dp-groups=yes
---
Maximum (NB in msec): 798
Average (NB in msec): 429.750000
Maximum (SB in msec): 15
Average (SB in msec): 12.998533
Maximum (northd-loop in msec): 3835
Average (northd-loop in msec): 564.875986
6: ovn-northd basic scale test -- 500 Hypervisors, 50 Logical
Ports/Hypervisor -- ovn-northd -- dp-groups=no
---
Maximum (NB in msec): 1074
Average (NB in msec): 593.875000
Maximum (SB in msec): 14
Average (SB in msec): 13.655273
Maximum (northd-loop in msec): 4973
Average (northd-loop in msec): 771.102605
The only one slower is test 6 which I will look into.
The rest are > 5% faster.
A.
On 30/09/2021 00:56, Han Zhou wrote:
On Wed, Sep 15, 2021 at 5:45 AM <[email protected]
<mailto:[email protected]>> wrote:
>
> From: Anton Ivanov <[email protected]
<mailto:[email protected]>>
>
> Restore parallel build with dp groups using rwlock instead
> of per row locking as an underlying mechanism.
>
> This provides improvement ~ 10% end-to-end on ovn-heater
> under virutalization despite awakening some qemu gremlin
> which makes qemu climb to silly CPU usage. The gain on
> bare metal is likely to be higher.
>
Hi Anton,
I am trying to see the benefit of parallel_build, but encountered
unexpected performance result when running the perf tests with command:
make check-perf TESTSUITEFLAGS="--rebuild"
It shows significantly worse performance than without parallel_build.
For dp_group = no cases, it is better, but still ~30% slower than
without parallel_build. I have 24 cores, but each thread is not
consuming much CPU except the main thread. I also tried hardcode the
number of thread to just 4, which end up with slightly better
results, but still far behind "without parallel_build".
no parallel |
parallel (24 pool threads) | parallel with (4 pool
threads)
|
|
1: ovn-northd basic scale test -- 200 Hypervisors, 200 | 1:
ovn-northd basic scale test -- 200 Hypervisors, 200 | 1:
ovn-northd basic scale test -- 200 Hypervisors, 200
--- | --- | ---
Maximum (NB in msec): 1058 | Maximum (NB in msec): 4269 |
Maximum (NB in msec): 4097
Average (NB in msec): 836.941167 | Average (NB in msec):
3697.253931 | Average (NB in msec): 3498.311525
Maximum (SB in msec): 30 | Maximum (SB in msec): 30 |
Maximum (SB in msec): 28
Average (SB in msec): 25.934011 | Average (SB in msec):
26.001840 | Average (SB in msec): 25.685091
Maximum (northd-loop in msec): 1204 | Maximum (northd-loop in
msec): 4379 | Maximum (northd-loop in msec): 4251
Average (northd-loop in msec): 1005.330078 | Average
(northd-loop in msec): 4233.871504 | Average (northd-loop in
msec): 4022.774208
| |
2: ovn-northd basic scale test -- 200 Hypervisors, 200 | 2:
ovn-northd basic scale test -- 200 Hypervisors, 200 | 2:
ovn-northd basic scale test -- 200 Hypervisors, 200
--- | --- | ---
Maximum (NB in msec): 1124 | Maximum (NB in msec): 1480 |
Maximum (NB in msec): 1331
Average (NB in msec): 892.403405 | Average (NB in msec):
1206.189287 | Average (NB in msec): 1089.378455
Maximum (SB in msec): 29 | Maximum (SB in msec): 31 |
Maximum (SB in msec): 30
Average (SB in msec): 26.922632 | Average (SB in msec):
26.636706 | Average (SB in msec): 25.657484
Maximum (northd-loop in msec): 1275 | Maximum (northd-loop in
msec): 1639 | Maximum (northd-loop in msec): 1495
Average (northd-loop in msec): 1074.917873 | Average
(northd-loop in msec): 1458.152327 | Average (northd-loop in
msec): 1301.057201
| |
5: ovn-northd basic scale test -- 500 Hypervisors, 50 L| 5:
ovn-northd basic scale test -- 500 Hypervisors, 50 L| 5:
ovn-northd basic scale test -- 500 Hypervisors, 50
--- | --- | ---
Maximum (NB in msec): 768 | Maximum (NB in msec): 3086 |
Maximum (NB in msec): 2876
Average (NB in msec): 614.491938 | Average (NB in msec):
2681.688365 | Average (NB in msec): 2531.255444
Maximum (SB in msec): 18 | Maximum (SB in msec): 17 |
Maximum (SB in msec): 18
Average (SB in msec): 16.347526 | Average (SB in msec):
15.955263 | Average (SB in msec): 16.278075
Maximum (northd-loop in msec): 889 | Maximum (northd-loop in
msec): 3247 | Maximum (northd-loop in msec): 3031
Average (northd-loop in msec): 772.083572 | Average
(northd-loop in msec): 3117.504297 | Average (northd-loop in
msec): 2833.182361
| |
6: ovn-northd basic scale test -- 500 Hypervisors, 50 L| 6:
ovn-northd basic scale test -- 500 Hypervisors, 50 L| 6:
ovn-northd basic scale test -- 500 Hypervisors, 50
--- | --- | ---
Maximum (NB in msec): 1046 | Maximum (NB in msec): 1371 |
Maximum (NB in msec): 1262
Average (NB in msec): 827.735852 | Average (NB in msec):
1135.514228 | Average (NB in msec): 970.544792
Maximum (SB in msec): 19 | Maximum (SB in msec): 18 |
Maximum (SB in msec): 19
Average (SB in msec): 16.828127 | Average (SB in msec):
16.083914 | Average (SB in msec): 15.602525
Maximum (northd-loop in msec): 1163 | Maximum (northd-loop in
msec): 1545 | Maximum (northd-loop in msec): 1411
Average (northd-loop in msec): 972.567407 | Average
(northd-loop in msec): 1328.617583 | Average (northd-loop in
msec): 1207.667100
I didn't debug yet, but do you have any clue what could be the
reason? I am using the upstream commit 9242f27f63 which already
included this patch.
Below is my change to the perf-northd.at <http://perf-northd.at> file
just to enable parallel_build:
diff --git a/tests/perf-northd.at <http://perf-northd.at>
b/tests/perf-northd.at <http://perf-northd.at>
index 74b69e9d4..9328c2e21 100644
--- a/tests/perf-northd.at <http://perf-northd.at>
+++ b/tests/perf-northd.at <http://perf-northd.at>
@@ -191,6 +191,7 @@ AT_SETUP([ovn-northd basic scale test -- 200
Hypervisors, 200 Logical Ports/Hype
PERF_RECORD_START()
ovn_start
+ovn-nbctl set nb_global . options:use_parallel_build=true
BUILD_NBDB(OVN_BASIC_SCALE_CONFIG(200, 200))
@@ -203,9 +204,10 @@ AT_SETUP([ovn-northd basic scale test -- 500
Hypervisors, 50 Logical Ports/Hyper
PERF_RECORD_START()
ovn_start
+ovn-nbctl set nb_global . options:use_parallel_build=true
BUILD_NBDB(OVN_BASIC_SCALE_CONFIG(500, 50))
Thanks,
Han
--
Anton R. Ivanov
Cambridgegreys Limited. Registered in England. Company Number 10273661
https://www.cambridgegreys.com/
--
Anton R. Ivanov
Cambridgegreys Limited. Registered in England. Company Number 10273661
https://www.cambridgegreys.com/
_______________________________________________
dev mailing list
[email protected]
https://mail.openvswitch.org/mailman/listinfo/ovs-dev
