Hi,
change done this morning, but to typical times I still see a raise in
CPU-load, though the number of flows kept being low:
20120608-094530: lookups: hit:21369696007 missed:12901230041
lost:210760086 flows: 588 SHORT_FLOWS=329, TOP=mem: 6848 cpu: 47
20120608-094540: lookups: hit:21369710090 missed:12901242161
lost:210760086 flows: 515 SHORT_FLOWS=253, TOP=mem: 6848 cpu: 45
20120608-094550: lookups: hit:21369723892 missed:12901279759
lost:210760086 flows: 538 SHORT_FLOWS=299, TOP=mem: 6848 cpu: 98
20120608-094600: lookups: hit:21369738394 missed:12901351923
lost:210760086 flows: 573 SHORT_FLOWS=309, TOP=mem: 6848 cpu: 100
20120608-094610: lookups: hit:21369756196 missed:12901416693
lost:210760086 flows: 524 SHORT_FLOWS=284, TOP=mem: 6848 cpu: 96
20120608-094620: lookups: hit:21369770278 missed:12901482470
lost:210760086 flows: 609 SHORT_FLOWS=378, TOP=mem: 6848 cpu: 96
20120608-094630: lookups: hit:21369784860 missed:12901548691
lost:210760086 flows: 627 SHORT_FLOWS=346, TOP=mem: 6848 cpu: 98
20120608-094640: lookups: hit:21369798128 missed:12901611209
lost:210760086 flows: 558 SHORT_FLOWS=317, TOP=mem: 6848 cpu: 97
20120608-094650: lookups: hit:21369812453 missed:12901670167
lost:210760086 flows: 661 SHORT_FLOWS=387, TOP=mem: 6848 cpu: 98
20120608-094700: lookups: hit:21369827834 missed:12901730154
lost:210760086 flows: 796 SHORT_FLOWS=531, TOP=mem: 6848 cpu: 100
.
.
.
( ovs-vsctl (Open vSwitch) 1.7.90)
it's now at these figures for about 12 minutes, calming down as I'm
writing, without changing the no. of flows etc...
Any idea? I have started with the file-log option, so I could get some
debug-logging next time this is happening, if it's useful.
Thnx n regards,
Oliver.
On 06/07/2012 12:22 PM, Oliver Francke wrote:
Hi Justin,
thnx for the explanations.
Here is an excerpt of a scenario, when CPU-load goes up, though within our
network the lost-figures don't normally change:
20120607-114420: lookups: hit:21149788628 missed:12736368714 lost:210746961
flows: 3280 SHORT_FLOWS=2451, TOP=mem: 19m cpu: 39
20120607-114425: lookups: hit:21149799408 missed:12736372737 lost:210746961
flows: 4654 SHORT_FLOWS=3831, TOP=mem: 19m cpu: 45
20120607-114430: lookups: hit:21149810014 missed:12736374681 lost:210746961
flows: 2769 SHORT_FLOWS=1907, TOP=mem: 19m cpu: 18
20120607-114435: lookups: hit:21149821758 missed:12736378269 lost:210746961
flows: 4160 SHORT_FLOWS=3231, TOP=mem: 19m cpu: 49
20120607-114440: lookups: hit:21149831635 missed:12736381096 lost:210746961
flows: 3871 SHORT_FLOWS=2995, TOP=mem: 19m cpu: 18
20120607-114445: lookups: hit:21149842697 missed:12736384730 lost:210746961
flows: 4099 SHORT_FLOWS=3241, TOP=mem: 19m cpu: 35
20120607-114450: lookups: hit:21149853788 missed:12736387554 lost:210746961
flows: 3769 SHORT_FLOWS=2944, TOP=mem: 19m cpu: 12
20120607-114455: lookups: hit:21149862246 missed:12736389740 lost:210746961
flows: 2589 SHORT_FLOWS=1809, TOP=mem: 19m cpu: 18
20120607-114500: lookups: hit:21149875807 missed:12736392636 lost:210746961
flows: 3311 SHORT_FLOWS=2489, TOP=mem: 19m cpu: 29
20120607-114505: lookups: hit:21149893590 missed:12736396998 lost:210746961
flows: 5187 SHORT_FLOWS=4066, TOP=mem: 19m cpu: 53
20120607-114510: lookups: hit:21149904797 missed:12736402095 lost:210746961
flows: 6230 SHORT_FLOWS=5171, TOP=mem: 19m cpu: 37
20120607-114515: lookups: hit:21149915723 missed:12736407377 lost:210746961
flows: 6054 SHORT_FLOWS=4950, TOP=mem: 19m cpu: 45
20120607-114520: lookups: hit:21149928325 missed:12736412748 lost:210746961
flows: 6422 SHORT_FLOWS=5326, TOP=mem: 19m cpu: 31
20120607-114525: lookups: hit:21149938705 missed:12736415973 lost:210746961
flows: 4072 SHORT_FLOWS=2993, TOP=mem: 19m cpu: 43
20120607-114530: lookups: hit:21149949606 missed:12736422759 lost:210746961
flows: 7633 SHORT_FLOWS=6338, TOP=mem: 19m cpu: 94
20120607-114535: lookups: hit:21149964017 missed:12736452506 lost:210746961
flows: 11739 SHORT_FLOWS=10993, TOP=mem: 19m cpu: 96
20120607-114540: lookups: hit:21149976648 missed:12736480881 lost:210746961
flows: 15925 SHORT_FLOWS=15143, TOP=mem: 19m cpu: 98
20120607-114545: lookups: hit:21149988896 missed:12736508350 lost:210746961
flows: 13592 SHORT_FLOWS=12888, TOP=mem: 19m cpu: 98
20120607-114550: lookups: hit:21150002168 missed:12736538481 lost:210746961
flows: 15581 SHORT_FLOWS=14800, TOP=mem: 19m cpu: 100
20120607-114555: lookups: hit:21150016018 missed:12736566873 lost:210746961
flows: 11541 SHORT_FLOWS=10865, TOP=mem: 19m cpu: 98
20120607-114600: lookups: hit:21150029226 missed:12736594616 lost:210746961
flows: 14313 SHORT_FLOWS=15555, TOP=mem: 19m cpu: 100
20120607-114605: lookups: hit:21150049470 missed:12736623781 lost:210746961
flows: 14113 SHORT_FLOWS=13341, TOP=mem: 19m cpu: 100
20120607-114610: lookups: hit:21150061782 missed:12736651311 lost:210746961
flows: 13490 SHORT_FLOWS=12613, TOP=mem: 19m cpu: 99
20120607-114615: lookups: hit:21150074821 missed:12736677656 lost:210746961
flows: 12209 SHORT_FLOWS=11518, TOP=mem: 19m cpu: 97
20120607-114620: lookups: hit:21150087942 missed:12736704949 lost:210746961
flows: 11863 SHORT_FLOWS=11182, TOP=mem: 19m cpu: 84
20120607-114625: lookups: hit:21150101016 missed:12736731540 lost:210746961
flows: 11214 SHORT_FLOWS=10475, TOP=mem: 19m cpu: 97
20120607-114630: lookups: hit:21150114324 missed:12736758289 lost:210746961
flows: 10456 SHORT_FLOWS=10931, TOP=mem: 19m cpu: 98
20120607-114635: lookups: hit:21150128318 missed:12736785776 lost:210746961
flows: 11338 SHORT_FLOWS=10645, TOP=mem: 19m cpu: 98
this now last for a couple of minutes, then falls down again. Nothing critical
so far.
Regards,
Oliver.
Am 07.06.2012 um 09:52 schrieb Justin Pettit:
On Jun 6, 2012, at 2:52 AM, Oliver Francke wrote:
@Justin: Any other recommendations?
Are you also having many short-lived flows? If you're in the range I mentioned
in my response to Kaushal (roughly 120,000 flow setups per second), then the
forthcoming 1.7.0 release may be enough for you.
If it's worth, I could try to start a new thread, but talking about high
CPU-load, how do you all handle something like SYN-FLOOD attacks and stuff like
that?
Each datapath has 16 queues that connect the kernel to userspace. We assign
each port to one of those queues, which will help prevent a port from starving
the other ports. Our use-case is to prevent one VM from starving out the
others. In Kaushal's case, he using OVS more like a bump-in-the-wire than a
vswitch, meaning that he's not concerned with a bad actor at the port level.
We've got a couple of people traveling this week, but when they get back, I
plan to discuss how we may be able to provide finer-grained control over flow
setups for vswitch deployments, since our current approach is rather coarse and
can lead to queue collisions. I've also written Kaushal off-line to see if I
can get more information about his situation.
--Justin
_______________________________________________
discuss mailing list
[email protected]
http://openvswitch.org/mailman/listinfo/discuss
_______________________________________________
discuss mailing list
[email protected]
http://openvswitch.org/mailman/listinfo/discuss
--
Oliver Francke
filoo GmbH
Moltkestraße 25a
33330 Gütersloh
HRB4355 AG Gütersloh
Geschäftsführer: S.Grewing | J.Rehpöhler | C.Kunz
Folgen Sie uns auf Twitter: http://twitter.com/filoogmbh
_______________________________________________
discuss mailing list
[email protected]
http://openvswitch.org/mailman/listinfo/discuss
