Hello Alec,
My development was leaded to test 2 NFVBench (and TRex processes) in parallel
but not for performances results, only for development test environment sharing.
So I just tested NFVBench functionalities were OK with such environment sharing.
For instance, we have 1 server with 8 CPUs and 4 NICs. Each instance of
NFVBench is able to used separated 4 CPUs and 2 NICs.
Example of conf for 1st instance :
generator_profile:
- name: trex-local
tool: TRex
ip: 127.0.0.1
cores: 1
software_mode: false
limit_memory: 1024
zmq_pub_port: 4500
zmq_rpc_port: 4501
interfaces:
- port: 0
pci: "0000:08:00.2"
switch_port:
- port: 1
pci: "0000:08:00.3"
switch_port:
intf_speed: 1Gbps
platform:
master_thread_id: 0
latency_thread_id: 1
dual_if:
- socket: 0
threads: [2]
And for 2nd instance :
traffic_generator:
default_profile: trex-local2
generator_profile:
- name: trex-local2
tool: TRex
ip: 127.0.0.1
cores: 1
software_mode: false
limit_memory: 1024
zmq_pub_port: 4510
zmq_rpc_port: 4511
interfaces:
- port: 0
pci: "0000:08:00.0"
switch_port:
- port: 1
pci: "0000:08:00.1"
switch_port:
intf_speed: 1Gbps
platform:
master_thread_id: 4
latency_thread_id: 5
dual_if:
- socket: 0
threads: [6]
In future, we can deploy multiple instances of NFVBench on baremetal servers
with the aim of having measurement with simultaneous background traffic.
But this scenario was not yet tested.
De : Alec Hothan (ahothan) [mailto:[email protected]]
Envoyé : jeudi 14 février 2019 08:16
À : MENGUY Francois Regis TGI/OLN
Cc : [email protected]
Objet : Re: [opnfv-tech-discuss] #nfvbench Add TRex parameters to tune
performance and allocate ressources
Hi Francois,
Can you provide some results on how you could scale trex (i.e. how many
instances and how many cores per instance, how well you could scale the pps
with/without optimization).
Thanks
Alec
From: "[email protected]<mailto:[email protected]>"
<[email protected]<mailto:[email protected]>>
Date: Wednesday, February 13, 2019 at 6:33 AM
To: "Alec Hothan (ahothan)" <[email protected]<mailto:[email protected]>>
Cc:
"[email protected]<mailto:[email protected]>"
<[email protected]<mailto:[email protected]>>
Subject: RE: [opnfv-tech-discuss] #nfvbench Add TRex parameters to tune
performance and allocate ressources
Hello Alec,
I’ve committed my work on TRex parameters tuning.
Please let me know if you have any question and if you want me to add
documentation in the RST files.
I added also some response inline ☺
Regards,
Francois Regis Menguy
[email protected]<mailto:[email protected]>
De : Alec Hothan (ahothan) [mailto:[email protected]]
Envoyé : vendredi 18 janvier 2019 16:32
À : MENGUY Francois Regis TGI/OLN
Cc :
[email protected]<mailto:[email protected]>
Objet : Re: [opnfv-tech-discuss] #nfvbench Add TRex parameters to tune
performance and allocate ressources
Inline…
From: "[email protected]<mailto:[email protected]>"
<[email protected]<mailto:[email protected]>>
Date: Tuesday, January 15, 2019 at 1:24 AM
To: "Alec Hothan (ahothan)" <[email protected]<mailto:[email protected]>>
Cc:
"[email protected]<mailto:[email protected]>"
<[email protected]<mailto:[email protected]>>
Subject: RE: [opnfv-tech-discuss] #nfvbench Add TRex parameters to tune
performance and allocate ressources
In fact we had the case with a shared platform during NFVBench tool testing.
Ressources such as huge page were completely used by one TRex instance, we were
unable to start other NFVBench instance on same compute node.
[alec]
Yes that is correct, I believe this is actually a behavior if the DPDK library
used by Trex.
The only workaround I know of is to artificially limit the number of huge pages
available on the box before starting Trex.
[françois]
limit_memory property permit to limit huge pages consumption by TRex process
So possible use cases :
- Operation: measurement with simultaneous background traffic
- Dev: multiple simultaneous tests for dev or config tuning purposes
In addition, even if we were on docker container, NFVBench process connect to
first TRex process even if it is in another docker container, so localhost IP
is not efficient to prevent connecting to another TRex instance from another
previous NFVBench process.
[alec]
This might be because you start your container in network host mode
(--net=host), can you try to start in the default bridge mode (just remove
–net=host)
[françois]
I have try without this docker property but still the issue
I will test with cgroups if I reproduced the case.
Our test on huge pages and platform parameters configuration for TRex
subprocess were OK on our side without impact on global server configuration as
far as I can see.
[alec]
Cool!
[françois]
I was not able to use correctly cgroups for our use cases. Only my work on TRex
properties working as expected for me.
BR,
Alec
Regards,
Francois Regis Menguy
[email protected]<mailto:[email protected]>
De : Alec Hothan (ahothan) [mailto:[email protected]]
Envoyé : samedi 12 janvier 2019 18:53
À : MENGUY Francois Regis TGI/OLN
Cc :
[email protected]<mailto:[email protected]>
Objet : Re: [opnfv-tech-discuss] #nfvbench Add TRex parameters to tune
performance and allocate ressources
That is an interesting use case although we need to make clear that running
multiple instances on the same server will likely not provide linear
performance increase.
Can you provide more information about your concurrency target? I.e. how many
instances are you planning to run and on what hardware resources (RAM, cores,
NIC)?
Regarding resource partitioning easiest would be to take advantage of
containerization (cgroups) with the goal of being able to run multiple
instances of the nfvbench container (each running the nfvbench orchestration
and 1 instance of trex).
This should take care of the nfvbench to trex communication which stays
internal to each container instance (the 127.0.0.1 subnet is specific to each
container instance - so no need to have different zmq ports as they will run on
distinct subnets - to be confirmed)
The externally visible resources across container instances would still need to
be solved through container parameters:
• Nfvbench REST service port - already supported (if used)
• Trex NIC and ports selection – already supported in nfvbench config
• Huge pages configuration in Trex might require some tweaking and
impact the server overall huge page configuration
• Platform latency and master thread id, socket and threads - we will
need to check with the trex team how to work out core partitioning across
containers
Thanks
Alec
_________________________________________________________________________________________________________________________
Ce message et ses pieces jointes peuvent contenir des informations
confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce
message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages
electroniques etant susceptibles d'alteration,
Orange decline toute responsabilite si ce message a ete altere, deforme ou
falsifie. Merci.
This message and its attachments may contain confidential or privileged
information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete
this message and its attachments.
As emails may be altered, Orange is not liable for messages that have been
modified, changed or falsified.
Thank you.
_________________________________________________________________________________________________________________________
Ce message et ses pieces jointes peuvent contenir des informations
confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce
message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages
electroniques etant susceptibles d'alteration,
Orange decline toute responsabilite si ce message a ete altere, deforme ou
falsifie. Merci.
This message and its attachments may contain confidential or privileged
information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete
this message and its attachments.
As emails may be altered, Orange is not liable for messages that have been
modified, changed or falsified.
Thank you.
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#22811):
https://lists.opnfv.org/g/opnfv-tech-discuss/message/22811
Mute This Topic: https://lists.opnfv.org/mt/29007935/21656
Mute #nfvbench: https://lists.opnfv.org/mk?hashtag=nfvbench&subid=2783016
Group Owner: [email protected]
Unsubscribe: https://lists.opnfv.org/g/opnfv-tech-discuss/unsub
[[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
