Re: Throughput/Goodput analysis on RTEMS
On Sun, Jul 11, 2021 at 6:42 PM Chris Johns wrote: > > On 10/7/21 1:28 pm, Vijay Kumar Banerjee wrote: > > On Fri, Jul 9, 2021 at 8:25 PM Chris Johns wrote: > >> > >> On 2/7/21 4:40 am, Vijay Kumar Banerjee wrote: > >>> I'm planning to do a throughput analysis on the RTEMS network stacks > >>> and I'm looking for some suggestions on the tools/applications for > >>> that if anyone has done something like that before. > >> > >> This is a great project. > >> > >>> If such application has not been used with RTEMS, then I might be > >>> willing to port it to RTEMS or write one from scratch as a net-demo > >>> maybe. Any resource/advice for that is welcome and much appreciated. > >> > >> Throughput is one parameter when dealing with networking and it is > >> important in > >> some applications while latency is another parameter that can be critical. > >> Consider a voice switch with operators using Push To Talk (PTT) buttons to > >> activate a radio's transmitter, an enable packet needs to be delivered > >> within > >> 10msec max in all cases. > >> > > This is an interesting point. Thanks for mentioning latency. > > In RTEMS and in an RTOS in general I also consider the latency for separate > socket paths to separate readers and/or concurrent readers and writers as > something important. A lot of stacks maintain a single giant lock for the > stack > and that serialises the processing through the stack. Libbsd does not have > this > limitation with it's fine grain locking. A protocol such as PTP benefits from > deterministic handling of packets. > Interesting. I'm making notes, I'll look at latency from different protocols. > >> I would look at removing the networking fabric from the analysis because > >> it is > >> subjective and can be effected by the NIC hardware, PHY configuration plus > >> any > >> externally connected equipment. > >> > > I'm approaching this by running different network stacks over the same > > hardware. I have a uC5282 that runs legacy networking and I have > > ported the driver to libbsd nexus device (dhcp doesn't work yet) and > > I'm able to see some difference in the round trip time over loopback > > with different stacks. > > Will your analysis look at architectures that support SMP and have various > cache > configurations? On a Zynq libbsd creates over 4000 locks and so there may be a > performance hit on smaller single core systems compared with multi-core > architectures that can run concurrent threads in the stack at the same time. > Yes, we plan to look at multicore architecture as well. Getting the target to support all three stacks would be the real challenge. > I think the 5282 may bias the results in a particular way and other archs will > bias in a different way. I do not see this as wrong or a problem, rather it is > something that needs to be accounted for so readers of the analysis do not get > the wrong impression. > We intend to analyze boards over multiple arch. > >> In terms of network stack performance for RTEMS you need to consider the > >> protocol, buffering used, size of the buffer pool, transmit and receive > >> paths, > >> they will have different characteristics, and target's memory effects such > >> as > >> caches. On top of this is filtering, ie packet filtering, and the types of > >> access. > >> > > Thanks for these interesting attributes to consider. I have done > > preliminary analysis over ICMP with packets of same size over > > different network stacks on the same board. > > > >> With libbsd there are a number of sysctl settings that effect the > >> performance. > >> How will you manage these? > >> > > I'm not sure. I didn't know about the possible performance difference > > based on sysctl settings. This would be relevant for any user and I > > would like to explore it. Could you please point to some resources or > > examples? > > There are a lot of posts on the net on this topic (search: "freebsd tune > network > sysctl") and the values can be technical but they do make a difference for > different data workflows. There is no single set of values what work > universally > and for libbsd this is OK because I encourage users to explore the settings > and > see what works for them. There can be a memory used vs performance trade off. > We > support the sysctl API and there is a sysctl command for the shell. > I found some nice articles, thanks. I'll explore it further. Thank you for the great points, it helped me get a better idea of what I should be looking at. Best regards, Vijay > Chris ___ devel mailing list devel@rtems.org http://lists.rtems.org/mailman/listinfo/devel
Re: Throughput/Goodput analysis on RTEMS
On 10/7/21 1:28 pm, Vijay Kumar Banerjee wrote: > On Fri, Jul 9, 2021 at 8:25 PM Chris Johns wrote: >> >> On 2/7/21 4:40 am, Vijay Kumar Banerjee wrote: >>> I'm planning to do a throughput analysis on the RTEMS network stacks >>> and I'm looking for some suggestions on the tools/applications for >>> that if anyone has done something like that before. >> >> This is a great project. >> >>> If such application has not been used with RTEMS, then I might be >>> willing to port it to RTEMS or write one from scratch as a net-demo >>> maybe. Any resource/advice for that is welcome and much appreciated. >> >> Throughput is one parameter when dealing with networking and it is important >> in >> some applications while latency is another parameter that can be critical. >> Consider a voice switch with operators using Push To Talk (PTT) buttons to >> activate a radio's transmitter, an enable packet needs to be delivered within >> 10msec max in all cases. >> > This is an interesting point. Thanks for mentioning latency. In RTEMS and in an RTOS in general I also consider the latency for separate socket paths to separate readers and/or concurrent readers and writers as something important. A lot of stacks maintain a single giant lock for the stack and that serialises the processing through the stack. Libbsd does not have this limitation with it's fine grain locking. A protocol such as PTP benefits from deterministic handling of packets. >> I would look at removing the networking fabric from the analysis because it >> is >> subjective and can be effected by the NIC hardware, PHY configuration plus >> any >> externally connected equipment. >> > I'm approaching this by running different network stacks over the same > hardware. I have a uC5282 that runs legacy networking and I have > ported the driver to libbsd nexus device (dhcp doesn't work yet) and > I'm able to see some difference in the round trip time over loopback > with different stacks. Will your analysis look at architectures that support SMP and have various cache configurations? On a Zynq libbsd creates over 4000 locks and so there may be a performance hit on smaller single core systems compared with multi-core architectures that can run concurrent threads in the stack at the same time. I think the 5282 may bias the results in a particular way and other archs will bias in a different way. I do not see this as wrong or a problem, rather it is something that needs to be accounted for so readers of the analysis do not get the wrong impression. >> In terms of network stack performance for RTEMS you need to consider the >> protocol, buffering used, size of the buffer pool, transmit and receive >> paths, >> they will have different characteristics, and target's memory effects such as >> caches. On top of this is filtering, ie packet filtering, and the types of >> access. >> > Thanks for these interesting attributes to consider. I have done > preliminary analysis over ICMP with packets of same size over > different network stacks on the same board. > >> With libbsd there are a number of sysctl settings that effect the >> performance. >> How will you manage these? >> > I'm not sure. I didn't know about the possible performance difference > based on sysctl settings. This would be relevant for any user and I > would like to explore it. Could you please point to some resources or > examples? There are a lot of posts on the net on this topic (search: "freebsd tune network sysctl") and the values can be technical but they do make a difference for different data workflows. There is no single set of values what work universally and for libbsd this is OK because I encourage users to explore the settings and see what works for them. There can be a memory used vs performance trade off. We support the sysctl API and there is a sysctl command for the shell. Chris ___ devel mailing list devel@rtems.org http://lists.rtems.org/mailman/listinfo/devel
Re: Throughput/Goodput analysis on RTEMS
On Fri, Jul 9, 2021 at 8:25 PM Chris Johns wrote: > > On 2/7/21 4:40 am, Vijay Kumar Banerjee wrote: > > I'm planning to do a throughput analysis on the RTEMS network stacks > > and I'm looking for some suggestions on the tools/applications for > > that if anyone has done something like that before. > > This is a great project. > > > If such application has not been used with RTEMS, then I might be > > willing to port it to RTEMS or write one from scratch as a net-demo > > maybe. Any resource/advice for that is welcome and much appreciated. > > Throughput is one parameter when dealing with networking and it is important > in > some applications while latency is another parameter that can be critical. > Consider a voice switch with operators using Push To Talk (PTT) buttons to > activate a radio's transmitter, an enable packet needs to be delivered within > 10msec max in all cases. > This is an interesting point. Thanks for mentioning latency. > I would look at removing the networking fabric from the analysis because it is > subjective and can be effected by the NIC hardware, PHY configuration plus any > externally connected equipment. > I'm approaching this by running different network stacks over the same hardware. I have a uC5282 that runs legacy networking and I have ported the driver to libbsd nexus device (dhcp doesn't work yet) and I'm able to see some difference in the round trip time over loopback with different stacks. > In terms of network stack performance for RTEMS you need to consider the > protocol, buffering used, size of the buffer pool, transmit and receive paths, > they will have different characteristics, and target's memory effects such as > caches. On top of this is filtering, ie packet filtering, and the types of > access. > Thanks for these interesting attributes to consider. I have done preliminary analysis over ICMP with packets of same size over different network stacks on the same board. > With libbsd there are a number of sysctl settings that effect the performance. > How will you manage these? > I'm not sure. I didn't know about the possible performance difference based on sysctl settings. This would be relevant for any user and I would like to explore it. Could you please point to some resources or examples? > Chris ___ devel mailing list devel@rtems.org http://lists.rtems.org/mailman/listinfo/devel
Re: Throughput/Goodput analysis on RTEMS
On 2/7/21 4:40 am, Vijay Kumar Banerjee wrote: > I'm planning to do a throughput analysis on the RTEMS network stacks > and I'm looking for some suggestions on the tools/applications for > that if anyone has done something like that before. This is a great project. > If such application has not been used with RTEMS, then I might be > willing to port it to RTEMS or write one from scratch as a net-demo > maybe. Any resource/advice for that is welcome and much appreciated. Throughput is one parameter when dealing with networking and it is important in some applications while latency is another parameter that can be critical. Consider a voice switch with operators using Push To Talk (PTT) buttons to activate a radio's transmitter, an enable packet needs to be delivered within 10msec max in all cases. I would look at removing the networking fabric from the analysis because it is subjective and can be effected by the NIC hardware, PHY configuration plus any externally connected equipment. In terms of network stack performance for RTEMS you need to consider the protocol, buffering used, size of the buffer pool, transmit and receive paths, they will have different characteristics, and target's memory effects such as caches. On top of this is filtering, ie packet filtering, and the types of access. With libbsd there are a number of sysctl settings that effect the performance. How will you manage these? Chris ___ devel mailing list devel@rtems.org http://lists.rtems.org/mailman/listinfo/devel
Re: Throughput/Goodput analysis on RTEMS
On Fri, Jul 2, 2021 at 10:51 AM Christian Mauderer wrote: > > Hello Vijay, > > On 01/07/2021 22:16, Vijay Kumar Banerjee wrote: > > Hi Kinsey, > > > > On Thu, Jul 1, 2021 at 12:57 PM Kinsey Moore > > wrote: > >> > >> On 7/1/2021 13:40, Vijay Kumar Banerjee wrote: > >>> Hi all, > >>> > >>> I'm planning to do a throughput analysis on the RTEMS network stacks > >>> and I'm looking for some suggestions on the tools/applications for > >>> that if anyone has done something like that before. > >> > >> Stephen Clark has recently added TTCP tests/commands to LibBSD which I > >> think would be a great starting point. > >> > > Thanks for the point! It looks interesting and I'll try it out. > > If GPL is no problem (which shouldn't be the case for a specific > throughput test application) you could also take a look at nuttcp. > That's basically a fork of a fork of ttcp. It might has some new > features that are not there in the original and quite old ttcp. > Thanks Christian, I tried ttcp and nuttcp on Linux and this looks great. GPL is certainly not a problem in our use case, so nuttcp is a good idea as well. > Beneath the ttcp family, there is also iperf (2 or 3) which has a BSD > license. Note that despite iperf2 and iperf3 are named similar, they are > both different projects and both are actively maintained. > Thanks. I haven't checked iperf2/3 yet, but I will certainly give it a shot. Best regards, Vijay > Best regards > > Christian > > > > > > > Best regards, > > Vijay > >> > >> Kinsey > >> > >> ___ > >> devel mailing list > >> devel@rtems.org > >> http://lists.rtems.org/mailman/listinfo/devel > > ___ > > devel mailing list > > devel@rtems.org > > http://lists.rtems.org/mailman/listinfo/devel > > ___ devel mailing list devel@rtems.org http://lists.rtems.org/mailman/listinfo/devel
Re: Throughput/Goodput analysis on RTEMS
On Thu, Jul 1, 2021 at 3:44 PM Joel Sherrill wrote: > > Have you seen this on benchmarking? > https://datatracker.ietf.org/doc/html/rfc8238 > Thank you for this link. This is insightful and gives a clear idea for the benchmarking work. > On Thu, Jul 1, 2021 at 3:16 PM Vijay Kumar Banerjee wrote: > > > > Hi Kinsey, > > > > On Thu, Jul 1, 2021 at 12:57 PM Kinsey Moore > > wrote: > > > > > > On 7/1/2021 13:40, Vijay Kumar Banerjee wrote: > > > > Hi all, > > > > > > > > I'm planning to do a throughput analysis on the RTEMS network stacks > > > > and I'm looking for some suggestions on the tools/applications for > > > > that if anyone has done something like that before. > > > > > > Stephen Clark has recently added TTCP tests/commands to LibBSD which I > > > think would be a great starting point. > > > > > Thanks for the point! It looks interesting and I'll try it out. > > > > > > Best regards, > > Vijay > > > > > > Kinsey > > > > > > ___ > > > devel mailing list > > > devel@rtems.org > > > http://lists.rtems.org/mailman/listinfo/devel > > ___ > > devel mailing list > > devel@rtems.org > > http://lists.rtems.org/mailman/listinfo/devel ___ devel mailing list devel@rtems.org http://lists.rtems.org/mailman/listinfo/devel
Re: Throughput/Goodput analysis on RTEMS
Hello Vijay, On 01/07/2021 22:16, Vijay Kumar Banerjee wrote: Hi Kinsey, On Thu, Jul 1, 2021 at 12:57 PM Kinsey Moore wrote: On 7/1/2021 13:40, Vijay Kumar Banerjee wrote: Hi all, I'm planning to do a throughput analysis on the RTEMS network stacks and I'm looking for some suggestions on the tools/applications for that if anyone has done something like that before. Stephen Clark has recently added TTCP tests/commands to LibBSD which I think would be a great starting point. Thanks for the point! It looks interesting and I'll try it out. If GPL is no problem (which shouldn't be the case for a specific throughput test application) you could also take a look at nuttcp. That's basically a fork of a fork of ttcp. It might has some new features that are not there in the original and quite old ttcp. Beneath the ttcp family, there is also iperf (2 or 3) which has a BSD license. Note that despite iperf2 and iperf3 are named similar, they are both different projects and both are actively maintained. Best regards Christian Best regards, Vijay Kinsey ___ devel mailing list devel@rtems.org http://lists.rtems.org/mailman/listinfo/devel ___ devel mailing list devel@rtems.org http://lists.rtems.org/mailman/listinfo/devel ___ devel mailing list devel@rtems.org http://lists.rtems.org/mailman/listinfo/devel
Re: Throughput/Goodput analysis on RTEMS
Have you seen this on benchmarking? https://datatracker.ietf.org/doc/html/rfc8238 On Thu, Jul 1, 2021 at 3:16 PM Vijay Kumar Banerjee wrote: > > Hi Kinsey, > > On Thu, Jul 1, 2021 at 12:57 PM Kinsey Moore wrote: > > > > On 7/1/2021 13:40, Vijay Kumar Banerjee wrote: > > > Hi all, > > > > > > I'm planning to do a throughput analysis on the RTEMS network stacks > > > and I'm looking for some suggestions on the tools/applications for > > > that if anyone has done something like that before. > > > > Stephen Clark has recently added TTCP tests/commands to LibBSD which I > > think would be a great starting point. > > > Thanks for the point! It looks interesting and I'll try it out. > > > Best regards, > Vijay > > > > Kinsey > > > > ___ > > devel mailing list > > devel@rtems.org > > http://lists.rtems.org/mailman/listinfo/devel > ___ > devel mailing list > devel@rtems.org > http://lists.rtems.org/mailman/listinfo/devel ___ devel mailing list devel@rtems.org http://lists.rtems.org/mailman/listinfo/devel
Re: Throughput/Goodput analysis on RTEMS
Hi Kinsey, On Thu, Jul 1, 2021 at 12:57 PM Kinsey Moore wrote: > > On 7/1/2021 13:40, Vijay Kumar Banerjee wrote: > > Hi all, > > > > I'm planning to do a throughput analysis on the RTEMS network stacks > > and I'm looking for some suggestions on the tools/applications for > > that if anyone has done something like that before. > > Stephen Clark has recently added TTCP tests/commands to LibBSD which I > think would be a great starting point. > Thanks for the point! It looks interesting and I'll try it out. Best regards, Vijay > > Kinsey > > ___ > devel mailing list > devel@rtems.org > http://lists.rtems.org/mailman/listinfo/devel ___ devel mailing list devel@rtems.org http://lists.rtems.org/mailman/listinfo/devel
Re: Throughput/Goodput analysis on RTEMS
On 7/1/2021 13:40, Vijay Kumar Banerjee wrote: Hi all, I'm planning to do a throughput analysis on the RTEMS network stacks and I'm looking for some suggestions on the tools/applications for that if anyone has done something like that before. Stephen Clark has recently added TTCP tests/commands to LibBSD which I think would be a great starting point. Kinsey ___ devel mailing list devel@rtems.org http://lists.rtems.org/mailman/listinfo/devel
