Hi Zheng, > Sorry for the delay, and thanks a lot for all your comments. First I want to > clarify that the performance number we measured in the tech report is for > the "routing" application. Because the routing application needs to generate > multiple flow configuration messages for all switches along the path one > packet is going to take, its performance is considered to be worse than that > of the "switch" application. We also use our simulator (which supports LLDP > packets exchanging between neighbor switches so that the routing application > could work) to evaluate the "switch" application of NOX, and get a > throughput of 90K rps on a 3.1Ghz machine. But for the "routing" > application, even after I turn on the ndebug option (by running ./configure > --enable-ndebug and make, and I hope this is the right way to do it), the > throughput is still around 20K rps. All the numbers in the tech report are > for the "routing" application. I hope this makes sense to you.
Thanks for your reply. Using the routing application explains some of your observation. However, I think benchmarks should be based on the switch application or a no-op application to illustrate the overhead of the controller itself (i.e., the underlying framework) as the baseline. I think the reason for routing application performing badly here is due to binding lookups (this is just a guess) and for sure it could be tuned to perform significantly better than 20Krps. > About the latency, we measure it in an end-to-end style. That is, we measure > the start time-stamp for a request before we call the "socket.send" > function, and measure the end time-stamp after we receive the > flow_mod/packet_out for that particular request. I agree that NOX will > throttle the connection and the latency of a request within NOX is very > small. It's just the latency we measure is the end-to-end delay, which > includes the queuing delay in both the sending buffer and receiving buffer. Regarding latency measurements, it is tricky to measure stuff precisely at a sub-millisecond resolution. One should be very careful with the resolution of timers and avoid being affected by the operating system scheduler. If you are seeing latencies close to the scheduler quantum size (under Linux typically 10ms) your measurement might be affected by the scheduler. There is no real workaround for this under a non-realtime kernel; however, running the controller, benchmarker, and the latency measurement tool with CPU affinity set (non-overlapping CPU sets -- using taskset -c X) and with a high-priority FIFO scheduler (chrt -f X under Linux) relieves many of these side effects. Btw, I was talking about the end-to-end latency previously. It is quite possible to reduce that > Furthermore, I would like to know that is the code of HyperFlow, or the NOX > multi-threading patch you mentioned, going to be available? Because I really > want to study the difference of all the existing systems, and hopefully my > effort could contribute to this community :) I think we could make the NOX multi-threading patch publicly available soon as a branch on noxrepo. The code needs to be reviewed and tested by the NOX team and the community before making its way to the mainstream NOX. I, for sure, appreciate your efforts very much. The only thing is that if we had a framework to compare controller performance we could better understand the trade-offs in controller design. Rob's oflops/cbench package is a great start, and we could all contribute to it. Cheers, Amin _______________________________________________ nox-dev mailing list [email protected] http://noxrepo.org/mailman/listinfo/nox-dev_noxrepo.org
