At 09:16 04/05/2007, Olaf Bergmann wrote: >Di-Shi Sun wrote: >> Hi All, >> >> We have performed a benchmark test on OpenSER V1.1, V1.2 and SER 2.0 to >> understand and compare the performance of the three releases in a >> simulated production environment. > >Nice, thanks for this interesting piece of work. > >> Summary of the test results: >> ============================ >> * The performance of OpenSER V1.2 and SER 2.0 are not materially >> different, however, there are two minor differences. >> - SER V2.0 requires less memory. >> - OpenSER V1.2 has less post dial delay. > >Could you please comment on the PDD graph? For my understanding, the >6+ seconds are caused by your failure scenarios? I wonder why the >SER graph seems to be constant while the OpenSER looks like exponential?
I have been struggling with the measurement too (actually I'm even missing PDD definition in the document). In a private conversation with authors I learned that the test scenario is actually about randomized- order forking, with some of the destinations being unavailable. That explains why SER is having a constant failure rate but it does not explain why openser is doing better initially (perhaps blacklisting turned on by default in openser?) and going like exponential later. Some few more results would be good in this context too (graph showing the actual delay as opposed to percentage exceeding a threshold -- which is fine for the 'big picture' but hard to disaggregate for tracing what's actually going on). Other thing which came out of a private chat with authors is that ser measurements are in SER's debugging mode (which is set by default on CVS). SER is compiled with PKG_MALLOC, DBG_QM_MALLOC while openser without (F_MALLOC). Otherwise for sake of completenss, I would enjoy attached (open)ser config files (among others it shall reveal if black-listing is turned on as that should have dramatic impact on the results), described call-flows, and described scenario (I mean details about this randomized-order forking). Apart from this piece of critique, I think it is a great contribution and with some work along the lines I have suggested it will be an excellent document. -jiri -- Jiri Kuthan http://iptel.org/~jiri/ _______________________________________________ Devel mailing list Devel@openser.org http://openser.org/cgi-bin/mailman/listinfo/devel
