Hi Charles, > I for one am certainly interested in these changes. There are a lot of > them that are generally useful (e.g., your performance improvements), and > also applicable to any interested in user-model based-benchmarking. Glad to hear you're sharing that point of view.
> The biggest logistical problem I see with this patchset is that there has > probably been quite a bit of divergence of your tree and the SIPp tree in > the meantime. For example, your network changes are going to conflict > with the network changes that I posted on the list a month ago. If we had > more visibility into each other's efforts, hopefully there would have been > peer review and less duplicated effort. I agree the network changes will be the biggest challenge initially. But just to put this back into perspective, I would like to mention that we only started coding just a little bit more than a month ago. So we haven't diverged that much. It is of course unfortunate that you've just been contributing significant changes to an area that we also completely revamped. But since we started our research on a SIPp based approach only a month ago, we couldn't really have avoided this. Just bad timing... One important thing to consider here is that we did not take care so far of other protocols/transports than UDP (we think our design will allow an easy integration of the others, but we did not 'port' the TCP and TLS implementations over just yet). So a combination of our efforts will hopefully be very beneficial. > > The manager feeds the same set of scenarios to each SIPp instance before > > starting a run and then instructs the SIPp instances to change the > > scenario attempt rate according to a configuration file. Each run > > specifies the occurrence rate of each scenario, as well as the rate of > > scenario attempts, as constant or as increasing steps. > This seems very useful. Does the manager also handle statistics > reporting? Right now, we run about half a dozen instances of SIPp, > collect all of the results, and have to combine them after the fact. > Having statistics reporting in the manager would be a big win. Our current implementation sends some of the statistics counters to the manager, mainly the total number of scenario attempts and the number of failed attempts. This is then used by the manager to determine the percentage of Inadequately Handled Scenario (IHS) so it can decide whether to do a next step or if the maximum load of the SUT has been reached. However, we implemented a very generic framework for communication between the manager and SIPp instances as well as between SIPp instances themselves and the reporting to the manager could therefore very easily be extended to cover a wider set of statistics. Our current approach though is to only report live the data that the manager really need to know in order to proceed. The rest of the data from which we produce a report (html with gnuplot graphs) is gathered at the end of the run from the local files created on the various test systems. I look forward to cooperate with you to hopefully merge all our improvements in a common code base. Regards, -David ------------------------------------------------------------------------- This SF.net email is sponsored by DB2 Express Download DB2 Express C - the FREE version of DB2 express and take control of your XML. No limits. Just data. Click to get it now. http://sourceforge.net/powerbar/db2/ _______________________________________________ Sipp-users mailing list Sipp-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/sipp-users
