On Mon, Sep 7, 2015, at 06:46 AM, Prashant Upadhyaya wrote: > Hi guys, > > I was wondering how to define the performance of an ACS in terms of > its ability to handle all those CPE's out there. > > Eg. should it be defined as the time taken to complete a typical > Inform cycle during BOOT or BOOTSTRAP. > Something like -- > TimeStart > TCP connection initiation > CPE Informs with BOOT > No Firmware upgrade > GPV > SPV > SPA > . > . > TCP connection end > TimeEnd > > Clearly an ACS is 'fast' if the difference between the TimeStart and > TimeEnd is minimum. Further, the time difference makes sense when 'N' > CPE devices attack the ACS simultaneously. Then cumulatively, to > services those N CPE's, the ACS will take a total time say T. > Can we then say that the performance of ACS is defined by the > parameter N/T which represents a throughput of servicing 'CPE's per > second' by the ACS ? > > Or am I clearly off the mark and there is a more meaningful way of > defining the performance of an ACS, kindly do share with your > experience.
A better metric to use is the number of TR-069 sessions completed in a given period of time. The actual time a single session takes is irrelevant because throughput is what matters. Ideally you want to test that by spawning a number of virtual CPEs using the CPE simulator. A few notes: - Edit the simulator source code to specify the number of devices to spawn and the inform interval. - You can spawn a 1000 devices at 10 minute inform interval or a 100 devices at 1 minute inform interval. The results should be identical, unless; - you need to test for database memory bottleneck. For that you may want to test using a DB server with small amount of memory and calculate your actual memory requirements based on that. - Not all sessions are equal. For better simulation, make sure your presets are setup in the test environment similar to what you'd have in production. Divide the times in your presets (e.g. parameter refresh rate) by the same value you use to divide the inform interval. For example, if you expect to have 100k devices at 100 minutes inform interval, you can test server performance using 1000 CPE simulators at 1 minute inform interval. Make sure the times in your presets are divided by 100 as well. - The first time the CPE connects to the ACS, the ACS fetches all device parameters and that's a heavy process. So make sure you do your measures after all devices have been fully registered. You'll probably need to spawn the simulators in batches (e.g. 100 at a time). I have added these notes to the wiki (https://github.com/zaidka/genieacs/wiki/Measuring-GenieACS-Performance). Feel free to contribute to it. Let us know if you get any interesting results :) Zaid _______________________________________________ Users mailing list [email protected] http://lists.genieacs.com/mailman/listinfo/users
