Hi Zaid, You mention 'CPE Simulator' -- do we already have one along with GenieACS. If yes, can you please point me to that.
Regards -Prashant On Tue, Sep 8, 2015 at 8:45 PM, Zaid Abdulla <[email protected]> wrote: > On Mon, Sep 7, 2015, at 06:46 AM, Prashant Upadhyaya wrote: >> Hi guys, >> >> I was wondering how to define the performance of an ACS in terms of >> its ability to handle all those CPE's out there. >> >> Eg. should it be defined as the time taken to complete a typical >> Inform cycle during BOOT or BOOTSTRAP. >> Something like -- >> TimeStart >> TCP connection initiation >> CPE Informs with BOOT >> No Firmware upgrade >> GPV >> SPV >> SPA >> . >> . >> TCP connection end >> TimeEnd >> >> Clearly an ACS is 'fast' if the difference between the TimeStart and >> TimeEnd is minimum. Further, the time difference makes sense when 'N' >> CPE devices attack the ACS simultaneously. Then cumulatively, to >> services those N CPE's, the ACS will take a total time say T. >> Can we then say that the performance of ACS is defined by the >> parameter N/T which represents a throughput of servicing 'CPE's per >> second' by the ACS ? >> >> Or am I clearly off the mark and there is a more meaningful way of >> defining the performance of an ACS, kindly do share with your >> experience. > > A better metric to use is the number of TR-069 sessions completed in a > given period of time. The actual time a single session takes is > irrelevant because throughput is what matters. Ideally you want to test > that by spawning a number of virtual CPEs using the CPE simulator. A few > notes: > > - Edit the simulator source code to specify the number of devices to > spawn and the inform interval. > > - You can spawn a 1000 devices at 10 minute inform interval or a 100 > devices at 1 minute inform interval. The results should be identical, > unless; > > - you need to test for database memory bottleneck. For that you may want > to test using a DB server with small amount of memory and calculate your > actual memory requirements based on that. > > - Not all sessions are equal. For better simulation, make sure your > presets are setup in the test environment similar to what you'd have in > production. Divide the times in your presets (e.g. parameter refresh > rate) by the same value you use to divide the inform interval. For > example, if you expect to have 100k devices at 100 minutes inform > interval, you can test server performance using 1000 CPE simulators at 1 > minute inform interval. Make sure the times in your presets are divided > by 100 as well. > > - The first time the CPE connects to the ACS, the ACS fetches all device > parameters and that's a heavy process. So make sure you do your measures > after all devices have been fully registered. You'll probably need to > spawn the simulators in batches (e.g. 100 at a time). > > I have added these notes to the wiki > (https://github.com/zaidka/genieacs/wiki/Measuring-GenieACS-Performance). > Feel free to contribute to it. Let us know if you get any interesting > results :) > > Zaid > _______________________________________________ > Users mailing list > [email protected] > http://lists.genieacs.com/mailman/listinfo/users _______________________________________________ Users mailing list [email protected] http://lists.genieacs.com/mailman/listinfo/users
