Thanks DZ for your detailed guidance! > Umar, > > I have been using pretty standard TestClient bots to do "performance > testing".. > > I have also used the existing command set to pre-program complex > activities, > like a continuous 6 model runway fashion show for in-world video > recording. > > Existing TestClient > commands<http://lib.openmetaverse.org/wiki/TestClient_0.7.0_commands> > > The existing commands I have found most useful to facilitate individual > control of the bot behavior are > > setmaster - Lets you assign an inworld object that can send IM's > to > the bot to initiate commands. > > script - lets you maintain your command scripts in a directory > relative to the testclient executable. > > Load testing generally consisted of 3 stages.... > > Generating and equipping the avatars.... The login credentials need > to > be available, and the avatars need to represent the kind > of population you expect. Performance differences between default > avatars, and those with prim and/or HUD attachments are significant. > > Logging in the Test population.. The avatar creation sequence is > very > load intensive... TestClient bots are most prone to operation timeout > failures. > Logging in significant numbers of avatars at once will prevent any > significant number of successful logins.. Scripting this activity is > the > fastest/easiest > way I have found to pre-load the test regions.. My experience has > been that running 10 bots per TestClient "console" provides an efficient > way to > balance the need to get bots logged in at a reasonable rate, and > being > able to keep most of the logged bots active if a TestClient "console" > fails. > > The final step of the login process is to have the bot touch an > inworld > object... This object is scripted to use the touch_start event to > "register" the UUID > of the bot and act as its "master", by sending the "performance > action" commands via IM or initiating scripts to direct the avatar > > Syncronizing the test... Once all the bots have been registered to > the > controller devices... I log in to the region using a standard viewer. > By scripting the "controllers' to provide me with menu access to > commands, I can initiate and terminate group activities. > Combining the in-world object ability to send IM commands with the > TestClient access to remote scripts has allowed complex load testing > scenarios > to be designed quickly and repeated across multiple platform > releases. > > > Some examples..... > > Using 4 testclient "consoles" running on my home desktop, i have logged > in 40 testclient avatars to an opensim region hosted on a remote server > and > integrated into another remote grid. I built a "reasonable" central > building of approximately 8000 prims. I divided the region into 4 > quadrants > and logged > 10 avatars into the approximate center of each quadrant. > > Initially I had each of the 10 bots randomly chat, or perform an > animation. > By varying the time between each of the operations I was able to get a > feel > for the performance by attempting to move between the groups and > listen/watch for the success rates of the activites. Additional > activities > like uploading images, sending IM messages grid wide, transferring and > collecting inventory items, and (now) teleporting to specific locations > around the grid were easily included in the scripts and provided excellent > simulations of real user loads. > > With more experience with the available commands I was able to build > "walking" paths for the avatar groups to follow.. Adapting the existing > scripts I set a home point in respective corners for each of the avatars > and designed a simple square for them to walk in. It was then a simple > measure to watch as groups of 5 avatars would walk from one corner to the > next in relative synchronization until the region became "overloaded".. > At > that point the avatars would either "miss" command and stop, or loose > connections with the region and I would see them walking off the edges or > disappearing from the scene. > > > I did not spend any time developing standardized "performance measurement" > mechanisms... Tests evolved from "How many can I get logged in and able > to > register on a controller" to how long can I walk 40 bots around a region > before "something" fails. Of course my results are also specific to the > configuration of my desktop/home network/region server/grid service > connections.... so I can guarantee... Your mileage WILL vary.. > > > > > > > > > > On Wed, May 25, 2011 at 9:29 AM, Umar Farooq <[email protected]> wrote: > >> Dear Dan, >> >> Thanks! I tested the teleports (goto command) and it looks fine now. >> >> We are trying to determine the overall load of the system with the >> introduction of users, contents and their activities starting with an >> unpopulated world on a single simulator simulating a number of regions. >> I >> just need to know what methods could be used to determine the load based >> on these activities. I saw somewhere you mentioned about a ScienceSim >> patch that collects statistics, could you please let me know if I can >> get >> and use it. >> >> We would be waiting for the exciting commands to be incorporated into >> libomv TestClient. >> >> Best wishes and have a good day. >> >> Umar >> >> > Umar, >> > >> > There was a bug in TestClient where no region crossings would work. >> That >> > goes for the 'goto' command for teleporting as well as just walking >> across >> > a border with 'forward' or other movement commands. That bug was fixed >> > yesterday by lkalif and is in the latest public libomv svn source. >> I've >> > only tested it with walking across borders, not with the 'goto' >> teleports. >> > >> > For metrics, what question do you want to answer about OpenSim that >> you >> > are scaling the clients? That will partially determine which metrics >> you >> > need to capture. We have tried to find the bottlenecks to scaling a >> scene >> > (objects, scripts, physics, clients, etc) and wanted to know if CPU or >> > network bandwidth was the limiting factor. We have measured CPU and >> > bandwidth as well as line-level profiling of various OpenSim modules. >> > >> > Rather than releasing our TestClient patches directly, I am working >> with >> > the libopenmetaverse team to commit them to their public SVN. For now, >> the >> > public client has the best available peformance until I can merge our >> > commands to automate the wandering movements. >> > >> > ~Dan >> > >> > >> > >> > -----Original Message----- >> > From: [email protected] >> > [mailto:[email protected]] On Behalf Of Umar Farooq >> > Sent: Monday, May 23, 2011 2:13 AM >> > To: [email protected] >> > Subject: Re: [Opensim-dev] Creation of 100+ users and TestClient >> Logins! >> > >> > Dear Dan, >> > >> > I am using Robust, and running a private grid for my experiments. I >> have >> > noticed the issue of sequential execution of actions, and am using a >> > number of different TestClient instances using different connections. >> > >> > I have noticed that when we use goto or teleport command for bots and >> then >> > apply a command, the server is responds that an action is performed >> but >> > the bots are not updating in the scene, any comment on that? >> > >> > Which metric would be best to check load of a system? CPU usage, >> Network >> > Utilisation and anything else? Sorry if that is too stupid to ask! >> > >> > It would be great to have the code available whenever it is possible. >> > Thanks again for your help. >> > >> > Best wishes and have a great day. >> > >> > Umar >> > >> >> Umar, >> >> Are you using Simian or Robust to create your accounts? If the >> grid >> >> required authentication (email, captcha, etc) you will probably need >> to >> >> work with the grid operator to get test accounts. On Simian, it's >> easy >> >> to create test accounts if you can disable the authentications >> >> temporarily. >> >> >> >> As for logging them in, I suggest not putting more than about 25 >> bots >> >> in >> >> a single TestClient instance. There are many functions in TestClient >> >> which iterate over all bots sequentially performing an action and you >> >> will find them not responding to scene updates in time if you have >> too >> >> many in one executable. >> >> >> >> We are planning to make our modified TestClient available but it is >> >> not >> >> at this time. It's actually older than the released TestClient from >> >> libomv but it has a few commands for making bots perform interesting >> >> actions independently rather than all bots doing the same thing. >> >> >> >> ~Dan >> >> >> >> >> >> -----Original Message----- >> >> From: [email protected] >> >> [mailto:[email protected]] On Behalf Of Umar >> Farooq >> >> Sent: Wednesday, May 18, 2011 2:29 AM >> >> To: [email protected] >> >> Subject: [Opensim-dev] Creation of 100+ users and TestClient Logins! >> >> >> >> Dear All, >> >> >> >> To use TestClient for adding bots, it requires that the users are >> first >> >> created as authentication is required during login. Is there any >> simple >> >> way to create lots of users and then login them via a single command >> to >> >> the simulator? Thanks alot for your help and guidance! >> >> >> >> Best wishes, >> >> >> >> Umar >> >> >> >> -- >> >> Umar Farooq >> >> >> >> Research Scholar, >> >> Virtual Humans Group, >> >> School of Computing Sciences, >> >> University of East Anglia, >> >> Norwich, United Kingdom. >> >> >> >> Cell: +4497853398184 >> >> >> >> _______________________________________________ >> >> Opensim-dev mailing list >> >> [email protected] >> >> https://lists.berlios.de/mailman/listinfo/opensim-dev >> >> _______________________________________________ >> >> Opensim-dev mailing list >> >> [email protected] >> >> https://lists.berlios.de/mailman/listinfo/opensim-dev >> >> >> > >> > >> > -- >> > Umar Farooq >> > >> > Research Scholar, >> > Virtual Humans Group, >> > School of Computing Sciences, >> > University of East Anglia, >> > Norwich, United Kingdom. >> > >> > Cell: +4497853398184 >> > >> > _______________________________________________ >> > Opensim-dev mailing list >> > [email protected] >> > https://lists.berlios.de/mailman/listinfo/opensim-dev >> > _______________________________________________ >> > Opensim-dev mailing list >> > [email protected] >> > https://lists.berlios.de/mailman/listinfo/opensim-dev >> > >> >> >> -- >> Umar Farooq >> >> Research Scholar, >> Virtual Humans Group, >> School of Computing Sciences, >> University of East Anglia, >> Norwich, United Kingdom. >> >> Cell: +4497853398184 >> >> _______________________________________________ >> Opensim-dev mailing list >> [email protected] >> https://lists.berlios.de/mailman/listinfo/opensim-dev >> > _______________________________________________ > Opensim-dev mailing list > [email protected] > https://lists.berlios.de/mailman/listinfo/opensim-dev >
-- Umar Farooq Research Scholar, Virtual Humans Group, School of Computing Sciences, University of East Anglia, Norwich, United Kingdom. Cell: +4497853398184 _______________________________________________ Opensim-dev mailing list [email protected] https://lists.berlios.de/mailman/listinfo/opensim-dev
