Re: [Opensim-dev] Problem with the bots
A comparison of 0.6.9 and 0.7.0.1 is written up at http://www.sciencesim.com/wiki/doku.php/vwperf/stress_profile_20100820 . The stress tests themselves are being made available at http://www.sciencesim.com/wiki/doku.php/vwperf/scalability_tests . We use a modified version of TestClient to create all the wandering bots for the stress tests. One of the significant TestClient modifications is increasing the baking timeout values to several minutes to give it time to let everyone bake without baking retries. The avatar stress test referenced above logs in 25 bots and waits a minute before starting another TestClient to log in another 25. The previous 25 are barely baking before the next are starting to log in. That's one TestClient image per 25 bots -- the 25 bots will free up their baking image memory and settle down to wandering in about 5 minutes. I usually run three client systems to drive the simulator and rotate between them starting 100 (25*4) bots on each. -- mb On Mon, Sep 6, 2010 at 3:01 AM, i...@nii.ac.jp wrote: Hello, Actually I would like to know a little bit more about the test you have done to assess the number of bot that can be runned on a simulation. My main concern is the time we have to wait to log several bot. Indeed, I can log bots and ensure their appearance will be Ok, but it requires a lot of time. (in OpenSim 0.7.0), for instance, to log 25 bots, it takes me almost 10 minutes if I want everybody to be baked. I am using the Testclient Object (inherited from GridClient). What about the time it took you to have 100 avatars on a sim ? And another question, according to http://www.sciencesim.com/wiki/doku.php/vwperf/profiling_069_070 The test have been made on 6.9 ? What about 7.0 ? Thanks a lot for your help IT ___ Opensim-dev mailing list Opensim-dev@lists.berlios.de https://lists.berlios.de/mailman/listinfo/opensim-dev ___ Opensim-dev mailing list Opensim-dev@lists.berlios.de https://lists.berlios.de/mailman/listinfo/opensim-dev
Re: [Opensim-dev] Problem with the bots
Hello, Thanks a lot for the answer Dan. Now I know what to expect. And I have another question. You are talking about independant bots, do you mean that you can control each bot in a single Testclient instance independantly ? (for instance, make a bot move forward while another move backward, or don't move ?) To do so, I wasn't using testclient put the opensim Scenepresence Object (scenepresence.controlingclient) that enable to actually have indepedant bots. I would like to know how independant are you bots. Thanks you very much IT It takes TestClient 5-15 minutes to bake 25 clients if they are all logging in at the same time (depends on hardware and other load on the client box). In our experiments, we use several multi-core servers each running multiple TestClient processes where each process hosts 25 bots. We can usually log in 500-1000 bots to our grid within 20 minutes and get all of them fully baked and ready for action. Instead of running 100 bots in a single TestClient, consider running 4 TestClients and have each one connect 25 bots. TestClient uses lightweight threads to handle packets from the simulator and if you have 100 bots moving around then there are a LOT of packets to process. A machine with more cores or hardware threads will improve your experience dramatically when simulating lots of independent bots in TestClient. ~Dan -Original Message- From: opensim-dev-boun...@lists.berlios.de [mailto:opensim-dev-boun...@lists.berlios.de] On Behalf Of i...@nii.ac.jp Sent: Monday, September 06, 2010 3:01 AM To: opensim-dev@lists.berlios.de Subject: Re: [Opensim-dev] Problem with the bots Hello, Actually I would like to know a little bit more about the test you have done to assess the number of bot that can be runned on a simulation. My main concern is the time we have to wait to log several bot. Indeed, I can log bots and ensure their appearance will be Ok, but it requires a lot of time. (in OpenSim 0.7.0), for instance, to log 25 bots, it takes me almost 10 minutes if I want everybody to be baked. I am using the Testclient Object (inherited from GridClient). What about the time it took you to have 100 avatars on a sim ? And another question, according to http://www.sciencesim.com/wiki/doku.php/vwperf/profiling_069_070 The test have been made on 6.9 ? What about 7.0 ? Thanks a lot for your help IT ___ Opensim-dev mailing list Opensim-dev@lists.berlios.de https://lists.berlios.de/mailman/listinfo/opensim-dev ___ Opensim-dev mailing list Opensim-dev@lists.berlios.de https://lists.berlios.de/mailman/listinfo/opensim-dev ___ Opensim-dev mailing list Opensim-dev@lists.berlios.de https://lists.berlios.de/mailman/listinfo/opensim-dev
Re: [Opensim-dev] Problem with the bots
check this out IT: http://www.youtube.com/watch?v=aB29KXgsMLQ On Mon, Sep 6, 2010 at 9:07 PM, i...@nii.ac.jp wrote: Hello, Thanks a lot for the answer Dan. Now I know what to expect. And I have another question. You are talking about independant bots, do you mean that you can control each bot in a single Testclient instance independantly ? (for instance, make a bot move forward while another move backward, or don't move ?) To do so, I wasn't using testclient put the opensim Scenepresence Object (scenepresence.controlingclient) that enable to actually have indepedant bots. I would like to know how independant are you bots. Thanks you very much IT It takes TestClient 5-15 minutes to bake 25 clients if they are all logging in at the same time (depends on hardware and other load on the client box). In our experiments, we use several multi-core servers each running multiple TestClient processes where each process hosts 25 bots. We can usually log in 500-1000 bots to our grid within 20 minutes and get all of them fully baked and ready for action. Instead of running 100 bots in a single TestClient, consider running 4 TestClients and have each one connect 25 bots. TestClient uses lightweight threads to handle packets from the simulator and if you have 100 bots moving around then there are a LOT of packets to process. A machine with more cores or hardware threads will improve your experience dramatically when simulating lots of independent bots in TestClient. ~Dan -Original Message- From: opensim-dev-boun...@lists.berlios.de [mailto: opensim-dev-boun...@lists.berlios.de] On Behalf Of i...@nii.ac.jp Sent: Monday, September 06, 2010 3:01 AM To: opensim-dev@lists.berlios.de Subject: Re: [Opensim-dev] Problem with the bots Hello, Actually I would like to know a little bit more about the test you have done to assess the number of bot that can be runned on a simulation. My main concern is the time we have to wait to log several bot. Indeed, I can log bots and ensure their appearance will be Ok, but it requires a lot of time. (in OpenSim 0.7.0), for instance, to log 25 bots, it takes me almost 10 minutes if I want everybody to be baked. I am using the Testclient Object (inherited from GridClient). What about the time it took you to have 100 avatars on a sim ? And another question, according to http://www.sciencesim.com/wiki/doku.php/vwperf/profiling_069_070 The test have been made on 6.9 ? What about 7.0 ? Thanks a lot for your help IT ___ Opensim-dev mailing list Opensim-dev@lists.berlios.de https://lists.berlios.de/mailman/listinfo/opensim-dev ___ Opensim-dev mailing list Opensim-dev@lists.berlios.de https://lists.berlios.de/mailman/listinfo/opensim-dev ___ Opensim-dev mailing list Opensim-dev@lists.berlios.de https://lists.berlios.de/mailman/listinfo/opensim-dev -- Michael Emory Cerquoni - Nebadon Izumi @ http://osgrid.org ___ Opensim-dev mailing list Opensim-dev@lists.berlios.de https://lists.berlios.de/mailman/listinfo/opensim-dev
Re: [Opensim-dev] Problem with the bots
Thanks a lot for your help, both of you Justin CC and Dan, Actually I really kept under review the evolution of Opensim dealing with 'running a lot of avatars'. The challenge is to have a reliable control of these bots, for crowd simulation purpose for instance. Sincere thanks for you help I.T In our load testing, we use TestClient from OpenMetaverse which is implemented using GridClient. We have contributed several changes in the past few weeks related to running a hundred or more bots in a single TestClient instance. The main reason we see incomplete appearance is that TestClient does not have time to bake each appearance before the appearance handshake times out between TestClient and the server. All client threads become consumed baking/encoding/decoding image data. They all slow down and new baking requests appear to time out. There are 2 workarounds for this we have used with equal success: 1. Run fewer than 25 clients per TestClient instance. 2. Increase the baking timeouts in TestClient (passed as params to GridClient's AppearanceManager) to a large value (10+ minutes). We routinely run 300-400 bots against a single region using 4 TestClient instances with 100 bots each. We also run 1000+ bots on our research grid using 40 TestClient instances with 25 bots each or 10 instances of 100 each. In both of these situations, every avatar appears baked in the viewer. Dan lake Intel Labs -Original Message- From: opensim-dev-boun...@lists.berlios.de [mailto:opensim-dev-boun...@lists.berlios.de] On Behalf Of Justin Clark-Casey Sent: Tuesday, August 24, 2010 4:40 PM To: opensim-dev@lists.berlios.de Subject: Re: [Opensim-dev] Problem with the bots On 23/08/10 08:08, i...@nii.ac.jp wrote: Hello Everybody, I made some additionnal test using bots (GridClient)(using different viewer, different database on Opensim 0.7.0). My observation is this one : When we log a lot of bots (50 for instance), the appearance of the bots is not completely loaded for each bot. Some of them are fully loaded, while some others have just hair, eye, or just a grey part of skull loaded (or a ghost with SL viewer). This problem is occuring on 0.7.0, and didn't existed on 0.6.8 Actually, I already talked about that, but, the new fact is that when a bon log into the simulation, it is set to the HOME location, and not to its previous position into the simulation. Maybe these problem are bound to each others. I wrote this to understand, if somebody could tell me what information is stored about each avatars, and where is it stored (for instance the last position of the avatar). For instance, where is stored the appearance of the avatar, and when it is loaded. Last and home positions are stored in a table called GridUser. Appearance information effectively comes from the body part and clothing entries in an avatar's inventory and the assets that they link to. This is stored in inventoryitems (and the assets table, of course). I've seen problems with appearance with multiple bots on Intel load tests as well. Don't know why this is. -- Justin Clark-Casey (justincc) http://justincc.org http://twitter.com/justincc ___ Opensim-dev mailing list Opensim-dev@lists.berlios.de https://lists.berlios.de/mailman/listinfo/opensim-dev ___ Opensim-dev mailing list Opensim-dev@lists.berlios.de https://lists.berlios.de/mailman/listinfo/opensim-dev ___ Opensim-dev mailing list Opensim-dev@lists.berlios.de https://lists.berlios.de/mailman/listinfo/opensim-dev
Re: [Opensim-dev] Problem with the bots
If you don''t need individual bakes for your test you might consider hacking the libomv baker to upload pre-baked files instead of making new ones for each bot. -d On Tue, Aug 24, 2010 at 4:54 PM, Lake, Dan dan.l...@intel.com wrote: In our load testing, we use TestClient from OpenMetaverse which is implemented using GridClient. We have contributed several changes in the past few weeks related to running a hundred or more bots in a single TestClient instance. The main reason we see incomplete appearance is that TestClient does not have time to bake each appearance before the appearance handshake times out between TestClient and the server. All client threads become consumed baking/encoding/decoding image data. They all slow down and new baking requests appear to time out. There are 2 workarounds for this we have used with equal success: 1. Run fewer than 25 clients per TestClient instance. 2. Increase the baking timeouts in TestClient (passed as params to GridClient's AppearanceManager) to a large value (10+ minutes). We routinely run 300-400 bots against a single region using 4 TestClient instances with 100 bots each. We also run 1000+ bots on our research grid using 40 TestClient instances with 25 bots each or 10 instances of 100 each. In both of these situations, every avatar appears baked in the viewer. Dan lake Intel Labs -Original Message- From: opensim-dev-boun...@lists.berlios.de [mailto: opensim-dev-boun...@lists.berlios.de] On Behalf Of Justin Clark-Casey Sent: Tuesday, August 24, 2010 4:40 PM To: opensim-dev@lists.berlios.de Subject: Re: [Opensim-dev] Problem with the bots On 23/08/10 08:08, i...@nii.ac.jp wrote: Hello Everybody, I made some additionnal test using bots (GridClient)(using different viewer, different database on Opensim 0.7.0). My observation is this one : When we log a lot of bots (50 for instance), the appearance of the bots is not completely loaded for each bot. Some of them are fully loaded, while some others have just hair, eye, or just a grey part of skull loaded (or a ghost with SL viewer). This problem is occuring on 0.7.0, and didn't existed on 0.6.8 Actually, I already talked about that, but, the new fact is that when a bon log into the simulation, it is set to the HOME location, and not to its previous position into the simulation. Maybe these problem are bound to each others. I wrote this to understand, if somebody could tell me what information is stored about each avatars, and where is it stored (for instance the last position of the avatar). For instance, where is stored the appearance of the avatar, and when it is loaded. Last and home positions are stored in a table called GridUser. Appearance information effectively comes from the body part and clothing entries in an avatar's inventory and the assets that they link to. This is stored in inventoryitems (and the assets table, of course). I've seen problems with appearance with multiple bots on Intel load tests as well. Don't know why this is. -- Justin Clark-Casey (justincc) http://justincc.org http://twitter.com/justincc ___ Opensim-dev mailing list Opensim-dev@lists.berlios.de https://lists.berlios.de/mailman/listinfo/opensim-dev ___ Opensim-dev mailing list Opensim-dev@lists.berlios.de https://lists.berlios.de/mailman/listinfo/opensim-dev ___ Opensim-dev mailing list Opensim-dev@lists.berlios.de https://lists.berlios.de/mailman/listinfo/opensim-dev
Re: [Opensim-dev] Problem with the bots
On 23/08/10 08:08, i...@nii.ac.jp wrote: Hello Everybody, I made some additionnal test using bots (GridClient)(using different viewer, different database on Opensim 0.7.0). My observation is this one : When we log a lot of bots (50 for instance), the appearance of the bots is not completely loaded for each bot. Some of them are fully loaded, while some others have just hair, eye, or just a grey part of skull loaded (or a ghost with SL viewer). This problem is occuring on 0.7.0, and didn't existed on 0.6.8 Actually, I already talked about that, but, the new fact is that when a bon log into the simulation, it is set to the HOME location, and not to its previous position into the simulation. Maybe these problem are bound to each others. I wrote this to understand, if somebody could tell me what information is stored about each avatars, and where is it stored (for instance the last position of the avatar). For instance, where is stored the appearance of the avatar, and when it is loaded. Last and home positions are stored in a table called GridUser. Appearance information effectively comes from the body part and clothing entries in an avatar's inventory and the assets that they link to. This is stored in inventoryitems (and the assets table, of course). I've seen problems with appearance with multiple bots on Intel load tests as well. Don't know why this is. -- Justin Clark-Casey (justincc) http://justincc.org http://twitter.com/justincc ___ Opensim-dev mailing list Opensim-dev@lists.berlios.de https://lists.berlios.de/mailman/listinfo/opensim-dev
Re: [Opensim-dev] Problem with the bots
In our load testing, we use TestClient from OpenMetaverse which is implemented using GridClient. We have contributed several changes in the past few weeks related to running a hundred or more bots in a single TestClient instance. The main reason we see incomplete appearance is that TestClient does not have time to bake each appearance before the appearance handshake times out between TestClient and the server. All client threads become consumed baking/encoding/decoding image data. They all slow down and new baking requests appear to time out. There are 2 workarounds for this we have used with equal success: 1. Run fewer than 25 clients per TestClient instance. 2. Increase the baking timeouts in TestClient (passed as params to GridClient's AppearanceManager) to a large value (10+ minutes). We routinely run 300-400 bots against a single region using 4 TestClient instances with 100 bots each. We also run 1000+ bots on our research grid using 40 TestClient instances with 25 bots each or 10 instances of 100 each. In both of these situations, every avatar appears baked in the viewer. Dan lake Intel Labs -Original Message- From: opensim-dev-boun...@lists.berlios.de [mailto:opensim-dev-boun...@lists.berlios.de] On Behalf Of Justin Clark-Casey Sent: Tuesday, August 24, 2010 4:40 PM To: opensim-dev@lists.berlios.de Subject: Re: [Opensim-dev] Problem with the bots On 23/08/10 08:08, i...@nii.ac.jp wrote: Hello Everybody, I made some additionnal test using bots (GridClient)(using different viewer, different database on Opensim 0.7.0). My observation is this one : When we log a lot of bots (50 for instance), the appearance of the bots is not completely loaded for each bot. Some of them are fully loaded, while some others have just hair, eye, or just a grey part of skull loaded (or a ghost with SL viewer). This problem is occuring on 0.7.0, and didn't existed on 0.6.8 Actually, I already talked about that, but, the new fact is that when a bon log into the simulation, it is set to the HOME location, and not to its previous position into the simulation. Maybe these problem are bound to each others. I wrote this to understand, if somebody could tell me what information is stored about each avatars, and where is it stored (for instance the last position of the avatar). For instance, where is stored the appearance of the avatar, and when it is loaded. Last and home positions are stored in a table called GridUser. Appearance information effectively comes from the body part and clothing entries in an avatar's inventory and the assets that they link to. This is stored in inventoryitems (and the assets table, of course). I've seen problems with appearance with multiple bots on Intel load tests as well. Don't know why this is. -- Justin Clark-Casey (justincc) http://justincc.org http://twitter.com/justincc ___ Opensim-dev mailing list Opensim-dev@lists.berlios.de https://lists.berlios.de/mailman/listinfo/opensim-dev ___ Opensim-dev mailing list Opensim-dev@lists.berlios.de https://lists.berlios.de/mailman/listinfo/opensim-dev
[Opensim-dev] Problem with the bots
Hello Everybody, I made some additionnal test using bots (GridClient)(using different viewer, different database on Opensim 0.7.0). My observation is this one : When we log a lot of bots (50 for instance), the appearance of the bots is not completely loaded for each bot. Some of them are fully loaded, while some others have just hair, eye, or just a grey part of skull loaded (or a ghost with SL viewer). This problem is occuring on 0.7.0, and didn't existed on 0.6.8 Actually, I already talked about that, but, the new fact is that when a bon log into the simulation, it is set to the HOME location, and not to its previous position into the simulation. Maybe these problem are bound to each others. I wrote this to understand, if somebody could tell me what information is stored about each avatars, and where is it stored (for instance the last position of the avatar). For instance, where is stored the appearance of the avatar, and when it is loaded. Thanks a lot for your cooperation, I greatly appreciate it, 'Arigatou gozaimasu' from Japan ___ Opensim-dev mailing list Opensim-dev@lists.berlios.de https://lists.berlios.de/mailman/listinfo/opensim-dev
