Greetings Sean,
We appreciate your work and understand the problems you may had.
The metrics you added are present on test code, with some extensive
rework. Hopefully achieving same goals (bugs are still possible).
As i think it is now clear to all, the statistics reported to
viewers just had other aspects that needed to be taken into consideration.
Regards,
From: [email protected]
[mailto:[email protected]] On Behalf Of Sean M
Sent: Thursday, November 12, 2015 16:23
To: [email protected]
Subject: Re: [Opensim-dev] Still on Sim and Phys Frames per Second (FPS)
Greetings everyone,
Please allow me to provide some background and motivation on why the MOSES Team
submitted corrections to the simulation's statistics gathering. When we first
became interested in determining the scalability of OpenSim, very little
information could be found on the web, publications, and through the
developers' IRC chat. Our investigation determined that we must do an
exhaustive study on our own because the information was not available. To our
surprise, we noticed early on that several questionable and incorrect
implementations of metrics resided in the code; this was a bit concerning
because grid owners, researchers, and curious users all relied on the accurate
statistics reporting. The biggest concern to us was that researchers have
published work containing these invalid statistics without knowing that the
gathered statistics were "fudged" and incorrect. [It should be noted that in
academia and research communities, researchers depend on and refer to previous
publications as the basis of their work. If the referenced data misleads
conclusions and reporting, an entire research thread can be deemed false,
wasting time and money and doing serious harm to the community.]
To make OpenSim's statistics more accurate and valid to measure, the MOSES Team
dedicated financial support and development hours to improve the simulator for
everyone. To do this, we first provided the core developers with a
statement-of-work that was a preview of the statistics development that we
anticipated to make and welcomed feedback. We then followed the community's
process to submit the code back to the project in three code patches. The first
phase corrected the frame rate reporting, which was originally multiplied by a
static/hard-coded value of 5, noted in the code to be a "hack" that must be
corrected in the future, and was not acknowledged anywhere on the OpenSim
website or any other documentation to be artificially boosted. From our
development, the "fudge" factor was removed, other noted invalid metrics were
corrected, and the simulator was thoroughly tested for both operational
correctness and user scalability. After the first phase of work was verified,
we submitted the code, provided further details of what was submitted, and
listened to your feedback to ensure acceptance.
With the metrics we added and corrected, the community now has enhanced and
valid statistics gathering. From the MOSES Team alone, we have provided 7
peer-reviewed publications (listed below) that uses the statics we have given
back to the community since July. From the work, you now know how OpenSim's
user scalability is affected with increased vertical hardware scaling: various
hardware configurations, allocations, and limitations. We have also provided
the methodology to generate predictive models to allow grid owners know what
hardware is needed to support a target amount of simultaneous users on a single
region. Without correcting the invalid metrics that resided inside of OpenSim,
we could not have given back to the OS community this type of detailed
analysis. More broadly, from our talks at conferences, workshops, and through
our journal publications, we have brought the attention of OpenSim to other
simulation enthusiasts by spreading the word of this extensive, research-able
open-sourced project. All of this published research and OS awareness stems
from the work that the MOSES team has contributed back to the OpenSim community.
[1] Sean C. Mondesire, Jonathan Stevens, Rebecca Leis, and Douglas B. Maxwell,
“Resource Allocation Predictive Modeling to Optimize Virtual World Simulator
Performance,” In Proceedings of the IEEE ICMLA’15 Workshop on Machine Learning
for Predictive Models in Engineering Applications (MLPMEA), Miami, FL, December
9-11, 2015.
[2] Sean C. Mondesire, Jonathan Stevens, and Douglas B. Maxwell, "Network
Bandwidth's Effect on Virtual World Simulator Performance Optimization," In
Proceedings of the Interservice/Industry Training, Simulation and Education
Conference (IT/TSEC '15), December 2015.
[3] Sean C. Mondesire, Jonathan Stevens, and Douglas B. Maxwell, "An Analysis
of Increased Vertical Scaling in Three-Dimensional Virtual World Simulation,"
In Proceedings of the 8th EAI International Conference on Simulation Tools and
Techniques 2015 (SimuTools '15), August 2015.
[4] Jonathan Stevens, Sean C. Mondesire, Rebecca Leis, and Douglas B. Maxwell,
“An Empirical Analysis of Virtual World Fidelity’s Impact on Simulator Network
Performance,” Journal of Advanced Research in Modeling and Simulation, Vol. 2,
No. 1, August 2015.
[5] Jonathan Stevens, Sean C. Mondesire, Rebecca Leis, and Douglas B. Maxwell,
“Human Entities' Effect on Server Performance in Distributed Virtual World
Training,” In Proceedings of the 2015 Fall Simulation Interoperability Workshop
(SIW), Orlando, FL, USA, Aug. 31-Sept. 4, 2015.
[6] Sean C. Mondesire, Rebecca Leis, Jonathan Stevens, and Douglas B. Maxwell,
"Analyzing Virtual World Region Fidelity on Scalability and Simulation
Performance," Open Journal of Modeling and Simulation (OJMSi), Vol. 3, No. 3,
July 27, 2015.
[7] Sean C. Mondesire, Jonathan Stevens, and Douglas B. Maxwell, "Vertical
Scalability Benchmarking in Three-Dimensional Virtual World Simulation," In
Proceedings of the 47th Summer Computer Simulation Conference 2015 (SummerSim
'15), July 2015.
Best regards,
Sean Mondesire, Ph.D.
MOSES Team
On Thu, Nov 12, 2015 at 5:04 AM, GarminKawaguichi
<[email protected]> wrote:
Well done !
Le 12/11/2015 03:43, Nicky Perian a écrit :
Returned lag meter to Kokua with adjusted values
when on opensim grids.
https://bitbucket.org/NickyP/kokuant/commits/c9c2099513d4ee0e2b023199efaff4a049a7cc05
Comment message follows if you don't care to follow the link.
[OPENSIM] Return Lag Meter. Fudge factor added for server section of Lag Meter
to compensate for the removal of a server side fudge factor. The trigger on SL
grids is 20 for red, between 20 and 30 for yellow and above 30 for green.
On SL grids with nominal 45 fps 20 is 44.44 % and warning point is 66.67 %.On
OS grids with nominal 55 fps 20 is 36.3 % and warning point is 54.5 %. On OS
there was a bias to not turn red or yellow until performance was worse
than SL points. Maybe that is one reason why the fudge factor was put in the
first place. With this change the bias to let performnace get worse than SL is
still present and the value for red is 4 and yellow is between 4 and 6.
While on OS 20 and 30 are multipled by (11/55).
--
_______________________________________________
Opensim-dev mailing list
[email protected]
http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-dev
_______________________________________________
Opensim-dev mailing list
[email protected]
http://opensimulator.org/cgi-bin/mailman/listinfo/opensim-dev
