Hi, Thanks for your help! I went over these ideas and now understand my problem better.
If I explain my simulation, it may help. Basically, I have a simulation where various "agents" have to find a target. The simulation is terminated once the target has been found. The current measurement of performance is "iterations taken" - time. The simulation settings are kept constant, aside from the starting positions which are random. The simulation is repeated until the confidence interval reaches a certain percentage [this statement may be wrong once you read then next step!] The simulation then changes a parameter (such as number of "agents") and is then repeated to sample the new population. This is done quite a few times. What I really need to do, is to prove with a certain confidence, that the MEAN time taken from Simulation A comes from a different population than from Simulation B, C, D, E etc. >From my undestating, this may imply that I need to concentrate on the accuracy of the mean of simulation run wrt the real population mean (unknown) and then compare this to other simulation runs with different populations. Some suggestions have included doing a ANOVA analysis. Comparing multi variances was also suggested, but this apparenly can only be done with 2 populations. On top of this, there is a big requirement on computational efficiency - each simulation needs to stop when the results are accurate enough for the next step. So is confidence in the mean the solution (and how do I do that), or is it comparing various simulation runs together (and using what method) or is it something else, or a combination. Does this explain enough? If anyone requires any more info, just ask. Sorry if this explanation or question sounds vague - I am just starting to find my way around stats!!!!! Many thanks! [EMAIL PROTECTED] (Jay Warner) wrote in message news:<[EMAIL PROTECTED]>... > the real question is, 'how much accuracy (precision, variance) is > suitable?' > > If you were to repeat the simulation run (i.e., a test) a total of n > times, then you could say that the true mean elapsed time was x-bar +/- > (certain amount), with say 95% confidence. > > That is, if you were to then repeat the whole process, n times again, 95% > of the time the x-bar would fall within the +/- (certain amount) you had > calculated. The average of your mean elapsed time is probably Normal, so > this equation can be used. If you want to predict the one next elapsed > time from the next simulation run, then you have to believe that your > individual times are Normally distributed, or do some deeper analysis. > If that's confusing, I'm sorry, but it comes from what you asked. > > You can do the simulation run n times, and _estimate_ a value for mean > elapsed time that could be confirmed only by say 100*n runs. Does this > sound like what you want? > > The eq. for the 'certain amount' is given by > > certain amount = s*z/sqrt(n) > > where s = stdev of your n run times, z = 1.96 for 95% confidence, and n = > number of simulation runs. > > Pick a confidence interval ('certain amount') that you like, then solve > for n to decide how many runs you will need to make. Statistics cannot > tell you what confidence interval is suitable to your problem - that is a > technical issue. It can tell you now many n's you need to reach that > confidence interval. > > Is this what you were looking for? > > Cheers, > Jay > > PS: Yes, I know 'accuracy' and 'precision' refer to different things. > But you used the first of these words in a way which I infer meant the > latter, so I opened the first sentence in that manner. > > Gooseman wrote: > > > Hi, > > > > I am writing a computer simulation, and I really would appreciate some > > advice about statistics side of things! > > > > Each simulation run has fixed settings, but there is some randomness > > involved (e.g. start position). As a result, each simulation scenario > > needs to be run until the universal mean (say time taken for objective > > to be met) varience is reduced. > > > > The simulation has just one output that needs measurement - time > > taken, and there is no transient state. > > > > The question is, what accuracy is acceptable, and how can I guartee > > that the varience is small enough to be accurate, while being > > efficient on computing power. Any methods, techniques etc. gladly > > welcome, as I am new to stats!! > > > > Thanks. > > > > ================================================================= > > Instructions for joining and leaving this list, remarks about the > > problem of INAPPROPRIATE MESSAGES, and archives are available at > > http://jse.stat.ncsu.edu/ > > ================================================================= > > -- > Jay Warner > Principal Scientist > Warner Consulting, Inc. > 4444 North Green Bay Road > Racine, WI 53404-1216 > USA > > Ph: (262) 634-9100 > FAX: (262) 681-1133 > email: [EMAIL PROTECTED] > web: http://www.a2q.com > > The A2Q Method (tm) -- What do you want to improve today? > > > > > > ================================================================= > Instructions for joining and leaving this list, remarks about the > problem of INAPPROPRIATE MESSAGES, and archives are available at > http://jse.stat.ncsu.edu/ > ================================================================= ================================================================= Instructions for joining and leaving this list, remarks about the problem of INAPPROPRIATE MESSAGES, and archives are available at http://jse.stat.ncsu.edu/ =================================================================
