Hi Li (or Gavin) I tried to tackle this issue a few years ago, in an Endangered Species context, where the burden of proof and the difference between Type I and II error are important, but often unrecognized concerns. Specifically, I used equivalence tests, in lieu of a standard null model regression test (i.e. H-null = no linear relationship), to examine whether the water quality in a large lake was likely to be harming two endangered fishes. My intention was for the article to be as clear and intuitive as humanly possible, so maybe it will be of help to you? If you'd like to see it, but don't have access to 'BioScience', just let me know, and I'll send you a copy.
McGarvey, D.J. 2007. Merging precaution with sound science under the Endangered Species Act. BioScience 57(1):65-70. ** - Dan McGarvey On Sun, Feb 7, 2010 at 2:21 PM, Gavin Simpson <gavin.simp...@ucl.ac.uk>wrote: > On Sat, 2010-02-06 at 21:44 -0800, Li An wrote: > > Dear Ecologers, > > > > In testing ecological models, we often use t-test as a way to compare > > our model results with observed data. If they are close enough, we > > obtain more confidence about our model. However, in most traditional > > situations, we put "no difference" as the null and regarded it as the > > default. This means that unless we find substantial evidence, we would > > retain the null hypothesis. For instance, we can use this type of test > > to examine if a drug has a noticeable effect. > > > > In our model performance situation (testing observed data = predicted > > numbers from a model, assuming data independence), I argue that we > > should keep the alternative hypothesis as the default, making every > > effort to find substantial evidence to support the null hypothesis (if > > unable, we retain the alternative hypothesis related to inequality > > between the model predictions and the data). In this case, we can still > > use the traditional test statistic such as z or p values, but interpret > > the results differently. Rather than using the criterion of p > 0.05 (or > > Z<1.96 or t < a big number) to retain the null hypothesis, we should use > > a more strict standard--e.g., p > a much larger number (e.g., 0.9) or z > > < a much smaller number (e.g.,0.125), to retain the null hypothesis > > about equality between the model predictions and the data. This seems > > mofrea philosophical issue. Does this make sense? > > > > Li >
