"Magill, Brett" wrote:
>
> The more general concern about significance testing notwithstanding, I have
> a question about the use of testing, or other inferential statistical
> techniques, in experiments using human subjects, or any other research
> method that does not use probability sampling...
>
> Now, all of these tests that we run--whether from ANOVA, regression,
> difference of means, correlations, etc.--are based on the assumption that we
> have sampled from a population using some probability sampling procedure.
> And the meaning of p is inextricably linked to the properties of the
> sampling distribution.
>
> However, little experimental research with human subjects is done using a
> sample. Most often, in my experience, these studies use volunteer subjects
> or naturally existing groups. These subjects are then randomly assigned to
> treatment and control groups. Yet, in every instance that I know of,
> results are presented with tests of significance. It seems to me that
> outside of the context of probability sampling, these tests have no meaning.
> Despite this, presentation of such results with tests of significance are
> common.
>
> Is there a reasonable interpretation of these results that does not rely on
> the assumption of probability sampling? It seems to me that simply
> presenting and comparing descriptive results, perhaps mean differences,
> betas from a regression, or some other measure of effect size without a test
> would be more appropriate. This would however be admitting that results have
> no applicability beyond the participants in the study. Moreover, these
> results say nothing about the number of subjects one has, which p might help
> with regard to establishing minimum believability. Yet, conceptually, p
> doesn't work.
>
> Am I missing the boat here? Significance testing in these situations seem to
> go over just fine in journals. Appreciate your clarification of the issue.
Try
http://www.tufts.edu/~gdallal/rand.htm
=================================================================
Instructions for joining and leaving this list and remarks about
the problem of INAPPROPRIATE MESSAGES are available at
http://jse.stat.ncsu.edu/
=================================================================
