I wrote:
> > Internet users are not a random sample of any other population.
> > Internet users who browse to a given site are not a random sample of
> > internet users. And internet users who bother to fill out a form are not
> > a random sample of anything. Your student will have data representative
> > of a population whose nature he is just about completely ignorant of,
> > and which he cannot determine. This is not science, and it is not made
> > more scientific by using the newest toys to take the "sample".
>
and Johannes Hartig responded:
> If collecting data via internet is "not science" because of lacking
> "representativity" then most current research in psychology, where
> most samples are convenience samples of students, is "not science".
You said it, not me. <grin>
I would agree with you that any study done using the
guinea-pig-for-credit pool is at least suspect, and is only as valid as
the assumption that the subjects are typical of the population to which
the inference is extended. If indeed - as you imply - most current
research in psychology _is_ done on such subjects and the
representativeness of the subjects _cannot_ be upheld, then no, it is
not science, it is sympathetic magic. [Sympathetic magic for dummies:
Take something resembling the subject but different. Name it after the
subject. Conduct a ritual. Hope that the subject will somehow be
affected.]
> IMHO online samples are more
>heterogenous than student samples and therefore offering a chance to get more
>generalizable results in many fields of research than the often used student
>samples.
The idea is not to get a "heterogeneous" sample. If that were the case,
a sample of 100 gathered in equal parts from a UFOlogists' meeting, a
monastery, street people, the Board of Governors of the local
university, and a Grade 2 class would be just about the best sample
anybody ever studied - even if the purpose was to study marital
infidelity in the Armed Forces. Clearly this is not the case.
The goal is to get a randomized sample from the population. Granted,
that is not easy to do. Even the census bureau cannot get 100%
responses, with the law behind them. However, while almost every sample
in the social sciences has _something_ ofthe convenience sample about
it, few methodologies are as blatantly bad as Internet sampling. (About
the only worse example I can think of is the Shere Hite style survey in
which a few hundred thousand copies of a survey are distributed in
magazines, of which one or two percent are returned by those who feel
strongly enough to pay their own postage.)
To summarize:
*One of the assumptions that most statistical methods (presumably
including the ones this hapless grad student intends to use in SPSS)
make is that the data are a true random sample from the population in
question. This assumption is moderately robust to mall deviations from
randomness, but cannot be ignored.
*That assumption would not be met, even approximately, by an Internet
survey, except under very unusual conditions. [A perfectly valid study
of (say) customers of Amazon.com could be done by sampling from the
electronic forms they fill in - but I have a hunch that this is not what
was being proposed!]
*Proceeding with the inference and publishing the results, not
realizing that the assumptions of the inferential method used were not
even approximately met, would be incompetence, and possible grounds for
failure. Doing so while aware of that fact would be academic fraud, and
possible grounds for expulsion.
*For this reason, students (and others) should be discouraged from
using such "sampling methods".
There is no point looking for your keys on the street where the
streetlights are; you have to look on the street where you dropped them.
-Robert Dawson
=================================================================
Instructions for joining and leaving this list and remarks about
the problem of INAPPROPRIATE MESSAGES are available at
http://jse.stat.ncsu.edu/
=================================================================
