i don't want folks to think i am against research ... i am not. but, i do
honestly think that we do too much of it ... we force too much to be done
... we force "publishing" and, the only real criterion is ... were you able
to get it published (in a decent outlet of course)? not only that ... most
of the rewards in academe come from doing this ... and racking up the
tallies for pubs ... that is ... the balance has tipped FAR too much to the
side of perks for pubs ...
good scholarship is more than that
social science research has a big barrier to hurdle ... and that is ...
what is the most important impact any of it can have? ... in general, the
answer to this is very limited ... at best. so, in this context, we don't
need more piecemeal projects ... research tidbits so to speak ... we need:
1. better definitions of what is valuable to do ... and what is not
2. projects that go on for longer periods of time ... to look at sustained
effects
3. groups of students/faculty/researchers working TOGETHER, even at the
dissertation stage ... on larger projects that have more potential for impact
4. as mundane as it may be, given #1, we need more replication studies ...
and not think that every new study has to break new ground
5. we need much more training in methodology ... broadly speaking ... (in a
time where there seems to be so many efforts being made to reduce such
training)
and on and on
and in the area of journal editorial policies ... perhaps we need to think
about a trained cadre of PROFESSIONAL reviewers ... who get paid for their
professional efforts
and, to make that editorial job easier to carry out, i would suggest that
(unless there is some overriding issue of huge importance) ... i would
FORBID anyone from submitting more than one GOOD paper a year ... (not try
to insist that they submit and publish more)
anyone attending the aera meeting this year in seattle ... and lugging
around the program from hotel to hotel ... will remember that is was at
least 3/4" thick (maybe closer to 1") ... crammed to the hilt of sessions
of research papers ... on narrow topics ...
i think it is time for our profession to take a long hard look at this
"volume" of activity ... and see if we can't come to some agreement about
far FEWER areas that we should do research in ... that's right ... cast off
many that have not and will not lead us to anything important ... and
concentrate our resources in a more comprehensive way in more limited areas
... with more people working together in a sustained effort ... over longer
periods of time and THEN think about putting together a monograph ...
summarizing what one did (the team that is) ... and what one found ... and
what the real import of all this is
some will say well, how do you KNOW that something won't be important ...
down the road?
we know ... trust me ... we know
however, we seem not of a mind to say ... research in that and this area
... is priority ... and these other areas ... no dice
At 09:44 AM 5/4/01 -0700, Carl Huberty wrote:
> Why do articles appear in print when study methods, analyses,
> results, and conclusions are somewhat faulty? [This may be considered as
> a follow-up to an earlier edstat interchange.] My first, and perhaps
> overly critical, response is that the editorial practices are faulty. I
> don't find Dennis Roberts' "reasons" in his 27 Apr message too
> satisfying. I regularly have students write critiques of articles in
> their respective areas of study. And I discover many, many, ... errors
> in reporting. I often ask myself, WHY? I can think of two reasons: 1)
> journal editors can not or do not send manuscripts to reviewers with
> statistical analysis expertise; and 2) manuscript originators do not
> regularly seek methodologists as co-authors. Which is more prevalent?
> For whatever it is worth ...
>
>Carl Huberty
==============================================================
dennis roberts, penn state university
educational psychology, 8148632401
http://roberts.ed.psu.edu/users/droberts/drober~1.htm
=================================================================
Instructions for joining and leaving this list and remarks about
the problem of INAPPROPRIATE MESSAGES are available at
http://jse.stat.ncsu.edu/
=================================================================
