---- Original Message ----- On Sun, 17 Jun 2007 18:17:03 -0400, "Miguel Roig" wrote: [SNIP] >In sum, in those cases I have no doubts that the project was the students' own > even if I suspect that the student has received considerable assistance from > a mentor. However, in other cases, a student submission is written at a > professional or near-professional level, the literature review shows a > fairly thorough grasp of relevant issues, and the data are analyzed with > fairly sophisticated statistical techniques (e.g., hierarchal regression, > MANOVAs, ANCOVAS, structural equations). Some of these papers are of such > high quality that, frankly, one begins to wonder the extent of the student's > contribution to the paper.
I think that there are at least two questions or issues that have to be addressed with respect to the points you raise: (1) How important is it that the student actually be able to sit down at a computer (or, almost unimaginable today, with paper and pencil and an electronic calculator) and do the specific statistical analysis by him/herself? And not just following a cookbook algorithm from a text or from a copy of program code that can serve as a model for the analysis to be done? (2) How important is it that the student is able to understand the conceptual issues involved in doing a specific analysis, what sorts of questions are being addressed and why, what are the pros and cons of certain analyses and whether one should do several different analyses to see the subtleties of how different analyses provide different pieces of information? Point (1) above is basically a technical skills issues: does a person know enough about the programming aspects of how to tell a statistical program to do a specific analysis in order to get specific results. Though this is a high-level technical skill (indeed, an undergraduate with a high level of programming ability in SPSS or SAS or Stat is almost guaranteed to get a job involving that skill while the student's knowledge of psychology may seem to be largely irrelevant) I don't think this is the kind of knowledge that most professors are really interested developing, regardless of how profitable it might be in the "real" world because it focuses on only one small part of the research process and assumes that some one else (hopefully a supervisor) has deterimined which scientific questions are being asked, why they are being asked, what are the implications of different results, and how to integrate the results into the other findings of the relevant research area. Point (2) focues on a "deeper" set of issues: does the student understand why a specific hypothesis or set of hypotheses are being tested and why specific statistical analyses are necessary to evaluate these hypotheses. The student may have a fairly clear idea as to what the issues are but may be unable to undertake the analysis him/herself. One might say that Point (2) is really what we as teachers are really interested in developing (at least I think so) and may serve as the foundation for the student to use this knowledge and skill to address new problems in the future. Let me give you an example regarding point (2) which may show how a student may have a relatively deep conceptual appreciation of theoretical position but would be limited in what he/she could actually do. Recently I taught a seminar course on Altemeyer's Authoritarianism research and one component focused on how to evaluate the psychometric properties of his RWA scale. Altermeyer has argued that there is a single latent variable or factor (in the factor analysis sense) which represents three differents aspects of being authoritarian (i.e., conventionality, authoritarian submission, and authoritarian agression). Altermeyer has explicitly said that he has written the items/questions for the RWA to mix together two or all three of these aspects. The question arises as to what is the "dimensionality" of the RWA scale: is there a single underlying factor representing a broad authoritarianism variable or are there three correlated factors that represent the construct? I explained how observed correlations between items on the RWA depend upon the loadings these items have on the underlying factor and what the correlation matrix should look like if (a) there is a single factor, (b) there are three independent factors), and (c) three correlated factors. I think that most of the students were able to appeciate how the observed correlation matrix depended upon the underlying factor structure and I explained what the appropriate statistical analysis would be (i.e., confirmatory factor analysis via a structural equation program such as AMOS, Lisrel, or whatever). However, I also point out that from a methodological perspective, researchers have criticized Altemeyer's items because although he usually extracts a single factor in his factor analyses, the question remains as to whether this is due to a single authoritarian factor or all of the items being a mish-mash of three factors that give the illusion of the operation of a single factor. We reviewed some work by Funke (2003) who used a German version of RWA scale but had re-written the items to reflect only one aspect of authoritarianism (e.g., only conventionalism, only authoritarian submission, or only authoritarian agression). If Altermeyer is correct, then these three sets of questions should load on a single factor. If authoritarianism as conceived by Altemeyer is flawed, that there are actually three correlated factors or "dimensions" to his version of authoritariansim, then a three factor model should fit the data better. Indeed, through the use of a confirmatory factor analysis that specified three seperate but correlated factors, Funke found evidence for a three factor model of authoritarianism. In class, I reviewed the structural model (path diagram) for the single factor and three factor models as well as some of the implications of the three factor model (e.g., if a person was high on conventionality but low on authoritarian submission and authoritarian agression, would that person still be an "authoritarian"?). I think that the presentation above is probably comprehensible to many advanced undergraduates with previous exposure to statistics and methods. I think that students given this presentation would appreciate the difference between a single factor model of authoritarianism and a three factor model even though they probably have no idea of how to actually do the appropriate statistical analysis (i.e., structural equation modeling) to test these models. Getting back to the original point, I think that it possible for a smart undergraduate to develop a high level of skill in the use of a particular statistical package or two, especially under the right supervision. I also think that it is possible for a smart undergraduate to grasp relatively deep conceptual models that represent different mathematical representations of a theoretical construct even if they are somewhat weak in statistics. I think that it is the rare undergraduate who can combine these two capabilities in a meaningful and productive way (indeed, it is unclear how many graduate students eventually develop this capability). Perhaps I have gone far afield from the original concern of trying to determine whether a student is actually able to do either (1) or (2) or both above. If one were expert in these issues, the it might be useful to have a talk with the student and see how much he/she actually knows and what the limits are on this knowledge. If the student can't answer relatively basic questions of either skill (how did you get SPSS' GLM to perform a multifactor ANOVA) or theory (why are you saying that there should be only one factor/variable that should be operating in this situation), then I think that one would have a basis to wonder if the work is truly the student's. Having the student provide intermediate work products (e.g., output, notes, etc.) may also be relevant. But one should probably also talk to the research supervisor and see how guidance was provided or whether the supervisor did the work that the student is claiming to be his/her own. Just a few thoughts. Take care, -Mike Palij New York University [EMAIL PROTECTED] Refs Funke, F. (2005). The dimensionality of Right-Wing Authoritarianism: Lessons from the dilemma between theory and measurement. Political Psychology, 26(2), 195-218. > I actually have some evidence indicating that some students are given > unmerited authorship (see > http://facpub.stjohns.edu/%7Eroigm/presentations/student%20authorship%20in%2 > 0EPA%2006.ppt). However, it is with respect to students' knowledge of > advanced statistical techniques that I now want to pick your brains. So, > here are my questions for the group: What are the most advanced data > analysis techniques that you are covered in the Statistics course offered in > your department? Does your department offer an advanced statistics course > and what areas do you cover in those courses? For both questions, a link to > a syllabus or course description will be sufficient. > > TIA > Miguel --- To make changes to your subscription go to: http://acsun.frostburg.edu/cgi-bin/lyris.pl?enter=tips&text_mode=0&lang=english
