TIPSters, I have received permission from the author to post this report. Enjoy. Sorry about its length. Miguel >===== Original Message From COGDOP Council of Graduate Departments of Psychology <[EMAIL PROTECTED]> ===== Fellow chairs, Should any of you need some holiday reading, I invite you to take a look at the following report that I initially prepared for internal consumption here at New Mexico State University. However, I suspect that our concerns are not parochial and that changes in student attitudes, work ethic, and performance are present across the nation to varying degrees. If you have any ideas, I would like to hear them. Perhaps you have already had some experience with my proposed solutions or have discovered additional ones. Cheers, Ken Paap ******* Shifting Winds in Undergraduate Attitudes and Performance (12/19/00) Ken Paap New Mexico State University Most universities are expending resources in "outcomes assessment" and, more often than not, the most important component is a direct measure of "learning." Although "learning assessment" measures vary both within and between universities, imagine that your institution has refined a good technique and that this measure will be consistently applied over the next quarter century. What changes in student learning (performance) would be desired and expected? I suggest that faculty and administrators alike would predict significant improvements over time. No change would be considered a disappointment. An actual downturn in student learning would be appalling to contemplate. Now turn back the clock to 1975, the year I started as a new assistant professor at New Mexico State University. My aspirations for myself and my university were reasonably rosy and on many dimensions the next 25 years have been very successful. The quality of my faculty colleagues at NMSU is better now, most departments have deep pools of talented individuals. Consistent with that, external funding for the university has boomed. But what about the performance of our undergraduate students. For several years a small cohort of faculty in Science Hall at New Mexico State University have grumbled and bemoaned that our undergraduates are not what they used to be. They are less prepared for college level work, are less willing to work hard (even to attend class), more likely to work long hours off campus, and perform at levels below the standards of the "good old days". Unfortunately, we have not had "learning assessment" measures in place for the past 25 years and objective changes in student learning can not be determined. However, it is possible to tap the institutional memory of a campus. In this article I report the results of a survey of senior faculty regarding changes in student performance. This is followed by a discussion of possible causes for the performance decline. The survey and a recent report by Adelman (1999) on predictors of degree completion sets the stage for a set of proposed solutions. THE SURVEY Fifty-three responses were obtained from senior faculty at NMSU. The survey asked faculty to compare current levels of student performance to those observed when the individual faculty member began teaching at NMSU. Forty-nine responses were obtained for the following item: The overall performance (exams, projects, papers, presentations, classroom participation, etc.) of undergraduate students at NMSU is: 1. considerably better 0 2. somewhat better 2 3. about the same 11.5 4. somewhat worse 21 5. considerably worse 14.5 than the average performance when I first start teaching at NMSU. The numbers to the right of each category indicate the frequency of responses. Four individuals responded that performance was in between two adjacent categories, e.g., "somewhere between a 4 and a 5. I counted these as half a response to each category. Using the number associated with each category label, e.g., "considerably worse" = 5, the following measures of central tendency were computed: mean 3.98 median 4 mode 4 I have also conducted two follow up analyses. A. College. There were more responses from faculty in the College of A&S (37) than from other colleges (13) for the simple reasons that A&S is big and I know more people in my home college. Although the median (4) and the mode (4) were identical in both subsets, the mean of the A&S faculty (3.98) is significantly (by a two-tailed t-test) greater than the mean of the other colleges combined (3.58), p = .039. This difference in means suggests that A&S faculty have perceived a greater downward shift in performance than faculty from other colleges. Perhaps A&S faculty, on average, have more experience teaching lower division courses where the problems are likely to be more acute. B. First Year at NMSU. Across the entire sample of 49 faculty, both the mean and median starting year was 1978 with a range of 1961 to 1988. The best fitting straight line to the plot of rating against first year at NMSU has a slope of -.035. Thus, there is a tendency for faculty to perceive higher relative performance as their starting year increases. In a subsequent analysis I sorted the faculty into 3 groups based on their years of service. The means for this analysis are as follows: Mean Rating 1961-1975 mean=1971 n=16 4.3 1976-1983 mean=1979 n=18 3.9 1984-1988 mean=1984 n=15 3.7 The trend across time appears quite meaningful, but there's quite a bit of variability within each category. Thus, it isn't surprising that the only difference between means that is statistically significant (by a 1-tailed t-test) at a conventional level of .05 is the difference between the oldest group (mean = 4.3) and the youngest group (mean = 3.7), p = .027. CONCLUSIONS FROM SURVEY NMSU faculty that have been teaching for at least 12 years believe, on average, that student performance is "somewhat worse" now than it was when they first began to teach at NMSU. Faculty who have been here longer and faculty in the College of Arts & Sciences indicate a greater decline in performance than younger colleagues and colleagues in other colleges. It is, of course, difficult to interpret these results. From one perspective the responses were obtained from a group of Grumpy Old Persons (henceforth the GOP's perspective) whose impressions and recollections may not be accurate. That is, if NMSU had consistently obtained the same set of objective learning assessment measures from students over the past 3 decades, those hypothetical results may well differ from the subjective ratings reported in this survey. Unfortunately, no such objective data exists. >From another perspective we could be viewed as a Council Of Wisepersons (henceforth the COW's perspective, -perhaps appropriate to our tradition as a land grant university). In the absence of three decades of learning assessment measures the collective wisdom and institutional memory of the senior faculty may provide the best evidence we have. With respect to the GOP's versus COW's interpretation, one of us (Skipper Botsford) was actually able to bring some objective data to the discussion. Skipper has been teaching introductory microbiology since 1975 and by consulting his archives of old grade books was able to report the following: Spring 1975: 77 completed the course with 2 F's. Fall 1975: 70 completed the course with no F's. Right now in 2000: 55 will complete with about 10 F's. POSSIBLE CAUSES FOR A PERFORMANCE DECLINE I was very impressed with the detailed and thoughtful responses that many colleagues included in their responses. Many colleagues spoke to the possible causes of the performance decline. By far, the most common attribution is that students now have DIFFERENT ATTITUDES and EXPECTATIONS with respect to how hard they must work in order to achieve desired grades: " I give take home quizzes on Fridays to be turned in on Monday. By using their texts and their notes, they should be able to score well on these quizzes. They do no better than the students in other sections taking "in-class" quizzes. Some students do not even bother to do the quizzes. Many students show little ability at thinking through problems and are upset when I explain that there is no mechanical way to do the problems; that they have to think about and puzzle over the concepts. Further, some students really believe the nonsense that they are our "customers" in that they feel they have paid for the grade and thus do not have to work for the grade" "I thought that I was the only one who was experiencing difficulty motivating my students. In both of my classes, about 1/3 of the students do not attend on a regular basis. Of course, there are always a few students who are wonderful, and make it all worth while, but it's sad that so many don't seem to care at all. They turn up for the exams, but I end up giving out so many Fs that I feel guilty." "There are TWO distinct groups in nearly every survey I now teach. Currently, about 1/4 to 1/3 of the class is hopeless in its performance and most in this group have high absences. I do not believe that they read the assigned material or study. Most have trouble writing. The remaining 2/3 to 3/4 of the class I would say is probably not that much lower in quality from what I was teaching 15 years ago.... Based on input from students about the rigor of a large corpus of classes at NMSU, an undergraduate who is functionally illiterate, very crafty, or just plain lazy can meet this standard with little or no effort." "Part of the problem is something that has been learned - an attitude that it is the instructor's job to teach rather than the student's job to learn. If they fail my course it is because I failed to teach them, I was too demanding, the material was too unreasonably difficulty, etc. The fact that they missed most of the lectures, failed to open the textbook until the night before an exam (if then), and didn't do any of the homework, etc., clearly has nothing to do with their failure. Attendance in my lecture sessions typically runs about 60-75%, with the lowest attendance occurring, predictably, on Fridays. It can't be that my lectures are THAT bad, since the 25-40% absenteeism rate begins with the first day of classes and holds steady throughout the semester. (By the way, in my current class, the top 10% gradewise have missed an average if 2.4 lectures, the next 20% an average of 6.9, the middle 40% an average of 8.8, and the remainder an average of 10.2. That's based upon about 40 lectures at which I've taken attendance so far.)" "Currently 1/2 of my undergrad students are failing, all but 2 of my grad students are getting Bs or below, and I have 3 Cs and a D in my honors class. Everyone is furious with me. I want them to attend class, complete the assignments, and do the readings. They view this as a ridiculous request." "When I first came here students might do poorly on the first exam they took in C J 101, Intro. to C J. I used to ask: have you read the book? To which I would get a number of no answers. Now I don't even ask that. I ask: did you buy the book? I would guess that somewhat less than half of my students actually purchase the lone text for this course!" " As a broad generalization, students seem more over committed and more personally offended by having to study than in prior years. Two faculty members in my department have commented that they had looked back at some exams they gave ten or more years ago and wouldn't think of giving such "hard" tests now." " Students expect to be able to take a full load (or even an overload) of classes, work 30- 40 hours a week and make A's and B's. The expectations of many students is that they should not have to work hard, attend class often, or be on time when they do come to class." "I do think what has changed is the level of civility in classroom performance and student-teacher interaction. Students are altogether too willing to confess that they are not prepared and to brazen out their unpreparedness in the classroom. I think they used to make more effort to fake preparation out of respect or fear." " With respect to discipline, I now find a laid-back attitude with respect to attending class, tardiness, lack of interest in attending the class when an exam is returned and reviewed, picking up a graded exam since the review class was skipped, etc." "I had an interesting experience with a student the other day. He seems to believe that because he completed the assignment, completed each step, he deserved an A. Although he received a B for his work, he couldn't believe that I wouldn't give him an A for his work. It was clearly inferior to the work of the A students, but he was incredulous that I thought quality had something to do with his grade. I've seen this a few times in recent years. I cant explain it." " My main concern about the present crop of students is that many of them operate under the illusion that learning is the professors responsibility, and the corollary that students do not need to study outside the classroom. I had one student who claimed be be taking four classes and to be working 60 hours/week. He was upset that I required so much work." The degree of spontaneous comment on STUDENT PREPARATION for college was far less and showed less agreement: " I've worked closely with freshmen the whole time I've been here and I think our new students are actually better prepared than before." " I would have to say that their level of preparation is appallingly worse. I teach an introductory astronomy course (Astr 110G) section, sometimes two, almost every semester - and have done so since 1975 (!). I find the "typical" student (usually a freshman or sophomore) incapable of understanding mathematics at the level of very basic algebra, and sometimes unable to do basic arithmetic. They tend to be utterly ignorant of basic scientific ideas; and are equally uninformed about history, literature, and current events. They cannot really read; their writing is embarrassing. Apparently things like composition, spelling, etc., are no longer taught. On the other hand, they are not stupid, just totally uneducated and lacking in basic learning skills." "Haven't seen a major decline because overall the students have been weak all along, especially in reading ability (which leads to all sorts of problems)." " A much larger segment of the local population seems to be coming to NMSU than used to. If we tracked only those students today who correspond to the students we had 20 years ago or more, that would be a more valid test. We are now seeing people we wouldn't have seen before at all. They didn't get any education at all then, and so anything they get now might be an improvement!" " I have noticed a change in preparedness (in agriculture) of our students but that is more due to a shift in the demographics, i.e., more students coming from urban areas than rural. Also, in the last three or so years, I have noticed more marginal students enrolled in my introductory course. I think the lottery scholarship program is allowing more students to try college that otherwise might go the route of vocational education in the various community colleges. So, we may be seeing some dilution effect occurring. Overall, I believe the top-end students are better prepared in math, I still don't think they can write." THE ADELMAN REPORT ON DEGREE COMPLETION Relevant to my current thinking and the possible courses of action that I describe at the end of this report is an excellent recent report by Clifford Adelman (1999, Answers in the Tool Box: Academic Intensity, Attendance Patterns, and Bachelor's Degree Attainment, Office of Educational Research and Improvement, U. S. Department of Education). Adelman reports a longitudinal study of a large national sample (N > 13,000) of students from the time they were in the 10th grade in 1980 to roughly age 30 in 1993. Of the subset that attended 4-year colleges at some time, 63% earned a bachelor's degree. The report evaluates 24 predictors, i.e., factors that may be causally related to earning a degree. These include demographic factors such as gender, race, ethnicity, and socioeconomic status and also factors that are specific to education such as high-school preparation, high-school GPA/rank, and ACT/SAT scores. >From among the factors that are known when students graduate from high-school, by far, the most powerful predictor is the intensity and quality of the high-school "curriculum". It is far more important than test grades or high-school GPA/Rank. The study constructs a value of "curriculum intensity" for each student on a 40 point scale. The value is determined by evaluating the high-school transcript. In order to gain a feel for the "intensity" scale it is informative to consider the minimum criteria for obtaining the top mark of 40: a) 3.75 or more Carnegie units of mathematics, with no remedial math, b) 3.75 or more Carnegie units of English, with no remedial courses, c) 2.0 or more units of core laboratory science or 2.5 or more units of all science, d) 2.0 or more units of foreign language, e) 2.0 or more Carnegie units of history or 1.0 unit of history and 1.0 unit of either civics or other social studies, and f) more than 1 advanced placement course. When students are sorted into quintiles on the basis of their "curriculum intensity" scores there are dramatic differences with respect to the percentage who earn bachelor's degrees: Highest 2nd 3rd 4th Lowest 70% 44% 19% 5% 3% The "curriculum intensity" factor can compensate for racial or socioeconomic factors. For example, for the entire cohort of students who entered 4-year colleges directly from high school here are the percentage of degree earners: White 75.4% Black 45.1% Latino 60.8% But, for that subset of students with "curriculum intensity" scores that fall in the top 40% of all scores, the disadvantages are substantially reduced: White 85.7% Black 72.6% Latino 79.3%. Comparing these two sets of scores one can see the impact of a quality high-school curriculum is greater for African Americans and Latinos than for white students. Of all the components of "curriculum intensity" measure, none has such an obvious and powerful relationship to ultimate completion of degrees as the highest level of mathematics one studies in high school. If we asked simply what percentage of students at each rung of the high-school math ladder (viz., algebra 1 - geometry - algebra 2 - trigonometry - advanced math) earned a bachelor's degree, the largest leap takes place between algebra 2 and trigonometry: a nearly 23% increase among all high school graduates. Said another way, the odds (ratio versus everybody else) of earning a bachelor's degree increase from 1.54 for those taking algebra 2 to 3.83 for those taking trigonometry. NMSU's current requirement is for 3 units of high school math taken from algebra 1, geometry, algebra 2, trigonometry, or advanced math. Thus, students can meet this requirement without taking the important leap to trigonometry. The large set of possible predictors are analyzed in a series of both logistic and ordinary least squares regression analyses. At the time of high-school graduation the best predictors and their associated R squared values are: Curriculum Intensity .29 Test Scores (ACT,SAT,etc) .04 Rank/GPA .02 Socioeconomic Status quintile .01 Aspirations/Plans .004 Race .003 Sex .002 There are several significant predictors that become available after students have established a collegiate record, but the amount of academic resources (a composite measure that depends mostly on curriculum intensity) brought from high-school is still the best predictor: Academic Resources .17 Continuous Enrollment Once Started .10 Proportion Dropped/Incomplete .06 Low Freshmen Year Credits .02 Freshmen GPA .02 Parenthood .01 The optimistic message in the Adelman report is that the most important factor in predicting degree completion rates is a factor that can be changed with the right synergy among school systems, colleges, universities, and the state government. POSSIBLE ACTIONS AND SOLUTIONS Based on the evidence summarized above the following proposals are offered. 1. Increasing the admission requirements with respect to high school preparation. Based on the Adelman report we should expect that higher requirement will enhance: a) student readiness to perform college level work, b) student expectations regarding effort required to perform college level work, c) the retention rate of those students who do enroll at NMSU, d) the degree completion rate. As a preliminary subgoal I propose to work with the Office of Institutional Research to evaluate the high-school transcripts of our students using the Adelman 40-point scale. We need to know the distribution of "curriculum intensity" for our students and determine the functional relationship between this factor and measures like grades, credits earned per year, and degree completion. 2. Orientations for new students should occur at multiple levels (university or college, department, and introductory survey classes). These orientations should repeatedly and consistently stress that performance standards at NMSU are high (very different from high school) and that students will have to work hard in order to graduate. 3. Grading standards should be reasonable, but rigorous, across the entire campus. This has no chance of working unless a broad consensus of faculty agree to maintain or raise standards (as the individual case need be) and to reinforce the core message that learning requires effort. I suspect that a dramatic change can only be accomplished by pursuing all three of these proposals in parallel. We also need broad support from all constituencies. To that end, I will be taking these proposals to the Deans Advisory Council, the Faculty Senate, and a convocation of student leaders. It may also be worthwhile to obtain input from alumni and major recruiters of our bachelor's level graduates. It is possible that the Music Department has already demonstrated that some improvements can accrue from preaching the gospel of hard work and then insisting that students stand and deliver. Here's a comment from Jerry Ann Alt: "I've been at NMSU since 1986. Student performance has always been surprisingly weak. After much recruiting in the high schools, I understood why and that it was not likely to change. I began to structure our program so that students understood that expectations would be very different here than where they'd come from. Our department has worked very cohesively on this and we've had good results. I'm always amazed at how little our students will do when given the opportunity. What we try to do to change that is set very high performance goals and work in a uniform manner to see that they are achieved. If we supervise the students much more than professors would normally do, we are successful. When we ask them to work independently, we usually are not successful." Increasing our entrance requirements and our standards will, no doubt, have a short-term adverse effect on enrollments. The long-term expectation (hope?) is that high-schools (in some places in partnership with junior colleges) will increase the availability of key courses and encourage more high-school students to take them. If students are better prepared in high-school and properly aculturated during their first year at NMSU, then retention and graduation rates will eventually go up. If NMSU can earn a reputation for academic attitudes and performance that favorably competes with a good private college or with UNM then we will have a recruiting tool that we can all be proud of. Remember, ''retention'' has two main meanings: retention n 1: the act of keeping in your possession 2: the power of retaining and recalling past experience We need to focus on the second, just as much as on the first. Ken Paap [EMAIL PROTECTED] Department of Psychology, New Mexico State University <>< <>< <>< <>< <>< <>< <>< <>< <>< <>< <>< <>< <>< Miguel Roig, Ph.D. Voice: (718) 390-4513 Assoc. Prof. of Psychology Fax: (718) 442-3612 Dept. of Psychology [EMAIL PROTECTED] St. John's University [EMAIL PROTECTED] 300 Howard Avenue http://area51.stjohns.edu/~roig���� Staten Island, NY 10301���������� ><> ><> ><> ><> ><> ><> ><> ><> ><> ><> ><> ><> ><>
