TEACHSOC: Re: To Del Thomas

[Kristen Hefley]

Hi Del.   Thanks for your response.  Please excuse my delay in responding.  I was away for a couple of days.  In the interest of several personal E-mails I received, I am posting this to the listserv so others may contribute to this conversation if they would like to.   I understand what software beta-testing is… I have beta-tested myself in the past.  I am just slightly confused by the beta-test metaphor.  When you beta-test software, you are helping to uncover the “bugs” in the program.  In talking about teaching activities, you want to know not only what the “bugs” in the program (activity) are, but also how it affects the learners.  In other words, the ultimate goal with software is to make the software the best it can be.  With teaching activities, the ultimate goal is to make the students the “best they can be,” if you’ll forgive the word choices here.  In the latter case, this is contingent upon the population of students you have, unlike with beta-testing software.  Without random samples, you can only know about your specific population of students.   So it seems a little confusing to me when you say, “Where is the science?”  I think we have to keep in mind that giving an activity to our students and then fine-tuning it based on their input is not scientific.  We may have found something that works with us as the professors, at the particular type of institution at which we teach, with the demographics of students who attend college in our area, and under the social and cultural conditions of the region, but we don’t want to expect that others would be able to use it in the exact same way.  In other words, it appears that each professor will have to do their own beta-testing.   Yet, by the postings on this listserv, it appears that a lot of what you respond to about beta-testing HAS been “beta-tested,” (according to your definition) as they are activities that professors have already been using for a long time and have modified to better meet the class needs, or are references to published works.  For example, in the journal Teaching Sociology, the authors usually submit some kind of evidence that the activity did what it was designed to do—i.e., the authors may ask the students about their learning or experiences with the activity as you suggest or may use another method, such as comparison of test responses to questions before and after.   Finally—as a side note—I’m not convinced that asking students “What did you learn?” is the only way (or even the best) way to gauge learning.  I’m not a psychologist, but I do know that sometimes people are not consciously aware of what they know or think.  I think a learning statement by students (or the equivalent) is useful and probably should be implemented, but it should not be the only standard by which we conduct our classes.  I have seen studies where students feel like and say they are learning when they are entertained or are having fun, but their knowledge level actually does not change very much, if at all, on “objective” tests or questionnaires, especially over the long-term.   I’m not sure if we will resolve this, but I did want to provide more input on what I was referring to in the first E-mail and provide another side to the issue.  Thank you again for responding.   Take care, Kristen   -----Original Message----- From: Del Thomas Ph D [mailto:[EMAIL PROTECTED]] Sent: Wednesday, May 31, 2006 9:36 AM To: [EMAIL PROTECTED] Cc: teachsoc@googlegroups.com Subject: Re: TEACHSOC: To Del Thomas   Hi khefley, I'm pleased that u see a value in testing. I have been involved in beta testing for a couple of decades.  Mainly software.  I thought that if we take that much care with games (that cost $40) and word processors why not instructional materials for which students pay thousands.  The texts we use could be considered software for the brain.    In general while many of the instructional material used in the classroom are heavily vetted they are not tested.  In particular they are not tested on the user population. Beta testing is just one of many types of testing.  However, there are a few things about beta testing you should consider.  Beta testing does not require random samples. There are very specific protocols available,  however, none of these are required to start testing instructional materials. ( I include on my website a way to do this is in class as a project.)  The main thing is to test the materials on a user group to improve.... or understand them.   For one thing it gives ownership to students.    [EMAIL PROTECTED] wrote: I see on your website that many of the things that you do also do not appear to be "beta-tested" either. Please tell me If that is of such concern to you, I am curious as to why you don't make a living out of beta testing popular or seemingly useful assignments?  what things on the website were not tested.   As I said Beta testing in only one way of testing... I have all students submit a learning statement with each assignment which is a measure of what they have learned.  The software I use has been beta tested as have the exercises. [EMAIL PROTECTED] wrote: If that is of such concern to you, I am curious as to why you don't make a living out of beta testing popular or seemingly useful assignments?   Is this meant to suggest that I am the only one concerned about the use of untested materials in the classroom?  As for whether or not untested activitiesare poor choices.....  will we ever know without testing?  I will be happy to aid and answer questions that go beyond the example on my website. Best Del    [EMAIL PROTECTED] wrote: Del: I have watched you over the years say over and over again, "This isn't beta tested."  If that is of such concern to you, I am curious as to why you don't make a living out of beta testing popular or seemingly useful assignments?  I see on your website that many of the things that you do also do not appear to be "beta-tested" either.  This doesn't make them poor choices for activities.  Yet because we only teach the students we are given (which are far from random samples), there is little way to truly "beta-test" as a professor or teacher.  That is only the tip of the iceberg of my thoughts on this matter, but I have to admit that every time I see you say that, I wonder why you do not try to conduct your own research.  Your results could be valuable to most teachers/professors, but as almost NOTHING we do has been researched about its effects, it becomes meaningless to say that. Thanks...   style='margin-left:36.0pt'>    --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Teaching Sociology" group. To post to this group, send email to teachsoc@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/teachsoc -~----------~----~----~----~------~----~------~--~---