Dear All, On Mon, Nov 21, 2016 at 05:10:42PM -0500, Jory Schossau wrote: > A twist on this could be to ask people to write down what they thought the > code would do...?
yes -- with suitable snippets of code that might very well complement the "would you know how to do X" questions and provide a way to moderate them for realism... > Karin > > True, but what I like about the recall experiment is that it's easy to > automate with a fuzzy matcher because the rubric is so clear. But while > chess board recall seemed to translate well to skill level, I don't know > off hand if that is the case with programming. Luis Fernandez-Izquierdo > already brought this point up in the blog post comments. For instance, > someone with eidetic memory will appear to be an advanced programmer, > though that case is easy to throw out with a simple question. It might just > be a matter of how much code is on screen to rule out all but the extreme > outliers. True, anyone with an unusual ability to remember lines of stuff without understanding them will be misgauged by this type of approach. I'd speculate that departures from the template text may be informative beyond some fuzzy string distance approach -- e.g. if the template is a shell for loop, mistyping the loop variable name once or omitting the "$" where it is necessary would suggest they didn't recognise the variable or aren't familiar with how the shell uses "$" for dereferencing, while using a different name consistently would suggest they recognised the variable but couldn't / didn't bother to memorise the exact name. Departures from the template could be quite like "plausible distractors", but with the difference that they're made by the takers themselves and open in that sense. Best regards, Jan > - Jory > _______________________________________________ > Discuss mailing list > [email protected] > http://lists.software-carpentry.org/listinfo/discuss -- +- Jan T. Kim -------------------------------------------------------+ | email: [email protected] | | WWW: http://www.jtkim.dreamhosters.com/ | *-----=< hierarchical systems are for files, not for humans >=-----* _______________________________________________ Discuss mailing list [email protected] http://lists.software-carpentry.org/listinfo/discuss
