PPIG Annual Conference in York ........ ROLL UP! ROLL UP!
Hi everyone Are you coming to PPIG? We sure hope so because we've got a nice program set up with some extremely interesting papers, plus: - a panel from industry-savvy experts on how cognitive factors turn up in unlikely-seeming places; - an invited paper by Gerrit van der Veer, one of the first authors to write about cognitive psychology of programming, since when he has chaired every important HCI conference you can think of; - and a presentation by Gordon Fletcher, anthropologist turned CS/ IT expert, on research methods using qualitative data. Not to mention York, a much-admired city, with a guided walk (weather permitting) and formal dinner in Meltons Too, an excellent restaurant in an old building accompanied by period live music. On the night before we've arranged a pub evening, and for those who wish to arrive on the evening of the 5th (e.g. to take part in the doctoral consortium, or just to hang out) there's casual grub and drinks at my house. If you are coming, please would you speedily register for the meeting? I know it's a little while away, but the university likes to get its room registrations in early. Help calm their restive minds - register NOW. website: http://www.cs.york.ac.uk/ppig2011/ NB I'm away next week: queries to Alistair Edwards, please, the co- chair. (Please forward) Thomas Green co-chair Visiting Professor University of Leeds, University of York -- The Open University is incorporated by Royal Charter (RC 000391), an exempt charity in England Wales and a charity registered in Scotland (SC 038302).
PPIG 2011
The dates for submitting papers to PPIG 2011 are approaching! Submission templates are now on the website, which is here: http://www.cs.york.ac.uk/ppig2011/ Regrettably the template for LaTex still makes you use the horrible numbered format, so that readers have to keep skipping to the back to find out who [1] is. Is there anyone out there who is competent to change it to the Name (year) format, AND who has time and inclination to do so? if so, I'd be very grateful if you could send me an improved version. Let me know if the templates give you any hassle. Registration details shouldn't be long now. Come and see the City of York. Thomas Green Visiting Professor University of York -- The Open University is incorporated by Royal Charter (RC 000391), an exempt charity in England Wales and a charity registered in Scotland (SC 038302).
Re: Magic features of programming languages
Kevlin Henney wrote: To really demonstrate autoboxing you need to allow the compiler to convert from int to Integer: Integer x = 1000; Integer y = 1000; However, if you are after interesting counterintuitive corner cases, change the constant to 100: Integer x = 100; Integer y = 100; A direct equality comparison will now compare true because, by default, the JVM caches the Integer objects for values from -128 to +127 (the range can be extended as an optimisation). In other words, your corner case has a corner case. Magic. Back in the sixties, the autocode for the Atlas machine at Harwell had a fine piece of magic. As an outsider, I could book for an occasional week there, and while I was there I could run programs twice a day iirc. I once spent two of those 7 days trying to find out why my program wouldn't work, panicking desperately about meeting my target, before discovering that of its 128 registers (called B-lines), the one I had chosen to use was fitted with a hardware conversion to return log-2 of any quantity stored in it. All the higher-numbered B-lines silently did special things, apparently. Great if you knew about them. Tough otherwise. (For those of you who came late to the party and missed the early days, an 'autocode' was a slightly-Englished version of machine code. Bit like a penny-farthing - if you stayed on, people admired you, but when you fell off it really showed.) Thomas -- The Open University is incorporated by Royal Charter (RC 000391), an exempt charity in England Wales and a charity registered in Scotland (SC 038302).
Re: URGENT: Testing Inclination to Programming
On 19 Mar 2011, at 09:55, Stefano Federici wrote: what I claim is the easiest programming environment ever designed so far). Er, yes. You might need to restrict what you mean by 'programming' . I regard using spreadsheets as programming. But Scratch is very good at its job, to be sure. I take it then that you're trying to out-do Scratch. Does this sound reasonable? Yes, it's probably the best you can do. I think the worst threat to generalisability is probably the risk of 'experimenter effect', where the students do better in the group that you want to do better. I don't know how to minimise that risk. If Sally Fincher or Mark Guzdial is listening, or anyone else with a good knowledge of these issues, I hope they'll join in. Good luck! Make sure to tell us how it goes. Thomas 73 Huntington Rd, York YO31 8RL 01904-673675 http://homepage.ntlworld.com/greenery/ -- The Open University is incorporated by Royal Charter (RC 000391), an exempt charity in England Wales and a charity registered in Scotland (SC 038302).
Re: Redefining the word language
And where do people want to put Inform 7, the interactive fiction language that is a subset of English and has some semantic inference built in? http://inform7.com/ Is it a programming language? a restricted natural language? both? neither? Thomas On 4 Mar 2011, at 17:17, Kari Laitinen wrote: The definition of 'language' depends on who you are talking to. I think that based on this discussion and earlier discussions, it is not always clear what the term programming language means. From the compilation point of view the term is clear, i.e., the lexical rules and the syntax of the language specify accurately what the language is. Kernighan and Ritchie used the term in that sense in their famous book, thus excluding printf from the language. From a human point of view, the term is less clear. A computer program usually contains names (identifiers) that must be understood by a person who wants to know how that particular program works. If a program contains definitions such as int nwhite, nother; int ndigit[10]; can we say that the names nwhite, nother, and ndigit, belong to the used programming language? If the answer is 'yes', all lexically correct names belong to the programming language, resulting in that the programming language is a huge set of symbols. If the answer is 'no', one might ask that into which language these names belong if they do not belong to the used programming language. They are not English words, if English words are those that can be found in an English dictionary. As both of these answers are somehow 'not good', I ended up proposing the idea that each computer program or any other document could be seen as containing its own language. Thus the above names would belong to the language of the program in which they were used. A name such as 'nwhite' can mean different things in different programs. In the program from which I copied it it was used to count white space characters. In a chess-playing application it might be used to count white chess pieces. A symbol can have a different meaning depending on in which language it is used. In the paper http://www.naturalprogramming.com/to_read/estimating_understandability_etc.pdf I have shown that these 'new' languages can be used to compare different naming styles in computer programs, and to show how computer programs relate to other software documents. Writing computer programs is a creative activity. If a computer program is considered to contain its own language, part of the programming project is then to create the language that is used in the program. -- The Open University is incorporated by Royal Charter (RC 000391), an exempt charity in England Wales and a charity registered in Scotland (SC 038302). 73 Huntington Rd, York YO31 8RL 01904-673675 http://homepage.ntlworld.com/greenery/
Re: evalutation of new tools to teach computer programming
Depending on your aims, you might want to measure transfer to other problems: that is, do participants who used tool A for the sorting task, then do better when tackling a new problem, possibly with a different tool, than participants who used tool B? You might also want to look at memory and savings: how do the participants manage two months later? Occasionally cognitive tasks like yours show no effect at the time but produce measurable differences when the same people do the same tasks later. Pretty hard to create a truly fair test, but things to think about are controlling for practice and order effects, which should be easy, and controlling for experimenter expectation effects. The hardest thing to balance for is sometimes the training period: people using a new tool have to learn about it, and that gives them practice effects that the controls might not get. Sometimes people create a dummy task for the control condition to avoid that problem; or you can compare different versions of the tools, with differing features. I suggest you try to avoid the simple A vs B design and instead look for a design when you can predict a trend: find A, B, C such that your theory says A B C. The statistical power is much better. Don't forget to talk to the people afterwards and get their opinions. Sometimes you can find they weren't playing the same game that you were. Good luck Thomas Green On 1 Mar 2011, at 11:20, Stefano Federici wrote: Dear Collegues, I need to plan an evaluation of the improvements brought by the usage of specific software tools when learning the basic concepts of computer programming (sequence, loop, variables, arrays, etc) and the specific topic of sorting algorithms. Which are the best practises for the necessary steps? I guess the steps should be: selection of test group, test of initial skills, partition of the test group in smaller homogenous groups, delivery of learning materials by or by not making use of the tools, test of final skills, comparative analysis. What am I supposed to do to perform a fair test? Any help or reference is welcome. Best Regards Stefano Federici - Professor of Computer Science University of Cagliari Dept. of Education and Philosophy Via Is Mirrionis 1, 09123 Cagliari, Italy - Tel: +39 349 818 1955 Fax: +39 070 675 7113 -- The Open University is incorporated by Royal Charter (RC 000391), an exempt charity in England Wales and a charity registered in Scotland (SC 038302). 73 Huntington Rd, York YO31 8RL 01904-673675 http://homepage.ntlworld.com/greenery/
Re: evalutation of new tools to teach computer programming
On 1 Mar 2011, at 14:29, Stefano Federici wrote: Do you have any references to similar evalutations? Now that Alan and Chuck have made the text of 'Psychology if Programming' available again, you might start there. Also look for work by Jorma Sajaniemi in Finland, and for lots of work by Margaret Burnett and her students in USA. @John Daughtry: Rather than asking which tool is best, you may be better served by seeking to empirically describe and explain the underlying trade-offs As an experimentalist, I'd see that as the goal of a _program_ of research. Any one study is usually about individual effects. I agree, though. It's certainly much much better to understand _why_ effects are happening, than to get one result and say That's That; and if John's subtext is that some studies in the literature are indeed rather shallow, I have to agree. Er, a spot of self-promotion here . the various types of comparison I did in the past led to a framework which attempts to make some sense of the underlying trade-offs, the cognitive dimensions framework, developed by me and lots of other people. Stefano, if you simply want to know whether your new tool works, then you probably just need to do an experiment and stop; but if you want to know why it works (or doesn't), you might take a look at that framework. There's a resources page here: http://www.cl.cam.ac.uk/~afb21/CognitiveDimensions/ CDs analysis is quite quick, though very vague. It's actually quite possible that it would reveal problems you've overlooked . Thomas -- The Open University is incorporated by Royal Charter (RC 000391), an exempt charity in England Wales and a charity registered in Scotland (SC 038302).
Re: How to do a small readability experiment?
Hi Richard Here are some detailed remarks about technique that you just might find useful. If it's grandmother and egg time, please accept my apologies. 1) Given your hypothesis, I don't think you need to use huge programs. If you get a significant result from something N lines long, you can expect a larger effect from something 10N lines long, unless there's something unusual happening. 2) There's a difference between reading for detail or verbatim recall versus reading for gist, well documented in the psycholinguistics literature. Françoise Détienne showed that the difference also applies in program comprehension, but I forget the details of her studies. You might want to use more than one measure, one for gist (like telling a story about the program) and one for detailed comprehension. Judith Good has done excellent work on telling stories about programs. My own work has mainly been on reading for detail. 3) If you want to look at detail, rather than gist, you're probably going to be looking at times or errors as the outcome measure. Time scores are quite nice to use, because they're much finer grain than number of errors. But it's essential that the participants know whether you want them to be incredibly accurate or incredibly fast - there's a trade-off - one option is present them a scenario, such as safety-critical software, and explain that you want them to be accurate even at the cost of slow performance. Or the other way round, if preferred. Using reading for gist, e.g. telling stores about the program, will require more work from the experimenter in assessing each reply. 4) When reading for detail, there's a difference between reasoning 'upstream' and reasoning 'downstream'. For an ordinary imperative structure, 'downstream' means in the direction of program flow, which is the easy way to reason. I did a study on answering questions about conditional structures, with professional programmers, using varieties of syntax for the conditional structure. Given something like if P then if Q then A else B else if R then ... etc, the downstream direction is : Given the truth-values for P, Q, R ..., what does this program do? The upstream direction is: The program did action B, what must have been the truth-values? What I found was very little difference (using time-to-answer) for the downstream direction, but large and very significant differences for upstream. I suggest therefore that you try to include both directions in your tests. 5) A technique you might consider for detail comprehension is a version of the cloze procedure (fill in the blank). I did an unpublished study using forced-choice from about 4 alternatives. Iirc the participants were told what the program was intended to do, and had to find one of the alternatives that gave the correct result - so for instance, if the program was to find the mean of N numbers, you could have a blank in the summation, and offer different operators; or you have a blank in the divisor, and offer N, or N+1, or something else. (My test programs were a good deal larger, of course.) You have to be careful that the answer can't be found on purely syntactic reasoning. This might be quite an attractive method for you, assuming you want to use something like time scores, because correct reasoning could be quite complex and therefore even small differences in readability might show up. 5) If you use time scores, you'll probably get skewed distributions (but not nearly as skewed as error scores usually seem to be, at least in my studies). Anova is said to be reasonably robust for small amounts of skew, as long as it's always in the same direction, but a technique I have frequently used is to supplement anova on the raw scores with anova on transformed scores - if the skew is positive (usual case), you can use a log transform, for example. If you get the same pattern of significances, you're home and dry. If you don't, er, it depends. 6) There are alternatives to complete Latin squares. Incomplete squares confound the controlled variables; if you can accept the confounding, you can save experimental effort. You'd need to look at a book on experimental design for more info. 7) Don't forget to talk to the participants afterwards and ask what they thought, how they did the tasks, and so on. You may find some surprises - I certainly have done. Can't think of anything else just now but open to questions. Good luck and make sure the results are made public! Thomas Green On 16 Jul 2010, at 07:42, Richard O'Keefe wrote: On Jul 16, 2010, at 2:20 AM, Alan Blackwell wrote: Hi Richard, A couple of clarifications - your primary experimental manipulation is 'style', is that right? Could you explain what this means? I thought I already said explicitly
