Re: [ppig-discuss] Beginner friendly error messages

[Mark Levison]

Linda - when you have run some experiments on this amazing victims (oops
kids) could you please share what you learn? I have some small small
influence at my daughter's middle/high school. They're interested, eager
and unschooled. I'm eager, overskilled and suffer the curse of knowledge.
30 yrs of professional coding experience makes me useless in helping them
help kids learn. Please share what you learn.

Cheers
Mark

On Mon, Apr 11, 2016 at 7:46 PM, Linda McIver 
wrote:

> Wow, so many familiar names in my email, I feel I should start making
> comments about getting the band back together. Ceilidh, anyone? :-)
>
> On 6 April 2016 at 20:22, Alan Blackwell  wrote:
>
>>
>> > On 6 Apr 2016, at 07:04, Linda McIver  wrote:
>> >
>> > The main thing is that I want to teach them real skills that they can
>> continue to lose.
>>
>> :-)
>>
>
> This was, sad to say, unintended, but possibly Freudian! :)
>
>>
>> Nice to hear from you Linda! We have been thinking hard about error
>> messages in Sonic Pi, which I guess has some similarity to your situation:
>> 1) It is a domain specific language; 2) We’re very concerned that students
>> should move past errors quickly, rather than becoming demotivated or having
>> learning obstructed by irrelevant questions.
>>
>> I am amazed how your second point is discounted or devalued by the
> programming language community. If we allow students to be put off by their
> first interactions with the language, we risk losing the very diversity we
> claim to be so concerned about attracting. People will never find out if
> they like programming and are good at it if they are chased away by this
> kind of perplexing language. I'm glad to find through this thread that
> people are thinking about it and making progress, and I wonder how we can
> transmit that progress to mainstream systems?
>
>
>> Our main approach has been to observe, based on lots of practical
>> classroom usage, what kinds of error situation frequently arise out of the
>> things students try to do, and then intercept and replace the standard
>> (Ruby) compiler message with a piece of domain-specific advice. There have
>> also been some cases where we were able to make particular kinds of error
>> impossible, by preprocessing the student’s text before compiling to
>> effectively provide more natural point-replacements for Ruby syntax.
>>
>> For errors that drop through these, students still get the standard Ruby
>> errors, which may not be very useful (though of course we provided the best
>> support we could to try and direct students to a likely location of the
>> problem in their source code). However, the cases that we did manage to
>> trap probably accounted for 80 or 90% of the typical student problems, so
>> there are probably some easy wins.
>>
>>
> I think that low hanging fruit would be enough to get us through, do you
> have any studies on the impact of your changes?
>
> Thanks all for such an interesting discussion. I'm unlikely to have the
> resources to research the area myself, so I'm very keen to learn from
> others who are systematically exploring such an important field, so that we
> can implement something to make life easier for our year 10s.
>
>
>
> --
>
> Exploring Life, Parenting and Social Justice:
> http://lindamciver.wordpress.com/
> Computational Science Education: http://computeitsimple.wordpress.com/
>
>
> Dr Linda McIver
> Teacher & Freelance Writer
> --
> Buy Fair Trade - Change the world one coffee at a time
>
> --
> You received this message because you are subscribed to the Google Groups
> "PPIG Discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to ppig-discuss+unsubscr...@googlegroups.com.
> To post to this group, send email to ppig-discuss@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>



-- 

[image: headshot-square-300x300]
 *Mark Levison* | 1
(877) 248-8277 | Twitter  | LinkedIn
 | Facebook

Certified ScrumMaster Training: Vancouver
 | Edmonton
 | Ottawa
 | Montreal
 | Toronto

Certified Product Owner & Private Training also available ~ Our Training
Schedule 
Agile Pain Relief Consulting  | Notes from a
Tool User 
Proud Sponsor of Agile Tour Gatineau Ottawa  and Agile
Coach Camp Canada 

-- 
You received this message because you are subscribed to the Google Groups "PPIG 
Discuss" group.
To unsubscribe from thi

Re: [ppig-discuss] Beginner friendly error messages

[Linda McIver]

Wow, so many familiar names in my email, I feel I should start making
comments about getting the band back together. Ceilidh, anyone? :-)

On 6 April 2016 at 20:22, Alan Blackwell  wrote:

>
> > On 6 Apr 2016, at 07:04, Linda McIver  wrote:
> >
> > The main thing is that I want to teach them real skills that they can
> continue to lose.
>
> :-)
>

This was, sad to say, unintended, but possibly Freudian! :)

>
> Nice to hear from you Linda! We have been thinking hard about error
> messages in Sonic Pi, which I guess has some similarity to your situation:
> 1) It is a domain specific language; 2) We’re very concerned that students
> should move past errors quickly, rather than becoming demotivated or having
> learning obstructed by irrelevant questions.
>
> I am amazed how your second point is discounted or devalued by the
programming language community. If we allow students to be put off by their
first interactions with the language, we risk losing the very diversity we
claim to be so concerned about attracting. People will never find out if
they like programming and are good at it if they are chased away by this
kind of perplexing language. I'm glad to find through this thread that
people are thinking about it and making progress, and I wonder how we can
transmit that progress to mainstream systems?


> Our main approach has been to observe, based on lots of practical
> classroom usage, what kinds of error situation frequently arise out of the
> things students try to do, and then intercept and replace the standard
> (Ruby) compiler message with a piece of domain-specific advice. There have
> also been some cases where we were able to make particular kinds of error
> impossible, by preprocessing the student’s text before compiling to
> effectively provide more natural point-replacements for Ruby syntax.
>
> For errors that drop through these, students still get the standard Ruby
> errors, which may not be very useful (though of course we provided the best
> support we could to try and direct students to a likely location of the
> problem in their source code). However, the cases that we did manage to
> trap probably accounted for 80 or 90% of the typical student problems, so
> there are probably some easy wins.
>
>
I think that low hanging fruit would be enough to get us through, do you
have any studies on the impact of your changes?

Thanks all for such an interesting discussion. I'm unlikely to have the
resources to research the area myself, so I'm very keen to learn from
others who are systematically exploring such an important field, so that we
can implement something to make life easier for our year 10s.



-- 

Exploring Life, Parenting and Social Justice:
http://lindamciver.wordpress.com/
Computational Science Education: http://computeitsimple.wordpress.com/


Dr Linda McIver
Teacher & Freelance Writer
--
Buy Fair Trade - Change the world one coffee at a time

-- 
You received this message because you are subscribed to the Google Groups "PPIG 
Discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to ppig-discuss+unsubscr...@googlegroups.com.
To post to this group, send an email to ppig-discuss@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [ppig-discuss] Beginner friendly error messages

[Michael Sloan]

When we demand that error messages are plaintext, there is a direct
tradeoff between how intimidating / overwhelming the error is and how
informative it is.  Quite simply, the more information that the compiler
provides, the more.  For example, C++ template errors will happily dump
their guts, giving you a hopelessly detailed account of what went wrong.

I think this tricky tradeoff can be greatly mitigated by making error
messages interrogatable! Take a look at this part of David Christiansen's
talk about his work on Idris's pretty-printer:

https://www.youtube.com/watch?v=m7BBCcIDXSg&t=10m50s

In dependently typed systems, you can check lots of properties of your
programs, but this comes with having very complicated types, and
correspondingly type errors.  This makes it almost necessary for
productivity to do something better about the presentation of errors.

This is not limited to compilation errors.  For example, what if your
runtime errors provided some of the program values in an interrogatable
fashion?  I think that could be very handy, particularly if it is difficult
to attach a debugger to the process (e.g. debugging a rare failure in a
distributed system)

-Michael

On Tue, Apr 5, 2016 at 9:49 PM, Linda McIver  wrote:

> Thanks all for pointers to useful info.
>
> To clarify my data science unit: It will be a basic intro to data analysis
> and visualisation, with some spreadsheet work and some programming work.
>
> Richard: I wholeheartedly agree that incorrect (but terribly
> well-intentioned) error messages are worse than unfriendly ones! But I feel
> we could simply rewrite the existing ones to make them more intelligible
> and less intimidating. I am aware that this is far less straightforward
> than it sounds!
>
> I may have found my next area of research, although as I am a high school
> teacher these days, research is rather lower on my agenda than it used to
> be.  I finished a PhD on introductory programming 15 years ago, and I
> confess I am a little startled at how little progress there appears to have
> been in this area. Surely I am missing something???
>
> On 6 April 2016 at 11:41, Richard A. O'Keefe  wrote:
>
>> On 6/04/16 12:47 pm, Luke Church wrote:
>>
>>> Based on observational studies, I've generally recommended to the Dart <
>>> http://www.dartlang.org/> team that the verbose errors state:
>>>
>>> - *What happened* (e.g. File 'stuff, ln 43:' class B doesn't implement
>>> method foo)
>>> - *Why it is a problem* (e.g. class B extends class A, which has an
>>> abstract method foo. This means that class B doesn't have an implementation
>>> of foo)
>>> - *What the user can do next* (e.g. consider implementing foo, or making
>>> class B abstract)
>>>
>>
>> If I may add an anecdote, I once used a programming system
>> whose authors had put enormous effort into writing "helpful"
>> error messages.  But I found them infuriating.  Much of the
>> text would explain what the program thought you were trying
>> to do, but it was almost always wrong about that.
>>
>> Take this particular example as an example:  maybe the real
>> problem is that the student was trying to call food(), not
>> foo(), in which case advice to make B abstract is worse than
>> useless.
>>
>> An error is detected when the system expects something and
>> something else happens.  Saying clearly *where* the error
>> was is important (and I know my own programs are not very
>> good at this), as is saying what the system expected and what
>> it thinks it got.
>>
>> In the Haskell world, one group went to the trouble of building
>> a whole new system called Helium, in part so that they could
>> provide human-friendly error messages.
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "PPIG Discuss" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to ppig-discuss+unsubscr...@googlegroups.com.
>> To post to this group, send an email to ppig-discuss@googlegroups.com.
>>
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
>
> --
>
> Exploring Life, Parenting and Social Justice:
> http://lindamciver.wordpress.com/
> Computational Science Education: http://computeitsimple.wordpress.com/
>
>
> Dr Linda McIver
> Teacher & Freelance Writer
> --
> Buy Fair Trade - Change the world one coffee at a time
>
> --
> You received this message because you are subscribed to the Google Groups
> "PPIG Discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to ppig-discuss+unsubscr...@googlegroups.com.
> To post to this group, send email to ppig-discuss@googlegroups.com.
>
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups "PPIG 
Discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to ppig-discuss+unsubscr...@googlegroups.com.
To post to this group, send an email to ppi

Re: [ppig-discuss] Beginner friendly error messages

[ok]

>> On 6/04/16, Linda McIver wrote:

> A couple of reasons. 1. It is widely used in real data science.


"It is widely used by experts" does not mean
"it is suitable for beginners."  All sorts of things are used
in real data science, including the seriously weird SQL.

> 2. nice clean human-readable syntax
> (this is where R seems to hit the wall)

Python has, um, quirks of its own.  Which is why Python 3
exists, but Python 2 isn't going away like it was supposed
to.  So we have the unfortunate case of "Python" programs
that don't work in "Python" (both ways around).

The sadly amusing thing here is that R syntax is a straight
steal from S, and S syntax was largely adapted from C
*in order to have something programmers would be comfortable
with*.  And I must say that compared with GLIM, GenStat, and
SAS, it was a success.  R syntax really isn't as bad as it
seems.  You could call it "C: the good bits" + keyword
parameters.  I think the main barrier for experienced
programmers is the heavy used of "vectorised" forms with
subscripting overloaded in a number of very useful ways,
and the whole-object thinking you have to do.  So maybe
the semantics is the key issue.

Which has me wondering if you evaluated Octave?
(I have a copy of Octave, but almost never used it, so I
have no idea of the quality of its error messages.)

> 3. We
> will do some spreadsheet stuff, but they won't all have access to Excel,

They COULD all have access to LibreOffice.

Spreadsheets are interesting because of Ray Panko's
work on errors and work on spreadsheets.
"What we know about spreadsheet errors."
http://panko.shidler.hawaii.edu/My%20Publications/Whatknow.htm
"there has long been ample evidence that errors in spreadsheets are
pandemic. Spreadsheets, even after careful development, contain errors in
one percent or more of all formula cells. In large spreadsheets with
thousands of formulas, there will be dozens of undetected errors. Even
significant errors may go undetected because formal testing in spreadsheet
development is rare and because even serious errors may not be apparent."

So there's something worse than poor error messages,
and that's *missing* error messages.

This is one reason why I wish DrRacket included a a statistics
(sub)language.
Scheme, like Python and R, is a dynamically typed language, but
DrRacket works hard to try to detect some type errors at
compile time.  (This was one of the strengths of ABC.  It did
not require or even allow the programmer to provide type
annotations, but it did do quite strong type inference/checking,
and this was because it was meant for beginner use.  It's a real
pity that Guido van Rossum abandoned that for Python.)  The
problem with run-time errors is that they are reported a long
time after you wrote the mistake that caused them.

Which leads me to F#.
F# is an interesting candidate for doing data science
because
 - it is an interactive programming language
 - that runs under .NET and Mono
 - which compiles via the .NET intermediate language to
   tolerably good native code, making it practical for
   large amounts of data
 - which has kit available for "data science"
   (search for F# for data science) e.g. http://fslab.org/
 - having an indentation-based syntax these days
 - and is as concise as Python or more so
 - but is strongly typed so more errors are caught early.

I don't know how well novices would cope with its error
messages.  What I'm suggesting here is that the question
of WHEN an error is reported may be as important as HOW
it is reported, and that F#-for-data-science vs Python-
for-data-science might be one way of exploring that.
(The F# compiler is open source, so it should be possible
to reword its error messages.)

There's another issue, which is that syntax-colouring
editors (which I personally hate with a visceral hatred)
may help novices.  For example, Squeak and Pharo have
editors where when you type an identifier, it is red if
the compiler doesn't recognise it, black if it does, so
most spelling mistakes are instantly reported.  Bracket
matching is another way of unobtrusively reporting some
errors very early.

So how are your students writing Python?
Inside IDLE?  Notepad?  Some other way?

> and I want them to learn some basic programming as well, in particular for
> data extraction/cleaning type tasks that rather stretch excel's
> capabilities. I know you can program macros to do most of this stuff, but
> hey presto we are suddenly teaching programming in a
> less-than-beginner-friendly syntax and system.

Indeed.

>
> The main thing is that I want to teach them real skills that they can
> continue to lose.
  
Ah, aging.

> Even worse, many of them
> are outright frightened by computation.

There is something worse than a scientist frightened by
computation, and that's one who isn't.  It's OK when it's
an MSc or PhD student, but any project with serious
funding that has nontrivial coding to do would do well to
hire someone who is good 

Re: [ppig-discuss] Beginner friendly error messages

[Alan Blackwell]

> On 6 Apr 2016, at 07:04, Linda McIver  wrote:
> 
> The main thing is that I want to teach them real skills that they can 
> continue to lose.

:-)

Nice to hear from you Linda! We have been thinking hard about error messages in 
Sonic Pi, which I guess has some similarity to your situation: 1) It is a 
domain specific language; 2) We’re very concerned that students should move 
past errors quickly, rather than becoming demotivated or having learning 
obstructed by irrelevant questions.

Our main approach has been to observe, based on lots of practical classroom 
usage, what kinds of error situation frequently arise out of the things 
students try to do, and then intercept and replace the standard (Ruby) compiler 
message with a piece of domain-specific advice. There have also been some cases 
where we were able to make particular kinds of error impossible, by 
preprocessing the student’s text before compiling to effectively provide more 
natural point-replacements for Ruby syntax.

For errors that drop through these, students still get the standard Ruby 
errors, which may not be very useful (though of course we provided the best 
support we could to try and direct students to a likely location of the problem 
in their source code). However, the cases that we did manage to trap probably 
accounted for 80 or 90% of the typical student problems, so there are probably 
some easy wins.

Alan
--
Alan Blackwell
Professor of Interdisciplinary Design, University of Cambridge
Further details from www.cl.cam.ac.uk/~afb21/




-- 
You received this message because you are subscribed to the Google Groups "PPIG 
Discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to ppig-discuss+unsubscr...@googlegroups.com.
To post to this group, send an email to ppig-discuss@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

RE: [ppig-discuss] Beginner friendly error messages

[Enda Dunican]

Hello All,

Good to hear from so many familiar PPIG contributors

>>This seems to work reasonably well in verbose form, but does need to be 
>>accompanied by a >>terser format, especially for experts who seem to on a 
>>first pass only pay attention to where >>something went wrong, and ignore the 
>>rest.

I agree with Luke here and find that using verbose descriptions of code 
statements works well in programming teaching/learning in general. These 
verbose descriptions can become terser and terser as the student moves further 
along the proficiency continuum.  In fact, this approach can be 
developed/tailored by the individual student and can be very useful from a 
self-regulated learning perspective.

>>I agree this experience is critical in education. I conjecture that it's a 
>>necessary component of >>maintaining self-efficacy amongst students. There 
>>have been studies in the past indicating that >>this is an important factor 
>>in success.

Once again I agree with Luke and I feel that the whole area of SRL in novice 
computer programming education is under-researched.



messages, which surprises me.
Am I missing something? Is there a treasure trove somewhere? If not, is there 
at least some solid research on which we could base the design of a beginner 
friendly Python interpreter?
Any and all clues gratefully received.
Linda

--

Exploring Life, Parenting and Social Justice: http://lindamciver.wordpress.com/
Computational Science Education: http://computeitsimple.wordpress.com/

Dr Linda McIver
Teacher & Freelance Writer
--
Buy Fair Trade - Change the world one coffee at a time
--
You received this message because you are subscribed to the Google Groups "PPIG 
Discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to 
ppig-discuss+unsubscr...@googlegroups.com.
To post to this group, send email to 
ppig-discuss@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "PPIG 
Discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to 
ppig-discuss+unsubscr...@googlegroups.com.
To post to this group, send email to 
ppig-discuss@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups "PPIG 
Discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to ppig-discuss+unsubscr...@googlegroups.com.
To post to this group, send an email to ppig-discuss@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [ppig-discuss] Beginner friendly error messages

[Dan Sumption]

I'm not aware of any research around this, but the Elm programming language
is doing a lot to improve error messages - it's still far from perfect, but
better than other compilers/interpreters I've used.

See http://elm-lang.org/blog/compiler-errors-for-humans for more info.

On 6 April 2016 at 07:04, Linda McIver  wrote:

> Thanks very much, awesome resources.
>
> On 6 April 2016 at 15:53, Richard A. O'Keefe  wrote:
>
>> On 6/04/16 4:49 pm, Linda McIver wrote:
>>
>> Finally, why Python?  If your intention is to teach *data science*,
>> there is a case to be made for (shudder, nausea) Excel.  The
>> R programming language is admittedly somewhat arcane, but
>> "R Commander" (http://socserv.mcmaster.ca/jfox/Misc/Rcmdr/)
>> should be a lot easier for beginners to cope with.
>>
>> I'm not saying that Python is *wrong*, especially not with NumPy,
>> SciPy, and so on available, just that it's not *obviously* the best
>> choice for teaching *data science*.  (For teaching *programming*,
>> that's another question.)
>>
>>
>>
>>
>>
>>
> A couple of reasons. 1. It is widely used in real data science. 2. nice
> clean human-readable syntax (this is where R seems to hit the wall) 3. We
> will do some spreadsheet stuff, but they won't all have access to Excel,
> and I want them to learn some basic programming as well, in particular for
> data extraction/cleaning type tasks that rather stretch excel's
> capabilities. I know you can program macros to do most of this stuff, but
> hey presto we are suddenly teaching programming in a
> less-than-beginner-friendly syntax and system.
>
> The main thing is that I want to teach them real skills that they can
> continue to lose. I find this is crucial for motivation. Also I am
> astounded by how many of the scientists I collaborate with can't even begin
> to contemplate doing anything remotely like this. Even worse, many of them
> are outright frightened by computation. I teach at a science school. Most
> of the kids will go into science of some form or other. I want them to
> develop confidence with computing, and a basic understanding of data
> science.
>
> --
>
> Exploring Life, Parenting and Social Justice:
> http://lindamciver.wordpress.com/
> Computational Science Education: http://computeitsimple.wordpress.com/
>
>
> Dr Linda McIver
> Teacher & Freelance Writer
> --
> Buy Fair Trade - Change the world one coffee at a time
>
> --
> You received this message because you are subscribed to the Google Groups
> "PPIG Discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to ppig-discuss+unsubscr...@googlegroups.com.
> To post to this group, send email to ppig-discuss@googlegroups.com.
>
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups "PPIG 
Discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to ppig-discuss+unsubscr...@googlegroups.com.
To post to this group, send an email to ppig-discuss@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [ppig-discuss] Beginner friendly error messages

[Linda McIver]

Thanks very much, awesome resources.

On 6 April 2016 at 15:53, Richard A. O'Keefe  wrote:

> On 6/04/16 4:49 pm, Linda McIver wrote:
>
> Finally, why Python?  If your intention is to teach *data science*,
> there is a case to be made for (shudder, nausea) Excel.  The
> R programming language is admittedly somewhat arcane, but
> "R Commander" (http://socserv.mcmaster.ca/jfox/Misc/Rcmdr/)
> should be a lot easier for beginners to cope with.
>
> I'm not saying that Python is *wrong*, especially not with NumPy,
> SciPy, and so on available, just that it's not *obviously* the best
> choice for teaching *data science*.  (For teaching *programming*,
> that's another question.)
>
>
>
>
>
>
A couple of reasons. 1. It is widely used in real data science. 2. nice
clean human-readable syntax (this is where R seems to hit the wall) 3. We
will do some spreadsheet stuff, but they won't all have access to Excel,
and I want them to learn some basic programming as well, in particular for
data extraction/cleaning type tasks that rather stretch excel's
capabilities. I know you can program macros to do most of this stuff, but
hey presto we are suddenly teaching programming in a
less-than-beginner-friendly syntax and system.

The main thing is that I want to teach them real skills that they can
continue to lose. I find this is crucial for motivation. Also I am
astounded by how many of the scientists I collaborate with can't even begin
to contemplate doing anything remotely like this. Even worse, many of them
are outright frightened by computation. I teach at a science school. Most
of the kids will go into science of some form or other. I want them to
develop confidence with computing, and a basic understanding of data
science.

-- 

Exploring Life, Parenting and Social Justice:
http://lindamciver.wordpress.com/
Computational Science Education: http://computeitsimple.wordpress.com/


Dr Linda McIver
Teacher & Freelance Writer
--
Buy Fair Trade - Change the world one coffee at a time

-- 
You received this message because you are subscribed to the Google Groups "PPIG 
Discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to ppig-discuss+unsubscr...@googlegroups.com.
To post to this group, send an email to ppig-discuss@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [ppig-discuss] Beginner friendly error messages

[Richard A. O'Keefe]

On 6/04/16 4:49 pm, Linda McIver wrote:
To clarify my data science unit: It will be a basic intro to data 
analysis and visualisation, with some spreadsheet work and some 
programming work.

I sought (Research beginner understand error message)

http://ics.p.lodz.pl/~stolarek/_media/pl:research:stolarek_understanding_basic_haskell_error_messages.pdf

reports on trouble with Haskell error messages and explains how to
read them.  (The examples actually look pretty good to me, but I
am not a beginner.)  A review of this sort might be helpful to your
students.

https://cs.brown.edu/~sk/Publications/Papers/Published/mfk-measur-effect-error-msg-novice/paper.pdf

is the paper Mark Guzdial mentioned, I think.

http://digitalcommons.morris.umn.edu/cgi/viewcontent.cgi?article=1029&context=horizons

looks further at DrRacket (and also at C++).

http://www.cs.bham.ac.uk/research/projects/cogaff/sloman.beginners.html

is about what beginners need in general, by someone who had a great
deal to do with the POP-11 (POPLOG) system and teaching it.  He
has little to say about error messages, but a fair bit of attention was
given to them in POP-11, as I recall, and the editor VED was supposed
to help a lot.  (Me, I was completely baffled by it...)  This 
reminds me

that some teaching environments used language-sensitive editors that
tried to catch syntax errors as you typed.

http://se.ethz.ch/~meyer/publications/teaching/compiler-errors.pdf

One of the authors is Bertrand Meyer, inventor of the Eiffel
programming language, very keen on helping people to write
correct programs, and amongst other things developer of a
course (and associated book) using Eiffel, with some novel
ideas.  "The results indicate that more detailed messages do
not necessarily simplify the understanding of errors but that
it matters more where information is placed and how it is
structured."

http://www.chargueraud.org/research/2015/ocaml_errors/ocaml_errors.pdf

This is specifically about type-error messages, a special problem for
functional languages (SML, OCaml, F#, Clean, Haskell, Mercury) using
type inference, where types can be quite complex.  Looks like a
nice piece of work, but not really relevant to Python.  (The worse
for Python!)

http://www.hindawi.com/journals/ahci/2010/602570/

I suppose the Human-Computer Interaction field is where we should
expect such things to be published.  I think you would agree with
"It is striking that there is little concern on devising techniques to
help the user of the system ... to do their job properly.  This issue
is ignored, not only in compiler textbooks (no matter how recent
... or advanced ... they may be), but also, and most importantly,
by current research directions.  This paper has a useful survey of
previous work.

http://www.nomensa.com/blog/2010/4-rules-displaying-error-messages-user-experience-perspective

is about error messages in form processing, but
1. The error message needs to be short and meaningful
2. The placement of the message needs to be associated
  with (the thing that's complained of).

https://web.cs.wpi.edu/~kfisler/Research/#novices

mentions work Kathi Fisler is doing in this area; surprise surprise
it's in the context of DrRacket.

http://gracelang.org/applications/home/

Grace has already been mentioned in this thread.
To me it looks harder to get started with than Python.

http://ints.io/daveagp/research/2015/error-frequency.pdf

Asks which error messages are commonest because the
author wanted to "write(e) error explanations for novices".
Has "large data sets [for] Python and Java".
Now this paper says "There is copious literature on writing
good error messages" and lists 6 papers.  Explanations
were written for 36 messages.  One of the papers is a
1974 one "What the compiler should tell the user" that I
was meaning to mention anyway, although it's rather
batch-oriented.

This might actually be directly useful to you.


Finally, why Python?  If your intention is to teach *data science*,
there is a case to be made for (shudder, nausea) Excel.  The
R programming language is admittedly somewhat arcane, but
"R Commander" (http://socserv.mcmaster.ca/jfox/Misc/Rcmdr/)
should be a lot easier for beginners to cope with.

I'm not saying that Python is *wrong*, especially not with NumPy,
SciPy, and so on available, just that it's not *obviously* the best
choice for teaching *data science*.  (For teaching *programming*,
that's another question.)





--
You received this message because you are subscribed to the Google Groups "PPIG 
Discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to ppig-discuss+unsubscr...@googlegroups.com.
To post to this group, send an email to ppig-discuss@googlegroups.com.
For more options, visit https://gr

Re: [ppig-discuss] Beginner friendly error messages

[Linda McIver]

Thanks all for pointers to useful info.

To clarify my data science unit: It will be a basic intro to data analysis
and visualisation, with some spreadsheet work and some programming work.

Richard: I wholeheartedly agree that incorrect (but terribly
well-intentioned) error messages are worse than unfriendly ones! But I feel
we could simply rewrite the existing ones to make them more intelligible
and less intimidating. I am aware that this is far less straightforward
than it sounds!

I may have found my next area of research, although as I am a high school
teacher these days, research is rather lower on my agenda than it used to
be.  I finished a PhD on introductory programming 15 years ago, and I
confess I am a little startled at how little progress there appears to have
been in this area. Surely I am missing something???

On 6 April 2016 at 11:41, Richard A. O'Keefe  wrote:

> On 6/04/16 12:47 pm, Luke Church wrote:
>
>> Based on observational studies, I've generally recommended to the Dart <
>> http://www.dartlang.org/> team that the verbose errors state:
>>
>> - *What happened* (e.g. File 'stuff, ln 43:' class B doesn't implement
>> method foo)
>> - *Why it is a problem* (e.g. class B extends class A, which has an
>> abstract method foo. This means that class B doesn't have an implementation
>> of foo)
>> - *What the user can do next* (e.g. consider implementing foo, or making
>> class B abstract)
>>
>
> If I may add an anecdote, I once used a programming system
> whose authors had put enormous effort into writing "helpful"
> error messages.  But I found them infuriating.  Much of the
> text would explain what the program thought you were trying
> to do, but it was almost always wrong about that.
>
> Take this particular example as an example:  maybe the real
> problem is that the student was trying to call food(), not
> foo(), in which case advice to make B abstract is worse than
> useless.
>
> An error is detected when the system expects something and
> something else happens.  Saying clearly *where* the error
> was is important (and I know my own programs are not very
> good at this), as is saying what the system expected and what
> it thinks it got.
>
> In the Haskell world, one group went to the trouble of building
> a whole new system called Helium, in part so that they could
> provide human-friendly error messages.
>
> --
> You received this message because you are subscribed to the Google Groups
> "PPIG Discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to ppig-discuss+unsubscr...@googlegroups.com.
> To post to this group, send an email to ppig-discuss@googlegroups.com.
>
> For more options, visit https://groups.google.com/d/optout.
>



-- 

Exploring Life, Parenting and Social Justice:
http://lindamciver.wordpress.com/
Computational Science Education: http://computeitsimple.wordpress.com/


Dr Linda McIver
Teacher & Freelance Writer
--
Buy Fair Trade - Change the world one coffee at a time

-- 
You received this message because you are subscribed to the Google Groups "PPIG 
Discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to ppig-discuss+unsubscr...@googlegroups.com.
To post to this group, send an email to ppig-discuss@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [ppig-discuss] Beginner friendly error messages

[Richard A. O'Keefe]

On 6/04/16 12:47 pm, Luke Church wrote:
Based on observational studies, I've generally recommended to the Dart 
 team that the verbose errors state:


- *What happened* (e.g. File 'stuff, ln 43:' class B doesn't implement 
method foo)
- *Why it is a problem* (e.g. class B extends class A, which has an 
abstract method foo. This means that class B doesn't have an 
implementation of foo)
- *What the user can do next* (e.g. consider implementing foo, or 
making class B abstract)


If I may add an anecdote, I once used a programming system
whose authors had put enormous effort into writing "helpful"
error messages.  But I found them infuriating.  Much of the
text would explain what the program thought you were trying
to do, but it was almost always wrong about that.

Take this particular example as an example:  maybe the real
problem is that the student was trying to call food(), not
foo(), in which case advice to make B abstract is worse than
useless.

An error is detected when the system expects something and
something else happens.  Saying clearly *where* the error
was is important (and I know my own programs are not very
good at this), as is saying what the system expected and what
it thinks it got.

In the Haskell world, one group went to the trouble of building
a whole new system called Helium, in part so that they could
provide human-friendly error messages.

--
You received this message because you are subscribed to the Google Groups "PPIG 
Discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to ppig-discuss+unsubscr...@googlegroups.com.
To post to this group, send an email to ppig-discuss@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [ppig-discuss] Beginner friendly error messages

[Richard A. O'Keefe]

That's rather odd.  Guido van Rossum, inventor of Python,
was if memory serves me correctly, worked for several
years on the ABC programming language developed at the CWI
by Guerts, Meertens and Pemberton as amongst other things a
beginners' language.  I gather van Rossum didn't like it,
but because of ABC's beginner emphasis surely they must have
thought about error messages a lot?

One thing about ABC (which can be downloaded from
http://homepages.cwi.nl/~steven/abc/implementations.html)
is that the error messages are in a file that's easy to
rewrite and replace.  I know that Python provides a
gettext module for Python programmers to use; presumably
it uses gettext to report errors itself, in which case it
should be possible to rewrite the error messages yourself
to something beginners find easier.

--
You received this message because you are subscribed to the Google Groups "PPIG 
Discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to ppig-discuss+unsubscr...@googlegroups.com.
To post to this group, send an email to ppig-discuss@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [ppig-discuss] Beginner friendly error messages

[Guzdial, Mark]

The Racket group won a SIGCSE Best Paper Award a few years back about their 
careful design of better error messages for students.


Strangely enough, the ITICSE Best Paper Award went to another paper a few years 
later, that claimed that improving error messages actually didn't help students 
at all.  The second paper did not use any of the techniques of the Racket paper 
(nor even reference that paper).


A blog post about that strange set of affairs -- with links to ACM DL for both 
papers: 
https://computinged.wordpress.com/2014/07/29/enhancing-syntax-error-messages-appears-ineffectual-if-you-enhance-the-error-messages-poorly/



- Mark



From: ppig-discuss@googlegroups.com  on behalf 
of Luke Church 
Sent: Tuesday, April 5, 2016 8:47 PM
To: Linda McIver; ppig-discuss@googlegroups.com
Subject: Re: [ppig-discuss] Beginner friendly error messages

Hi Linda,

Error messages generally serve as a way of trying to realign a gap between what 
the user thought the computer was going to do on their behalf and the model 
inside the computer. This is hard-ish, as if we knew what the user wanted, we 
would probably have just done that instead :)

Based on observational studies, I've generally recommended to the 
Dart<http://www.dartlang.org/> team that the verbose errors state:

- What happened (e.g. File 'stuff, ln 43:' class B doesn't implement method foo)
- Why it is a problem (e.g. class B extends class A, which has an abstract 
method foo. This means that class B doesn't have an implementation of foo)
- What the user can do next (e.g. consider implementing foo, or making class B 
abstract)

This seems to work reasonably well in verbose form, but does need to be 
accompanied by a terser format, especially for experts who seem to on a first 
pass only pay attention to where something went wrong, and ignore the rest.

It's worth noting that being able to answer each of these questions well 
requires careful attention during the design of the interpreter to keep the 
necessary information around.

I agree this experience is critical in education. I conjecture that it's a 
necessary component of maintaining self-efficacy amongst students. There have 
been studies in the past indicating that this is an important factor in success.

Please keep in touch with how you get on with the education-friendly 
interpreter. If I remember correctly the folks from 
Grace<http://gracelang.org/applications/> have given this problem some thought 
and might have more insights.

Hope this helps,

Luke

On Wed, Apr 6, 2016 at 12:18 AM Linda McIver 
mailto:linda.mci...@gmail.com>> wrote:
Hello PPIGers,

I am trying to design a data science course for year 10 students that will be 
taught to, and in some cases *by* beginners. We'll be using Python both for its 
data science credentials and its user friendliness, but the error messages are 
a big barrier to success. Students hit one incomprehensible error message and 
run screaming in the opposite direction.

I recall some research on error messages and their user friendliness, but I 
still can't find any interpreters with beginner friendly error messages, which 
surprises me.

Am I missing something? Is there a treasure trove somewhere? If not, is there 
at least some solid research on which we could base the design of a beginner 
friendly Python interpreter?

Any and all clues gratefully received.

Linda

--

Exploring Life, Parenting and Social Justice: http://lindamciver.wordpress.com/
Computational Science Education: http://computeitsimple.wordpress.com/

Dr Linda McIver
Teacher & Freelance Writer
--
Buy Fair Trade - Change the world one coffee at a time

--
You received this message because you are subscribed to the Google Groups "PPIG 
Discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to 
ppig-discuss+unsubscr...@googlegroups.com<mailto:ppig-discuss+unsubscr...@googlegroups.com>.
To post to this group, send email to 
ppig-discuss@googlegroups.com<mailto:ppig-discuss@googlegroups.com>.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "PPIG 
Discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to 
ppig-discuss+unsubscr...@googlegroups.com<mailto:ppig-discuss+unsubscr...@googlegroups.com>.
To post to this group, send email to 
ppig-discuss@googlegroups.com<mailto:ppig-discuss@googlegroups.com>.
For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups "PPIG 
Discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to ppig-discuss+unsubscr...@googlegroups.com.
To post to this group, send an email to ppig-discuss@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [ppig-discuss] Beginner friendly error messages

[Luke Church]

Hi Linda,

Error messages generally serve as a way of trying to realign a gap between
what the user thought the computer was going to do on their behalf and the
model inside the computer. This is hard-ish, as if we knew what the user
wanted, we would probably have just done that instead :)

Based on observational studies, I've generally recommended to the Dart
 team that the verbose errors state:

- *What happened* (e.g. File 'stuff, ln 43:' class B doesn't implement
method foo)
- *Why it is a problem* (e.g. class B extends class A, which has an
abstract method foo. This means that class B doesn't have an implementation
of foo)
- *What the user can do next* (e.g. consider implementing foo, or making
class B abstract)

This seems to work reasonably well in verbose form, but does need to be
accompanied by a terser format, especially for experts who seem to on a
first pass only pay attention to where something went wrong, and ignore the
rest.

It's worth noting that being able to answer each of these questions well
requires careful attention during the design of the interpreter to keep the
necessary information around.

I agree this experience is critical in education. I conjecture that it's a
necessary component of maintaining self-efficacy amongst students. There
have been studies in the past indicating that this is an important factor
in success.

Please keep in touch with how you get on with the education-friendly
interpreter. If I remember correctly the folks from Grace
 have given this problem some thought
and might have more insights.

Hope this helps,

Luke

On Wed, Apr 6, 2016 at 12:18 AM Linda McIver  wrote:

> Hello PPIGers,
>
> I am trying to design a data science course for year 10 students that will
> be taught to, and in some cases *by* beginners. We'll be using Python both
> for its data science credentials and its user friendliness, but the error
> messages are a big barrier to success. Students hit one incomprehensible
> error message and run screaming in the opposite direction.
>
> I recall some research on error messages and their user friendliness, but
> I still can't find any interpreters with beginner friendly error messages,
> which surprises me.
>
> Am I missing something? Is there a treasure trove somewhere? If not, is
> there at least some solid research on which we could base the design of a
> beginner friendly Python interpreter?
>
> Any and all clues gratefully received.
>
> Linda
>
> --
>
> Exploring Life, Parenting and Social Justice:
> http://lindamciver.wordpress.com/
> Computational Science Education: http://computeitsimple.wordpress.com/
>
>
> Dr Linda McIver
> Teacher & Freelance Writer
> --
> Buy Fair Trade - Change the world one coffee at a time
>
> --
> You received this message because you are subscribed to the Google Groups
> "PPIG Discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to ppig-discuss+unsubscr...@googlegroups.com.
> To post to this group, send email to ppig-discuss@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups "PPIG 
Discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to ppig-discuss+unsubscr...@googlegroups.com.
To post to this group, send an email to ppig-discuss@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [ppig-discuss] Beginner friendly error messages

[Huw Lloyd]

I'm not clear on the full shape of your problem, Linda. What will the
programs be used for, for instance?  Are you using programming as a means
of analysing data?

Working through a more thorough description may help you, even if the
emails on this list do not.

Best,
Huw

On 6 April 2016 at 00:18, Linda McIver  wrote:

> Hello PPIGers,
>
> I am trying to design a data science course for year 10 students that will
> be taught to, and in some cases *by* beginners. We'll be using Python both
> for its data science credentials and its user friendliness, but the error
> messages are a big barrier to success. Students hit one incomprehensible
> error message and run screaming in the opposite direction.
>
> I recall some research on error messages and their user friendliness, but
> I still can't find any interpreters with beginner friendly error messages,
> which surprises me.
>
> Am I missing something? Is there a treasure trove somewhere? If not, is
> there at least some solid research on which we could base the design of a
> beginner friendly Python interpreter?
>
> Any and all clues gratefully received.
>
> Linda
>
> --
>
> Exploring Life, Parenting and Social Justice:
> http://lindamciver.wordpress.com/
> Computational Science Education: http://computeitsimple.wordpress.com/
>
>
> Dr Linda McIver
> Teacher & Freelance Writer
> --
> Buy Fair Trade - Change the world one coffee at a time
>
> --
> You received this message because you are subscribed to the Google Groups
> "PPIG Discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to ppig-discuss+unsubscr...@googlegroups.com.
> To post to this group, send email to ppig-discuss@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups "PPIG 
Discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to ppig-discuss+unsubscr...@googlegroups.com.
To post to this group, send an email to ppig-discuss@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [ppig-discuss] Beginner friendly error messages

[Derek M Jones]

Linda,


I am trying to design a data science course for year 10 students that will
be taught to, and in some cases *by* beginners. We'll be using Python both


Is this a late April fool's post?

--
Derek M. Jones   Software analysis
tel: +44 (0)1252 520667  blog:shape-of-code.coding-guidelines.com

--
You received this message because you are subscribed to the Google Groups "PPIG 
Discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to ppig-discuss+unsubscr...@googlegroups.com.
To post to this group, send an email to ppig-discuss@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [ppig-discuss] Beginner friendly error messages

[Raoul Duke]

I guess there is a dearth, a void, a vast sucking sound, a gap, a
fundamental problem with most all programming systems when it comes to
error message UX; it is still an area of research. :-(

e.g. http://lambda-the-ultimate.org/node/5280

-- 
You received this message because you are subscribed to the Google Groups "PPIG 
Discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to ppig-discuss+unsubscr...@googlegroups.com.
To post to this group, send an email to ppig-discuss@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.