Neil,

Hate to inject some data into this polemic, but a paper at ICSE this year 
studied this very topic.


I'm a great fan of data:
http://shape-of-code.coding-guidelines.com/2012/06/22/background-to-my-book-project-empirical-software-engineering-with-r/

Salman, Misirli, and Juristo: Are Students Representatives of Professionals in 
Software Engineering Experiments?

I have read this paper.

The paper provides convincing evidence for what I am saying.
Professionals were quicker at doing tasks (because they had had
more practice) and have performances with smaller standard deviation
(because their performances have converged).

Quoting the full conclusion:

   "o Professionals write higher quality code than students to implement
tasks when subjects apply a development approach with which they
are already familiar.

   o Students and professionals perform similarly when they apply a
development approach in which they are inexperienced."

Since so much of programming is repetitive professionals have
obvious differences in performance, compared to students.

Most of the experiments in software engineering employ students as subjects. 
This raises concerns about the realism of the results acquired through students 
and adaptability of the results to software industry.  Aim: We compare students 
and professionals to understand how well students represent professionals as 
experimental subjects in SE research. Method:The comparison was made in the 
context of two test-driven development experiments conducted with students in 
an academic setting and with professionals in a software organization. We 
measured the code quality of several tasks implemented by both subject groups 
and checked whether students and professionals perform similarly in terms of 
code quality metrics.  Results: Except for minor differences, neither of the 
subject groups is better than the other. Professionals produce larger, yet less 
complex, methods when they use their traditional development approach, whereas 
both subject groups perform similarly when they apply a
 new ap
proach for the first time.  Conclusion: Given a carefully scoped experiment on 
a development approach that is new to both students and professionals, similar 
performances are observed. Further investigation is necessary to analyze the 
effects of subject demographics and level of experience on the results of SE 
experiments.

Slides: 
http://homepages.dcc.ufmg.br/~figueiredo/disciplinas/2015a/sqm/paper08mauricio.pdf

On Jun 15, 2015, at 6:55 AM, Derek M Jones <de...@knosof.co.uk> wrote:

Richard,

Concerning student subjects in SE experiments:
people use the subjects they can afford.  If you

Then perhaps research based on these papers should come
with a health warning that the work only exists to increase
the authors paper count so they can get promoted and has no
connection with industrial practice.



--
Derek M. Jones           Software analysis
tel: +44 (0)1252 520667  blog:shape-of-code.coding-guidelines.com

--
You received this message because you are subscribed to the Google Groups "PPIG 
Discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to ppig-discuss+unsubscr...@googlegroups.com.
To post to this group, send an email to ppig-discuss@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Reply via email to