Hate to inject some data into this polemic, but a paper at ICSE this year
studied this very topic.
I'm a great fan of data:
Salman, Misirli, and Juristo: Are Students Representatives of Professionals in
Software Engineering Experiments?
I have read this paper.
The paper provides convincing evidence for what I am saying.
Professionals were quicker at doing tasks (because they had had
more practice) and have performances with smaller standard deviation
(because their performances have converged).
Quoting the full conclusion:
"o Professionals write higher quality code than students to implement
tasks when subjects apply a development approach with which they
are already familiar.
o Students and professionals perform similarly when they apply a
development approach in which they are inexperienced."
Since so much of programming is repetitive professionals have
obvious differences in performance, compared to students.
Most of the experiments in software engineering employ students as subjects.
This raises concerns about the realism of the results acquired through students
and adaptability of the results to software industry. Aim: We compare students
and professionals to understand how well students represent professionals as
experimental subjects in SE research. Method:The comparison was made in the
context of two test-driven development experiments conducted with students in
an academic setting and with professionals in a software organization. We
measured the code quality of several tasks implemented by both subject groups
and checked whether students and professionals perform similarly in terms of
code quality metrics. Results: Except for minor differences, neither of the
subject groups is better than the other. Professionals produce larger, yet less
complex, methods when they use their traditional development approach, whereas
both subject groups perform similarly when they apply a
proach for the first time. Conclusion: Given a carefully scoped experiment on
a development approach that is new to both students and professionals, similar
performances are observed. Further investigation is necessary to analyze the
effects of subject demographics and level of experience on the results of SE
On Jun 15, 2015, at 6:55 AM, Derek M Jones <de...@knosof.co.uk> wrote:
Concerning student subjects in SE experiments:
people use the subjects they can afford. If you
Then perhaps research based on these papers should come
with a health warning that the work only exists to increase
the authors paper count so they can get promoted and has no
connection with industrial practice.
Derek M. Jones Software analysis
tel: +44 (0)1252 520667 blog:shape-of-code.coding-guidelines.com
You received this message because you are subscribed to the Google Groups "PPIG
To unsubscribe from this group and stop receiving emails from it, send an email
To post to this group, send an email to email@example.com.
For more options, visit https://groups.google.com/d/optout.