Hate to inject some data into this polemic, but a paper at ICSE this year
studied this very topic.
Salman, Misirli, and Juristo: Are Students Representatives of Professionals in
Software Engineering Experiments?
Most of the experiments in software engineering employ students as subjects.
This raises concerns about the realism of the results acquired through students
and adaptability of the results to software industry. Aim: We compare students
and professionals to understand how well students represent professionals as
experimental subjects in SE research. Method:The comparison was made in the
context of two test-driven development experiments conducted with students in
an academic setting and with professionals in a software organization. We
measured the code quality of several tasks implemented by both subject groups
and checked whether students and professionals perform similarly in terms of
code quality metrics. Results: Except for minor differences, neither of the
subject groups is better than the other. Professionals produce larger, yet less
complex, methods when they use their traditional development approach, whereas
both subject groups perform similarly when they apply a new approach for the
first time. Conclusion: Given a carefully scoped experiment on a development
approach that is new to both students and professionals, similar performances
are observed. Further investigation is necessary to analyze the effects of
subject demographics and level of experience on the results of SE experiments.
> On Jun 15, 2015, at 6:55 AM, Derek M Jones <de...@knosof.co.uk> wrote:
>> Concerning student subjects in SE experiments:
>> people use the subjects they can afford. If you
> Then perhaps research based on these papers should come
> with a health warning that the work only exists to increase
> the authors paper count so they can get promoted and has no
> connection with industrial practice.
You received this message because you are subscribed to the Google Groups "PPIG
To unsubscribe from this group and stop receiving emails from it, send an email
To post to this group, send an email to email@example.com.
For more options, visit https://groups.google.com/d/optout.