Hi Mark,
I have recently finished a controlled experiment comparing
graduate student programmers that just received a standard
PSP training course to others that received other training of
similar size (and including programming practice) but not PSP.
All subjects wrote the same program and improved it until
they passed an acceptance test or gave up.
Each failed acceptance test costed them DM 10 of a maximum
of DM 50 which they received when passing the first acceptance
test. (DM 10 is about US$ 6)
The fastest subjects required about 4 hours, the slowest over 20.
We have recorded information on working schedules (work days,
elapsed days, total worktime), compilations (total number, number
of successful ones), time and success of acceptance test
and for many of the subjects have even recorded all of the
source program versions that were compiled.
We also have pre-estimations of worktime and program size from the
subjects, some biographical (programming experience) data, as
well as subjective postmortem quality estimates of the product.
I am currently in the process of evaluating the data and the programs,
which is a rather laborious task. (There is an almost infinitely
large number of details that are special for a few of the subjects
or that have gone wrong, such as missing data.)
It will take several more weeks before I have finished the
initial evaluation and have written something up.
In particular, I feel it will be difficult to create fair criteria
for comparing different things (e.g. how do you trade off more
subjects that gave up in group A against longer average work
time til success in group B?)
When I have completed a writeup (a detailed TR probably) I will
send a message to this list.
I would be highly interested in
- the design of your measurement tool (what does it measure? how?)
- your models and hypotheses (what do you expect?)
- your results (what have you found?)
- learning why MindAmp is pursuing such research (from what I
see on your webpage, programming research does not seem to fit)
Lutz
Lutz Prechelt http://wwwipd.ira.uka.de/~prechelt/ | Whenever you
Institut f. Programmstrukturen und Datenorganisation | complicate things,
Universitaet Karlsruhe; D-76128 Karlsruhe; Germany | they get
(Phone: +49/721/608-4068, FAX: +49/721/608-7343) | less simple.
>>> Ever had negative research results? http://wwwipd.ira.uka.de/fnr <<<
