What you are doing is pointing out that research that is not robust is as poorly reproducible as research that is insufficiently documented.
If I have to worry about specific compiler versions giving vastly different results in my analysis then it is not the fault of the system so much as my choice of analysis methods being somewhat less than robust. (Yes there are cases where there are specific packages that are wrong - I mean the different cases of 'right'). So teaching reproducible research is also about teaching robust research, that is not dependent on the exact microenvironment but can be replicated anywhere similar. ..d -----Original Message----- From: Discuss [mailto:[email protected]] On Behalf Of David Jones Sent: 14 January 2016 15:46 To: Titus Brown <[email protected]> Cc: Software Carpentry Discussion <[email protected]> Subject: Re: [Discuss] Reproducible computing -- some thoughts and questions On 14 January 2016 at 15:18, C. Titus Brown <[email protected]> wrote: > > (Agree with most of this; I'm just hesitant to say anything nice about > ugly heavy binary images :) I think everyone is a little bit hesitant to say anything nice about ugly heavy binary images. And I think part of that stems from the fact that the things we do really do depend on a whole load of ugly heavy things. And the way to capture that is to capture a whole load of ugly heavy binary things. We're reluctant to face the reality. When we try and optimise that (by say, a Dockerfile, or a list of conda package versions) we're building a model, and we're saying, at least implicitly, I don't care about various minor uninteresting things (for example that security updates change the exact set of files that are "ubuntu:14.04"). Of course, a binary VM image or the nested tar files of a Docker image are only a model too. I can easily write a script whose output depends on the CPU ID of the CPU in my laptop. Clearly that's never going to be reproducible. I think it's useful to bear in mind that to some extent reproducibility is a modelling exercise. "All models are wrong but some are useful" - George Box drj _______________________________________________ Discuss mailing list [email protected] http://lists.software-carpentry.org/mailman/listinfo/discuss_lists.software-carpentry.org The University of Dundee is a registered Scottish Charity, No: SC015096 _______________________________________________ Discuss mailing list [email protected] http://lists.software-carpentry.org/mailman/listinfo/discuss_lists.software-carpentry.org
