Reproducibility

2010-04-30 Thread Teemu Ikonen
On Fri, Apr 30, 2010 at 2:08 AM, Michael Hanke michael.ha...@gmail.com wrote: Debian: The ultimate platform for neuroimaging research [...] However, it is hard to blame the respective developers, because the sheer number of existing combinations of operating systems, hardware, and library

Re: Reproducibility

2010-04-30 Thread Michael Hanke
Hi, On Fri, Apr 30, 2010 at 10:01:23AM +0200, Teemu Ikonen wrote: On Fri, Apr 30, 2010 at 2:08 AM, Michael Hanke michael.ha...@gmail.com wrote: Debian: The ultimate platform for neuroimaging research [...] However, it is hard to blame the respective developers, because the sheer number

Re: Reproducibility

2010-04-30 Thread Andreas Tille
On Fri, Apr 30, 2010 at 07:07:21AM -0400, Michael Hanke wrote: This nice abstract inspired me to think about reproducibility of program runs. If one runs e.g. Debian unstable the OS code which can potentially affect the results of calculations can change almost daily. Reproducing results

Re: Reproducibility

2010-04-30 Thread Andreas Tille
On Fri, Apr 30, 2010 at 09:30:16AM -0300, David Bremner wrote: Yes, that's the problem. For stable releases though, we have the time, and we can (I suspect) get the compute cycles to run heavy regression tests. Would that be a worthwhile project? Well, it is not me who raised this problem

Re: Reproducibility

2010-04-30 Thread Brett Viren
Teemu Ikonen tpiko...@gmail.com writes: Does anyone here have good ideas on how to ensure reproducibility in the long term? Regression testing, as mentioned, or running some fixed analysis and statistically comparing the results to past runs. We worry about reproducibility in my field of

Re: Reproducibility

2010-04-30 Thread Adam C Powell IV
On Fri, 2010-04-30 at 14:18 +0200, Andreas Tille wrote: I can confirm that this is actually the reason why at Sanger Institute (even if there are three DDs working) plain Debian (and specifically the Debian Med packages) is not used. FYI, I uploaded a new version of the Med packages on Monday

Re: Reproducibility

2010-04-30 Thread Antonio Paiva
Those of you interested in reproducibility might be interested in VisTrails. These is a start-up commercializing the software but most of it is free and development is open source, available from http://www.vistrails.org/index.php/Downloads. I remember that the software keeps track of the

Re: Reproducibility

2010-04-30 Thread Johan Grönqvist
2010-04-30 16:29, Michael Hanke skrev: Usually we have some version in stable and some people will use it. [...] In Debian we have the universal operating system that incorporates all software and 'stable' is a snapshot of everything at the time of release -- and this is not what scientists

Re: Reproducibility

2010-04-30 Thread Yaroslav Halchenko
On Fri, 30 Apr 2010, Johan Grönqvist wrote: That is why we have backports.org and neuro.debian.net that offer at least the latest and greatest for 'stable'. But this is still not enough. To me (IMHO) that feels like _the_ solution, when combined with the debian snapshot service. Exactly that

Re: Reproducibility

2010-04-30 Thread Yaroslav Halchenko
On Fri, 30 Apr 2010, Antonio Paiva wrote: http://www.vistrails.org/index.php/Downloads. I remember that the software keeps track of the libraries, OS, and CPU that the code is using to get the results. Best, António Rafael C. Paiva Post-doctoral fellow SCI Institute, University of Utah