i havent followed the whole discussion but agree that it would be nice to separate our testing data entirely from the source, as yarik suggests, at least for ANTs.
i have been struggling a bit with testing lately, specifically, testing combinations of parameters on combinations of data. is there any new/better approach to this? if i am going to put effort into this, then would like to improve the testing situation, in general, if possible. brian On Thu, Feb 27, 2014 at 12:30 AM, Yaroslav Halchenko <[email protected]>wrote: > > On Thu, 27 Feb 2014, Matthew McCormick wrote: > > >> @Yaroslav, the script we use for creating the ITK tarballs [3] may be > > >> a useful reference for creating an ANTS data tarball. > > > > Thank you Matthew, we might make use of its parts... but while at it I > > > would like to raise a concern actually about shipping those relatively > > > large data volumes used for testing WITHIN the source tarballs... Do > > > you expect those to get modified (grow/change) with each ITK release? > > > ATM those data samples occupy big (if not majority) of the > > > compressed tarball IIRC. > > > > May be that data could get a life of its own? ;) i.e. may be we could > > > better package "working set" of test images used by ITK/ANTs (else?) > and > > > distribute a dedicated package with only data? I eventually see it > > > possible that data(set) stabilizes more than code which keeps > developing > > > reusing those data files > > > > P.S. That is what Michael (of NeuroDebian) has done to other big > > > projects such as AFNI, FSL etc -- we barely ever update the data > > > packages but code packages get updated quite often now without > incurring > > > any major penalty of dragging the same data around > > > > The testing data is included with the source tarballs so that the > > source tarball can be downloaded and the tests can be readily run > > without any extra effort and without a network connection. > > > The testing data will vary with releases. Most of the data are not > > input images but baseline, i.e. ground truth, images used when > > verifying the output of a test. > > Fair enough if data is to change with releases and tarballs remain the > ultimate form of "sources" -- thank you Matthew. > > Brian -- is that to be the case with ANTs? then would you be so kind to > provide similar tarball distribution bundled with data? > > Cheers! > -- > Yaroslav O. Halchenko, Ph.D. > http://neuro.debian.net http://www.pymvpa.org http://www.fail2ban.org > Senior Research Associate, Psychological and Brain Sciences Dept. > Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755 > Phone: +1 (603) 646-9834 Fax: +1 (603) 646-1419 > WWW: http://www.linkedin.com/in/yarik >
