Dear Matthew and others, Thank you for the interesting discussion. I was not on the HCP mailing list*, but Thomas Yeo was kind enough to suggest that I respond to the two primary points you made about the 2009/2010 registration evaluation studies:
1. "the Klein et al 2010 study’s findings are based on manual defined gyral and sulcal labels that will frequently have little to do with the areal organization of the cerebral cortex. In fact, we have shown that as one aligns cortical folds more tightly, functional alignment may actually decrease some (Robinson et al 2014). The 2009/2010 studies were interested in evaluating anatomical registration methods, and as such, used anatomical labels as gold standard data. We know little about how anatomical boundaries correspond to function/connectivity/receptor maps, and even less about how anatomical correspondence across brains relates to the correspondence of function/connectivity/receptors across brains. It seems to me that if one wishes to compare different brains, it would be best to do so within a given modality, then try to reconcile the intermodal mappings if desired. 2. "Neither the Klein et al 2010 nor the volume only paper that preceded it in 2009 considered distortion in their ranking of algorithms." This is true. I raise a number of caveats in the 2009 Discussion's Caveats section, and the one most relevant portion is: "The evaluation measures and analysis methods used in this paper are predicated on the assumption that, at the macroscopic scale of topographic anatomical regions, there are correspondences across a majority of brains that can effectively guide registrations. It is very important to stress that we cannot make inferences about the accuracy of registrations within these macroscopic regions. Therefore our overlap evaluation measures not only ignore misregistration within a labeled region but are insensitive to folding in the deformations, which would impact studies such as deformation-based morphometry. More generally, our evaluation measures rely on information which is not directly included in the images, which is good for evaluating the registrations, but they do not inform us about the intrinsic properties of the spatial transformations. Example measures of the intrinsic properties of spatial transformations include inverse consistency error, transitivity error, and “mean harmonic energy” (where the Jacobian determinant of the transformation is averaged over the volume)." Cheers, @rno *It's a shame people don't use publicly accessible forums for scientific debate. Isn't that what Neurostars.org and other stack overflow forks are for? On Sat, Aug 13, 2016 at 11:17 AM, Glasser, Matthew <glass...@wustl.edu> wrote: I’ve seen this study come up several times and there are a few things to consider about it: There is a significant literature that has shown that surface-based alignment is better than volume-based methods (e.g. Fischl et al 1999, Anticevic et al 2008, Fischl et al 2008, Van Essen et al 2012, Frost et al 2012, Tuchola et al 2012, Smith et al 2013). But the Klein et al 2010 study is the only one that I am aware of that has come to a different conclusion. Thus, it is worth considering why this study may have come to a discordant conclusion and whether it really is “the best paper” on this topic. One important difference between the literature which has shown converging evidence of the superiority of surface-based alignment to volume and the Klein et al 2010 study is that these studies based on their findings on measures tied closely to cortical areas or the areas themselves (such architecture, function, connectivity, or topography). On the other hand, the Klein et al 2010 study’s findings are based on manual defined gyral and sulcal labels that will frequently have little to do with the areal organization of the cerebral cortex. In fact, we have shown that as one aligns cortical folds more tightly, functional alignment may actually decrease some (Robinson et al 2014). When evaluating the quality of a registration, there are two important considerations: 1) Accuracy of alignment and 2) Distortion induced by the alignment. The best approach will maximize accuracy of alignment while minimizing the distortion induced by the alignment (keeping it within neurobiologically reasonable limits). Neither the Klein et al 2010 nor the volume only paper that preceded it in 2009 considered distortion in their ranking of algorithms. Thus, the best performing algorithms in these studies may well simply be the ones with the most distortion. As I mentioned above, however fitting cortical folds very tightly (leading to higher distortion) doesn’t improve functional alignment (and indeed we found that we could achieve much better multi-modal areal feature alignment than folding-based approaches with less distortion than a standard FreeSurfer folding-based registration). Given the discordance between folding and areas, I don’t know that a paper that focuses on aligning folding-based labels really relates to the question of aligning cortical areas to an areal parcellation, regardless of the above issues. The whole point of cortical registration is to align cortical areas across subjects (and ideally the topographic organization within these areas) as well as is feasible. Doing so makes group average results much more interpretable, both visually and in terms of statistical sensitivity. If folks want to compare volume-averaged data with the multi-modal parcellation, I’d rather the burden of inaccuracy be bourn by the volume-averaged data than making the parcellation less accurate to enable such comparisons. The recommended way to compare data to this parcellation is to align across subjects on the surface, ideally driving the alignment based on areal features (e.g. architecture, connectivity, and topography like in MSMAll) instead of cortical folds. This will allow the most definitive comparisons. Peace, Matt. From: Chris Gorgolewski <krzysztof.gorgolew...@gmail.com> Date: Friday, August 12, 2016 at 1:31 PM To: Matt Glasser <glass...@wustl.edu> Cc: "Horn,Andreas (BIDMC - Neurology)" <aho...@bidmc.harvard.edu>, Timothy Coalson <tsc...@mst.edu>, "Reid, A.T. (Andrew)" <a.r...@psych.ru.nl>, "hcp-users@humanconnectome.org" <hcp-users@humanconnectome.org> Subject: Re: [HCP-Users] Surface parcellation to volume Is there any comprehensive quantitative comparison of volume and surface based (potentially multimodal) registration methods? The best paper I know of (Klein 2010 - http://www.sciencedirect.com/science/article/pii/S105381191000114X) recommended using custom study templates over external templates (like fsaverage or MNI152). The authors could not, however, recommend volume over surface (or other way around) due to resampling errors. Here's the relevant paragraph: "The resampling tests demonstrate that, for almost every region, the resampling error is too great to distinguish between the performance of top-ranking volume and surface registration methods (SyN, FreeSurfer, and Spherical Demons, all using customized optimal average templates). Based on these results, it may not be possible to directly compare evaluations of these surface and volume registration methods using the present resampling methods, when considering the full surface or full volume or the full extent of their label boundaries." I was wondering if there is some other literature I'm missing that overcomes the aforementioned resampling problems and provides a quantitative comparison between the two registration approaches. Best, Chris On Fri, Aug 12, 2016 at 10:36 AM, Glasser, Matthew <glass...@wustl.edu> wrote: > > Use of diffusion fiber orientation information might indeed improve volume-based alignment of the white matter and is worth pursuing. > > I don’t think using areal features in the volume will address the core limitation of volume-based cortical areal registration. This will not change the fundamental issue of incompatibilities in folding patterns across subjects creating topological matching issues. This sort of thing occurs in 2D on the surface as well, though much less frequently, where the spatial relationships between an area and its neighbors are so different that one would have to tear the surface to align the areas. When this happens something like the cortical areal classifier is needed to achieve correspondence across subjects. For the same reason that topological incompatibilities are not fixable in 2D on the surface, the more frequent ones that occur in 3D in the volume will also not be fixable. > > The overall point is that when we compare across subjects, we need to be sure that we are comparing like with like. If we are not doing that we aren’t making a valid comparison. > > Peace, > > Matt. > > From: "Horn,Andreas (BIDMC - Neurology)" <aho...@bidmc.harvard.edu> > Date: Friday, August 12, 2016 at 8:49 AM > To: Timothy Coalson <tsc...@mst.edu> > Cc: "Reid, A.T. (Andrew)" <a.r...@psych.ru.nl>, " hcp-users@humanconnectome.org" <hcp-users@humanconnectome.org>, Matt Glasser <glass...@wustl.edu> > Subject: Re: [HCP-Users] Surface parcellation to volume > > Dear Tim and Matt, > > thank you very much for your detailed and insightful answers. I learned a lot and agree to nearly everything you said. Especially, I totally agree that we should not use T1w -> T1w Template nonlinear volumetric warps + smoothing nowadays anymore. Regarding the sulcus-on-gyrus mismatches for techniques such as DARTEL/Shoot/ANTs that Matt pointed out, I wonder to what extent the inclusion of FA and e.g. rs-fMRI eigenvector centrality maps in multispectral warps could minimize such mismatches. I agree that it would be nontrivial to add a real connectome (i.e. edges) to the volumetric deformation problem. And again, I lack empirical data to be able to say how much impact either method would really have on results – a fair comparison study would be great and important to the field in my view. On the other hand, I agree that MSM is an awesome technique and why should we not just use it since it’s available. > > Best, Andy > > > Am 11.08.2016 um 17:49 schrieb Timothy Coalson <tsc...@mst.edu>: > > Sorry, I was not precise enough in my language - my earlier comments should be considered in the context of cortex only (the MMP v1.0 is cortex-only). > > The "volume-based group-average methods" methods I meant to refer to are when people analyze the whole brain, including cortical data, by doing some T1w MNI registration, and then smooth the volumes (to partially make up for deficiencies in the cortical registration), and then average all signal across subjects (including cortical), still in the volume. This has a whole host of problems, but some people persist in doing things this way. > > Other replies inline. > > Tim > > > On Thu, Aug 11, 2016 at 10:03 AM, Horn,Andreas (BIDMC - Neurology) < aho...@bidmc.harvard.edu> wrote: >> >> Hi Tim, >> >> Yes, of course it’s compared to an average surface – but isn’t that the final goal of brain mapping to somehow define regions within a well-defined space? > > > I would say it is more specific than that - the ideal goal is to define, for each subject, the locations in each atlas region. In the case of the HCP MMP v1.0, it only defines cortical regions, in terms of areal features (for instance, functional connectivity). A group-average cortical surface has very little of the folding (geometric definition) you will find in any contributing subject, which makes it a notably inferior method for transferring cortical data to or from a nonlinear-registered volume template (we aren't enthusiastic about the MMP cortical data being used as a volume file, but there are better ways to get there than group average surfaces). > >> >> I totally agree with you and Matt that there are a lot of advantages of surface based processing, especially when we are predominantly interested in only the cortex. However, I find it a bit too dogmatic to say that something is only feasible using surface-based analyses and that surface-based approaches are the (only) thing the field should be doing. > > > Our method of generating the HCP MMP v1.0 relied heavily on the MSMAll surface registration. It was a very critical step to being able to do what we did, and I don't know of volume registrations that achieve comparable areal feature alignment in cortex (and most of the ones that I know people use only register anatomy, and don't try to use things like functional connectivity). Perhaps Matt or David have done a more thorough survey of volume methods for registering areal features in cortex. > >> >> Some things are definitely easier using surfaces (since we can reduce the 3D problem to a 2D problem by projecting the surface to a sphere). Also, it’s much easier to inflate resolution since the data points are drastically reduced. However, my feeling is that volume based approaches have also improved a lot over the last years with multispectral diffeomorphic processes that are often segmentation based, i.e. reduce the warping techniques between single subject’s cortices and an average mean to a more or less 2-D problem as well. In the end, any warp is a set of coordinates projected to another set of coordinates, no matter if doing this on a surface or a volume, right? The warp is just sometimes more constrained on a surface. > > > Surface registration also doesn't require changing the anatomical shape of the cortex in order to enable cross-subject comparisons or group averaging. This makes it easier to regularize the registration in a way that is not penalized for an unusual folding pattern. As before, Matt or David may be in a better position to comment regarding the current state of the field in volume registration of cortical areal features. > >> >> I’d be very interested in good comparison studies that show superior results using the most advanced surface-based techniques in comparison to most-advanced volume-based techniques (like e.g. multispectral ANTs SyN deformations using the OASIS templates or MNI 2009b NLIN series or similar). > > > This would be an interesting comparison to do (and perhaps include a more "traditional" volume method as well to put any performance difference in perspective). > >> >> I’d still guess that the surface based approaches would be superior on the cortex but I wonder how much impact it would really have. Really curious about how you did this in the upcoming Nature Neuroscience article and to which volumetric analyses you compared your results. > > > As I said, my comments were intended in the context of cortex, that is, where the HCP MMP v1.0 is defined. It is likely that for a future version including subcortical components, we would use volume registration and voxel representation for those, as they are not as challenging for volume registration (don't have cortical folding variability). > >> >> Personally, I am interested in deep brain stimulation and small subcortical structures like the subthalamic nucleus. This structure is not visible on T1 (but T2) and is not represented on tissue probability maps at all (there is an enhanced TPM including it by Bogdan Draganski available as a side-note). In my view, the surface-based world it not at all ready to deal with such structures (correct me if I’m wrong). > > > As I understand it, most of these structures do not have a sheetlike nature the way cortex does, and thus we probably would not advocate using surfaces to represent them. > >> >> When assessing connectivity from these structures to the rest of the brain, it makes it a lot easier to stick to the volume-based approach (and not do volume-to-surface projections at all). Moreover, we are often merely interested in connectivity to „motor“, „sensory“, „limbic“ and „associative“ regions. It could be seen as methodological over engineering to implement volume-to-surface based methods for such trivial parcellations. So this could maybe illustrate an example where it is – at least in my view – still totally fine to use multispectral volumetric deformations for connectometric analyses. > > > I was not intending to say that subcortical structures should be analyzed on a surface. > >> >> Then, the volumetric version I put up on figshare is really for comparative reasons with atlases that used different techniques and are available in MNI space. > > > Using group average cortical surfaces (which lack significant folding definition) to generate it means it won't align well with any subject's cortex post-registration (because it doesn't align well with the template's cortex features), which is what I was trying to say (with the comment about viewing overlaid on the template slices). Thus, this representation of it will fall short of other volume atlases in terms of cortical overlap with the volume template. > >> >> I guess this is something many people are interested in. For instance, the histological atlas by the Jülich group exhibits anatomical detail and has been used by the field by coregistering nonlinear warps to it for decades – totally accepting the fact that histology was originally based on different brains than used to construct the MNI templates. In my view, we’d do nothing else with your averaged anatomy atlas if we would compare our results to your map. > > > Per my other comments, this comparison will be compromised when using any group-average cortical surface to translate any cortical data between surface and volume (either direction). > >> >> We should be aware of likely mismatches in classifications in the same way as we have always been using e.g. the SPM anatomy toolbox or comparisons to the Harvard Oxford atlas. Still, such a comparison could be helpful (in my view). >> >> I hope we may agree on a few points I raised. Of course, if the volumetric MMP version bothers you, I’ll gladly put it offline again. > > > People obviously want a volume representation of it, despite the caveats of it being hard to faithfully represent as a volume, and I don't know whether we have reached a decision on whether there is a method of generating it that we think is a reasonable approximation. I'll defer to Matt or David on the question of whether it bothers us. > >> >> Best, Andreas >> >> >> -- >> Andreas Horn, MD >> Laboratory for Brain Network Imaging and Modulation >> Berenson-Allen Center for Noninvasive Brain Stimulation >> Department for Neurology, Beth Israel Deaconess Center >> Harvard Medical School >> 330 Brooklin Avenue, Kirstein Building KS 158 >> 02215 Boston >> >> t: +1 6174077649 >> w: http://www.brainnetworkstim.com >> >> Am 08.08.2016 um 18:18 schrieb Timothy Coalson <tsc...@mst.edu>: >> >> Thanks for putting a note on that page about how we don't recommend volume-based group-average methods. >> >> It should be noted that the similarity between the two representations in that figure is due to the use of a group average surface for display, so that the surface representation shown is also lacking in folding definition. Display of the volume data as a slice overlaid on a T1 volume would show this lack of folding more clearly. >> >> However, it looks like the coloring scheme has been changed. Do the left and right labels still have different values in your version? >> >> Tim >> >> >> On Mon, Aug 8, 2016 at 1:38 PM, Andreas Horn <andy_h...@icloud.com> wrote: >>> >>> Hi Andrew, >>> >>> I made a projection here: >>> https://figshare.com/articles/HCP-MMP1_0_projected_on_MNI2009a_GM_volumetric_in_NIfTI_format/3501911 >>> >>> Best, Andy >>> >>> Am 08.08.2016 um 14:34 schrieb David Van Essen <vanes...@wustl.edu>: >>> >>> Hi Andrew, >>> >>> 1) As noted in a previous thread, -cifti-separate should solve this problem. >>> >>> On Jul 20, 2016, at 7:37 PM, Chris Gorgolewski < krzysztof.gorgolew...@gmail.com> wrote: >>> Awesome - this did the trick. Thanks! >>> On Wed, Jul 20, 2016 at 5:03 PM, Timothy Coalson <tsc...@mst.edu> wrote: >>>> >>>> Use -cifti-separate with the -label repeatable option to make the left and right cortex gifti label files. >>>> Tim >>>>> >>>>> >>> 2) As noted in other recent hap-users threads, mapping the HPC_MMP1.0 surface parcellation via a group average midthickness to a group-average volume pays a steep price in the fidelity of spatial relationships, particularly in regions of high individual variability in folding patterns. We have a paper in press (Nature Neuroscience, appearing Aug 28) that discusses this and related issues and suggests alternative analysis strategies for more faithfully preserving spatial fidelity. >>> >>> David >>> >>> On Aug 8, 2016, at 9:14 AM, Reid, A.T. (Andrew) <a.r...@psych.ru.nl> wrote: >>> >>> Hi all, >>> >>> For comparison purposes, we want to project the excellent new surface parcellation to a NIFTI volume. We tried to do this in two steps using wb_command: >>> >>> 1. Convert CIFTI to GIFTI: >>> wb_command -cifti-convert -to-gifti-ext Q1-Q6_RelatedParcellation210.L.CorticalAreas_dil_Colors.32k_fs_LR.dlabel.nii glasser_labels.gii >>> >>> 2. Project labels to volume (using the nearest-vertex option): >>> wb_command -label-to-volume-mapping glasser_labels.gii Q1-Q6_RelatedParcellation210.L.midthickness_MSMAll_2_d41_WRN_DeDrift.32k_fs_LR.surf.gii MACM_F1_RostroMiddle_red.nii EssaiMap.nii -nearest-vertex 3 >>> >>> >>> Unfortunately, this gives an error: >>> >>> ERROR: input surface and label file have different number of vertices >>> >>> Most likely because the labels are for both hemispheres, and the surface is only for the left hemisphere. >>> >>> Not sure where to go from here. Is there a command to combine surfaces, or conversely to split the labels? Is there a combined surface file somewhere available? >>> >>> Thanks, >>> >>> Andrew >>> >>> _______________________________ >>> >>> Andrew Reid >>> Postdoctoral Fellow >>> Department of Cognitive Artificial Intelligence >>> Donders Institute for Brain, Cognition and Behaviour >>> Radboud University Nijmegen >>> Web: http://andrew.modelgui.org/ >>> Tel: +31 (0)24 36 55931 ________________________________ The materials in this message are private and may contain Protected Healthcare Information or other information of a sensitive nature. If you are not the intended recipient, be advised that any unauthorized use, disclosure, copying or the taking of any action in reliance on the contents of this information is strictly prohibited. If you have received this email in error, please immediately notify the sender via telephone or return mail. _______________________________________________ HCP-Users mailing list HCP-Users@humanconnectome.org http://lists.humanconnectome.org/mailman/listinfo/hcp-users _______________________________________________ HCP-Users mailing list HCP-Users@humanconnectome.org http://lists.humanconnectome.org/mailman/listinfo/hcp-users
