I wanted to do the same and had to make some changes to PyMVPA here: https://github.com/bpinsard/PyMVPA/tree/gnbsearchlight_confusiontable using it with: errorfx = ConfusionMatrix(labels=ds.uniquetargets) slght = GNBSearchlight(clf, prtnr, qe, errorfx=errorfx)
On Fri, Aug 28, 2015 at 2:23 PM, marco tettamanti <[email protected]> wrote: > Thanks again! > I am on Debian testing (well, reverted on stable now, because of troubles > with gcc5) and have version 2.3.1. > I will give a try to the one from git. > Best, > Marco > > > PyMVPA: > Version: 2.3.1 > Hash: d1da5a749dc9cc606bd7f425d93d25464bf43454 > Path: /usr/lib/python2.7/dist-packages/mvpa2/__init__.pyc > Version control (GIT): > GIT information could not be obtained due > "/usr/lib/python2.7/dist-packages/mvpa2/.. is not under GIT" > SYSTEM: > OS: posix Linux 4.1.0-1-amd64 #1 SMP Debian 4.1.3-1 > (2015-08-03) > Distribution: debian/stretch/sid > > > *Yaroslav Halchenko* debian at onerussian.com > <pkg-exppsy-pymvpa%40lists.alioth.debian.org?Subject=Re%3A%20%5Bpymvpa%5D%20Confusion%20Matrix%20for%20each%20Node%20with%0A%20sphere_gnbsearchlight&In-Reply-To=%3C20150828161509.GS19455%40onerussian.com%3E> > *Fri Aug 28 16:15:09 UTC 2015* > ------------------------------ > > On Fri, 28 Aug 2015, marco tettamanti wrote: > > >* Dear Yaroslav, > *>* thank you very much for your reply. I have made several attempts, > trying > *>* to guess a solution, but it seems I always get a > *>* 'TypeError: 'NoneType' object is not callable'. > * > oh shoot... forgotten that this one was implemented after the last 2.4.0 > release: in upstream/2.4.0-34-g55e147e this June... we should release I > guess. what system are you on and what version of pymvpa currently? > if you could use/try the one from git directly... ? > > >* Case 1: > *>* slght = sphere_gnbsearchlight(clf, partitioner, radius=slradius, > *>* space='voxel_indices', errorfx=None, postproc=mean_sample()) > * > not the problem here BUT there should be no mean_sample() if errorfx is > None -- you wouldn't want to average labels ;) > > -- > Yaroslav O. Halchenko, Ph.D.http://neuro.debian.net http://www.pymvpa.org > http://www.fail2ban.org > Research Scientist, Psychological and Brain Sciences Dept. > Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755 > Phone: +1 (603) 646-9834 Fax: +1 (603) 646-1419 > WWW: http://www.linkedin.com/in/yarik > > > > > On 08/28/2015 05:28 PM, marco tettamanti wrote: > > Dear Yaroslav, > thank you very much for your reply. I have made several attempts, trying > to guess a solution, but it seems I always get a > 'TypeError: 'NoneType' object is not callable'. > > Any further advice is greatly appreciated! > Best, > Marco > > > Case 1: > slght = sphere_gnbsearchlight(clf, partitioner, radius=slradius, > space='voxel_indices', errorfx=None, postproc=mean_sample()) > slght_map = slght(fds) > > In [70]: slght = sphere_gnbsearchlight(clf, partitioner, radius=slradius, > space='voxel_indices', errorfx=None, postproc=mean_sample()) > > In [71]: slght_map = slght(fds) > [SLC] DBG: Phase 1. Initializing partitions using > <NFoldPartitioner> on <Dataset: 108x111@float32, <sa: > chunks,targets,time_coords,time_indices>, <fa: voxel_indices>, <a: > imgaffine,imghdr,imgtype,mapper,voxel_dim,voxel_eldim>> > [SLC] DBG: Phase 2. Blocking data for 18 splits and 3 labels > [SLC] DBG: Phase 3. Computing statistics for 54 blocks > [SLC] DBG: Phase 4. Deducing neighbors information for 111 ROIs > [SLC] DBG: Phase 4b. Converting neighbors to sparse matrix > representation > [SLC] DBG: Phase 5. Major loop > [SLC] DBG: Split 0 out of 18 > [SLC] DBG: 'Training' is done > [SLC] DBG: Doing 'Searchlight' > [SLC] DBG: Assessing accuracies > --------------------------------------------------------------------------- > TypeError Traceback (most recent call last) > <ipython-input-71-1146d298ca06> in <module>() > ----> 1 slght_map = slght(fds) > > /usr/lib/python2.7/dist-packages/mvpa2/base/learner.pyc in __call__(self, > ds) > 257 "used and auto training is > disabled." > 258 % str(self)) > --> 259 return super(Learner, self).__call__(ds) > 260 > 261 > > /usr/lib/python2.7/dist-packages/mvpa2/base/node.pyc in __call__(self, ds) > 119 > 120 self._precall(ds) > --> 121 result = self._call(ds) > 122 result = self._postcall(ds, result) > 123 > > /usr/lib/python2.7/dist-packages/mvpa2/measures/searchlight.pyc in > _call(self, dataset) > 141 > 142 # pass to subclass > --> 143 results = self._sl_call(dataset, roi_ids, nproc) > 144 > 145 if 'mapper' in dataset.a: > > /usr/lib/python2.7/dist-packages/mvpa2/measures/adhocsearchlightbase.pyc > in _sl_call(self, dataset, roi_ids, nproc) > 513 # error functions without a chance to screw up > 514 for i, fpredictions in enumerate(predictions.T): > --> 515 results[isplit, i] = errorfx(fpredictions, > targets) > 516 > 517 > > TypeError: 'NoneType' object is not callable > > > > > Similarly for other cases and combinations of them: > > Case 2: > slght = sphere_gnbsearchlight(clf, partitioner, radius=slradius, > space='voxel_indices', errorfx=ConfusionMatrixError(), > postproc=mean_sample()) > slght_map = slght(fds) > > > Case3: > class KeepConfusionMatrix(Node): > def _call(self, fds): > out = np.zeros(1, dtype=object) > out[0] = (fds.samples) > return out > > slght = sphere_gnbsearchlight(clf, partitioner, errorfx=None, > radius=slradius, space='voxel_indices', > postproc=ChainNode([Confusion(labels=fds.UT)])) > slght.postproc.append(KeepConfusionMatrix()) > slght_map = slght(fds) > > > Case4: > class KeepConfusionMatrix(Node): > def _call(self, fds): > out = np.zeros(1, dtype=object) > out[0] = (fds.samples) > return out > > slght = sphere_gnbsearchlight(clf, partitioner, errorfx=None, > radius=slradius, space='voxel_indices', > postproc=ChainNode([mean_sample(),Confusion(labels=fds.UT)])) > slght.postproc.append(KeepConfusionMatrix()) > slght_map = slght(fds) > > > > Case5: > class KeepConfusionMatrix(Node): > def _call(self, fds): > out = np.zeros(1, dtype=object) > out[0] = (fds.samples) > return out > > slght = sphere_gnbsearchlight(clf, partitioner, > errorfx=ConfusionMatrixError(), radius=slradius, space='voxel_indices', > postproc=ChainNode([mean_sample(),Confusion(labels=fds.UT)])) > slght.postproc.append(KeepConfusionMatrix()) > slght_map = slght(fds) > > > > Yaroslav Halchenko debian at onerussian.com > Fri Aug 28 13:16:38 UTC 2015 > > quick an possible partial reply > > 1. "not sure" -- if it pukes then probably not, although judging from > the code I foresaw arbitrary shape of the errorfx output > > 2. but you could make sphere_gnbsearchlight to return labels (not > errors) and then post-process to get those confusion matrices. Just > specify errorfx=None to it (not to CV). But you could also try > passing errorfx=ConfusionMatrixError and see how that goes > > Please share what you discover/end up with. > mvpa2/tests/test_usecases.py has more of usecase demos for gnb > searchlights which might come handy > > -- > Yaroslav O. Halchenko, Ph.D.http://neuro.debian.net http://www.pymvpa.org > http://www.fail2ban.org > Research Scientist, Psychological and Brain Sciences Dept. > Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755 > Phone: +1 (603) 646-9834 Fax: +1 (603) 646-1419 > WWW: http://www.linkedin.com/in/yarik > > > On 08/28/2015 01:48 PM, marco tettamanti wrote: > > Dear all, > is it possible to obtain confusion matrices for all nodes with > "sphere_gnbsearchlight", as was suggested before with "sphere_searchlight": > > slcvte = CrossValidation(clf, partitioner, errorfx=None, > postproc=ChainNode([Confusion(labels=fds.UT)])) > class KeepConfusionMatrix(Node): > def _call(self, fds): > out = np.zeros(1, dtype=object) > out[0] = (fds.samples) > return out > > slcvte.postproc.append(KeepConfusionMatrix()) > slght = sphere_searchlight(slcvte, radius=slradius, space='voxel_indices', > nproc=4, postproc=mean_sample()) > slght_map = slght(fds) > > > Thank you and best wishes, > Marco > > -- > Marco Tettamanti, Ph.D. > Nuclear Medicine Department & Division of Neuroscience > San Raffaele Scientific Institute > Via Olgettina 58 > I-20132 Milano, Italy > Phone ++39-02-26434888 > Fax ++39-02-26434892 > Email: [email protected] > Skype: mtettamanti > > > > > _______________________________________________ > Pkg-ExpPsy-PyMVPA mailing list > [email protected] > http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-exppsy-pymvpa > -- Basile Pinsard *PhD candidate, * Laboratoire d'Imagerie Biomédicale, UMR S 1146 / UMR 7371, Sorbonne Universités, UPMC, INSERM, CNRS *Brain-Cognition-Behaviour Doctoral School **, *ED3C*, *UPMC, Sorbonne Universités Biomedical Sciences Doctoral School, Faculty of Medicine, Université de Montréal CRIUGM, Université de Montréal
_______________________________________________ Pkg-ExpPsy-PyMVPA mailing list [email protected] http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-exppsy-pymvpa
