Date: Fri, 28 Aug 2015 14:40:47 -0400
From: basile pinsard <[email protected]>
To: Development and support of PyMVPA
<[email protected]>
Subject: Re: [pymvpa] Confusion Matrix for each Node with sphere_gnbsearchlight
I wanted to do the same and had to make some changes to PyMVPA here:
https://github.com/bpinsard/PyMVPA/tree/gnbsearchlight_confusiontable
using it with:
errorfx = ConfusionMatrix(labels=ds.uniquetargets)
slght = GNBSearchlight(clf, prtnr, qe, errorfx=errorfx)
On Fri, Aug 28, 2015 at 2:23 PM, marco tettamanti <[email protected]>
wrote:
Thanks again! I am on Debian testing (well, reverted on stable now,
because of troubles with gcc5) and have version 2.3.1. I will give a try
to the one from git. Best, Marco PyMVPA: Version: 2.3.1 Hash:
d1da5a749dc9cc606bd7f425d93d25464bf43454 Path:
/usr/lib/python2.7/dist-packages/mvpa2/__init__.pyc Version control (GIT):
GIT information could not be obtained due
"/usr/lib/python2.7/dist-packages/mvpa2/.. is not under GIT" SYSTEM: OS:
posix Linux 4.1.0-1-amd64 #1 SMP Debian 4.1.3-1 (2015-08-03) Distribution:
debian/stretch/sid *Yaroslav Halchenko* debian at onerussian.com
<http://onerussian.com> <pkg-exppsy-pymvpa%40lists.alioth.debian.org
<http://40lists.alioth.debian.org>?Subject=Re%3A%20%5Bpymvpa%5D%20Confusion%20Matrix%20for%20each%20Node%20with%0A%20sphere_gnbsearchlight&In-Reply-To=%3C20150828161509.GS19455%40onerussian.com
<http://40onerussian.com>%3E> *Fri Aug 28 16:15:09 UTC 2015*
--------------------------------------------------------------------------------
On Fri, 28 Aug 2015, marco tettamanti wrote:
* Dear Yaroslav,
*>* thank you very much for your reply. I have made several attempts,
trying *>* to guess a solution, but it seems I always get a *>*
'TypeError: 'NoneType' object is not callable'. * oh shoot... forgotten
that this one was implemented after the last 2.4.0 release: in
upstream/2.4.0-34-g55e147e this June... we should release I guess. what
system are you on and what version of pymvpa currently? if you could
use/try the one from git directly... ?
* Case 1:
*>* slght = sphere_gnbsearchlight(clf, partitioner, radius=slradius, *>*
space='voxel_indices', errorfx=None, postproc=mean_sample()) * not the
problem here BUT there should be no mean_sample() if errorfx is None --
you wouldn't want to average labels ;) -- Yaroslav O. Halchenko,
Ph.D.http://neuro.debian.net http://www.pymvpa.org http://www.fail2ban.org
Research Scientist, Psychological and Brain Sciences Dept. Dartmouth
College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755 Phone: +1
(603) 646-9834 Fax: +1 (603) 646-1419 WWW:
http://www.linkedin.com/in/yarik On 08/28/2015 05:28 PM, marco tettamanti
wrote: Dear Yaroslav, thank you very much for your reply. I have made
several attempts, trying to guess a solution, but it seems I always get a
'TypeError: 'NoneType' object is not callable'. Any further advice is
greatly appreciated! Best, Marco Case 1: slght =
sphere_gnbsearchlight(clf, partitioner, radius=slradius,
space='voxel_indices', errorfx=None, postproc=mean_sample()) slght_map =
slght(fds) In [70]: slght = sphere_gnbsearchlight(clf, partitioner,
radius=slradius, space='voxel_indices', errorfx=None,
postproc=mean_sample()) In [71]: slght_map = slght(fds) [SLC] DBG: Phase
1. Initializing partitions using <NFoldPartitioner> on <Dataset:
108x111@float32, <sa: chunks,targets,time_coords,time_indices>, <fa:
voxel_indices>, <a:
imgaffine,imghdr,imgtype,mapper,voxel_dim,voxel_eldim>> [SLC] DBG: Phase
2. Blocking data for 18 splits and 3 labels [SLC] DBG: Phase 3. Computing
statistics for 54 blocks [SLC] DBG: Phase 4. Deducing neighbors
information for 111 ROIs [SLC] DBG: Phase 4b. Converting neighbors to
sparse matrix representation [SLC] DBG: Phase 5. Major loop [SLC] DBG:
Split 0 out of 18 [SLC] DBG: 'Training' is done [SLC] DBG: Doing
'Searchlight' [SLC] DBG: Assessing accuracies
--------------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-71-1146d298ca06> in <module>() ----> 1 slght_map =
slght(fds) /usr/lib/python2.7/dist-packages/mvpa2/base/learner.pyc in
__call__(self, ds) 257 "used and auto training is disabled." 258 %
str(self)) --> 259 return super(Learner, self).__call__(ds) 260 261
/usr/lib/python2.7/dist-packages/mvpa2/base/node.pyc in __call__(self, ds)
119 120 self._precall(ds) --> 121 result = self._call(ds) 122 result =
self._postcall(ds, result) 123
/usr/lib/python2.7/dist-packages/mvpa2/measures/searchlight.pyc in
_call(self, dataset) 141 142 # pass to subclass --> 143 results =
self._sl_call(dataset, roi_ids, nproc) 144 145 if 'mapper' in dataset.a:
/usr/lib/python2.7/dist-packages/mvpa2/measures/adhocsearchlightbase.pyc
in _sl_call(self, dataset, roi_ids, nproc) 513 # error functions without a
chance to screw up 514 for i, fpredictions in enumerate(predictions.T):
--> 515 results[isplit, i] = errorfx(fpredictions, targets) 516 517
TypeError: 'NoneType' object is not callable Similarly for other cases and
combinations of them: Case 2: slght = sphere_gnbsearchlight(clf,
partitioner, radius=slradius, space='voxel_indices',
errorfx=ConfusionMatrixError(), postproc=mean_sample()) slght_map =
slght(fds) Case3: class KeepConfusionMatrix(Node): def _call(self, fds):
out = np.zeros(1, dtype=object) out[0] = (fds.samples) return out slght =
sphere_gnbsearchlight(clf, partitioner, errorfx=None, radius=slradius,
space='voxel_indices', postproc=ChainNode([Confusion(labels=fds.UT)]))
slght.postproc.append(KeepConfusionMatrix()) slght_map = slght(fds) Case4:
class KeepConfusionMatrix(Node): def _call(self, fds): out = np.zeros(1,
dtype=object) out[0] = (fds.samples) return out slght =
sphere_gnbsearchlight(clf, partitioner, errorfx=None, radius=slradius,
space='voxel_indices',
postproc=ChainNode([mean_sample(),Confusion(labels=fds.UT)]))
slght.postproc.append(KeepConfusionMatrix()) slght_map = slght(fds) Case5:
class KeepConfusionMatrix(Node): def _call(self, fds): out = np.zeros(1,
dtype=object) out[0] = (fds.samples) return out slght =
sphere_gnbsearchlight(clf, partitioner, errorfx=ConfusionMatrixError(),
radius=slradius, space='voxel_indices',
postproc=ChainNode([mean_sample(),Confusion(labels=fds.UT)]))
slght.postproc.append(KeepConfusionMatrix()) slght_map = slght(fds)
Yaroslav Halchenko debian at onerussian.com <http://onerussian.com> Fri
Aug 28 13:16:38 UTC 2015 quick an possible partial reply 1. "not sure" --
if it pukes then probably not, although judging from the code I foresaw
arbitrary shape of the errorfx output 2. but you could make
sphere_gnbsearchlight to return labels (not errors) and then post-process
to get those confusion matrices. Just specify errorfx=None to it (not to
CV). But you could also try passing errorfx=ConfusionMatrixError and see
how that goes Please share what you discover/end up with.
mvpa2/tests/test_usecases.py <http://usecases.py> has more of usecase
demos for gnb searchlights which might come handy -- Yaroslav O.
Halchenko, Ph.D.http://neuro.debian.net http://www.pymvpa.org
http://www.fail2ban.org Research Scientist, Psychological and Brain
Sciences Dept. Dartmouth College, 419 Moore Hall, Hinman Box 6207,
Hanover, NH 03755 Phone: +1 (603) 646-9834 Fax: +1 (603) 646-1419 WWW:
http://www.linkedin.com/in/yarik On 08/28/2015 01:48 PM, marco tettamanti
wrote: Dear all, is it possible to obtain confusion matrices for all nodes
with "sphere_gnbsearchlight", as was suggested before with
"sphere_searchlight": slcvte = CrossValidation(clf, partitioner,
errorfx=None, postproc=ChainNode([Confusion(labels=fds.UT)])) class
KeepConfusionMatrix(Node): def _call(self, fds): out = np.zeros(1,
dtype=object) out[0] = (fds.samples) return out
slcvte.postproc.append(KeepConfusionMatrix()) slght =
sphere_searchlight(slcvte, radius=slradius, space='voxel_indices',
nproc=4, postproc=mean_sample()) slght_map = slght(fds) Thank you and best
wishes, Marco -- Marco Tettamanti, Ph.D. Nuclear Medicine Department &
Division of Neuroscience San Raffaele Scientific Institute Via Olgettina
58 I-20132 Milano, Italy Phone ++39-02-26434888 Fax ++39-02-26434892
Email: [email protected] Skype: mtettamanti
--------------------------------------------------------------------------------
Pkg-ExpPsy-PyMVPA mailing list [email protected]
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-exppsy-pymvpa
--
Basile Pinsard
*PhD
candidate, *
Laboratoire d'Imagerie Biom?dicale, UMR S 1146 / UMR 7371, Sorbonne
Universit?s, UPMC, INSERM, CNRS
*Brain-Cognition-Behaviour Doctoral School **, *ED3C*, *UPMC, Sorbonne
Universit?s
Biomedical Sciences Doctoral School, Faculty of Medicine, Universit? de
Montr?al
CRIUGM, Universit? de Montr?al
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.alioth.debian.org/pipermail/pkg-exppsy-pymvpa/attachments/20150828/fc793f7b/attachment.html>
--------------------------------------------------------------------------------
