Re: [pymvpa] kNN and sensitivity map

[Yaroslav Halchenko]

On Tue, 31 May 2016, marco tettamanti wrote:

> Dear all,
> since kNN performs best on a particular dataset, I am trying to obtain a
> sensitivity map based on the code that I have used for other classifiers.
> However, I receive an error message, and I am not able to proceed further.
> I have been struggling with the documentation, but I still cannot find any
> solutions for this problem.
> Can somebody please help?

kNN (depending on your k) could be a highly non-linear classifier and it
wouldn't be as easy to estimate its sensitivity as in the case of linear
classifiers.  With relatively large k, decision surface could be
approximately linear and then some kind of 'trivial sensitivity' could
be assessed (I remember Francisco Pereira talking about that in one of
his paper) but we don't have it implemented.  

The only quick way to get a sensitivity for you would be to try

NoisePerturbationSensitivity

but it might take awhile depending on the size of your dataset ;)

Myself, haven't used that functionality for ages, so although
(unit)tested -- mileage might vary

-- 
Yaroslav O. Halchenko
Center for Open Neuroscience http://centerforopenneuroscience.org
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834   Fax: +1 (603) 646-1419
WWW:   http://www.linkedin.com/in/yarik

___
Pkg-ExpPsy-PyMVPA mailing list
Pkg-ExpPsy-PyMVPA@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-exppsy-pymvpa

Re: [pymvpa] kNN and sensitivity map

[Richard Dinga]

You cannot use sensitivity banalysis with KNN because kNN is not using or
producing any feature weights.

On Tue, May 31, 2016 at 5:39 PM, marco tettamanti 
wrote:

> Dear all,
> since kNN performs best on a particular dataset, I am trying to obtain a
> sensitivity map based on the code that I have used for other classifiers.
> However, I receive an error message, and I am not able to proceed further.
> I have been struggling with the documentation, but I still cannot find any
> solutions for this problem.
> Can somebody please help?
> Thank you and best wishes,
> Marco
>
>
>
> This is the relevant snippet:
>
> {fds == fmri dataset}
> clf = kNN(k=2, dfx=one_minus_correlation, voting='majority')
> partitioner = HalfPartitioner()
> fraction=1
> fselaov = SensitivityBasedFeatureSelection(OneWayAnova(),
> FractionTailSelector(fraction, mode='select', tail='upper'))
> fclf = FeatureSelectionClassifier(clf, fselaov)
> fselcvte = CrossValidation(fclf, partitioner, errorfx=lambda p, t:
> np.mean(p == t), postproc=mean_sample(), enable_ca=['confusion', 'stats'])
> res_fsel = fselcvte(fds)
> sensana = fclf.get_sensitivity_analyzer(postproc=maxofabs_sample())
>
>
>
>
> This is the error message:
>
> ---
> NotImplementedErrorTraceback (most recent call
> last)
>   in() > 1sensana =
> fclf.get_sensitivity_analyzer(postproc=maxofabs_sample())
> 2  #sensana = fclf.get_sensitivity_analyzer(postproc=absolute_features())
> 3  #sensana = fclf.get_sensitivity_analyzer(postproc=mean_sample())
> 4  cv_sensana=  RepeatedMeasure(sensana,  ChainNode((partitioner,
> Splitter('partitions',  attr_values=(1,)
> 5  sens=  cv_sensana(fds)
>
> /usr/lib/python2.7/dist-packages/mvpa2/misc/args.pyc
> indo_group_kwargs(self, *args_, **kwargs_) 73 if passthrough:
> kwargs__[k]  =  skwargs
> 74  if  assign:  setattr(self,  '_%s'  %  k,  skwargs)
> ---> 75return method(self,  *args_,  **kwargs__)
> 76  do_group_kwargs.func_name=  method.func_name
> 77  return  do_group_kwargs
>
> /usr/lib/python2.7/dist-packages/mvpa2/clfs/meta.pyc
> inget_sensitivity_analyzer(self, slave_kwargs, **kwargs) 329 return
> self.__sa_class__( 330 self,
> --> 331analyzer=self.__clf.get_sensitivity_analyzer(**slave_kwargs),
> 332  **kwargs)
> 333
> /usr/lib/python2.7/dist-packages/mvpa2/clfs/base.pyc
> inget_sensitivity_analyzer(self, **kwargs) 492 """Factory method to return
> an appropriate sensitivity analyzer for 493 the respective classifier."""
> --> 494raise NotImplementedError
> 495  496
> NotImplementedError:
>
>
>
>
> This are the code and OS versions:
>
> Current date:   2016-05-31 17:22
> PyMVPA:
>  Version:   2.5.0
>  Hash:  6a9d4060ad863f99170801854c272b61af51f015
>  Path:  /usr/lib/python2.7/dist-packages/mvpa2/__init__.pyc
>  Version control (GIT):
>  GIT information could not be obtained due
> "/usr/lib/python2.7/dist-packages/mvpa2/.. is not under GIT"
> SYSTEM:
>  OS:posix Linux 4.3.0-1-amd64 #1 SMP Debian 4.3.3-7
> (2016-01-19)
>  Distribution:  debian/stretch/sid
> EXTERNALS:
>  Present:   atlas_fsl, cPickle, ctypes, good
> scipy.stats.rv_continuous._reduce_func(floc,fscale), good
> scipy.stats.rv_discrete.ppf, griddata, gzip, h5py, hdf5, ipython, joblib,
> liblapack.so, libsvm, libsvm verbosity control, lxml, matplotlib, mdp, mdp
> ge 2.4, mock, nibabel, nipy, nose, numpy, numpy_correct_unique, pprocess,
> pylab, pylab plottable, pywt, pywt wp reconstruct, reportlab, running
> ipython env, scipy, scipy.weave, sg ge 0.6.4, sg ge 0.6.5,
> sg_fixedcachesize, shogun, shogun.mpd, shogun.svmocas, skl, statsmodels
>  Absent:atlas_pymvpa, cran-energy, datalad, elasticnet, glmnet,
> good scipy.stats.rdist, hcluster, lars, mass, nipy.neurospin, numpydoc,
> openopt, pywt wp reconstruct fixed, rpy2, shogun.krr, shogun.lightsvm,
> shogun.svrlight, weave
>  Versions of critical externals:
>   ctypes  : 1.1.0
>   h5py: 2.5.0
>   hdf5: 1.8.13
>   ipython : 2.3.0
>   joblib  : 0.9.4
>   lxml: 3.4.4
>   matplotlib  : 1.4.2
>   mdp : 3.5
>   mock: 1.3.0
>   nibabel : 2.0.2
>   nipy: 0.4.0.dev
>   numpy   : 1.8.2
>   pprocess: 0.5
>   reportlab   : 3.2.0
>   scipy   : 0.14.1
>   shogun  : 3.2.0
>   shogun:full : 3.2.0_2014-2-17_18:46
>   shogun:rev  : 197120
>   skl : 0.17.1
>  Matplotlib backend: module://IPython.kernel.zmq.pylab.backend_inline
>
>
> --
> Marco Tettamanti, Ph.D.
> Nuclear Medicine Department & Division of Neuroscience
> San Raffaele Scientific Institute
> Via Olgettina 58
> I-20132 Milano, Italy
> Phone ++39-02-26434888
> Fax ++39-02-26434892
> Email: tettamanti.ma...@hsr.it
> Skype: mtettamanti
>
>
> ___
> Pkg-ExpPsy-PyMVPA mailing list
> Pkg-ExpPsy-PyMVPA@lists.alioth.debian.org
>