Thanks again Yarik, for helping with RFE! While working on this again, I ran the script for RFE and uncommented line 62:
zscore(ds, chunks_attr=None) using chunks_attr=None as you suggested since I only have two samples per chunk. However, I got much lower accuracies which is odd... SMLR: 35.42% ACC SVM: 50% ACC RFE (overfit): 43.75% ACC RFE (overfit less?): 43.75% ACC RFE (half/splits+rank order): 50% ACC In looking at the documentation for zscoremapper, I wonder if it's possible for zscoremapper to zscore voxel-wise or per target per chunk/run? It seems like it only zscores sample/target wise. I appreciate your help! Thank you, Kimberly Zhou On Fri, Jul 8, 2011 at 7:00 AM, < [email protected]> wrote: > Send Pkg-ExpPsy-PyMVPA mailing list submissions to > [email protected] > > To subscribe or unsubscribe via the World Wide Web, visit > > http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-exppsy-pymvpa > > or, via email, send a message with subject or body 'help' to > [email protected] > > You can reach the person managing the list at > [email protected] > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of Pkg-ExpPsy-PyMVPA digest..." > > > Today's Topics: > > 1. Re: RFE and dataset splits (Yaroslav Halchenko) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Thu, 7 Jul 2011 11:04:08 -0400 > From: Yaroslav Halchenko <[email protected]> > To: [email protected] > Subject: Re: [pymvpa] RFE and dataset splits > Message-ID: <[email protected]> > Content-Type: text/plain; charset="utf-8" > > just to catch up on things: > > cons: indeed recipe was not correctly adopted from 0.4.x > pros: no changes needed in PyMVPA itself > > see attached a complete script with few ways for RFE, depending on what you > take for the sensitivity and for the error to decide on when to stop. I > was > going also implement a "biased" version, close to the original RFE version > from > Guyon, where stopping point is chosen by having a sneak view and the > cross-validation held-out set, which would have negative consequences on > significance testing, which is a corner-stone of machine-learning in > neuroimaging... so I haven't done that -- only truly unbiased, although > possibly slightly overfitting versions available ;-) > > we are yet to adjust the webpage describing the RFE in 0.6.x > > on the dataset you have provided, here is what I got with the script (d'oh > -- > forgotten about zscoring being commented out): > > SMLR: 58.33% ACC > SVM: 64.58% ACC > RFE (overfit): 66.67% ACC > RFE (overfit less?): 66.67% ACC > RFE (half/splits+rank order): 64.58% ACC > > altogether -- results are comparable with SMLR and I bet with a simple > nested > cross-validation to choose lm parameter for SMLR you might get better > results for lower computing cost > > > On Wed, 06 Jul 2011, Yaroslav Halchenko wrote: > > > neh -- that was a good call from me but still not it -- just read it > > nevertheless ;) > > -- > =------------------------------------------------------------------= > Keep in touch www.onerussian.com > Yaroslav Halchenko www.ohloh.net/accounts/yarikoptic > -------------- next part -------------- > A non-text attachment was scrubbed... > Name: dataset_rfe.py > Type: text/x-python > Size: 12162 bytes > Desc: not available > URL: < > http://lists.alioth.debian.org/pipermail/pkg-exppsy-pymvpa/attachments/20110707/692e692a/attachment-0001.py > > > > ------------------------------ > > _______________________________________________ > Pkg-ExpPsy-PyMVPA mailing list > [email protected] > http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-exppsy-pymvpa > > End of Pkg-ExpPsy-PyMVPA Digest, Vol 41, Issue 6 > ************************************************ >
_______________________________________________ Pkg-ExpPsy-PyMVPA mailing list [email protected] http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-exppsy-pymvpa
