I haven't checked, but make_scorer(v_measure_score) should suffice.
On 15 May 2014 01:12, Lee Zamparo <zamp...@gmail.com> wrote:
> Combining the helpful suggestions of Andy & Joel I'm tyring the following:
>
> # Make a scoring function for the pipeline
> v_measure_scorer =
>
> make_scorer(v_measure_score,labels_true=labels[:,0],labels_pred=kmeans.predict)
>
> # Parameters of pipelines are set using ‘__’ separated parameter names:
> estimator = GridSearchCV(pipe, dict(kpca__gamma=gammas),
> scoring=v_measure_scorer)
> estimator.fit(D_scaled)
>
> Was this what you were referring to Andy?
>
> Thanks,
>
> Lee.
>
> On Wed, May 14, 2014 at 1:27 AM, Andreas Mueller <t3k...@gmail.com> wrote:
> > I think you should use the make_scorer function. Using labels_ will not
> > work, as it will only have labels for the training split, while the
> > performance is measured on the test split.
> >
> > On May 14, 2014 2:28 AM, "Joel Nothman" <joel.noth...@gmail.com> wrote:
> >>
> >> Hi Lee,
> >>
> >> The scoring parameter, if not an existing scoring name, needs to be a
> >> function with the signature:
> >>
> >> fn(estimator, X, y_true) -> score which increases with goodness
> >>
> >> So I think you want to define:
> >>
> >> def score_clusters(estimator, X, y):
> >> return v_measure_score(y[:,0], kmeans.labels_))
> >>
> >> Then construct the GridSearchCV as:
> >>
> >> estimator = GridSearchCV(pipe, dict(kpca__gamma=gammas),
> >> scoring=score_clusters)
> >>
> >> It seems like there should be more predefined scorers available for
> >> clustering...
> >>
> >> Cheers,
> >>
> >> - Joel
> >>
> >>
> >> On 14 May 2014 09:10, Lee Zamparo <zamp...@gmail.com> wrote:
> >>>
> >>> Hi,
> >>>
> >>> I'm trying to use GridSearchCV and Pipeline to tune the gamma
> >>> parameter of kernel PCA. I'd like to use kernel PCA to transform the
> >>> data, followed by kmeans to cluster the data, followed by v-measure to
> >>> measure the goodness of fit of the clustering.
> >>>
> >>> Here's the relevant snippet of my script
> >>> -----
> >>> # Set up the kPCA -> kmeans -> v-measure pipeline
> >>> kpca = KernelPCA(kernel="rbf")
> >>> kmeans = KMeans(n_clusters=3)
> >>> pipe = Pipeline(steps=[('kpca', kpca), ('kmeans', kmeans)])
> >>>
> >>> # Range of parameters to consider for gamma in the RBF kernel for kPCA
> >>> gammas = np.logspace(-10,2,num=100)
> >>>
> >>> # Parameters of pipelines are set using ‘__’ separated parameter names:
> >>> estimator = GridSearchCV(pipe, dict(kpca__gamma=gammas),
> >>> scoring=v_measure_score(labels[:,0],kmeans.labels_))
> >>> estimator.fit(D_scaled)
> >>>
> >>> -----
> >>>
> >>> Yet I get an AttributeError claiming that the kmeans object has no
> >>> labels_ attribute.
> >>>
> >>> File "/home/lee/projects/SdA_reduce/utils/kernel_pca_pipeline.py",
> >>> line 86, in <module>
> >>> estimator = GridSearchCV(pipe, dict(kpca__gamma=gammas),
> >>> scoring=v_measure_score(labels[:,0],kmeans.labels_))
> >>>
> >>> AttributeError: 'KMeans' object has no attribute 'labels_'
> >>>
> >>> Does anyone have any tips on how I should restructure my snippet to
> >>> get my desired outcome?
> >>>
> >>> Thanks,
> >>>
> >>> Lee.
> >>>
> >>>
> >>>
> ------------------------------------------------------------------------------
> >>> "Accelerate Dev Cycles with Automated Cross-Browser Testing - For FREE
> >>> Instantly run your Selenium tests across 300+ browser/OS combos.
> >>> Get unparalleled scalability from the best Selenium testing platform
> >>> available
> >>> Simple to use. Nothing to install. Get started now for free."
> >>> http://p.sf.net/sfu/SauceLabs
> >>> _______________________________________________
> >>> Scikit-learn-general mailing list
> >>> Scikit-learn-general@lists.sourceforge.net
> >>> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
> >>
> >>
> >>
> >>
> >>
> ------------------------------------------------------------------------------
> >> "Accelerate Dev Cycles with Automated Cross-Browser Testing - For FREE
> >> Instantly run your Selenium tests across 300+ browser/OS combos.
> >> Get unparalleled scalability from the best Selenium testing platform
> >> available
> >> Simple to use. Nothing to install. Get started now for free."
> >> http://p.sf.net/sfu/SauceLabs
> >> _______________________________________________
> >> Scikit-learn-general mailing list
> >> Scikit-learn-general@lists.sourceforge.net
> >> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
> >>
> >
> >
> ------------------------------------------------------------------------------
> > "Accelerate Dev Cycles with Automated Cross-Browser Testing - For FREE
> > Instantly run your Selenium tests across 300+ browser/OS combos.
> > Get unparalleled scalability from the best Selenium testing platform
> > available
> > Simple to use. Nothing to install. Get started now for free."
> > http://p.sf.net/sfu/SauceLabs
> > _______________________________________________
> > Scikit-learn-general mailing list
> > Scikit-learn-general@lists.sourceforge.net
> > https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
> >
>
>
> ------------------------------------------------------------------------------
> "Accelerate Dev Cycles with Automated Cross-Browser Testing - For FREE
> Instantly run your Selenium tests across 300+ browser/OS combos.
> Get unparalleled scalability from the best Selenium testing platform
> available
> Simple to use. Nothing to install. Get started now for free."
> http://p.sf.net/sfu/SauceLabs
> _______________________________________________
> Scikit-learn-general mailing list
> Scikit-learn-general@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
>
------------------------------------------------------------------------------
"Accelerate Dev Cycles with Automated Cross-Browser Testing - For FREE
Instantly run your Selenium tests across 300+ browser/OS combos.
Get unparalleled scalability from the best Selenium testing platform available
Simple to use. Nothing to install. Get started now for free."
http://p.sf.net/sfu/SauceLabs
_______________________________________________
Scikit-learn-general mailing list
Scikit-learn-general@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
