oops, sorry that's an artifact from a draft of that mesage. Yes, I
went with your suggestion. Specifying any kwargs to v_measure_score
in make_scorer raises AttributeErrors.
Thanks again,
L.
On Thu, May 15, 2014 at 8:17 AM, Joel Nothman <joel.noth...@gmail.com> wrote:
> I don't think
> v_measure_scorer = make_scorer(v_measure_score, labels_pred=kmeans.predict)
> does what you think it does.
>
> You should stick to
> v_measure_scorer = make_scorer(v_measure_score)
>
>
>
> On 15 May 2014 22:11, Lee Zamparo <zamp...@gmail.com> wrote:
>>
>> Seems the estimator.fit method needs the true labels, and that I
>> shouldn't pass either the true lables or the predicted labels to
>> v_measure_score (passing either triggers an AttributeError). So now
>> I'm running with
>>
>> # Make a scoring function for the pipeline
>> v_measure_scorer = make_scorer(v_measure_score,
>> labels_pred=kmeans.predict)
>>
>> # Parameters of pipelines are set using ‘__’ separated parameter names:
>> estimator = GridSearchCV(pipe, dict(kpca__gamma=gammas),
>> scoring=v_measure_scorer)
>> estimator.fit(D_scaled,D_labels)
>>
>> It's been running overnight, hopefully I get a result this morning.
>> Thanks for all your help,
>>
>> L.
>>
>> On Wed, May 14, 2014 at 11:12 AM, Lee Zamparo <zamp...@gmail.com> wrote:
>> > Combining the helpful suggestions of Andy & Joel I'm tyring the
>> > following:
>> >
>> > # Make a scoring function for the pipeline
>> > v_measure_scorer =
>> >
>> > make_scorer(v_measure_score,labels_true=labels[:,0],labels_pred=kmeans.predict)
>> >
>> > # Parameters of pipelines are set using ‘__’ separated parameter names:
>> > estimator = GridSearchCV(pipe, dict(kpca__gamma=gammas),
>> > scoring=v_measure_scorer)
>> > estimator.fit(D_scaled)
>> >
>> > Was this what you were referring to Andy?
>> >
>> > Thanks,
>> >
>> > Lee.
>> >
>> > On Wed, May 14, 2014 at 1:27 AM, Andreas Mueller <t3k...@gmail.com>
>> > wrote:
>> >> I think you should use the make_scorer function. Using labels_ will not
>> >> work, as it will only have labels for the training split, while the
>> >> performance is measured on the test split.
>> >>
>> >> On May 14, 2014 2:28 AM, "Joel Nothman" <joel.noth...@gmail.com> wrote:
>> >>>
>> >>> Hi Lee,
>> >>>
>> >>> The scoring parameter, if not an existing scoring name, needs to be a
>> >>> function with the signature:
>> >>>
>> >>> fn(estimator, X, y_true) -> score which increases with goodness
>> >>>
>> >>> So I think you want to define:
>> >>>
>> >>> def score_clusters(estimator, X, y):
>> >>> return v_measure_score(y[:,0], kmeans.labels_))
>> >>>
>> >>> Then construct the GridSearchCV as:
>> >>>
>> >>> estimator = GridSearchCV(pipe, dict(kpca__gamma=gammas),
>> >>> scoring=score_clusters)
>> >>>
>> >>> It seems like there should be more predefined scorers available for
>> >>> clustering...
>> >>>
>> >>> Cheers,
>> >>>
>> >>> - Joel
>> >>>
>> >>>
>> >>> On 14 May 2014 09:10, Lee Zamparo <zamp...@gmail.com> wrote:
>> >>>>
>> >>>> Hi,
>> >>>>
>> >>>> I'm trying to use GridSearchCV and Pipeline to tune the gamma
>> >>>> parameter of kernel PCA. I'd like to use kernel PCA to transform the
>> >>>> data, followed by kmeans to cluster the data, followed by v-measure
>> >>>> to
>> >>>> measure the goodness of fit of the clustering.
>> >>>>
>> >>>> Here's the relevant snippet of my script
>> >>>> -----
>> >>>> # Set up the kPCA -> kmeans -> v-measure pipeline
>> >>>> kpca = KernelPCA(kernel="rbf")
>> >>>> kmeans = KMeans(n_clusters=3)
>> >>>> pipe = Pipeline(steps=[('kpca', kpca), ('kmeans', kmeans)])
>> >>>>
>> >>>> # Range of parameters to consider for gamma in the RBF kernel for
>> >>>> kPCA
>> >>>> gammas = np.logspace(-10,2,num=100)
>> >>>>
>> >>>> # Parameters of pipelines are set using ‘__’ separated parameter
>> >>>> names:
>> >>>> estimator = GridSearchCV(pipe, dict(kpca__gamma=gammas),
>> >>>> scoring=v_measure_score(labels[:,0],kmeans.labels_))
>> >>>> estimator.fit(D_scaled)
>> >>>>
>> >>>> -----
>> >>>>
>> >>>> Yet I get an AttributeError claiming that the kmeans object has no
>> >>>> labels_ attribute.
>> >>>>
>> >>>> File "/home/lee/projects/SdA_reduce/utils/kernel_pca_pipeline.py",
>> >>>> line 86, in <module>
>> >>>> estimator = GridSearchCV(pipe, dict(kpca__gamma=gammas),
>> >>>> scoring=v_measure_score(labels[:,0],kmeans.labels_))
>> >>>>
>> >>>> AttributeError: 'KMeans' object has no attribute 'labels_'
>> >>>>
>> >>>> Does anyone have any tips on how I should restructure my snippet to
>> >>>> get my desired outcome?
>> >>>>
>> >>>> Thanks,
>> >>>>
>> >>>> Lee.
>> >>>>
>> >>>>
>> >>>>
>> >>>> ------------------------------------------------------------------------------
>> >>>> "Accelerate Dev Cycles with Automated Cross-Browser Testing - For
>> >>>> FREE
>> >>>> Instantly run your Selenium tests across 300+ browser/OS combos.
>> >>>> Get unparalleled scalability from the best Selenium testing platform
>> >>>> available
>> >>>> Simple to use. Nothing to install. Get started now for free."
>> >>>> http://p.sf.net/sfu/SauceLabs
>> >>>> _______________________________________________
>> >>>> Scikit-learn-general mailing list
>> >>>> Scikit-learn-general@lists.sourceforge.net
>> >>>> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>
>> >>> ------------------------------------------------------------------------------
>> >>> "Accelerate Dev Cycles with Automated Cross-Browser Testing - For FREE
>> >>> Instantly run your Selenium tests across 300+ browser/OS combos.
>> >>> Get unparalleled scalability from the best Selenium testing platform
>> >>> available
>> >>> Simple to use. Nothing to install. Get started now for free."
>> >>> http://p.sf.net/sfu/SauceLabs
>> >>> _______________________________________________
>> >>> Scikit-learn-general mailing list
>> >>> Scikit-learn-general@lists.sourceforge.net
>> >>> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
>> >>>
>> >>
>> >>
>> >> ------------------------------------------------------------------------------
>> >> "Accelerate Dev Cycles with Automated Cross-Browser Testing - For FREE
>> >> Instantly run your Selenium tests across 300+ browser/OS combos.
>> >> Get unparalleled scalability from the best Selenium testing platform
>> >> available
>> >> Simple to use. Nothing to install. Get started now for free."
>> >> http://p.sf.net/sfu/SauceLabs
>> >> _______________________________________________
>> >> Scikit-learn-general mailing list
>> >> Scikit-learn-general@lists.sourceforge.net
>> >> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
>> >>
>>
>>
>> ------------------------------------------------------------------------------
>> "Accelerate Dev Cycles with Automated Cross-Browser Testing - For FREE
>> Instantly run your Selenium tests across 300+ browser/OS combos.
>> Get unparalleled scalability from the best Selenium testing platform
>> available
>> Simple to use. Nothing to install. Get started now for free."
>> http://p.sf.net/sfu/SauceLabs
>> _______________________________________________
>> Scikit-learn-general mailing list
>> Scikit-learn-general@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
>
>
>
> ------------------------------------------------------------------------------
> "Accelerate Dev Cycles with Automated Cross-Browser Testing - For FREE
> Instantly run your Selenium tests across 300+ browser/OS combos.
> Get unparalleled scalability from the best Selenium testing platform
> available
> Simple to use. Nothing to install. Get started now for free."
> http://p.sf.net/sfu/SauceLabs
> _______________________________________________
> Scikit-learn-general mailing list
> Scikit-learn-general@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
>
------------------------------------------------------------------------------
"Accelerate Dev Cycles with Automated Cross-Browser Testing - For FREE
Instantly run your Selenium tests across 300+ browser/OS combos.
Get unparalleled scalability from the best Selenium testing platform available
Simple to use. Nothing to install. Get started now for free."
http://p.sf.net/sfu/SauceLabs
_______________________________________________
Scikit-learn-general mailing list
Scikit-learn-general@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
