Hello,
I have a little question about Random Search vs Grid Search. I am used to
grid search over parameter increasing exponentially (lambda = 0.01, 0.1,
... 1000). When you use randomized search, what type of distribution do you
use ? If I use an uniform distribution, I would test hyper-parameter values
that are in the same order of magnitude.
Thank you,
Arnaud
2015-04-07 21:28 GMT+02:00 Andreas Mueller <t3k...@gmail.com>:
> I talks about it a bunch here:
> https://www.youtube.com/watch?v=0wUF_Ov8b0A
>
> The paper is here: http://www.jmlr.org/papers/v13/bergstra12a.html
>
> I'd be interested in the blog post.
> Really I don't think searching over losses and regularization will give
> you much usually.
> I have not really seen any experienced machine learner do it.
>
>
> On 04/07/2015 03:01 PM, Jason Wolosonovich wrote:
>
> Hi Roberto,
>
>
>
> I’m no expert by any means, but I was reading a blog post the other day
> that talked about using Random Search vs Grid Search. The gist of the
> article is that, since you can feed distributions to Random Search and it
> selects values randomly over the number of iterations you choose, it is a
> better initial choice when you’re not sure which parameters/combinations to
> use (which is usually my case J) and you’ll end up with Random Search
> finding more useful parameters faster than if you tell Grid Search to
> search over combinations (some of which may have no potential to help you).
> Then when you see the results of the Random Search, you can use that
> information to search a narrower range of values/parameters (a finer grid)
> exhaustively using Grid Search.
>
>
>
> Unfortunately, I thought I bookmarked the article but I can’t find it.
> I’ll keep looking though and send it out if I do.
>
>
>
> Additionally, the docs for the individual estimators in Sklearn tell you
> what parameters are not valid with each other, so you wouldn’t want to put
> those parameters together in your param_grid dictionary. For your
> dictionary (as others have already mentioned) just make sure that you only
> provide options in each of your dictionaries that can be used together. You
> can pass a list of dictionaries to param_grid like Sebastian just
> demonstrated.
>
>
>
> Check the links below as well, Random Search comes up with just about the
> same results as Grid Search, but faster/more efficiently. Hope this helps.
>
>
>
>
>
> Scikit Docs:
>
> http://scikit-learn.org/stable/modules/grid_search.html#grid-search-tips
>
>
> http://scikit-learn.org/stable/auto_examples/model_selection/randomized_search.html#example-model-selection-randomized-search-py
>
>
>
>
>
> -Jason
>
>
>
> *From:* Pagliari, Roberto [mailto:rpagli...@appcomsci.com
> <rpagli...@appcomsci.com>]
> *Sent:* Tuesday, April 07, 2015 9:24 AM
> *To:* scikit-learn-general@lists.sourceforge.net
> *Subject:* [Scikit-learn-general] CV with SVM
>
>
>
> not all combinations of cost/loss functions and dual are possible with
> SVM.
>
> when performing grid search with CV, does sklearn skip invalid
> combinations?
>
> Thank you,
>
>
> ------------------------------------------------------------------------------
> BPM Camp - Free Virtual Workshop May 6th at 10am PDT/1PM EDT
> Develop your own process in accordance with the BPMN 2 standard
> Learn Process modeling best practices with Bonita BPM through live
> exerciseshttp://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual-
> event?utm_
> source=Sourceforge_BPM_Camp_5_6_15&utm_medium=email&utm_campaign=VA_SF
>
>
>
> _______________________________________________
> Scikit-learn-general mailing
> listScikit-learn-general@lists.sourceforge.nethttps://lists.sourceforge.net/lists/listinfo/scikit-learn-general
>
>
>
>
> ------------------------------------------------------------------------------
> BPM Camp - Free Virtual Workshop May 6th at 10am PDT/1PM EDT
> Develop your own process in accordance with the BPMN 2 standard
> Learn Process modeling best practices with Bonita BPM through live
> exercises
> http://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual-
> event?utm_
> source=Sourceforge_BPM_Camp_5_6_15&utm_medium=email&utm_campaign=VA_SF
> _______________________________________________
> Scikit-learn-general mailing list
> Scikit-learn-general@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
>
>
------------------------------------------------------------------------------
BPM Camp - Free Virtual Workshop May 6th at 10am PDT/1PM EDT
Develop your own process in accordance with the BPMN 2 standard
Learn Process modeling best practices with Bonita BPM through live exercises
http://www.bonitasoft.com/be-part-of-it/events/bpm-camp-virtual- event?utm_
source=Sourceforge_BPM_Camp_5_6_15&utm_medium=email&utm_campaign=VA_SF
_______________________________________________
Scikit-learn-general mailing list
Scikit-learn-general@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
